I've had this problem by the current endgame, when I was handling thousands of buildings. Not only that, I once left my Architects out of control, and they queued tens of thousands of houses and tombs. It greatly surpassed the land limit, so it didn't let me build new buildings until I removed the queues!
Removing them by hand in stacks of 50 isn't very pleasant, that's why I suggest being able to remove them (or assign/build them) in higher stacks. Or even remove all of them! (A confirmation message would help for that)