inside the sentence, It had not been until the Fourth Dynasty that the most famous pyramids were built, had not been does not fit inside (it doesn't make sense).
Wasn't works, because wasn't is the same as was not
It was not until the Fourth Dynasty that the most famous pyramids were built.