Quality should be put over quantity. If your philosophy was correct on this then Africa would be thriving. The quality of education, housing, food and water are all more important than the sheer number of new people being born.
Africa is thriving, at least some countries in it are, and all are actively replacing the population of USA and Europe as we speak.
Also, China and India are both thriving, and China is even challenging USA.
You may prefer quality over quantity, but statistically, there is always a certain percentage of smart people among those who are born, so naturally the more people are born, the more smart people are born as well.
And if your birth rates are less than 2 per woman, then your population is shrinking and getting older, causing drain on economy.