An increase IN the birth rate OF the population.
In general, you could say An increase in/of the population. In the past OF seems to have been the favoured preposition, nowadays IN is in fashion.
But I'll go with Paul on this. I prefer IN the birth rate OF the population.