They don’t need to own all of them, just slightly more than the turnover/growth rate. If a community only has 2% of people moving in/around, and billionaires own 4% of the houses in an area, they can still jack up the prices, which changes the value of all the homes