I assume you still had access to your old phone and could approve the transfer from it. If not, then your phone number is your password, which is even worse, in my opinion (it’s basically public information).

With my limited knowledge of cryptography, this is how I understand it:

The distinction to make is that the user’s password is not the encryption key - it only gives access to the key. So even if the user has the same password on a new device, there would be no way to decrypt the data without the original key.

In order to maintain full privacy, data has to be encrypted on device before sending it through any server (whether to another participant in a chat, or for backup). This means that the encryption key has to be on device.

If that key was copied over to a location not controlled by the user (e.g. Telegram server), then that location would have access to the key and can decrypt any data encrypted by that key. In the same vein, if a user loses their phone then that encryption key must be lost, so encrypted data cannot be decrypted on a new phone.

Which means that the only way that Telegram can provide the chats on a new phone (when the user has no access to the old phone) is if they have access to the encryption key and can provide it to the new phone.