Yeah, the reverse would be super freaky. However, arranging circumstances like that would be very hard. Like, how do you prevent yourself from hearing your own speech for an extended period of time? Either way, that would be quite an experience once you switch back to normal and you can hear yourself normally again.