Do not downshift at all if you can help it. Yeah that noise sounds great b ut your tranny parts are saying WTF over?!!
Brakes are cheaper to replace than transmission parts.
As to what you broke, I dunno maybe pilot bearing or tob not liking you. It could also be something else.