It is designed so that the bolt cannot come all the way out while the intermediate shaft is mated to the column shaft (to retain steering control in the event of the nut backing off). The bolt has a notch in it that straddles a flat spot on the steering column shaft, so it cannot come out without damaging something. It will allow the intermediate shaft to be seperated when the nut is removed as you have already done, though it will most likely need some penetrant to ease the process. The bottom half of the rag joint is secured to the steering rack with a single pinchbolt.
The ragjoint pins are designed to allow some control of the car ,albiet sloppy, in the event of isolater failure. I believe you will have to drill the retaining rivets to seperate the joint, after removing the intermediate shaft from the car.
Like Mark, I nave never had to replace one yet, all mine came with the rubber joint.