Need help quick! Rear Lower Control Arm.

I'm in the middle of switching my springs, so I decided to do the lower control arms at the same time. The uppers looked tough to get too, but I figured I wouldn't have a problem with the lowers... well I am!

I am having problems getting the bolt out of the lower control arm to frame connection. I got the nut off no problem, then tried hammering out the bolt from where the nut was, but it won't budge. I can't get a wrench into the hole where the bolt is because my muffler is only about an inch or so away from it. I would take off my muffler, but my entire exhaust system is all welded. I also tried putting pliers on the end that the nut was and tried to turn the bolt from that side, but there's not enough room to move the pliers and I can't get a good enough grip. Do I need to line up the control arm in some way so that I can hammer it out like I was trying before?

So what do you guys do when you switch lower control arms? Any tricks? Is it impossible to do without taking the muffler off? Thanks, I need to finish this tonight.

Thanks guys, I'll give it a shot.