Suspension Rear sway bar hitting exhaust

Well, I guess you really can't install it upside down. It's the same either way actually.
When I drive my car and go over a medium sized bump or dip, I have a clunk noise. With the sway bar removed, no clunk. I can see on the sway bar itself, that there is a spot where the exhaust makes contact. Not much, but enough to be very annoying. I have a little less than 2" clearance between the exhaust and sway bar. I can also see where the exhaust was just barely touching the gas tank plastic cover. I've read a few posts stating that the Magnaflow is problematic in these 2 exact spots where mine is having issues. I didn't really notice a difference in the driving characteristics with or without the bar installed.