Ok, there are a lot of points here to tackle:
+ The function of the smog pump is simple; it pumps air into the exhaust system ONLY after a cold start to help light off the cats. (Catalysts need to heat up to 300 to 400 deg C before they can react.) The first stage pushes air through the heads into the exhaust ports to light the pup cats (coupled close to the header), then it switches to the main cats (under-floor). The engine typically runs rich during this period to give the exhaust unburned HC to mix and burn with the oxygen from the smog pump in the catalyst brick. The smog pump does NOT continuously pump air into the exhaust system. If it did, your cat would melt down half way through the first WOT.
+ Removing the smog pump will not cause any problems other than hurt your cold start emissions. It won't wear out your catalysts early or anything of the sort.
+ There is definitely some torque to be had by removing it, but the actual amount is debatable. The pump is just dumping the air after the cold start procedure has completed. This load is minimal compared to pumping it into the exhaust.
The final decision is kind of a judgment call. If you have to pass emissions tests, removing it will probably hurt your chances if their test is half-ways decent and involves a cold start. If you are running an off-road H/X pipe then you might as well pull it because it isn’t doing anything.
If you do pull it off make sure to cap all of the lines/ports securely. Fresh air making it into the exhaust system will throw off your O2 sensor signal and bias your combustion air/fuel to the rich side. Also, a continuous fresh air leak into or upstream of the cats will ultimately hurt the cats because they will run much hotter than normal.
I hope this helps!