First, a little background: EGR or exhaust gas recirculation is an emissions control system that is used to reduce NOx (oxides of nitrogen) emissions. Exhaust gas is basically inert (non-reactive) and, when fed into the combustion chamber in measured quantities, acts to cool the peak temperatures reached during combustion. High temperatures are where NOx are formed so keeping combustion temps down helps to reduce emissions. An added benefit of EGR is that because it reduces peak temperatures and pressures, you can run more part-throttle spark advance which helps efficiency some. Often, when the EGR system is disabled, people notice an increased detonation (spark-knock) propensity. EGR is not applied at idle nor is it applied at WOT so there is no power penalty from leaving the EGR active.
Why remove it? Good question. Some people object to the carbon film it leaves in the intake tract though they never really explain why it's such a problem. It certainly doesn't affect airflow. I think many people who remove (or want to remove it) don't understand how it works or when it is used and just think that routing exhaust back into the intake must automatically be a "bad thing."
Most people that delete it do it for looks...another reason to delete it is if you're doing a header install and the tube either breaks or wont thread into the new header...
In order to delete is you must turn the egr option off via a tuner otherwise you're car will have hesitation issues...a lot of people will claim you gas mileage will lower too..but the most possible amount of mpg's you will lose is 1/4 of a mpg...I've turned plenty of egr systems off and its very rare to see a loss in mpg's...