Let me tell you from experience NOT to remove the EGR. I removed mine when I completed the swap of the 5.0 into my '66 and it was definitely a mistake - even though I used a EGR eliminator plug.
If it was above 70 degrees outside and the car had fully warmed up, it would detonate at part throttle conditions, transitions from zero throttle to part throttle, idle, and deceleration from highway speeds.
Granted that my engine is not stock and I still have the stock tune - but the fact remains that when I reinstalled the EGR all those problems went away.
Do yourself a tuning favor and leave it on.