If you drive the car everyday I would not remove the inner bumper. If it is just a weekend/track car remove both front and rear inner bumper. Just be carefull when you drive the car on the street.
I doubt the bumper would make a big difference in a major crash but in a minor one it could. Yeah, your taking a chance, it all depends on what it all means to you. I will run no bumpers on a street driven car, but not a daily driver either.