City driving won't be affected much (once you get past stomping on the pedal so much). Some say you'll even save because the engine doesn't have to work as hard. I was getting around 19 mpg city with stock gears. With 4.10s, I'm getting 18 mpg.
At highway speeds, you'll see some loss if you try to maintain the same driving speeds as you did with stock gears. I used to drive 80 mph really easy and still get around 24 mpg. With 4.10s, my tach reads a hair over 3,000 RPM. No way I'll get 24 mpg holding 3,000 RPM for an hour. If I stay around 60 mph, my tach will be around 2,000 RPM. I could possibly get 24 mpg like that.
Now I understand why some fast looking cars stay in the slow lane and do 65 mph when everyone else is in the fast lanes doing 80-85 mph.