Facts:
20's will give you worse gas mileage, slower acceleration, decreased braking performance, rougher ride, more stress on drivetrain all due to increased rotational mass, decreased tire life and much greater potential to scratch and damage rim lip.
Not to mention more expensive tire...