I always open the bleeder and put a bleeder kit on the bleeder screw when I'm pushing the pistons back in the caliper.
Yeah, you should defenitely bleed your brakes. I got air in my master cylinder when I just replaced my rotors, pads, and calipers and even though I bled the wheels, the...