Valve Spring Retainer keeps breaking

I have an 89 Lx notch with a recently rebuilt 306,Crane 1.7 RR and a B303 cam. Now I only have a set of worked E6 heads (they came with the car) because when I rebuilt the motor I couldn't really find a better set. The engine has about 3000 miles on it but in the last 2 weeks has dropped the exhaust valve on cylinder 4 twice. Both times it has been because the retainer has broke which has also thankfully kept the valve from falling all the way into the cylinder. Do any of you have any ideas as to what may be causing this? I had the heads resurfaced, guides replaced and the springs checked to make sure they would work with the cam.

Well the machine shop replaced the valve guides when I first took them in. When it dropped the valve the first time I didn't even pull the head I just put a new retainer on. That makes sense though. I'll have to check to see if the rocker has been pushing on the retainer. Thanks for the suggestions.

I won't be able to check the heads out until later on this evening but what would cause the rocker to be pushing on the retainer? Could it be just a messed up valve guide or is there more to it?