007 I had adjusted the rod so short the brakes would not lock up at all. Pedal was almost to the floor. The shorter rod would allow a longer drive before acting up. By longer I mean 20 minutes on surface streets. When it was the correct length I could not go around the block without the front wheels locking up on gravel. The actual problem in this case is the plunger/ springs mechanism in the booster itself.
Symptoms you describe is what I was experiencing. Short drive everything fine. Longer it was driven the more it vibrated. I had not taken it on a long drive until a couple weeks ago because of the vibrations. That long drive is when it all came together for me. Before the vibrations felt like the entire car. On the long highway drive it was yanking the steering wheel left and right.
I had replaced the flywheel, transmission, wheels and tires, added and removed shims to axel and trans. Replaced master cylinder and brake lines. Again. Trying to get this under control. I do 99% my own work but I paid several shops to help me find it. All they did was recommend throwing parts at it in the end. No one thought brake booster. Purchased two angle finders myself when they said driveline angle is fine. Bought a lazer level to shoot lines down the chassis to make sure the engine was not crooked in the engine bay.
It all came down to the N.E.W booster Never Ever Working right. Somehow the mechanism to return the pedal hangs up. Not all at once. It slowly makes the pedal lower and lower. When cold I could not lift the pedal. When hot it pops back up when lifting it.