I can't seem to solve this engine problem!

Background:
'66 coupe with 289 2bbl
single vac distributor
manual steering/brakes/no AC
C4

Problem:
About a month ago, my car would start and idea little rough. When I'd put it into gear it bogs down and stalls. When driving away it would hesitate, then surge but once going faster it would drive close to normal (still hesitating at times even at cruising speeds). I drove it longer and the problem went away (I thought it was dirty fuel that cleared itself up). It happened again a week or so later so I used SeaFoam carb cleaner down the carb throat and after plumes of white smoke it cleared up and was fine. Then a few days or a week later, the same problem happened again. That brings us up to a few days ago.

I checked the vacuum pressure and it reads 15-16 (a little low). I sprayed starter fluid around the carb and manifold and didn't find any leaks to my knowledge. I then used a dwell meter and set it to 29 degrees and set the points gap to 0.018in. Then I unplugged the vacuum advance and plugged it and set the timing to 6 BTC. Then I'd plug the vacuum back in and the dwell would drop about 9 degrees and the idle would increase by about 100 rpm's. I was told that dwell should not change when the vac advance was plugged in or removed. So I bought a new distributor and installed/set dwell/gap/timing and the same problem happens. It idles very decently but when I put it in gear, it idles down and I hear a very diesely rattle (I assume coming from the exhaust below, as I sit in the car with my head out the door listening) and it hesitates and lurches forward. People prolly think I'm revving my engine at them trying to show them up when I drive slowly past in the parking lot!
Is it my transmission? I've found that the dipstick is covered in foamy fluid. This may be from when I had my fluids changed, they could have overfilled it. I read where someone also suggested replacing the coil. What do you guys think? Another thing I noticed is that I seat the carb fuel mixture screws and then back them out 1.5 turns, I can screw in the left one all the way and the engine will keep running, but the engine will stumble if I turn in the right screw too much (as to be expected). When you adjust the carb, do you screw in each screw at a time or do both screws have a turn at the same time?

*Should I have the car in gear when I adjust the timing and for that matter the idle speed as well?

I replaced the fuel filter a few days ago and also used that carb cleaner, but I still have issues when in gear. Today I realized something: I am getting about 8 inches of vacuum coming from my TIMED port that goes to the advance on the dizzy. I can push the throttle lever closed with my fingers and the reading goes down to about 1-2 inches. But blipping the throttle returns the reading back up to 8. In the past I added a second return spring because the idle wouldn't return all the way to low idle unless I tapped it shut with my foot when driving. Because I have to use two return springs to return the idle to normal, this makes me think that my throttle plate is warped or obstructed causing a vacuum leak that makes the advance change the timing at idle (which it shouldn't). Could this leak cause the hesitation and poor driving?

Is it possible to fix that bushing issue without paying for a new rebuild or a new carb?

I've just heard very good things about the Autolites and figured I'd try to stay with that but I will look into the holley's and also that bushing kit.

Thanks, D.Hearne for the knowledgable advice. I may go that route actually. I'll see what I can find on ebay and then I may put out a wanted add on here.

And thanks to everyone else for chiming in. I'll let you know what happens after I replace the carburetor to see if that relieves the problem I'm having.