Buddy of mine had that problem with a 90 GT. He kept driving it, then eventually the trans locked up and he had a lot of bad parts inside. I would get it fixed right away to avoid further damage.
It means the pocket bearings were starved of fluid causing them to heat up and score the mainshaft.
In 4th gear, the input and mainshaft are 1:1, so they don't spin, but the noise will be the loudest in lower gears and in reverse.
Cause is low fluid.
Sometimes, draining and refilling the fluid will silence it, but not always. If the mainshaft is scored, the trans is fine to drive on for many miles and will not blow...but it will sound annoying as hell