Good news, bad news.
I'm almost positive I know what your problem is because I chased it for 2 years, changed tie rods, ball joints ,rack bushings, new control arms twice.
On my car the control arm bushings bind and then shift in the control arms themselves you can see wear on the metal...