mine was the exact same way......then my motor went out all together.....i am going to replace it tommorrow and see what happens.....i thought it was probably my switch too,but i never got around to replacing it...i will let you know what happens tommorrow after i change out the motor whether it still works or not...you might have a loose wire somewhere that is causing a bad connection...
ok, ill try my hand at this one. Right now Im looking at the wiring schematic on page 12-35 of the Haynes manual (left picture).
1. Make sure you have 0 volts on the ground (black) wire
2. Second, compare the voltage from the yellow wire to the tan/light blue wire. they should be within each other
3. Now, do the same with teh White/black wire and the white/yellow wire
Its obvious that the switch is getting some power due to your driver side windows working. my guess is you have either a bad ground or a bad wire. good luck