I do pretty much the same thing - nothing but soap and water to wash. Never anything abrasive. I use a convertible top cleaner/conditioner with UV protection. I can't recall the name - blue & white bottle. I usually use quite a bit of it on a terry towel and rub it in pretty good - I keep my car near immaculate and it's amazing how much stuff comes out of the top.
A lot of people, including Ford and the manufacturer of the top, will recommend Raggtopp - never used it myself.
Jay