That sucks man. Now it's time to protect yourself. Start piecing together your story in as much detail as you can. It will be important to be able to point to every fact (especially every fact that the officer would not dispute) that is in your favor.
It sounds like the position of the cop might help you. Sounds like he was quite a distance off. So go back over that. How far away? How difficult are the sight lines from that position (obstructions, perspective, etc.)? If this really helps you, then get a street map, or print off Google maps. Make a presentation that illustrates where he was and how he couldn't have made a reasonable judgement from there. But think carefully about this. How would a judge view this argument? Would he disagree with your position, and think the cop clearly had a clear view? If so, you need to get real empirical (measurements and perspective).
Then the night time thing helps. Look up the sunset. Bring a copy from the local news paper's weather for the day/night. Or find the same on the internet. Anything that points out low light could help you (cloudy night, partial moon, etc).
Any street lights out at the location?
The sound level of your car might even help. By that I mean, if you have a loud car, the cop (like everyone) assumes that because it's loud that you are driving fast. But that's clearly not the case. So you might want to get sound level measurements on your car, compared to stock cars. Even if you can't get the measurements, think about developing this side of your story.
Finally, what is your driving record? If it's clear, then REALLY play that up in court. You need to say how much you respect the danger of driving, and how you only drive your car hard at the track (if true). And because you go to the track and care so much about your car, stress how that makes you appreciate just how important it is to drive really tamely on public roads.
BTW, I don't think it means anything that the cop came to your door. If anything, that builds his argument (you were driving so fast, that he couldn't pursue safely in the meighborhood). I remember when I was in HS, an idiot got a ticket for speeding on the way to school. He saw the cop just as he was getting to the parking lot. So he raced in to the parking lot, and ran into the building. He was thinking that if the cop didn't catch him while in the car, he couldn't get a ticket. Obviously that is BS.
Anyway, I can't make the decision for you on whether to fight this, but if you do, I hope some of this helps you start to frame your side of the story. Remember to focus on hard facts. Things that no one can dispute. Good luck.