If your only reason to take the spoiler off is because leafs and pine needles gets under it, run weatherstripping between the hatch and the spoiler. I did it for vibration issues with my subs, but it works for keeping stuff out of the spoiler void.
I have seen what Mustangs look like without...