You sure can, but before you do make sure you don't have to pass any emissions tests where you live.
The easiest would be just to have an exhaust shop cut them out & replace them with pipe (but be sure to ask them to save them for you - they are worth some $)
Im not too familiar on the mustang since I just upgraded from a different car, but you will (pretty sure) throw a CEL without the cat on as well. There are ways to try to trick the car to not show this though.
Also, realize the cat is there for a good reason (if you care about pollution at all),
The cut-out-the-cats solution is what the previous owner did on my '02.
You can trick the computer out of throwing the code with a set of MIL eliminators for like 50-70 bucks for the plug-in ones(try americanmuscle.com) or you can make some yourself. They make splice-in ones that are cheaper and some say they work better also.
The more expensive route would be to get a tuner (diablosport or superchips) which will allow you to just shut off the rear O2 sensors.
I also have a question about exhaust. I am new to the forum. I just bought a 1998 mustang gt and it has true dual exhuast but the restrictors are huge!!! Am i able to just cut them off or is there more to it? Sorry if this is a bad question just new to the muscle car thanks for everyones help.