It seems like the solenoid is welding the contacts together. You may be able to unstick the contacts, but you will need to completely remove the solenoid. Place it in your palm, stud side up: you may want to put a shop rag between your palm and the studs to protect your hand. Now smack the bottom of the solenoid hard against the pavement or something really solid. If they are not completely welded, it will unstick them.
Make sure that your battery is up to full charge, a low battery will cause the starter to pull more current. Charge the battery or use some jumper cables connected to a car with the engine running about 1500 RPM. If you use the jumper cables, run them connected for 5 minutes to help charge your battery.
Once you have eliminated the battery as being the problem, the starter is the only thing left.