Simple test. Start the car. If you are seeing over 12.6 volts (anywhere from 14.4 to 13.8 is normal) when you hook your DVM to both terminals on the battery, then the alternator is charging.
You might want to inspect your connections, check your batteries water level and look for any massive buildup on the lead plates. If your battery is over 4 years old, you should replace it anyway, just in case.
Scott