Battery voltage is a good indicator of the state of charge of the battery. A fully charged battery should read 12.6 V or more.
Testing the capacity of the battery with a voltmeter is more difficult. For example, one way is to test the voltage of the battery under the load is with the headlights on. If the battery voltage with engine off and headlights on drops from 12.6 V to less than 11 V in a couple minutes, the battery is weak.