There are a few ways you can connect the battery - there are two contacts on every TinyDuino processor board next to the main connector that are labelled GND and +, and battery power can be applied here. You can also apply battery power on the Proto1 board on the VBATT and GND pins.
VBATT (The battery voltage) can be between 2.7V and 5.0V. If the battery voltage can be higher than 5.0V, you will need to use a voltage regulator as cseng14 recommends. However single cell Li-ion batteries will be fine, since their nominal voltage is 3.7 volts and under a charge condition will get up to 4.3V max.
Thanks,
Ken
TinyCircuits