HI Nikhil,
True, a number of the higher powered board (like the WiFi, GPS, etc), draw too much power to be run off a coin cell battery. Alkaline batteries can certainly be used (like AAA or AA's), Adafruit and Sparkfun also have a number of small rechargeable Li-Ion packs that can be used with the TinyDuino that would be able to power the WiFi.
Thanks,
Ken
TinyCircuits