I am new to arduino and have yet to buy the kit, but was wondering if there is any on board storage, like an eprom and if so how much.

It is the same as an Uno. It has 32k flash, 2k RAM, and 1k eeprom.

Two answers, Flash and RAM (program memory) is dependent on the chip used, the Tiny uses the '328 chip which is 32K of flash and 2K of RAM. You can add a SD Card shield which will let you have 16 GB of SD space.