My TinyDuino setup does not boot up correctly when connected via pins RX/TX/GND to an ESP8266 board.  It prints a few lines of setup, resets, and does the same thing again and again.  Unplugging the ESP board at this point does not allow it to boot up correctly.  This behaviour continues until I switch the Arduino off and on again.

Also, when the TinyDuino power switch is set to OFF and I am viewing the serial monitor, while the ESP8266 board is connected I can see that the Arduino is repeatedly printing the first few characters of its setup message as if it is starting to boot up EVEN THOUGH IT IS SUPPOSED TO BE OFF.

If the Arduino boots up while not connected to the ESP board it can be plugged in and then functions properly (at least for a while).  However, constant unplugging is an issue while I'm debugging.

Has anyone experienced this before or know what is going on?

Sounds like the TX line on the ESP8266 is passing enough current that it's able to partially power up the TinyDuino processor through the RX pin. This will result in pretty strange behavior. Usually a setup like this have the same power source and are completely powered/unpowered at the same time. You may be able to add a resistor of 1-10K inline on the serial pins if you want to be able to power them separately.