Seems like my THUMBY was not provisioned right. It was most likely in some kind of factory test mode. To fix it I had to:
1. go to
https://tinycircuits.github.io/2. download firmware from UTILITIES -> DOWNLOAD FIRMWARE
3. power off THUMBY, plug in the USB
4. hold down the d-pad down button and power on THUMBY - this boots up the Pico in a mode that shows up as a drive on your PC
5. drag and drop the firmware into the THUMBY drive. This will disconnect THUMBY and leave it with no working screen
6. go back to
https://tinycircuits.github.io/7. CONNECT THUMBY
6. Now Filesystem tab shows empty
7. Click FORMAT to load all files again
8. Disconnect and reboot