We decided we would take a look at the BIOS settings today, so that the Windows Media PC wouldn't try to boot from the new USB drive. Well.
We ended up pulling the TV Tuner card out, to convince Windows that it should treat the LCD monitor as the primary. That worked fine, and we went into Setup (with nothing but the keyboard, mouse, and monitor hooked up. No dice - setup boot order does not have USB drives specified as being before the HD. Plug the USB drive in, go into setup - and the entire system halts - reboot without the drive plugged in, and it booted fine.
It's at this point that we just gave up - we decided to just boot with the drive unplugged, and plug it in when we are already booted up. Now sure, it's probably more the fault of the BIOS vendor, or HP (the system maker) than it is MS' fault. However - the fact remains that I wouldn't have this problem at all with a Mac. When we do eventually replace this system, it will be another Mac.