Need some help!

Hello,
Maybe somebody here can help me. I have a Western Digital External Hard drive I just installed (plugged in) to my machine at work. When I boot the machine with the external drive plugged (usb), it hangs, it goes through the boot sequence and right before it usually starts windows, it says something like " press any key to boot from device, (I think). Looks like when it is plugged in it is trying to boot from the external drive.
If I unplug it and reboot, I can then plug the USB External into the machine and it notices then recognizes it. I can deal with that but would like it to boot normally with the external connected all the time.
I googled around and found I might need to change a setting in the BIOS? If anybody can help me with what i need to do, it would greatly be appreciated... Thanks!
Dave