[Discuss] I think my server is running out of "something"

[david at thekramers.net (David Kramer)]

Quick overview of ground zero:

Home-built server/firewall/mail server/web server/MythTV back end/makes
coffee.
Motherboard: Abit IP35 Pro with Intel Core2Duo E6750  @ 2.66GHz
eth0 --> cablemodem
eth1 --> intranet
6 sata ports on motherboard (JBOD, not RAID)
  1 sata drive for root
  4 sata drives for MythTV recordings
  1 sata DVD-RW

I decided to add another 1TB drive for MythTV recordings, so I bought a
WD Caviar Black sata drive.  I plug it into the 6th and last sata port
and power back up, and the server won't boot.  Unplug the new drive, and
it boots.

I got an idea of trying to disconnect the DVD-RW drive and plug the new
drive back into sata port 6, and the server boots.  Spooky.  So once
again I unhook the new drive, plug in the DVD-RW drive again, and it boots.

Then I discovered that eth1 was dark.  No signal, lights aren't lighting
up.  So my server can get to the internet, but the rest of the house was
SOL.  I reboot, and then eth1 works, but eth0 is dark, so I can get to
my server from my other computers, but nothing can get out to the
internet.  I reboot AGAIN and finally both ethernet jacks are live.  I
backed away slowly thinking clean thoughts.  Again, this is several
reboots with no changes to hardware getting different results.

Current status is that the system is back up with all the original hard
drives working, and both ethernet ports working, but my brand spankin
new hard drive is staring at me longingly waiting to be deployed.

I have a theory that my server is running out of "something"
(interrupts?  DMAs?), and the luck of the draw is determining what
devices get what they need.  I can't think of another scenario where
devices would randomly work or not at boot, and adding a device disables
others.  What do you think?

What can I look at?
What can I try?
Thanks in advance.

Side note: I stick labels on all my drives with the install date.
Apparently some of my MythTV drives have been spinning almost
continuously since 2007.  That is impressive.  And scary.