Eve Fairbanks discusses the question in her latest Examiner column.
The article is worth a read, but the answer for 2006, I think comes down to multiple reasons:
1. Some Republicans were, frankly, stupid. John Hostettler and his way of running a campaign was the most inept you could imagine. George Allen's "macaca" comment also falls into that category as well.
2. The Congressional GOP went out of its way to undercut President Bush. They refused to take up Social Security reform. They utterly failed to do anything meaningful on immigration. They also proceeded to join in with labor unions to slap one of our closest Arab allies in the face. Then there was John McCain's antics.
3. Conservatives went beyond disagreement, and into open revolt in such a manner that they managed to tick off Republican women (see the Harriet Miers fiasco), Hispanics (immigration), and labeled those who stood by the President as party hacks and implied they had sold their souls.
4. The media dispensed with any pretense of objectivity, and was in open warfare against the Administration. The biggest part of this was how anythign that went wrong was the Administration's fault. Never mind that FEMA under Michael Brown had handled multiple Hurricanes in Florida well, they were responsible for thousands being trapped in New Orleans (no mention of the fact that Kathleen Blanco and Ray Nagin made Frank Jack Fletcher look like the model of a decisive leader in a crisis was really made). The coverage of Iraq was also misreported.
5. Dirty tricks. It is now known that Rahm Emanuel knew of the Foley e-mails long before they were reported - and didn't do anything about them. We also had a lot of leaks - each generating a media firestorm.
6. Just bad luck. Katrina was an act of god that got politicized.
The 2006 midterm election was the perfect storm in a very real sense. You had, on the one had, the normal "six-year itch". There was also the fact that the war on terror's Iraqi theater was not going as well as hoped despite the progress in 2005. Reverses in war happen. And it is no small thing that in five years, three state sponsors of terrorism are off the board - one voluntarily, the other two taken down.
But Republicans and conservatives did a lot of this to themselves. There was a lot of infighting - and conservatives went out of their way to insult some of those who honestly disagreed with them on issues. Conservatives irritated those who they needed to convince. I don't know much about anyone else, but if someone calls me a party hack, implies I have sold my soul, or insinuates that my support for the President's position on immigration is support for treason, I'm not going to rush to support them right away.
And if conservatives eager to get back to "first principles" persist in such, I can find other things to do with my time after I vote against them in primaries. Oh, I'll vote, but I'll be very vocal about my dissatisfaction in who I vote for.