Website-usability guru, Jakob Nielsen, says this should be the #1 tool in your usability toolbox. It's called the Thinking Aloud Test
and it's pretty straightforward.
Here's all you do: Sit some people down in front of your website, give them some tasks to do and have them "think out loud" while trying to perform them.
(One thing to note, says Nielsen, is that thinking out loud is easier said than done. You may have to keep prompting users to get useful comments out of them.)
Neilsen call this test a "window on the soul" that allows you to find out what users really think about your site. Where do they run into trouble? Where are the glitches? What is obvious to you that isn't to them? Listen and you will find out.
Aside from being an incredibly simple, inexpensive, common-sense way to test your site, it is also flexible and fairly reliable. He has found that "unless you blatantly bias users by putting words into their mouths, you'll still get reasonably good findings, even from a poorly run study."
The results will convince even the most "hard-boiled developers, arrogant designers and tight-fisted executives," he says.
Obviously, this method doesn't lend itself to detailed statistical analysis. And it won't find all your problems or design and content errors. That's why he suggests using other, more scientifically valid methods on occasion, as well.
