The intangibles of hiring programmers

When you first get a chance to have a prospective engineer come visit your office, it's critical to subtly show off all the smart people you have and all the big interesting projects they are working on. Make sure the candidate gets a chance to hang out with your existing engineers, chat about the problems you're working on etc. You'll get to evaluate each other in a more casual environment than a one-on-one interview and hopefully impress them with what working in your company is like.

Of course, these informal discussions are also your chance to evaluate the candidate, but be careful. You've been thinking about your projects every waking minute for a year while they just got exposed to them. Don't jump to conclusions based on their lack of amazing insights after only 5 minutes.

Ask the candidate about interesting problems they've solved. If they can't come up with any good problems they've worked on, or if they've come from a big company where they work on some really narrow problem then be careful. Have they ever thought up new features for products before or have they always had everything carefully spec'ed and spoon fed to them? Have they ever built a new type of product? Do they have (non-fanatically held) opinions about preferred tools, languages etc? Anyone really into programming will have these sorts of opinions.

Speaking of which, I think it is incredibly rare that a great programmer is someone who just programs 9-5 as a job and then forgets about it. Most of the really good programmers I've known were programming as a hobby before doing it professionally. When growing up they found ways to get access to computers at a local school, they wrote games on PCs, they built websites, whatever. The point is they love to program, they have done it for free, and they have done a lot of it. Like anything, programming every day for the past ten years, is what makes you a great programmer today.