> > 1. There's no guarantee your candidate did the work.
> Cheating can be revealed during the work trial period. The
> candidate should be aware that he takes a risk with such
> tricks.

True but there are problems with this approach. First, it is not always legally easy to let someone go once they are employed. Second, it's a waste of money so if you can avoid this, it is good. Third, it is often politically difficult to get rid of someone who is getting something done, even if it isn't what you'd hoped for. Fourth, if the candidate was relocated, there is a moral argument for not getting rid of the person. Finally, morale can degrade in an environment where people are suddenly shown the door. Even though everyone knows the rules, fear can still wreck things.

> > 2. You have no control over the resources the candidate
> > uses.
>
> Why control the resources? Internet usage not allowed? Why
> not? He would use it during the every day work, so why not
> allow it?

The ability to use the Internet (assuming that's the resource the candidate used) is pretty low on the skill level. When I interview, I am more interested in what the candidate knows how to do on her own. Of course, I must be careful about expectations: great developers do use other resources when they work and do not keep everything in their head. Still, I want a sense of what they do have in their head because that is very useful in gauging how good they really are. If someone does something every day, they are likely to have certain knowledge readily available. That's what I want to tap into.

It is also important to run the coding/problem solving task fairly. I help with syntax details; I encourage questions; I will act as an IDE at times. For example, a candidate may be hung up on obscure syntax and I will assist. Or he may ask, does Set have a contains method? And I will say 'yes.' These are the things that documentation is for and I don't expect the candidate to have that kind of detail in his head. I do expect him to know or figure out whether a Set or List is better for the problem, and that Java provides both.

> > 3. You do not get to observe how the candidate solves
> > problems.
>
> I am not a hiring expert, but I think the overall result
> is much more important than how he solved it? Solved the
> task at the first sight or tried 2-3 methods before the
> final solution.

Actually no. Getting the right answer isn't nearly as interesting as how they got there. Problem solving is a much more elusive skill than cut-and-paste coding. In fact, borrowing code well is a skill; those who do it poorly can wreak havoc on the code base.

> > 4. You have no idea how long the candidate worked on
> the
> > problem.
>
> That is true, but again: is it really a good indicator of
> the knowledge if someone solves something in 5 minutes or
> in 15 minutes?

Productivity and in-the-head knowledge are both important things to observe.

> It goes back to the ordinary school problems where the
> fastest replying students are thought to be the better,
> not taking the considerations like:
> - is he cheated?

It is much harder to cheat when you have to solve a problem immediately.

> - is he seen and solved the problem before so it is easy
> to remember the solution?

Hard to control for that in either case. It gives some comfort that a candidate has seen and can recall the answer to a problem. Better though is to pick problems that are not hackneyed to reduce the odds of this.

> - is he really thought over the problem?

In real time problem solving, you get to see how the candidate thinks about the problem. Yes, they are on the spot and can't think long, but it's those who don't think at all that you must avoid. Again, it comes back to how the person works.

> > 5. You do not get to collaborate with the candidate.
>
> That is a problem but I think one could have one hour
> interview with small problems and home task to replace the
> 4 hours on site tasks.

In four hours, you can accomplish a lot more than solving one problem. There are several opportunities to solve problems, there are opportunities to observe personality and behavior, and there is an opportunity to determine where the candidate might best fit in the organization, and there is the chance to sell the company.

> > 6. You cannot observe how the candidate communicates
> > technical information.
>
> During checking the solution he could comment on his
> solution.

That gets you part of the way there but not all the way. Communicating about what you really know is much easier than communicating about what you don't yet now. On the job, this is a big part of development.

> > 7. You cannot observe how the candidate handles stress.
>
> If I would be a employer I would try to bring the stress
> into the minimum. So it would not be extremely important
> to test against the 5% stressful work times.

I certainly don't want to create unnecessary or artificial stress in an interview. I want to provide an environment where the candidate can show his best. That said, interviews are stressful and solving a problem on the white board is even more stressful. But problem solving is a fair request (and a big part of the job) so it is valuable to observe a candidate under stress. Some handle it quite easily, and others do not. The 5% of stressful work time is often the most important 5% of the work time, how the candidate handles stress is very important to an employer.

> > So, although a take home test may be more similar to a
> > real work environment, it reveals very little. Live
> > problem-solving is an information-rich experience.
>
> Yes, there is no silver bullet here, but maybe a mixture
> would be better.

A mixture of approaches is good. Again, the problem with a take home test, IMO, is that it tells me very little. To the point where it isn't even worth the effort. If I were to use this approach, I would not put much weight into the results but it is additional information and can reveal interesting things.