I just had to comment on an interview with Linda Rensing about the similarities of gauging the effectiveness of agile with the placebo effect on InfoQ.

Unfortunately, I rather think that she misses the point. What’s brilliant about the medical community is that they have so much measurement in place that they are able to know how effective a specific treatment is in comparison to the placebo effect, and only those procedures that are proven to be better than placebo are kept. What’s missing from Linda’s interview is that software development doesn’t appear to be ready to apply the same level of measurement. So sure, talking about the process and spending a lot of energy introspecting on it should help. Making a change with the stated intent of it being for the better should help (there’s even the classic study with altering light levels in manufacturing that I can’t dig out a reference to at the moment). The crucial bit that’s missed here is how success is measured. If the only measure we have is by how well people think it’s going

The criteria by which we can say that agile has succeeded has to be whether it has outperformed the placebo effect. The problem is, you’ll need someone who is better at designing trials than me to work out how you do a double-blind test on the effectiveness of a process.