As an advocate of evidence-informed practice, I have seen fashions come and go in an effort to get governments, non-profit and commercial organisations value the contribution of research and evaluation to the success of innovation.
The latest is the use of terms such as ‘scientific method’, ‘experimentation’ and ‘A/B testing’ which has gained currency amongst the tech start-up community and is now seeping into public policy and charity organisations. If you have been involved in user testing, how informed were you about the purpose of the testing? Did you get any feedback on the outcome?
There is no doubt that....
a well-designed and implemented experiment to test a novel service or product can be the difference between success and failure. Pragmatic, real-world, adaptive and mixed-methodologically rigorous designs add value when built into solution development from the beginning.
A key test of any well-designed experiment is informed consent. This is a concept used in health care and academic research and should be equally applied in the ethnographic research that many tech start-up use as part of their process of development. Informed consent means that all participants understand the goal of the research they are taking part in, what will happen to their data and what effect if any participation may have on them. Framing a clear and simple-to-understand explanation to participants, forces teams to think clearly about their experiment, why they are doing it, and what the effects of doing it might be.
Introducing scientific method into innovation practice could greatly enhance the reliability of experimentation results. With it needs to come a respect for the participants in ‘user-testing’ as active participants in the development of new solutions for the challenges governments and other organisations are addressing.