Sep 01, 2020
When some statistics land on your desk, how do you examine them?
Chartered Statisticians will always present in plain English and walk you through the paper explaining as they go so that you can get the most from the data. However, some people like to hide behind complexities. This is certainly not required. Here are a few tips to decide if you are looking at a ‘magic solution’ being sold to you, or a statistical study that you can gain value from.
- Is there a good explanation of the methodology with Assumptions and Caveats? The assumptions and caveats will tell you what is included, what’s not, and what the expectations and limits are for this study.
- Is there a good plain English explanation of the problem and methodology, explaining why the specific methods were chosen.
- Have you heard of the statistical method or is it complex – if it is complex why is it being used? Most studies can be done with basic techniques, this allows the client to replicate them and understand more easily. Most of the time if the technique is complex it is not required.
- Is there a P Value? Is it explained within the context of the study? Some P values are presented but the error on the study is huge – thereby negating any conclusion based on the P Value. Be wary of the P value, or any metric that relies on deeper knowledge, current guidance is to not use the P value unless absolutely necessary. A good analyst/statistician will be able to explain in plain English rather than rely on metrics.
- Do any charts have axis titles and a title. Is the chart unnecessarily complex with stacked bars and lines or is it fairly easy to follow chart?
- Is the above chart explained in plain English with the ‘so what’ explained to you.
- Are you able to request and get the data? Sometimes studies that have taken a freedom with the truth don’t like to hand over the data.
- Does the study generally seem like it hangs together well, follows through and produces a conclusion that sounds reasonable.
- Does the study end with a good conclusion that draws together why the study was done, what data was collected, what methodology was used and the conclusions logically following from this?
- If you have any queries, can’t follow the study or it is not transparent then please ask questions or request a rewrite. A good analyst/statistician will be happy to help and will be keen to learn more about writing studies along the way.
In conclusion, if you start to feel sold to or, after asking questions, can’t quite get your head around what is being presented – run for the hills!