What is Measurement System Variation?

I will talk about how variation can crop into your data because of a faulty measurement system. And how you can ensure it doesn’t happen before you analyze the data set.

Traditionally, Lean is used to remove any and all types of wastes in your processes, and Six Sigma is used to remove variation from your processes. That being said, its important to know what variation means as well as the various type of variation that any process data might have. Please check out my post on What variation is and how to identify variation in your process. It gives a simple understanding of variation and this post will further drill down into the same.

Types of Variation in your data set

In the above mentioned post, we talked about variation in the time taken to prepare tea and the diameter of a critical part of an engine that is being manufactured. We also established that however critical, well defined, well structured or precise the process is, it will still have some amount of variation. This variation, if within the tolerance limits is OK, but if it is outside the tolerance limits, can hurt the business and customers alike. This is called Process Variation and it is attributable to the process itself. (Click here to read my post about what a process means and how to correctly represent on using SIPOC)

Process variation can further be classified as Special cause variation and common cause variation.

Special Cause Variation

As the name suggests, Special cause variation is the variation in your process which is caused due to an irregular and/or non recurring cause. It is something that occurs rarely, once in a while. For instance, imagine a day when you have have a natural disaster in your town, like a heavy flood. Because of this, the workers will not show up in the factory. The per day output for 2 or 3 days will drop considerable. This will cause variation in your daily productivity data. Here is another example. While you are in the middle of preparing tea, someone knocks on your door and it takes you 10 minutes to get back to your kitchen, But the stopwatch is still running, remember.

What do you do when you have special cause variation in your process. Well, first, you need to identify if there is any special cause variation. These, usually, are the outliers in your data set and are easy to identify. Once you identify the a particular data point as outlier, dig deeper and research to see if there are any special causes attributable to that data point. After sufficient proof, if it is a special cause, remove such data point from the data and proceed with your analysis. You don’t want such data points skewing your analysis for no reason, right?

Common Cause Variation

Once you remove all outliers attributable to special causes, you are left with the data set with variation attributable to common causes. This variation in the common, natural variation that exists in any process. This variation can be caused because of people or because of instruments, systems, material used in the process. This can also be a result of the environmental factors or the method used.

In a DMAIC project, we identify all such common cause variation. We find the reasons behind such variation in the process and drill down to the root causes. These root causes are then solved for, thereby reducing the common cause variation and improving the process.

In addition to understanding variation, we also need to understand how to measure this variation. There are different measures of variation in statistics. Please click to read about these measures of variation (opens in a new tab).

What is Measurement System Variation?

Now, think about it for a minute. Agree there is a variation in the process, atleast that’s what the collected data is showing, right? But then, what if the stopwatch used for capturing the time to prepare tea was faulty? Or. The caliper used to measure the diameter of the part was itself faulty? Or. What if the person operating the stopwatch wasn’t timing it correctly? Or. What if the people measuring the diameter using the caliper were not well trained? Hence, there were inconsistencies in their measuring approaches?

These are very valid and important questions. If any of those are true, it will end up with a data set which WILL have variation.

Now the tricky part is, irrespective of how much you improve the actual process, this particular variation in the data will never go away. Because its not related to the process but is because of a faulty measurement system. This is called Measurement System Variation.

The variation in a data set caused because of or attributable to a faulty measurement system used to collect data, is called Measurement System Variation.

How do you handle Measurement System Variation?

There is only one way to take care of Measurement System Variation. It is to not let it get into your project / process data. And there is a way to ensure that it doesn’t.

Before you go ahead and collect / collate data for any analysis, its important for you to first define the data requirement. And it starts with defining the metric itself. You should have a clear definition of your project metric at the most granular level and you should make everyone involved in the data collection process to understand this definition. In out example of preparing tea, you should clearly specify when to start the stopwatch and when to stop it. In the engine part manufacturing process, you should calibrate the calipers which will be used to measure the diameter and you should train your people on how to use it.

Data Collection Plan and templates

Data collection plan is an important tool that you can use for this effect. The more detailed your data collection plan is, the better are your chances to avoid measurement system variation. Your data collection plan should clearly define the WHAT, WHO, WHEN, HOW and WHERE of data collection. You should define these attribute at a granular level without leaving anyone in any doubt.

Once you define such detailed data collection plan, it is also advisable that you out together the required data collection templates. These templates will give the required structure to your data collection efforts. They will also ensure that the collected data is in the exact same structure as required. Without such templates, everyone will collect data the way they seem fit and will surely result in measurement system variation cropping in.

Measurement System Analysis

Once you have defined the project metric, put together the data collection plan and the required data collection templates. You need to test it as well. This can be done by analyzing your Measurement System on a sample data collected. Measurement System Analysis (MSA) is a tool used to check if your measurement gauge is accurate, repeatable and reproducible.

There are two types of measurement system analysis that you can do. An Attribute gauge RnR and a Continuous gauge RnR. RnR stands for Repeatability and Reproducibility. Attribute gauge RnR is done on discrete data set. Continuous gauge RnR is done on continuous data sets. Both these tests warrant a detailed post and I have covered them in a separate post. Please click on the links to read about them (opens in a new tab). Don’t forget to follow the blog and subscribe to get notified when new posts are up.

Once you do the measurement system analysis and conclude that you gauge passed, you can go ahead and collect data using the data collection plan and template that you built. In case it fails, you will need to rework on the above steps and repeat MSA till it passes. DO NOT COLLECT data using a failed measurement system, ever.

The data collected with the correct measurement system will ideally not have any measurement system variation. Atleast not that you will ever know of. Such data set should be used for all your data analysis to get correct results.

Sachin Naik

Passionate about improving processes and systems | Lean Six Sigma practitioner, trainer and coach for 14+ years consulting giant corporations and fortune 500 companies on Operational Excellence | Start-up enthusiast | Change Management and Design Thinking student | Love to ride and drive

Read More about Sachin