Part 26: Clause 9.1 - Monitoring, measurements, analysis and evaluation
Clause 9.1 is all about measuring how well we are doing. This is the “check” part of Deming’s Plan-Do-Check-Act cycle.
Starting at 9.1.1 the wording on what to measure is very vague. The standard leaves it up to you to determine:
- What needs to be monitored and measured;
- How you will measure it;
- When you will measure it;
- How to collect and analyse the results.
Based on my experience (much of which was using the older versions of ISO 9001), the “what you should measure” can be split into 3 areas:
- Your product and service
- Your QMS
- Your improvements
For example here at Mango we monitor and measure the quality of the product by the number of bugs released to the client, as well as the up-time of the servers for the clients/users. We use two time frames to monitor and measure our QMS - every week in SCRUM meetings we review our processes, and every month we carry out a Management Review to ensure that we deliver on what we/customers want. And finally, we measure our improvements to the system using Mango itself.
Next up is clause 9.1.2. This clause requires you to measure customer satisfaction. Or as the standard says, “… customers’ perceptions of the degree to which their needs and expectations have been fulfilled.”
So how do you do that? The standard has a handy list of examples:
- customer surveys,
- customer feedback on delivered products and services,
- meetings with customers,
- market-share analysis,
- warranty claims and
- dealer reports
I don’t recommend surveys. I have completed hundreds of surveys in my time, but the reality is that I feel I don’t have the time to do them properly. I complete them without much thought - I tick the boxes and move on. How useful to anyone is that kind of half-hearted feedback?
What I do recommend doing is meeting with clients one-on-one and getting real customer feedback on the performance of your product and service. This is incredibly valuable data that is worth its weight in gold.
For example, here at Mango we try and visit our clients on a regular basis – all over the world. Recently I travelled from my home in New Zealand to visit our clients in South Africa. I took my camera with me and asked each customer for an on camera review of Mango. This will be presented back to staff in New Zealand or sometimes even posted on our website.
Another way we measure customer satisfaction is to measure our customer “churn” or the client retention rate. We are currently at over 99% customer retention, which is very high in our industry.
The final clause is 9.1.3. This is where you take all the data previously discussed and analyse and evaluate it to see if things are in or out of control. You can use statistical techniques for doing that.
The standard again has a handy list of areas to evaluate:
- Product and service
- Customer satisfaction
- Quality management system
- Risks and opportunities
- Supplier and contractors
For example, here at Mango we evaluate each of those above with the following:
- Number of bugs
- Customer feedback interviews
- Review of processes
- Plan versus actual
- Risk register review
- Supplier performance
- Number of improvements open and closed
Here is a list of takeaways that will help you meet clause 9.1:
- Create measures of your product, QMS and improvements.
- Measure customer satisfaction to get great feedback on your business.
- Evaluate your business’s performance over time.
View previous blogs in this series "How to Implement a QMS and Achieve ISO 9001 Certification":