Quality Responsibility - 5G of Quality Audit

It is very true that software has undoubtedly dominated the market. There are building information modeling (BIM) systems that help with the design and development of constructing a bridge or a skyscraper. The anti-lock breaking systems (ABS) that everyone is familiar with is having software controls. It is increasingly becoming common for even a medical implant to include software with secured over-the-air (OTA) update capabilities. While all these functions of software are needed and great for the society, it has also reduced the notions of quality to be frequently limited to testing. This was apparent to me when I was moderating the Northeastern University's Third Global Symposium on Leadership and Project Management when I saw attendees ask the question, "Who is responsible for Quality?" 

Simply put, I responded, "Quality is everyone's responsibility!" It is not the responsibility of project manager, program manager, product manager, or product owner or most importantly the quality manager or tester. Despite the predominance of software, there are still physical products and ancillary services that have non-software components. In the raw materials received at a facility that are either assembled (nuts and bolts) or somehow used (cement, chemicals) within the manufacturing facilities or drug manufacturing, there are quality control inspections. Quality audits apply to walking around the facilities observing how people are using their safety goggles, helmets, protective gloves, or safety harness and understanding the reasons for their misusing or abusing them. So, I thought I will discuss what a quality audit is and the lean's 5G contribution to this domain of knowledge.

Quality Audit is a broader domain that involves the verification activity to validate quality of the products and processes. While quality control is often reactive controls (corrective actions) within the context of a product, quality audit extends to quality assurance monitoring (preventive actions) within context of a process. Together, these product audit and process audit also evaluate the system audit ensuring that the systems used in both these product and process controls work as expected. Such an audit is done at three levels:

• First Level Audit: Periodical evaluation by the teams supported by the auditors internal to the organization. In plan driven approaches, the project manager is accountable for this activity but the teams are responsible for this activity themselves. In change driven approaches, the self-governing team is responsible for this activity and the scrum master or the agile coach is responsible for ensuring that this is happening not only in the reviews and retrospectives but also in the subsequent iterations.   
• Second Level Audit: This is the first external audit. The focus is on the suppliers and vendors providing raw materials as well as on the knowledge workers (e.g.: consultants) providing expertise. The contracting organization performs this function on the suppliers used. This external audit is subject to the terms and conditions listed in the contractual documents. As a result, it could be the procurement department, PMO, or a combination of stakeholder groups internal to the contracting organization. These audits ae facilitated by the auditors, mandatory, and formal evaluation of the goods and services provided by the third party providers. Examples of security assessments such as penetration tests and proof of skills and talent management such as professional development certification are the starting points of the numerous procedures evaluated to uphold quality.
• Third Level Audit: This is the second external audit. This audit is more formal, more extensive, and more time consuming because these audits are conducted by a third party independent of the contracting or the contracted organization. Such a rigorous and robust evaluation looks at people, processes, and technology among many things to ensure compliance with a specific government regulation or industry certification. For instance, an organization may perform such an audit to confirm with ISO 9001 or SOC2 standard and/or GDPR or HIPAA regulation. Frequently, such a quality audit is done as part of a program component (project or subprogram) or portfolio component (operations) and so responsibility is multifold with the portfolio or program manager being the accountable owner. 

Understanding the 5G's proposed by Lean Manufacturing is paramount in understanding this quality audit. While these concepts apply to manufacturing mainly (Azzaouri, Yousfi, and Bouamrani, 2022), I believe these concepts are extensible to any domain or industry . These audits focus on five key elements:

1. Gemba: Actual place where the work is performed. Often called Gemba walks, the management terminology is called "Management by Walking Around" (MBWA) to perform "inspection" (observing behavior rather than what is told) and gather data as part of "non-verbal communication."
2. Gembutsu: Actual events that transpired. These are the issues that happened as documented in the issues log, defects, or the voice of customer or voice of business. It is important to note that this could also be things that didn't happen.
3. Genjitsu: Actual situation that contributed to the problem. This engages why the "Gembutsu" happened. This is the root cause analysis that can be evaluated by anyone of the seven quality control techniques and can also be superseded by the force-field analysis.  Examples include: no backlog refinement prior to planning, lack of a clear DoD, not using risks to prioritize, etc.
4. Genri: Actual hypothesis of the scientifically or universally accepted principles and frameworks with a factual correlation of observations. The Genjitsu analysis may involve some hypothesis such as "If we had a tool to perform X, we would not have seen this problem!" or "If we had enough capability, competency, or capacity, we would have avoided this from happening!" While these may be true, these observations may be based on universal facts and understanding the factual correlation is important (How much are we trained on the tool or why are not we using an alternative tool that works well with our processes?) is also important.   
5. Gensoku: Actual verification based on adopted operational standards and practices. Extending the corrective and preventive actions, alternative thinking, and risk driven development (Rajagopalan, 2023a), these thoughts are work done after the audit findings to correct the observations. These observations may be controls implemented are effective or are not relevant (green), observed gaps that needs improvement (yellow), implemented controls with material concerns (orange), and immediate remedial actions required to address ineffective or material issues (red). 

By examining these five 5G elements, quality professionals can gain a deeper understanding of the root causes of quality problems and develop more effective solutions. For example, in the case of a manufacturing defect, a quality control engineer might conduct a Gemba walk to observe the production process firsthand and identify any potential issues. They might also collect and analyze Gembutsu data, such as machine settings and operator logs, to determine the exact sequence of events that led to the defect. This information can then be used to analyze for Genjitsu such as missed maintenance schedule contributing machine calibration errors. When developing such corrective or preventive actions, we can build Genri driven hypothesis such as documenting clear processes and procedures or giving frequent training to build quality into the processes. Subsequently, processes can be audited per Gensoku thoughts to evaluate how controls are working to address the risks and issues. These observations may further lead one to look at Muda, Mura, and Muri contributing to the management debt (Rajagopalan, 2016).

As you can see from these discussions, quality is a function of risks (Rajagopalan, 2023b). Everyone contributes to risk and subsequently therefore to quality. 

What do you think?

References

Azzaouri, K., Yousfi, S., Bouamrani, M.L. (2022). Combining digitalisation with 5G lean tool for quality and competitiveness in automotive electrical wiring systems manufacturing. Moroccan Association for Applied Sciences and Innovation, 1-9. In Proceedings of 4th International Conference on Quantitative and Qualitative Methods for Economics, Management and Social Sciences, Istanbul, Turkey.

Rajagopalan, S. (2016). Management Debt: Cost of non-delivery and non-conformance. Retrieved from https://agilesriram.blogspot.com/2016/10/management-debt-costs-of-non-delivery.html

Rajagopalan, S. (2023a). Risk driven prioritization: Challenges to prioritization techniques. Retrieved from https://agilesriram.blogspot.com/2023/02/risk-driven-prioritization-challenges.html

Rajagopalan, S. (2023b). Quality is a function of risk. Retrieved from https://agilesriram.blogspot.com/2023/03/quality-is-function-of-risk.html

Quality is a function of Risk

I was recalling the statement, "Quality = f (Risk)," in one of my PMP training sessions and one of them asked how quality relates to risk. The premise behind this thought was on the iron-triangle thinking that quality is controlled by scope, schedule, and cost! It seems like we have a lot of work to do still in understanding about risk and its impact!  

As this person was in semi-conductor space, I reasoned risk is like the hard-wired interrupt that takes precedence over soft logic in the way microprocessor operates. That got the attention. So, I continued to make a connection on the immediate topic of the "Cost of Quality" we were discussing and reasoned out the importance of risk.

Dr. Sriram Rajagopalan's rendition of Quality is a function of Risk

In the diagram above, I have presented the cost of quality made up of two important branches. These are the cost of conformance to avoid risks happening in the first place and the cost of non-conformance to address risks that have happened. 

• To avoid risks as part of the cost of non-conformance, the best approach is to practice the "wisdom of the ages" saying, "Prevention is better than cure!" Here we take proactive steps to ensure quality planning (as part of the Quality trilogy) includes preemptive measures. This involves building quality using quality assurance (QA) with process oriented and proactive steps to train people, have multiple documentation (caters to multiple modes of learning), the appropriate equipment required and the required amount of time to do things correctly (e.g.: right-sizing stories to fit into the timebox, risk driven development methods to prioritize). 
• The next step is to evaluate how well our controls are working by performing quality audit on the work (PM/PO owns the quality audit). Here, quality control (QC) comes from the delivery team that comes in with reactive and product-oriented methods like testing (product testing), inspection (Gemba Walks), etc. 
• Now, if the errors are released such as not missed compliance or security considerations or misinterpreted requirements, or other forms of requests like change request or enhancements are noted, depending upon the triaging process, these could be show-stoppers disallowing the user to realize the intended benefit thus risking value delivery. So, rework may be required, or products may be discarded (prototyping or physical products) as scrap. These corrective actions are adding more time and cost and increase the opportunity cost of people unavailable for improving the benefit in the current project (working on newer functionality) or other business initiatives. Time may translate further into budget risks as available funding may be depleted to pay for contractors and infrastructure.  
• Finally, if these internal errors were not caught and were released to the customer, they become escaped defects! This impacts now the customer's value delivery life cycle as our faulty products may be used in their product assembly or our faulty code may be impacting their applications built. These translates into liabilities for the company, Warranty claims (ongoing free support, recall for the products at our expense) and perhaps even the business lost to competitors. 

As you can see, there are various forms of risks that interface the quality assurance, quality control, and escaped defects side of the equation with some additional risks foundational to the entire quality function in the company through its projects, program, and portfolio functions. The sooner they are addressed (as noted in the green color), the lesser the expenses are. As time passes through this cost of quality function from left to right, the intensity and visibility of risks through corrective and preventive actions (CAPA) to the business is high (as noted in color gradient going to red). 

So, am I not correct to say, "Quality = function(risk)"? Share your thoughts.

Project Quality: Whose job is it to guarantee it?

In a series of training that I was recently providing to project managers, software testers, and business analysts, I found discussions that focused on who was responsible for software quality in projects. The project managers thought it was a QA department role.  QA passed it back to requirements form business analysts who then delegated to directions from the project managers.  With so much literature on total quality management, Six Sigma, capability maturity model, and organizational project management maturity model, it dawned on me that it is this inherent lack of ownership for quality is the reason why quality is suffering in many projects.

Basic requirements of quality are often described as adherence to requirements and fitness for use.  When you look at the key deliverable of this statement, “requirements” stems from the project requirements. In traditional projects, this requirement may come from the client, business analyst, project manager, or the product owner at a minimum. In agile projects, this could be coming from the product owner or client, scrum master or coach, and the team at a minimum. The “fitness for use” stems from timely release of the futures to ensure that the client can benefit from the release.

But does this definition unambiguously tell whose responsibility quality is? The Chartered Quality Institute comes to rescue as it defines quality management as a company’s approach to “…understanding precisely what customers need and consistently delivering accurate solutions within budget, on time, and with minimum loss to society.” (“What is Quality”, n.d., para 1) This definition emphasizes quality within the project constraints budget and time limiting the focus to the scope of the project’s requirements but also to the needs of the society relating to the fitness for use. A project manager is not developing the code or involving in the execution of tests. But quality is the fourth constraint that is compromised as the other project constraints are modified. Therefore, the project manager is accountable for the quality. This is why Quality Audits are associated with the project manager (or product owner).

It reminds me of the famous parable on the virtue of citizenship from the “Adventures from the Book of Virtues.” Many subjects in the kingdom complained about many things in the kingdom and not taking true ownership for improvement. The king hid a pot of gold beneath a big boulder and waited to see who took leadership for removing the rock enhancing safety and quality experience for the others that passed by.  Quality is the same thing as the citizenship requirement. It is everyone’s job. The more structured approaches the organization uses in testing accuracy through automation, test case development, and test execution, the more the project manager becomes attuned to following through on quality, the more the team members will become focused on providing quality by design rather than expecting it to be accidentally evolving.

References

What is Quality? (n.d.) The Chartered Quality Institute.  Retrieved March 28, 2013, from http://www.thecqi.org/The-CQI/What-is-quality/

Agile – Is it a Panacea or a Placebo?

It is true that the fundamental twelve principles of Agile Manifesto (n.d.) have helped many projects succeed in organizations. These results are evident in the increasing 61% of agile practitioners promoting the adoption of Agile in organizations, as noted by the 7^th annual state of agile development survey conducted by Version One. But, several agile purists focus on some of the buzzwords, such as the Test Automation, making it as though Agile is the cure-all for all business problems.

Those in the technical world may understand the famous statement, “Not all business problems are nails to use the same hammer as the tool,” when common object request broker architecture (CORBA) was initiated to introduce by Object Management Group (OMG) the communication between software components regardless of the underlying language used to develop them. However, it is interesting that some agile practitioners are inclining towards test automation as a cure-all (panacea) treating all test cases alike.

Testing requires two basic requirements namely verification and validation. Verification is the process of evaluating test cases, test scripts, requirements or user stories to ensure that the process of testing is going to ensure that the required functionality is present to meet the user requirements. The focus of verification is on the process and is therefore an improvement process oriented towards assuring quality (QA). But validating is the ongoing approach to testing the software with the goal of identifying defects. These defects could be traced to the faulty code, inaccurate design, incomplete or ambiguous requirements, misinterpretation of the requirements, or a combination of these factors. Validating therefore aims at controlling quality (QC).

So, test automation can help in the validation to ensure that every incremental build is tested for the new features but also for the previous features. It is therefore not a placebo but is not a panacea for all business problems. Depending upon the unique business requirements, some requiring can’t be easily automated even if technology can allow it. For instance, testing the human interfacing with the interactive voice response (IVR) at various points in the call flow to check for the disposition code, checking for the line breaks in the email, or the accidental mix up of Far Eastern Asian Language characters because languages such as the Chinese, Japanese, and Korean share a common base do benefit from manual testing.

It is therefore important to evaluate what is the business need in test automation and identify areas that can lead to efficiency gain. Redman and Nielsen (2013) very nicely paraphrased Deming suggesting to carefully select the right process to automate explaining “… automating a process that produces junk just allows you to produce more junk faster". Deming might have inferred this long before the agile was conceived but some golden principles do stand the test of time.

References

Principles behind the Agile Manifesto (n.d.). Retrieved February 13, 2013, from https://agilemanifesto.org/principles.html

Redman, T.C., & Nielsen, D. (2013). Computerization in Health Care Demands High Data Standards. Harvard Business Review. Retrieved February 28, 2013, from http://blogs.hbr.org/cs/2013/02/computerization_in_health_care.html