When Quality Engineering Creates Chaos: Lessons from my Workshop at Leading with Quality 2026

When the promise of Quality Engineering meets reality

Last week, I had the pleasure of attending and speaking at Ministry of Testing’s Leading with Quality event, where I delivered a new workshop: “When Quality Engineering Creates Chaos: Preparing Teams for Sustainable Change.”

The workshop generated some fantastic discussions about the promise and challenges of Quality Engineering (QE), and, most importantly, what organisations can do to CREATE the conditions necessary for a sustainable QE transformation.

This blogpost, captures the key activities, questions, and insights explored during the workshop. It follows the same progression explored in the session:

Recognise the gap between the promise and reality of QE.
Diagnose the system conditions (hidden pitfalls) creating that gap.
Create the conditions for a more sustainable change.

Importantly, in the final stage, I introduce CREATE, a practical change-design framework that I developed to help teams examine and strengthen their QE transformation.

If you attended the workshop, or would like to use the exercise with your own team, you can:

Download the workshop slides here .

If you would like the accompanying commentary, reflections and practical questions, read on.

The central argument that ran through the workshop and this recap is this:

Quality Engineering rarely struggles because teams lack skill or commitment. It often struggles because organisations underestimate the system change required to make it work.
Product Quality Insights

Quality Engineering and transformation: the case I made

Across the software industry, organisations are adopting new operating models and ways of working. Alongside these changes, many teams are exploring or actively transitioning towards Quality Engineering.

These transformations usually begin with positive intentions. Organisations want to improve collaboration and alignment, receive feedback earlier, strengthen delivery confidence, and provide value to customers with greater speed and quality.

Yet, despite those intentions, some teams struggle to adapt. When it comes to Quality Engineering:

expectations may remain unclear;
different teams interpret QE differently;
New practices and tools may be introduced without improving the outcomes that matter;
In some cases, QE creates additional activity, confusion, and frustration rather than greater confidence;
and the outcomes do not always match what people hoped to achieve.

These challenges are not necessarily caused by a lack of capability or willingness. More often, something in the surrounding environment prevents the intended change from taking hold. That was the problem I invited attendees to explore.

These challenges are not necessarily caused by a lack of capability or commitment. More often, something in the surrounding environment prevents the desired change from taking hold. That was what I wanted to explore with attendees.

The workshop was designed around three key objectives – Recognise; Diagnose; and Create the conditions for sustainable change – which formed the basis of exploration and what we learned in the session.

Figure 1: The workshop objective and exploration structure

In the sections that follow, I will provide a summary of how these different aspects were explored during the workshop.

Activity one: Recognising the chaos

The first activity began with a simple question:

What has been your experience with Quality Engineering?

We explored several prompts:

Is your team or organisation exploring, adopting, or scaling QE?
What outcomes did you expect QE to help you achieve?
What has happened so far?
- What is working well?
- What challenges or surprises have emerged?

The discussion surfaced questions about what Quality Engineering means, how it differs from Quality Assurance, and how it relates to established ideas such as quality control and quality management.

What stood out was not simply the range of experiences in the room, but how often similar challenges appeared across different organisational contexts. Teams used different language, but unclear ownership, misaligned expectations, and insufficient system change surfaced repeatedly.

These conversations revealed a recurring gap between the promise and reality of Quality Engineering.

The promise and reality of Quality Engineering

When teams begin a Quality Engineering transformation, they usually expect:

shared ownership of quality;
smarter, more purposeful automation;
faster feedback;
fewer late surprises;
better collaboration;
greater confidence in delivery;
and improved customer outcomes.

These are reasonable goals. However, pursuing them can expose questions that were not sufficiently considered at the start:

If everyone owns quality, who makes the final decision about risk?
If developers take greater responsibility for testing, how does the tester’s role evolve?
If Quality Engineers are expected to influence design, when and how are they involved?
If automation expands, who owns its maintenance?
If different teams define quality differently, what are they collaborating towards?
If improvement actions are identified, who ensures they are prioritised?

Without clear answers, the transition to Quality Engineering can create confusion rather than alignment.

The promise may be shared ownership, while the reality becomes unclear ownership.

The promise may be smarter automation, while the reality becomes growing maintenance costs and larger automation backlogs.

The promise may be better collaboration, while the reality becomes more conversations but little agreement about what good looks like.

The promise may be continuous improvement, while the reality becomes recurring issues that are repeatedly discussed but rarely resolved.

This is where Quality Engineering can begin to create chaos.

Recognising the symptoms is important. But recognition alone does not explain why they are happening. Teams must look beneath the visible problems and diagnose the system conditions producing them.

Activity two: Diagnosing the chaos

For the second activity, I introduced a case study featuring a fictional digital product company, Proquains Ltd.

The company had announced a transition to Quality Engineering. Testers were renamed Quality Engineers, developers were encouraged to take greater responsibility for testing, and investment in automation increased. The organisation introduced more quality metrics and promoted shared ownership across its product teams.

Figure 2: Case study summary of the Proquains Ltd QE Transformation

On the surface, the transformation appeared to be progressing. However, after reviewing the case (described in the image above), attendees identified some positive intentions in the organisation’s approach, but also several warning signs and anti-patterns.

Although Proquains Ltd appeared to be adopting Quality Engineering, its surrounding system had not meaningfully changed. Before reading further, consider:

What appears positive about this transformation?
What warning signs can you identify?
What system conditions might be producing them?
What could the organisation have done differently?

The exercise demonstrated that signs of a struggling QE transformation are not always dramatic. From a distance, the programme may look successful: job titles have changed; automation coverage has increased; dashboards are being reviewed; teams are discussing shift-left, shared ownership, and quality at speed.

Yet, beneath those visible activities, a different picture may be emerging:

Quality Engineers are involved after important decisions have already been made.
Product, Engineering, Design, Operations, and Quality hold different perceptions of quality.
No one is clear about who owns release-risk decisions.
Automation increases, but delivery confidence does not.
Quality metrics are reported regularly, yet problems recur.
Improvement actions are repeatedly deprioritised.
Responsibilities expand without matching authority, capacity, or influence.
Teams become sceptical about whether QE is improving how they work.

These are rarely isolated problems. They are symptoms of a transformation in which practices have changed, but the surrounding environment has not.

To apply this diagnostic lens to their own organisations, I invited attendees to ask three questions::

What is the most significant risk to our QE transformation?
What symptoms of that risk can we currently observe?
What underlying system condition might be producing those symptoms?

The third question is especially important. It redirects attention away from blaming individuals or functions and towards understanding the system producing the outcome. That shift makes it easier to identify the hidden pitfalls undermining the transformation.

Revisiting hidden pitfalls of QE transformation

The following patterns are based on my experience and conversations with practitioners across the industry. They are not exhaustive, but they provide a useful starting point for diagnosing why QE may be struggling in a particular context.

1. Renaming roles without redesigning the system

Changing a tester’s title to Quality Engineer does not transform how quality is created. If workflows, incentives, decision rights, delivery pressures, and team behaviours remain unchanged, the transformation is largely cosmetic.

Quality Engineers may be expected to influence discovery and architecture while remaining absent from those conversations. They may be asked to improve quality throughout the lifecycle while continuing to receive work late in the delivery cycle.

The title changes, but the system does not. In a nutshell:

If we change the role associated with quality without changing the environment around it, we have not transformed quality. We have renamed it.

2. Increasing automation without purpose

More automation does not automatically create better quality. Automation creates value when it reduces meaningful risks, accelerates useful feedback, supports decisions, or improves delivery confidence. Without that alignment, it can produce brittle pipelines, high maintenance costs, false confidence, and additional cognitive load.

Before expanding automation, ask:

What risk are we trying to reduce?
Where would faster feedback provide the greatest value?
Which failures would be most costly to users or the organisation?
At what level should a particular check be automated?
How will automated tests be maintained, assessed, and retired?

Automation should support quality decisions. It should not become the objective of the transformation.

3. Collaborating without a shared vision

Collaboration creates value only when people are aligned around a meaningful goal. Different disciplines may perceive quality differently:

Product may prioritise speed and customer value.
Engineering may emphasise maintainability and architectural integrity.
Operations may focus on reliability and recovery.
Design may prioritise usability and accessibility.
Quality professionals may focus on risk and confidence.
Users may simply want a reliable product that meets their needs.

None of these perspectives is inherently wrong. Problems arise when they remain unspoken and unaligned. Teams may collaborate frequently while continuing to move in different directions. A QE transformation therefore needs a shared quality vision that answers:

What does quality mean in our context?
What do our users value most?
Which outcomes are we trying to improve?
How will we recognise progress?
Which principles will guide decisions and trade-offs?

Without a shared vision, collaboration can generate activity without alignment.

4. Measuring without learning

QE transformations often introduce more data: defect trends, test coverage, pipeline duration, automation results, incidents, and release metrics. But more measurement does not guarantee more learning.

If dashboards are reviewed without decisions being made, recurring problems continue. If metrics are used to judge individuals, teams become defensive and conceal issues. If success is measured mainly through activity, teams optimise for visible outputs rather than meaningful outcomes.

The question should not simply be: What can we measure? It should be:

What do we need to learn, and which decisions should this evidence help us make?

Metrics should reveal patterns, support learning, and guide improvement. They should not become reporting theatre or instruments of blame.

5. Sharing ownership without empowerment

“Quality is everyone’s responsibility” is a valuable principle, but responsibility without authority is not empowerment.

Quality professionals may be expected to improve outcomes while lacking access to early decisions. Teams may be held responsible for reliability without the capacity to address systemic weaknesses. People may identify serious risks but have little influence over scope, release, or prioritisation.

Empowerment means giving people the practical ability to affect outcomes through:

access to relevant information and environments;
involvement in early decisions;
clear decision rights;
time to address systemic risks;
influence over quality-related trade-offs;
and visible leadership support.

Shared ownership works only when people are enabled to act. Otherwise:

“Quality is everyone’s responsibility” can quickly become “quality is no one’s responsibility.”

6. Pushing improvement without prioritisation

Many teams already know what needs to improve. Retrospectives identify recurring problems. Incident reviews reveal contributing factors. Quality assessments expose weaknesses. Improvement actions are added to backlogs. Then feature delivery takes priority.

The issue is not always awareness. It is often capacity, ownership, and prioritisation. Continuous improvement must be part of delivery, not optional work attempted after everything else is complete. Otherwise, the same problems reappear and confidence in the transformation declines.

Improvement without prioritisation is aspiration, not change.

* Agreeing on QE without an actionable strategy

One more thing I’ve been seeing lately, which I’ve now added to my initial six patterns, is that: organisations and teams may agree that Quality Engineering is the right direction without agreeing on how it will work. They agree on the what, but not the how to achieve QE. For example, there may be no shared explanation of:

the problem QE is intended to solve;
how roles and expectations will change;
which outcomes matter;
what teams should stop, start, or continue doing;
how progress will be evaluated;
what support teams will receive;
and how learning will be translated into action.

Without an actionable strategy, teams create their own versions of Quality Engineering. Practices become inconsistent, confidence erodes, and meaningful adoption becomes increasingly difficult.

Quality Engineering is not a role transformation

Many organisations approach QE primarily as an evolution of testing. They rename roles, introduce new skills, expand automation, and redistribute testing responsibilities. These may form part of the transition, but they are not sufficient.

As my colleague Jit Gosai often encourages teams to consider, Quality Engineering asks a much bigger question:

How is quality created, maintained, and lost within our system?

Answering that requires teams to examine: how they think about quality; how they collaborate; how decisions are made; how risks are discussed; how failures are handled; how customer feedback is used; how improvement work is prioritised; and how incentives shape behaviour. It is for this reason I argue that:

Quality Engineering is not simply a role transformation. It is a system transformation.

If the system does not change, QE risks becoming something an organisation says it is doing rather than something that improves its outcomes.

From a quality function to a quality system

Another way I’ve been thinking about this shift is distinguishing between a quality function and a quality system.

A quality function centres responsibility around a particular team or role. Testers identify risks, execute checks, report defects, and provide confidence near the end of delivery. Quality is treated as something specialists protect or validate. Meanwhile, a quality system focuses on how the whole organisation creates quality outcomes.

So, within a quality system:

the team owns the outcome rather than one function owning quality;
learning becomes a primary signal, rather than defect counts alone;
automation becomes a shared risk-management asset;
confidence is designed throughout delivery, rather than validated only at the end;
Product, Engineering, Design, Operations, Support, and Quality align around a shared vision;
and feedback is translated into improvement.

Quality professionals continue to play an important role, but their contribution expands from downstream verification towards facilitation, coaching, risk analysis, systems thinking, and quality leadership.

The central question shifts from: Who owns quality? to:

How does quality emerge from the way we work together?

That is where sustainable Quality Engineering begins to take shape. But how can teams move from a quality function to a quality system? How should organisations redesign the environment to realise the intended promise of QE?

That was the focus of the final part of the workshop, where I introduced CREATE, a change framework for sustainable QE transformation.

The CREATE framework

Recognising the pitfalls of QE transformation is useful, but recognition alone does not create change. Organisations must intentionally establish the conditions in which Quality Engineering can succeed.

Drawing on established quality and change-management principles, as well as my experience of QE in practice, I developed CREATE as a structured lens for helping teams examine and strengthen their transformation.

Figure 3: Visual illustration of CREATE framework for QE transformation

Each element of CREATE responds directly to one or more of the failure patterns discussed above. For example:

No shared definition of quality requires teams to Create a shared quality vision.
Role changes without structural change require organisations to Redesign the system.
Responsibility without authority requires leaders to Enable empowerment.
Automation without purpose requires teams to Align practices with risk and value.
Reporting without learning requires teams to Track meaningful metrics.
Improvement without prioritisation requires organisations to Enable continuous improvement.

The additional (seventh) pitfall, the absence of an actionable strategy, is addressed by the CREATE framework as a whole.

CREATE is not simply another list of QE practices. Its six connected elements provide the foundations of an actionable transformation strategy.

Description of CREATE elements

C: Create a shared quality vision

Co-create a clear definition of quality for the organisation, its teams, and its customers. Agree on:

what quality means in your context;
which customer and business outcomes matter;
what good looks like;
how competing priorities will be balanced;
and how progress will be evaluated.

A shared vision does not remove every disagreement. It gives teams a common basis for navigating those disagreements. Without it, different functions optimise towards different outcomes and weaken the end-to-end system..

R: Redesign the system

Move beyond changes to titles and responsibilities. Examine: workflows; incentives; decision rights; team interactions; delivery cadences; release practices; and feedback mechanisms.

Do not ask only what Quality Engineers should do differently. Ask what the surrounding system must enable so that every discipline can contribute effectively to the shared quality vision.

QE succeeds through system redesign, not role renaming.

E: Enable empowerment

Pair responsibility with: authority; influence; capacity; access; capability development; and leadership support.

If people are expected to improve quality, they must be able to influence the decisions and conditions that shape it.

Empowerment does not mean allowing every person to make every decision. It means clarifying where decisions sit and giving people sufficient information, time, support, and influence to fulfil their responsibilities.

Responsibility without empowerment creates frustration. Over time, it can also create disengagement from the transformation itself.

A: Align practices with risk and value

Testing, automation, observability, and other quality practices should be driven by meaningful risks and desired outcomes aligned with business goals. Before introducing or expanding a practice, ask:

Which problem does this solve?
Which risk does it reduce?
Which decision does it support?
What value should it create?
How will we know whether it is working?

A practice should serve the system. It should not become the goal itself. Therefore, QE should not be treated as an isolated testing initiative. It requires meaningful engagement with stakeholders across Product, Engineering, Design, Operations, Support and Test disciplines.

This joined-up approach helps people understand not only which practices are being introduced, but why they matter and how they contribute to shared outcomes.

T: Track meaningful metrics

Use measurement to learn and improve, not merely to report activity or assign blame. Meaningful metrics should help teams: identify trends; understand outcomes; expose system weaknesses; make better decisions; and verify whether improvements are working.

The number of tests, defects, deployments, or automated checks may tell part of the story, but these measures should not be mistaken for quality itself.

Data becomes valuable when it changes what the team understands or does next.

E: Enable continuous improvement

Create mechanisms for experimentation, feedback, adaptation, and follow-through. This may include:

allocating protected capacity for improvement;
assigning clear ownership;
prioritising systemic fixes;
visualising progress;
evaluating whether changes produced the intended result;
and sharing lessons and successes.

Continuous improvement is not simply the act of holding retrospectives or documenting lessons. It depends on whether learning leads to action and whether that action produces a better outcome.

Quality improves when learning is translated into action.

The role of leadership

Leaders play a critical role in making these conditions real. They influence:

what receives attention;
which trade-offs are accepted;
whether improvement work is protected;
how decision rights are distributed;
whether teams are rewarded only for speed or also for resilience;
and whether people feel safe enough to expose weaknesses.

A sustainable QE transformation cannot depend on practitioner effort alone. Quality Engineers may influence the system, but they cannot compensate indefinitely for weak priorities, unclear ownership, or a lack of leadership support.

Leadership therefore means more than endorsing QE as a direction. It means creating the organisational conditions in which its principles can be practised consistently.

A QE readiness check

Before scaling or accelerating a QE transformation, leaders and teams should assess whether the surrounding environment is ready to support it.

During the workshop, attendees were invited to consider six questions to assess their team and organisational readiness:

Figure 4: Visualisation of QE readiness checklist

The purpose is not to achieve a perfect score before beginning. It is to identify the constraints most likely to undermine transformation.

The exercise was deliberately simple:

Answer Yes, No, or Mixed to each question.
Use the answers to identify the most significant constraint.

For every “No” or “Mixed” response, ask:

What symptoms can we currently observe?
What system condition is creating them?
What needs to change for this answer to become “Yes”?
Who needs to be involved?
What is the smallest meaningful action we can take?

Then complete this sentence:

The most important thing our organisation needs to change before further QE adoption is…

The answer can become the starting point for meaningful action. It turns a broad ambition to “improve QE” into a specific change that people can understand, own, and evaluate.

Three actions to take back to your team

1. Diagnose the system, not only the symptom

Quality Engineering is not simply a new version of testing. When QE struggles, look beyond the immediate symptoms. Examine how teams collaborate, how decisions are made, how risk is managed, how improvement is prioritised, and what behaviours the surrounding system encourages.

2. Connect QE practices to customer and organisational goals

Testing, automation, observability, and other quality practices should support outcomes that matter to customers and the organisation. Product, Engineering, Design, Operations, Support, and Quality need to understand how QE contributes to the organisation’s wider purpose. This connection provides a stronger basis for prioritisation, investment, and collective ownership..

3. Manage QE as an intentional change, not an operational announcement

Do not assume that announcing a new direction will produce new behaviour. Build feedback loops. Clarify ownership and decision rights. Empower teams. Protect improvement capacity. Make learning visible and translate it into action. Design the conditions that make the desired behaviour possible..

Final reflection

If Quality Engineering is creating confusion, resistance, or additional work, the first response should not necessarily be to abandon it or introduce more tools. Instead, examine the system around it.

Have roles changed without workflows changing? Has responsibility expanded without empowerment? Has automation grown without strategic purpose? Are teams collaborating without a shared vision? Are metrics being collected without producing learning? Are improvement opportunities being identified without being prioritised?

The chaos is often not caused by Quality Engineering itself. It results from introducing QE without creating the conditions it needs to succeed.

The question for leaders and teams is therefore not only: How do we implement Quality Engineering? It is:

What must our system enable differently for Quality Engineering to succeed?

Quality is not owned by a role. It is created, maintained, and sometimes lost through the way the system operates. That is why:

Sustainable Quality Engineering is not something you announce. It is something you intentionally design.

Where to begin

Choose one readiness question that your organisation currently answers with “No” or “Mixed.”

Then identify:

the symptoms you can observe;
the system condition creating them;
the people who need to be involved;
and the smallest meaningful change you can make next.

That first action does not need to transform the whole organisation. It needs to improve one condition that currently prevents quality from emerging through the way your teams work.

Feel free to use the diagnostic and reflective questions with your team to evaluate your readiness for Quality Engineering.

Continue the conversation

If you attended the workshop, found this recap valuable, or would like to explore running the session with your own team or organisation, please get in touch.

Enjoyed this post? Join the conversation on LinkedIn!