The public outages caused by Crowdstrike’s disastrous production update were a stark reminder of the cost of software delivery failures today. Crowdstrike’s stocks plummeted 14% overnight, as critical infrastructure battled to stay online.
Yet, every software delivery failure – from the public and catastrophic, to the “small” and systemic – undermines business operations, innovation and competitiveness. As Watts S. Humphrey observed in 2001, “every business is a software business” [1]. Whole generations of innovation and technology have come and gone since then, tightening this entwinement of software, hardware, and enterprise.
Mounting pressure on software delivery leaders
Software’s foundational role for modern enterprise brings increasing stress for software delivery managers. You are responsible for balancing the many moving parts in a modern software delivery ecosystem, making the difficult decisions about where to invest and when to shift focus. Ultimately, you will be accountable if your enterprise struggles or grows.
And it’s not just markets, investors and c-suites who are scrutinising your software delivery today: hefty fines and new regulation focused on business resilience show how legislators are taking seriously the importance of software to human lives.
Yet, for all the fine-tuning, big decisions and drive for continuous improvement, enterprises today struggle with the dual risks of declining software delivery quality and productivity. As paradoxical as it might seem, investing in developer productivity has undermined the agility and accuracy of their delivery.
Software delivery’s inner and outer loops
To untangle this paradox, we must understand how software delivery has separated into what Jyoti Bansal and I are calling software delivery’s “inner” and “outer” loops:
This is not the harmonious ecosystem that is idealized in the DevOps infinity loop. Instead, development has separated into a largely siloed “inner loop”, which comprises the initial creation and refinement of software code. Inner loop activities include coding, problem solving, reviewing requirements and debugging. The inner loop is the domain of the developer.
The outer loop
Software might be developed by software engineers, but it is not delivered by the inner loop alone. In enterprise software delivery, the inner loop is more or less effectively integrated into a fragile ecosystem. It is surrounded by software delivery’s “outer loop”.
The outer loop drives the initial ideation, feeds the inner loop with requisite assets, and ensures that the developed software is robust, reliable, and ready for real-world use:
Where the inner loop is the domain of the developer, the outer loop is the world of the business analyst, product owner, tester and data engineer. These outer loop practitioners are responsible for gathering and defining requirements, integrating and testing the developed code, deploying software, and maintaining it in production.
Inner/outer loop imbalance
The functions of software delivery’s inner and outer loops are symbiotically related, and every stage is essential to enterprise software delivery. Innovation hinges on the productivity and quality of both loops. Yet, enterprises have for a long time invested disproportionately in inner loop productivity.
Just consider advancements in developer productivity tools at your organisation: code completion, package managers and build automation were once considered the next big thing; now, AI is growing proficient in coding, with GitHub estimating that “92% of U.S.-based developers are already using AI coding tools,”[2] and developers enjoy a whopping 55% productivity gain by using their co-pilot [3].
By contrast, the outer loop has been left to struggle with a shared set of recurring painpoints:
- Manual artifact creation (user stories, test cases, diagrams, test data, environments);
- Repetitive “toil” and maintenance;
- Technical debt, as the outer loop cannot keep up with business needs and inner loop changes;
- Uncertainty and knowledge debt;
- A lack of measurability and no demonstrable release readiness.
The devastating impact of inner/outer loop imbalance
Changes to business requirements or inner loop code have a disproportionate impact on outer loop artifacts and labour. What might be a “quick” and apparently “simple” change in the inner loop generates vast workloads and potential debt in the outer loop. A myriad of requirements must be updated, test data and environments reprovisioned, and test phases rushed through.
As the outer loop lacks the accelerators or traceability needed to match inner loop velocity, these vast workloads create increasingly out-of-date artifacts and unfinished outer loop processes.
This translates to growing technical debt, knowledge debt, and mounting risks to release readiness and velocity. Any one productivity blocker in the outer loop has a knock-on effect on all interrelated functions. For example, integration, testing and deployment are bottlenecked when testing waits for data and environments. On the other side of development, when the outer loop feeds inaccurate requirements in, it creates a “garbage in, garbage out” scenario and buggy code in the inner loop.
Communication gaps and knowledge debt likewise grow as outer loop artifacts become increasingly out-of-date and incomplete, and as subject matter experts move teams or organisations. This creates uncertainty, which cannot be de-risked by “doing more” requirements gathering or testing. Outer loop activities are not formally linked to one another, nor to inner loop changes. This prevents formal measurability or assessments of release readiness. “Doing more” toil in the outer loop will not fix the problems when the fundamental relationship between outer and inner loop activities is broken.
The root cause of software delivery leaders’ concerns
Understanding this disparity between software delivery’s inner and outer loops has allowed me to diagnose and articulate the pain that I commonly hear from software delivery leaders across industries. These typically include:
- Slowing productivity and lost market share;
- Suboptimal release quality;
- A high turnover of the best talent.
AI will exacerbate – not solve – inner/outer loop imbalances
As Jyoti Bansal points out, AI adoption in the inner loop will amplify this disparity, as developers continue to use AI agents (e.g. Devon, Meta, CICERO) to evolve code at an ever faster rate.
Faster inner loop changes will exacerbate outer loop workloads and bottlenecks. There will be more functionality than ever to test in ever-shortening iterations. Meanwhile, AI-generated code will need more thorough testing to overcome hallucinations and inevitable errors, ensuring the software’s quality, stability and reliability. This will create a pressure cooker environment across the outer loop, as Quality Engineering requires more test data and environment refreshes, faster than ever.
Out-of-date, incomplete artifacts and requirements in the outer loop further risk a “garbage in, garbage out” scenario with AI adoption. Code generation will lack the requirements needed to produce quality software, while the underlying language models (LLMs) must be fed with accurate and up-to-date data. As long as the outer loop battles toil, technical debt and knowledge debt, these accurate requirements and data will remain unavailable – as will true AI readiness.
How software delivery leaders can rebalance inner and outer loops
Understanding how your delivery ecosystem has separated into inner and outer loops is not only a useful tool for diagnosing perennial challenges in your software delivery; it also offers a framework for rebalancing your ecosystem, maximising the innovation and competitiveness of your enterprise.
Invest in outer loop people, processes, and tools
First, focus continuous improvement efforts and budgets on holistically improving your outer loop. Hire individuals with key outer loop skills, like product management, integration and Quality Engineering. Build diverse teams, capable of addressing every interrelated facet of software delivery. These teams can then help shape the process and tools needed for success.
When investing in outer loop productivity, avoid repeating the mistake of investing disproportionately in any one function of delivery. This risks creating a new imbalance in your ecosystem, in which functions with the potential to be highly efficient are held back by less-well equipped functions.
Similarly, avoid treating or measuring teams or functions in silos. Focus instead on the relationships that exist between teams in your ecosystem, investing in harmonious communication, collaboration and traceability. Measure the overall output of your ecosystem, your “product”, not individual teams. Use concepts like “flow” and value streams to identify how can optimize your overall ecosystem.
5 priorities to start with when rebalancing your inner and outer loops
Which processes should your outer loop experts focus on to unlock these productivity and quality gains?
Curiosity have worked closely with software delivery leaders and large enterprises to improve their lifecycle productivity. We’ve found that 5 priorities best help rebalance enterprise software delivery’s inner and outer loops:
- A true shift left: This does not mean offloading more responsibilities to developers. It means performing critical thinking upfront, minimizing rework, bottlenecks and quality risks from the outset. Integrating quality engineering, “Ops” and requirements gathering, and performing them continuously, allow enterprises to resolve issues before they impact software’s overall quality, while minimizing the risk of scope and budget creep.
- Living documentation: Identifying a “ubiquitous” or “bridge” language from which different teams and stakeholders can collaborate fosters close collaboration and traceability, avoiding knowledge debt, technical debt and uncertainty. Adopting structured, logically precise formats like visual modelling additionally allow enterprises to uproot ambiguity and incompleteness in specifications and communication, while storing information in structured data that enables greater automation, integration and AI readiness.
- Continuous artifact creation: Maintaining information in structured formats – not more words and in heads – allows the automated creation and maintenance of many outer loop artifacts. Generating rigorous tests, data and ephemeral environments unlocks outer loop productivity. It avoids technical debt and toil, freeing the outer loop to apply its creativity, critical thinking and problem solving skills.
- Continuous quality: Automated artifact creation from agreed specifications additionally builds traceability across the outer loop, and into the inner loop. This enables measurability and optimisation, targeting outer loop activities on the latest changes in requirements and code. Measurable tests provide release readiness, confidence and quality, while collaboratively refined requirements further avoid and remove bugs.
- On demand data: Outer loop inefficiencies in test data management are often one of the greatest blockers to software delivery quality and productivity, as every inner and outer loop process requires complex, up-to-date data. Meanwhile, outdated, manual data management processes risk legislative non-compliance and block innovation through AI and beyond. Modernising manual test data management with self-service data automation is one of the fastest and biggest wins for productivity, quality, innovation and compliance.
Go slow to go fast with software delivery’s inner and outer loops
Long-standing oversight of outer loop productivity has created deep-set issues and debt. There are no overnight solutions or “quick fixes” to the imbalance between software delivery’s inner and outer loops, and the priorities suggested above are necessarily wide-reaching.
Fixing a fundamental imbalance in your ecosystem calls software delivery leaders to take a step back, going slow to go fast. Trying to crank out more deliverables or making “hero hires” will not fix the problem, as the fundamental relationship of the outer loop to inner loop development will remain broken. You need fundamental solutions, not painkillers to mask the symptoms.
The challenge of inner and outer loop misalignment is today being properly understood, as it is being diagnosed across large enterprises. The solutions are similarly being proven by their implementation in complex environments, with large enterprises regaining their agility and dominance. Balancing their delivery ecosystem, they become innovators and leaders in their respective industries.
Curiosity are here to help you diagnose and solve your enterprise’s inner/outer loop misalignment. Book a meeting or contact us to talk through the primary challenges that you are trying to solve in your software delivery. Our experts will help you identify the symptoms of inner/outer loop misalignment, building a roadmap to regain software delivery productivity and quality.
Turbocharge software delivery
Our outer loop experts will help you identify symptoms of inner/outer loop misalignment at your enterprise, crafting a roadmap to software delivery productivity and quality.
References
[1] Humphrey, Watts S., Winning With Software: An Executive Strategy (Boston, MA: Addison-Wesley Professional, 2001), 1.
[2] Shani, Inbal and Github Staff (Github:2023), “Survey reveals AI’s impact on the developer experience”. Retrieved from https://github.blog/2023-06-13-survey-reveals-ais-impact-on-the-developer-experience on 23/07/2024.
[3] Kalliamvakou, Eirini, “Research: quantifying GitHub Copilot’s impact on developer productivity and happiness”. Retrieved from https://github.blog/2022-09-07-research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness/ on 23/07/2024.
