A decade of scholarship on DevOps maturity reveals an increasingly diverse but still fragmented body of models. This section (i) summarizes the search and screening process that underpins our systematic literature review (full protocol in § 3), (ii) synthesizes the capability dimensions most frequently cited by prior work, and (iii) positions the proposed Lean-Agile DevOps Maturity Framework (LADMF) against ten seminal models from both academia and industry.

Foundational works and early empirical mappings of DevOps capability constructs establish the baseline we build on [1] - [8] , and the case-study research canon provides the methodological scaffolding for our multi-site design [9] . Domain-specific empirical analyses extend to delivery-pipeline practices and organizational change [10] - [12] , with subsequent surveys and frameworks refining maturity dimensions and validation approaches [13] - [15] . Recent scholarship broadens coverage to DevSecOps and architecturally agile delivery—including policy-as-code enforcement, threat modelling integration, and evolutionary architectures—thereby addressing gaps noted in earlier models [16] - [25] . Methodological and practice-oriented contributions on immutable infrastructure, GitOps controllers, compliance-as-code, and automated architecture fitness functions further operationalize maturity assessment [26] - [33] . Finally, studies on DevOps metrics and ROI, reliability engineering practices, and team-level psychological safety extend the evidence base against which we benchmark outcomes in this paper [34] - [40] .

Applying the search string (“DevOps” AND “maturity model”) OR (“continuous delivery” AND “capability model”) to IEEE Xplore, ACM DL, Scopus and SpringerLink returned 1428 records (2013-2024). After duplicate removal and title-abstract screening, 142 papers remained. Quality appraisal using Kitchenham’s checklist excluded studies scoring < 3/5, yielding 78 peer-reviewed articles for full-text analysis.

Early models (2013-2016) were primarily descriptive checklists derived from single company experience reports. Mid-period studies (2017-2020) introduced multi-domain structures—e.g. CALMS and CAMS—to capture culture and measurement aspects. Recent work (2021-2024) shows a shift toward data-driven validation yet still concentrates on the four canonical domains of culture, automation, measurement, and sharing. Security integration and architectural agility appear in only 15% and 18% of studies respectively, confirming the gap noted by Erich et al. [7] and Lwakatare et al. [8] and the case-study research canon provides the methodological scaffolding for our multi-site design [9] .

Coding of the 78 papers produced a catalogue of 27 discrete capability dimensions. The five most cited were Continuous Integration (79%), Automated Testing (74%), Continuous Deployment (68%), Telemetry (64%) and Change-Failure Recovery (59%). Less than one-fifth of papers explicitly addressed Threat-Modelling, Static-Code Analysis, or Modular Architecture, underscoring the limited treatment of security and design agility.

Table 1 contrasts ten frequently referenced maturity models against five criteria: level granularity, domain coverage, empirical validation, inclusion of security, inclusion of architecture, and use of quantified delivery metrics.

Three themes emerge:

Table 2 summarizes LADMF domain definitions and rationale.

We followed Yin’s five-step protocol to maximize construct, internal, and external validity. Cases were chosen using maximum-variation purposeful sampling to span industry context and baseline DevOps maturity.

Inclusion:

Exclusion:

We studied three large enterprises—F-Bank (finance), StreamMedia (digital media), Telecom (telecommunications), each comprising several value-stream teams (~150 developers in total). Baseline LADMF levels varied intentionally: Beginner (StreamMedia), Intermediate (F-Bank), and Advanced (Telecom). This distribution reduces the risk that findings reflect only high-maturity organizations.

Units were value-stream teams. Data sources combined: (i) deployment and incident telemetry, (ii) pipeline configurations, (iii) 26 semi-structured interviews, and (iv) documentary artifacts (runbooks, architecture diagrams).

Two researchers scored the LADMF rubric independently using artifact evidence; disagreements were resolved by discussion (Cohen’s κ = 0.82). Interview participants were recruited across roles (dev, SRE, QA, security, product) to avoid single-perspective bias.

A formal control group (non-DevOps organizations) was not included. Instead, we contextualized site KPIs against widely used industry benchmarks (DORA quartiles) to provide an external reference distribution rather than a causal counterfactual. We treat causal claims cautiously (see § 6.4). Table 3 maps research questions (RQs) to data sources.

Quantitative telemetry was exported from each site’s CI/CD analytics platform and normalized to DORA KPI definitions (deployment frequency, lead time, change failure rate, MTTR).

Qualitative data came from semi-structured interviews covering culture, process, tooling, and governance; all sessions were transcribed and member checked. Documentary artifacts (runbooks, architecture diagrams) supplied contextual detail.

All participants provided informed consent; organizational names are pseudonymized. Threats to validity are mitigated as follows: construct validity via multiple data sources; internal validity via pattern matching; external validity via industry variation; reliability via audit trail and shared artifacts.

Table 4 summarizes each domain’s scope and rationale, grounded in SLR frequency counts and interview coding.

*Percentage of 78 SLR papers that explicitly covered the domain.

Table 5 offers level descriptors that are tool-agnostic yet concrete enough for scoring. Each descriptor aligns with evidence patterns observed in the case sites and with DORA-style KPIs used in § 5.

*KPIs: deployment frequency, lead time for changes, mean-time-to-recover (MTTR), change-failure rate (CFR).

To operationalize LADMF, we created a rubric that assigns 0 - 5 points per capability dimension. Scores aggregate upward to domain totals (0 - 25) and an overall maturity index (0 - 150). Table 6 illustrates the rubric for the Deployment Automation domain; analogous rubrics for the remaining domains are included in Appendix A .

Assessment proceeds as follows:

The three case sites applied this workflow; § 5.2 quantifies observed KPI improvements and § 6 discusses qualitative enablers and inhibitors.

A PRISMA flow diagram of the screening stages is provided in Figure 3 below.

The most frequent capability dimensions are listed in Table 7 .

Security-related dimensions (bold) and architecture agility remain below 20% coverage, reinforcing the research gap addressed by LADMF.

The following section lists the observed KPI improvements in Table 8 .

At study entry, sites exhibited heterogeneous baseline LADMF levels—Beginner (StreamMedia), Intermediate (F-Bank), and Advanced (Telecom)—confirming that results are not limited to top-quartile maturity contexts. To probe selection bias, we compared KPI medians of our Beginner and Intermediate teams with DORA industry quartiles; values fell within the interquartile range, suggesting that the sample is not skewed toward exceptional performers. Nevertheless, the absence of a true non-adopter control limits causal inference (see §6.4).

Statistical tests—Pooled data across sites show a significant drop in MTTR when advancing a maturity level (Mann-Whitney U = 241, p = 0.003, r = 0.54). Spearman correlation between maturity index (0 - 150) and deployment frequency is ρ = 0.67 (p < 0.01). Lead-time distributions across maturity levels are visualized in Figure 4 .

Coding of 26 interviews yielded 42 first-order codes, collapsed into nine axial themes. Table 9 maps themes to exemplary quotations and the maturity domains they influence.

Across cases, leadership commitment and toolchain cohesion emerged as the strongest enablers, while legacy architecture and regulatory inertia were the main inhibitors.

Together, the quantitative and qualitative strands demonstrate that LADMF not only fills documented domain gaps but also tracks closely to outcome improvements that matter to business and risk stakeholders.

Our mixed-methods evidence sharpens the largely descriptive work of CALMS-MM and CNCF’s model by anchoring maturity levels to DORA-style KPIs. Compared with the 2024 DevSecOps-MM, LADMF extends coverage to architectural agility and offers a scoring rubric validated across three industries—an advance over single-case antecedents.

Our cases are large enterprises in finance, media, and telecom. The framework’s applicability to healthcare, public sector, and start-ups—and to very small teams—remains to be tested. We therefore avoid universal claims and provide replication materials to enable evaluation in additional settings.

Although we purposely sampled varying baseline maturities (Beginner, Intermediate, Advanced), participants may still be more engaged with DevOps than average organizations. To mitigate, we (i) pre-specified inclusion/exclusion criteria, (ii) used artifact-based scoring with dual independent raters, and (iii) benchmarked KPIs against external distributions. A non-adopter control group was not available; we treat observed associations as correlational.

KPI changes were assessed using rolling 90-day medians; we did not run a longitudinal experiment with random assignments. Improvements concurrent with maturity uplifts could be confounded by co-occurring initiatives (e.g., headcount, funding). Pattern matching across three industries and convergent qualitative explanations reduce, but do not eliminate, this threat.

Capability definitions were derived from a 78-paper corpus and reviewed with site SMEs. KPI operationalization followed DORA definitions. The remaining risk includes misclassification of incidents and team-reported practices.

A full audit trail (rubrics, instruments, extraction templates) is provided to support independent replication.

Limitations include the focus on three large enterprises, potential self-selection bias, and the cross-sectional nature of KPI measurement. Future work should automate telemetry-driven scoring, replicate the framework in additional sectors (e.g., healthcare, public-sector, start-ups), and conduct longitudinal studies to quantify long-term ROI of domain-specific improvements.

By fusing evidence from both scholarship and practice, LADMF offers organizations a defensible roadmap for continuous improvement—accelerating delivery while embedding security and architectural resilience at the core of the DevOps journey.

Given the study’s enterprise focus and correlational design, we encourage readers to treat results as general guidance rather than causal proof, and we provide a roadmap (§ 6.5) for achieving stronger generalizability and causal identification in future work.