Chapter 35: DevSecOps Metrics, Maturity, and Engineering Management

35.1 Why.

Welcome to Chapter 35.

I am Ravindra Bagale.

Raju had no engineering degree. He had a spreadsheet that was entirely green. Deployment frequency looked busy. The scanner said zero critical findings. He wanted Guru-ji to call the team elite and to stop asking questions.

Guru-ji did not look at the green cells first. He asked who had muted a finding last Tuesday, how many hours the squad spent closing noise, and which accepted risk had no expiry date. Raju's smile thinned. The green cells did not answer any of those questions. They only proved that a cell can be colored without a decision attached.

"Chapter 2 already taught the formulas," Guru-ji said. "Deployment frequency, lead time for changes, change failure rate, and time to restore live there, with the arithmetic and the bands. This chapter does not write them again. A lead's job is the decision the number forces. A green cell that nobody can act on is decoration. A red cell with an owner and a date is management. If you memorize the fraction and never change a sprint, you have a poster, not a program."

Raju still wanted a maturity badge. Guru-ji shut the laptop. "Suppose we are Paytm and a change looks fast only because the gate was warn-only for a sale. Chapter 2's speed number would smile, and the city would be worse. Suppose we are Swiggy and every pull request throws forty style notes. People click through, and the real note dies. Suppose we are Infosys and a client wants a maturity score with no owners and no expiries. That slide is a costume. Suppose we are Netflix and the error budget is fine while library debt is a year old. Different clocks. Suppose we are Amazon, Zomato, BHIM UPI, WhatsApp, or PayFast. Three questions remain. Can Chapter 2 explain the pace. Are humans drowning in non-risk. Does postponed debt have a name and a date."

35.2 What.

Point back, do not recite. Chapter 2 defines deployment frequency, lead time for changes, change failure rate, and time to restore, with formulas and bands. Use those. Do not retype them here. Practice the sentence after the arithmetic. If frequency and change failure both rose, ask which gate was skipped. If restore time improved because incidents were renamed, fix the dictionary. If lead time shrank because review became a stamp, restore the review and keep the slower honest number. Do not invent a fifth formula and call it DORA.

What a lead uses the numbers for. A lead is not a collector of charts. A lead chooses capacity. In a week a squad can ship features, pay security debt, tune noisy tools, or rehearse a restore. The numbers tell you which of those is being starved. A useful review fits on one page and ends in a choice a sprint board can feel.

The page has five lines and no more decoration than those lines need.

1. One delivery number from Chapter 2 that moved, and the plain reason.
2. One security exposure number: open reachable findings above your severity line, aged against the promise you learned in Chapter 25.
3. The false-positive load for the main gate, in human hours, not in a vendor percentage.
4. The security-debt balance: what you knowingly postponed, who owns it, and when it expires.
5. One decision: pause a feature, fund a fix, tune a rule, or accept a risk in writing.

If the page has no decision, it was not a review. It was a recital. Recitals feel like leadership and change nothing by Monday.

False-positive load. A false positive, here, is a finding that a competent reviewer with the code in front of them would not treat as a real exposure. It is not "anything that delays a release." Load is the cost of being wrong in the noisy direction. Each week, for each blocking gate, count findings opened, findings closed as not a risk with a reason a second person could audit, and minutes a human spent to reach that close. The load is those minutes, summed. A vendor percentage is not your load. A small percent of a flood can still be a week. A small percent of a thin gate can be an afternoon you can afford. When comments are mostly noise, people stop reading, and a real note, such as a token in a log, dies beside a naming nit. A suppression needs an owner and an expiry.

Separate three piles. Not a vulnerability: the tool misunderstood a framework or flagged a fixture. Not reachable: the pattern is real in the abstract, but no request, role, or build path can touch it, and you wrote why. Not ours this week: it is real, it is reachable, and you are choosing debt on purpose. Only the first pile is a false positive. The second might be a mistake, so it stays visible. The third is debt. Mixing them lets a team boast a flattering rate while dangerous rows sit beside test fixtures. Raju's green scanner number was mostly pile one. Pile three had been renamed until it vanished.

How a lead reduces load without blinding the gate. Sample closed findings every week. If the same rule produces the same non-risk ten times, fix the rule, the baseline, or the path filter. If a rule has not found a true risk in a quarter on this code, stop blocking on it and say so. If a rule sometimes saves you, keep it blocking and budget the minutes. Do not allow a personal ignore file the pipeline never sees. Centralize suppressions in the repository with a reason, a date, and a name. A suppression past its expiry fails the pipeline, the way an expired certificate should fail a handshake.

Watch the other failure. If you tune until the gate is silent, and the running system then shows what the gate used to catch, load fell for the wrong reason. Chapter 2 named vulnerability escape: issues that appear live after the pipeline had its chance. Plot load and escape together. Load down and escape flat is tuning. Load down and escape up is blindness. Put the signal back, and tell the engineering manager before the roadmap meeting tells them for you.

Security debt. Technical debt is a design shortcut: a module nobody dares to change. Security debt is a known exposure, or a missing control you already agreed mattered, still unpaid. It belongs in a ledger: a library not upgraded after a real advisory, a cloud role wider than Chapter 18's design, an exception to a signed-image rule, a log pipeline that drops the admin-grant event from Chapter 34, a waiver granted so a Zomato-shaped menu push or a Swiggy-shaped rush would not slip. Each row needs severity, reachability, an owner, an opened date, an expiry, and the harm if you are wrong. "We accept the risk," with none of those fields, is amnesia.

Interest is the open window. Pay an internet-facing high finding before an easy internal nit. Protect a slice of sprint time for debt and for tuning, on the order of ten to twenty percent, not as a law and not as a certificate rule. If interest spikes, the slice grows and the feature list shrinks. Netflix-shaped teams already stop a launch when an error budget is gone. A debt expiry deserves the same spine. Write the trade next to the Chapter 2 result.

Maturity without a costume. Public models such as OWASP SAMM and the questions gathered under BSIMM give organizations a shared vocabulary for practices. This course does not score you against those models, does not sell you a level, and does not claim that reading them certifies anyone. Borrow the idea of stages, and keep the language plain enough to say aloud.

Ad hoc. A hero runs a scan before a big release. Findings live in chat and die when the hero is on leave. Exceptions live in memory.

Repeatable. The same gates run on the main branch for this service. There is a queue where findings go. People know who to call on a Tuesday.

Measured. You can show false-positive load, the age of debt, and the Chapter 2 numbers for the same period. You can show that a gate ran, with a log or a pipeline result, not that a person promised it ran.

Managed. The measures change the plan. Expired exceptions fail closed in the way your policy describes. Tuning has an owner. On an Infosys-shaped engagement, a client can be shown queries, dates, and decisions, not a watercolor that says the program is mature.

You can be repeatable on identity and ad hoc on upgrades at the same time. Report by practice. Do not hide the weak one inside a single decimal.

Engineering management that is not a tool tour. A central group cannot read every pull request, at Amazon scale or on a small PayFast-shaped wallet. Chapter 2's champions are the translators inside the squad. Give them sprint time, a direct line, and the right to mark "not a risk" without punishment, plus the duty to escalate when unsure. If you score champions only by tickets closed, they will close noise and duck hard debt. Watch decision quality, whether load fell, and whether escape stayed flat. A champion who is the only person awake is a single point of failure in a culture costume.

Rituals stay small. A weekly half hour with the on-call lead, the champion, and the engineering manager is enough if the one-page review is written beforehand. Argue the decision. Leave with at most three named actions. Monthly, look at rows that expired or will expire. Quarterly, ask whether the gates still match the Chapter 3 threat model. A scanner tuned for last year's app will ignore this year's BHIM UPI-shaped callback or WhatsApp-shaped media worker. A calendar invite and a ledger in git are enough. If the ritual needs a slide team, it will die in a busy quarter.

Refuse a few popular habits out loud. Do not rank teams by raw finding counts. The team that scans earlier will look worse, and you will have trained people to scan less. Do not set a bonus on zero critical findings unless you also audit suppressions. You would be buying silence. Do not publish a leaderboard of developers by vulnerabilities introduced. Chapter 2's blameless lesson still holds in a leadership meeting. Ask which control was missing. Then fund it.

A teaching sample: one-page lead review. This block is for a lab notebook. The figures are invented so you can practice the sentences. They are not the real books of PayFast or of anyone else. Chapter 2's formulas are used, not rewritten on this page.

period: last 14 days, team lab-status
delivery: change failure rate moved up sharply (formula and bands: chapter 2)
why: two rollbacks after a gate was set to warn-only during a sale drill
security: 1 reachable high, age 6 days, owner named, expires on day 10
false-positive load: 6.5 engineer-hours, mostly one SAST rule on test fixtures
debt: signed-image exception for a lab legacy image, owner named, expires Friday
decision: gate returns to blocking; one day scheduled to fix the noisy rule; no new feature until the high is closed or extended in writing

The decision is the product of the page. The rest is evidence so the decision can be audited by your future self, who will not remember the mood of the meeting.

Speaking to people who do not live in the pipeline. Tell a product owner on a Paytm-shaped or PayFast-shaped service that the campaign can slip the finding, or the finding can slip the campaign, and you will not pretend both happened. Show an Infosys-shaped client the ages, the expiries, and the hours of noise. If a director wants one score, do not average a strong identity practice with a stale library practice. A clear trade can be steered. A blended color cannot. Chapter 39's exams do not replace this page.

35.3 How.

1. Open Chapter 2 and bookmark the four formulas. Do not copy them into a new sheet that silently changes a definition.
2. Pick one lab service you own and a two-week window, and write the window at the top of the page.
3. Compute one delivery number with the Chapter 2 formula and write the result in a sentence a product owner could hear.
4. List the security gates that can block a merge or a deploy for that service.
5. Copy the findings those gates opened in the window into your notes, and redact every secret.
6. Mark each finding as not a vulnerability, not reachable, or real debt. Do not use one bucket for all three.
7. Record the minutes spent on the "not a vulnerability" pile. That sum is your false-positive load.
8. For the noisiest rule, decide whether to fix it, narrow it, or stop blocking on it, and write the reason.
9. Put suppressions in one reviewed file with an owner, a reason, and an expiry. Remove personal ignore habits.
10. Open a debt ledger with severity, reachability, owner, opened date, expiry, and harm if you are wrong.
11. Move every real finding and every standing exception into that ledger.
12. Sort the ledger by interest, with internet-facing and high severity before easy internal nits.
13. Reserve a visible slice of the next sprint for the top debt row and for the noisy-rule fix.
14. Write the one-page review with exactly one decision and no second hidden decision in a footnote.
15. Show the page to a second person and ask what they would do differently. Write down the disagreement.
16. If you extend an expiry, require a written reason and a new date. Do not extend only in chat.
17. Define how you will notice escape: a live issue that a gate should have had a chance to see.
18. Compare escape notes with the load number. If escape rises while load falls, put the signal back.
19. Put the review on a weekly calendar hold with the champion and the person who can change the sprint.
20. In a monthly lab pass, treat a debt row past expiry as blocking for your own lab deploy until it is fixed or renewed.
21. Label one practice as ad hoc, repeatable, measured, or managed, and refuse a single blended score.
22. Delete any team ranking you drafted that uses raw finding counts.
23. Archive the page. In the next window, compare the decision with what the squad actually did.
24. Practice the spoken tradeoff once: the feature versus the expired debt. Say it aloud in one breath.
25. Stop. Do not query, scrape, or benchmark Swiggy, Paytm, Amazon, BHIM UPI, WhatsApp, Zomato, Netflix, Infosys, PayFast, or any system you do not own.

35.4 A real public incident.

Name. Target payment-card breach.

Date. Target said unauthorized access may have affected purchases from 27 November 2013 through 15 December 2013. Target publicly confirmed the breach on 19 December 2013.

Source. Target's press statement of 19 December 2013, and the U.S. Senate Committee on Commerce, Science, and Transportation staff report, "A 'Kill Chain' Analysis of the 2013 Target Data Breach," dated 26 March 2014. That report, drawing on public accounts including Bloomberg Businessweek reporting, described security alerts that were raised while the activity continued and that were not acted on in time. This chapter uses those public sources for the management lesson. It does not describe how the intrusion was carried out.

What happened. Target said about 40 million card accounts may have been impacted. The management lesson is signal and ownership. Public accounts say alerts fired and did not become a fast containment decision. A purchased detector without an owner for the next hour is a vanity metric. Generic alerts train a shrug, and a true page inherits it. That is false-positive load in the painful direction. The repair is a reviewed queue, a named decision, and a clock, the same shape as a debt row that can expire.

How to stop.

1. Treat a credible high-severity alert on a system you own as an incident until a named person downgrades it in writing.
2. Isolate the systems the alert names and preserve the logs before anyone tidies them.
3. Cut off the path you can already see, such as separating a suspected system from stores of customer or payment data, without waiting for a perfect narrative.
4. Tell the people who must act inside your organization, and tell customers and regulators when that duty is real.
5. Name one incident lead so two teams do not each assume the other is already responding.

How to prevent it again.

1. Measure the time from alert to a recorded decision, not only the count of alerts generated.
2. Sample closed alerts the way you sample false positives, so a habitual ignore becomes a visible number.
3. Give the on-call role the authority to pause a launch-shaped change while a serious page is open.
4. Keep detections few enough that a page is uncommon and taken seriously. Retire duplicate alerts.
5. Practice the first thirty minutes on a lab page you generate yourself, as Chapter 34 taught, and file the result on the one-page review beside the Chapter 2 number.
6. Do not probe a retailer, a payment network, or any system that is not yours in order to "see how alerting works."

This is for learning only, and only inside a system you are allowed to test.

35.5 A simple picture.

The four delivery numbers stay in Chapter 2, where their formulas live. Beside them a lead keeps false-positive hours, a debt ledger whose rows expire, and a single decision that can change a sprint. A maturity word is only a description of whether that page is real.

35.6 A short animation.

A debt row ages across the month. The meter fills. On the expiry day the row turns solid and a decision is due: pay, or extend in writing. A number that never comes due is not management.

35.7 Conclusion.

Raju deleted the all-green sheet. He kept one page with a worse change-failure story, six and a half hours of noise, and a debt row that expired on Friday. The page looked less like a victory and more like a job. Guru-ji preferred it. "That page can run a team. Chapter 2 remains the arithmetic. You remain the person who chooses what the arithmetic is for. In Chapter 36 we take a failure that crosses a pipeline, a cloud account, and a running service, and we practice finding the first broken belief in the timeline without inventing three causes at once."

• Use Chapter 2's formulas. Do not rewrite them to flatter a quarter.
• False-positive load is hours humans spend confirming that a finding is not risk.
• Security debt has an owner, a harm statement, and an expiry. Interest decides what you pay first.
• Maturity language describes evidence. It is not a badge and not one blended score.
• Target, confirmed on 19 December 2013, shows an alert without a decision is not yet a control.
• Own lab only.

END OF CHAPTER 35 MASTER CLASS