Chapter 33: Security Architecture and Enterprise Reference Design

33.1 Why.

Welcome to Chapter 33.

I am Ravindra Bagale.

Raju had no engineering degree. He had diagrams from every chapter, stacked until the corners curled. A host baseline. A pipeline. A catalog. A database role. A function. A phone. A model. A GitOps loop. He asked which picture was the architecture. Guru-ji took a fresh page and drew four horizontal bands.

"Architecture is the decision of which band may speak to which, and who pays when the decision is wrong," Guru-ji said. "A reference design is that decision written so the next team does not invent a sixth way."

Raju wanted one diagram that would impress a client. Guru-ji wanted boundaries. "Suppose we are Infosys and the client has many accounts, many teams, and one audit. Suppose we are Amazon and a shared platform account can touch every product account because it was convenient during a migration. Suppose we are Paytm or PayFast and the payment zone is not a zone, only a name on a slide. Suppose we are Swiggy or Zomato and a vendor integration lands inside the same network as the order store. Suppose we are Netflix and a region fails over into an account that never received the identity baseline. Suppose we are BHIM UPI or WhatsApp-shaped systems and the control plane that deploys is the same plane that holds customer data. If everything trusts everything, you do not have an architecture. You have a room."

33.2 What.

Reference architectures and trust boundaries. A reference architecture is a pattern your organization repeats: how an account is born, how a service is exposed, how a secret is fetched, how a log leaves the account. A trust boundary is a line where identity, network, or data assumptions change. Crossing it requires a check. Draw boundaries around the internet, around partners, around production, around the corporate laptops, and around the payment or message data. Do not draw a boundary you will not enforce. An unenforced line is decoration.

Shared services and team-owned services. Shared, or central, services are the ones every team should consume the same way: identity, the golden pipeline, the registry, the secret store, the log lake, the GitOps controllers for their own namespaces, the key management service. Team-owned services are the product: orders, payments, recommendations, notifications. Central teams must not become a bottleneck that forces teams into shadow accounts. Team freedom must not extend to inventing a second identity system. Write the catalog of shared services and the contract from Chapter 26. Cost and complexity are design constraints. A fourth copy of a registry "for security" that nobody patches is a new risk.

Multi-account and multi-cluster. Separate accounts or subscriptions by environment and by blast radius. Production is not a folder inside the sandbox account. Identity accounts, log-archive accounts, and deployment accounts have different jobs. Clusters follow the same idea. A lab cluster is not peered to production because a developer asked. Network connectivity is explicit, minimal, and logged. The reference design names which pairs may speak. Everything else is absent, not "temporarily allowed."

Four planes. The identity plane issues and checks who a human or a workload is. The network plane moves packets and refuses the rest. The data plane holds customer state and artifacts. The control plane changes the other three: pipelines, GitOps, infrastructure applies, DNS, certificate issuance. A compromise of the control plane becomes a compromise of the rest, which is why Chapter 32 scoped the controller. Do not store customer data in the control plane because a tool made it easy. Do not let the data plane administer the identity plane. Draw the four planes on one page and mark the allowed arrows. Paytm-shaped payment data has fewer arrows than a public marketing page. That difference is the architecture.

PKI, secrets, artifacts, telemetry. Enterprise PKI or a managed certificate authority gives services names they can verify. Private certificate authorities need the same protection as signing keys. Secrets architecture is centralized issuance, workload identity, and short lifetimes, with human break-glass logged. Artifact architecture is one or few registries, signed images, retained digests, and a rule that production can pull only from the promoted view. Telemetry architecture is a path from every account into an archive the application cannot delete, with retention that matches the duty. These are shared services. They have owners, SLOs, and an on-call. A reference design that says "use best practices" and does not name these owners is a poster.

Developer platform in the picture. Chapter 26 sits inside the reference as the way teams enter. The portal creates an account-shaped sandbox from a module. The module already peers nowhere, already ships logs, already uses the artifact registry. Exceptions are visible on the architecture review, not hidden in a ticket system nobody reads.

Disaster recovery and regions. Name the recovery point and recovery time for each tier of service. The second region is built from the same modules, not from a snowflake. Identity, keys, and log archive survive the loss of a region. Practice failover as Chapter 27 described, and include security checks in the practice. A backup account that is empty of logs and full of old admin keys is not a recovery design. Business continuity includes who can declare the failover and how customers hear about it. Netflix-shaped multi-region habits and Infosys-shaped client commitments should be written as tests, not as hope.

Regulated patterns. When payment, health, or personal data is in scope, the reference design shows the smaller zone: network segmentation, stricter roles, stronger retention, and a change rule with a second person. The zone is implemented in modules so a new payment service inherits it. A wiki that says "PCI zone" while the security group is the same as marketing is not a regulated pattern. It is a label.

Threat-modeling the decisions. For each arrow on the reference diagram, ask Chapter 3's question. What crosses, what if it is hostile, what control sits on the line, what evidence shows the control. Record architecture decisions in short notes: the decision, the options, the reason, the revisit date. When the product changes, revisit the note. An architecture that cannot change is a fossil. An architecture that changes without a note is a rumor.

Cost, scale, and complexity. Every boundary has an operational cost. Too few boundaries and one incident is the company. Too many and teams bypass them. Choose a small set you will actually run. Measure bypass the way you measured golden-path bypass. If everyone peers to production, the reference is fiction. Fix the module or change the reference. Do not add a seventh tool to hide the fiction.

A teaching sample: one decision note. Lab only.

decision: production and lab are separate accounts
reason: a lab role must not read production orders
options rejected: one account with naming conventions only
evidence: a deny you tested, a log in the archive account
revisit: when a second product team exists
planes: control plane cannot read the data plane's tables

Seminar notes. Raju drew boxes until the page was gray. Guru-ji erased half of them. A reference you cannot operate is not kind. It is a trap for the next hire. Keep shared services few and excellent: identity, registry, secrets, logs, the paved pipeline, the certificate authority you actually run. Everything else is a product service with a team name. Multi-account design earns its keep when a mistake in the sandbox cannot list production storage. Multi-cluster design earns its keep when a noisy experiment cannot schedule on the production-shaped nodes. If you cannot afford the split yet, write the decision note that says you are accepting that risk until a date, and name the compensating control. An undated "we will split later" is how flat trust becomes the culture.

The identity plane should survive a bad deploy. Break-glass accounts are few, hardware-backed, logged, and tested. The network plane should default to no path. Add paths in modules so a new service does not require a ticket to open the world. The data plane should be boring: private, classified, backed up elsewhere. The control plane should be powerful and narrow: it can change desired state for a namespace and cannot browse tables. When an arrow connects control to data, justify it in a decision note. Most arrows of that kind are habits, not requirements.

Enterprise PKI is a named service with an owner and a rotation drill. Wildcard certificates that cover every name you might ever use are convenient and blunt. Prefer names you meant. Artifact architecture is promotion: a digest is born in sandbox, scanned, signed, and only then visible to production pull credentials. Telemetry architecture is an archive account. Application roles write logs forward. They do not hold the delete right. Retention matches the duty you mapped in Chapter 24. Cost shows up here as storage. Sample high-volume debug in the lab, and do not sample away the admin audit.

Regulated patterns are modules, not slogans. A payment-shaped module sets private networking, a second reviewer on Git, a narrower role, and a shorter token. A content-shaped module can be looser and must not be copy-pasted into the payment zone because it was faster. Disaster recovery is a region or an account built from the same modules, with a tested recovery time. Multi-region without a security baseline is a second place to be breached and a first place to be surprised. Practice the failover when the budget allows, and include the deny tests so the recovery does not widen trust.

Threat-model the reference when you add a partner, a new data class, or a new control plane such as a feature-flag service or a model endpoint. Ask what crosses the boundary. Ask what evidence proves the check. Revisit notes on a date, not when an incident forces you. Complexity is the enemy. If the diagram needs a legend longer than the diagram, delete a tool. Amazon-shaped estates and Infosys-shaped client estates can be large and still have a one-page reference for a single product. The one page is what you are learning to draw. The rest is repetition of the page, funded and owned.

One more pass. Guru-ji made Raju read the diagram as if a new teammate had to build from it on a Monday. Every box needed an owner. Every arrow needed a protocol or a queue name in words, not a vendor sketch. Every plane needed a failure story. Identity down: how do operators get in. Network down: what fails closed. Data down: where the restore comes from. Control down: who is allowed to change things by hand, for how long, and how Git catches up. If a story was "we would figure it out," the box was not real yet. Reference designs are kind to the future on-call. They are not posters for a kickoff.

Cost sits in the decision note as a number you can feel in the lab: two accounts instead of one, a log archive that retains admin events, a second virtual machine for failover practice. Scale sits there too. A design that works for one service and collapses at ten needs a module, not a heroic copy. Operational complexity sits there as the question of who wakes up. If the answer is "the person who drew this," the design is a hobby. Assign the shared services. Let product teams own product services. Review the page when a new data class appears, including when someone wants to add a model or a mobile client. Those are new arrows. Chapter 31 and Chapter 30 do not float beside the city. They enter through the same gates.

A reference that names owners and denies is operable on Monday. A reference that only names products is a shopping list.

Write the deny you tested under the arrow it protects, so the picture and the lab stay the same object.

33.3 How.

1. Pick one synthetic product: lab orders with fake data.
2. Draw internet, partner, corporate, and production as separate boxes.
3. Draw identity, network, data, and control as four bands across the page.
4. Mark the only arrows you intend to allow.
5. Assign an owner to identity, registry, secrets, logs, and the product service.
6. Create two lab accounts or two clusters: one sandbox, one production-shaped.
7. Prove a sandbox role cannot read the production-shaped store. Save the deny.
8. Send logs to a third place the app role cannot delete.
9. Publish artifacts to one registry and allow the production-shaped cluster to pull only by digest.
10. Issue a lab certificate from a named authority and refuse a plain connection.
11. Put the GitOps controller in the control plane with a narrow role, as in Chapter 32.
12. Confirm the controller identity cannot query customer-shaped tables.
13. Write a decision note for the account split, including the option you rejected.
14. Write a second note for how a partner event would enter, and the check on that arrow.
15. Define a recovery point and recovery time for the lab service.
16. Place a backup in an account the app cannot administer.
17. Run a small failover or restore and verify the security baseline came with it.
18. Mark a pretend regulated zone around the payment-shaped table and show its role is narrower.
19. Review the diagram against the running lab and delete one arrow that exists only on the slide.
20. Store the diagram and the notes next to the catalog entry.
21. Schedule a revisit date.
22. Stop. Do not diagram your way into scanning Swiggy, Paytm, Amazon, BHIM UPI, WhatsApp, Zomato, Netflix, Infosys, PayFast, or any estate you do not own.

33.4 A real public incident.

Name. NotPetya disruption of A.P. Møller-Maersk.

Date. The destructive outbreak hit on 27 June 2017. Maersk's terminals, shipping operations, and corporate systems were widely disrupted. The company later described a financial impact in the hundreds of millions of dollars in its public results discussion that summer.

Source. Maersk's public statements and interim reporting in 2017, and public government statements attributing the NotPetya outbreak, including comments from United States and United Kingdom officials in 2017 and 2018. Contemporary reporting described a trusted software-update path and flat internal connectivity. This chapter does not explain how the program spread and does not provide any sample of it.

What happened. A destructive program entered through a path organizations already trusted, a software update channel used in one region, and then interrupted operations across a global company. Public accounts described ports stopped, systems rebuilt, and a recovery that depended on finding intact directory data when much of the internal computing estate had been damaged. The architectural lesson is not a malware lecture. It is boundaries and planes. A control plane and an identity system that sit in the same flat trust as every office and every port will fall together. A data backup that lives where the destructive program can delete it is not a recovery strategy. Segmentation, a separately protected identity and log archive, and a practiced rebuild from known images are reference-design choices. They are expensive. The outage showed the cost of not having them. Heroic recovery is not a control you should plan to need.

How to stop.

1. Isolate affected networks you own so destruction does not keep walking.
2. Use out-of-band communication when the primary identity path is untrusted.
3. Preserve any intact logs and directory data in a location the incident cannot overwrite.
4. Rebuild from known images and known code, not from machines you hope are clean.
5. Restore business services in a priority order you wrote before the bad day.

How to prevent it again.

1. Draw real trust boundaries between partners, office networks, and production.
2. Keep identity backups and log archives where the application and the office PC cannot delete them.
3. Treat update channels as a supply chain with pinning and a staged rollout.
4. Practice rebuild of the control plane from the reference modules.
5. Limit east-west traffic so one site is not every site.
6. Do the practice in a lab you own. Do not handle malware samples as a homework stunt.

This is for learning only, and only inside a system you are allowed to test.

33.5 A simple picture.

Four planes. Few arrows. Production data is not the control plane. The log archive is not writable by the app.

33.6 A short animation.

A change approaches a boundary. It waits. A reviewed stamp appears, and only then the arrow lights.

33.7 Conclusion.

Raju's sandbox role failed closed against the production-shaped table. The log archive sat somewhere the app could not reach. Guru-ji stacked the old diagrams under the new page. "Keep the page honest. In Chapter 34 we test the controls without becoming an attacker. A purple team proves a detection. It does not collect trophies, and it does not leave the lab."

• A reference design names boundaries, owners, and allowed arrows.
• The control plane is not a place to store customer data.
• Recovery is an account the incident cannot erase.
• Maersk's 27 June 2017 disruption is the flat-trust lesson.
• Own lab only.

END OF CHAPTER 33 MASTER CLASS