Chapter 26: Jenkins Security, RBAC and Hardening

26.7 Hardening checklist

By Ravindra Bagale · Last updated: 25 October 2026

Why: the problem

Think of HDFC Bank. Before an auditor or a customer demo, the team does not want to say "I think Jenkins is safe". They want a list with ticks and proof. This lesson joins all the earlier lessons into one hardening checklist. Each line has a check you can run, a good result, and a fix. You will also turn the main settings into Configuration as Code, so a new Jenkins starts hardened from minute one, and a weekly job checks that nothing drifted. Hardening is not a one-day task. Plugins change, people join and leave, and someone always clicks something at midnight. A checklist plus an automatic check is how real teams stay safe.

What we will build

You will build checklist.md with twelve lines. You will write harden-check.sh that tests most lines with curl and prints PASS or FAIL. You will put the security settings in jenkins.yaml and start a new Jenkins from it. Finally you will add a weekly Jenkins job that runs the check and fails if anything drifted.

What: every term in plain English

• Hardening checklist — A short list of security checks with a clear pass rule for each.
• Configuration as Code (JCasC) — A YAML file that sets Jenkins settings. Kept in Git and applied at start.
• Drift — When real settings slowly move away from the agreed settings.
• Break-glass account — One emergency admin account kept in a vault, used only when SSO is down.
• Audit log — A record of who changed what. The Audit Trail plugin writes it.
• Backup — A copy of jenkins_home to restore from after a mistake or attack.
• Markup formatter — How Jenkins shows text in descriptions. Safe HTML or plain text stops script injection.
• Disabled CLI over remoting — The old CLI protocol is removed in modern Jenkins; use the HTTP or SSH CLI with tokens.

How: step by step

1. Write the checklist

   nano checklist.md
   

   What it does: Twelve lines, each with check, good result, fix.

   Output

   (file saved)
   

2. Write the checker

   nano harden-check.sh
   

   What it does: curl checks for the lines that can be tested from outside.

   Output

   (file saved)
   

3. Run it on the lab

   bash harden-check.sh http://localhost:8080
   

   What it does: Shows PASS and FAIL lines.

   Output

   PASS anonymous blocked
   FAIL markup formatter is raw HTML
   

4. Fix the FAIL lines

   Manage Jenkins > Security > Markup Formatter: Safe HTML
   

   What it does: Each FAIL gets the fix from the checklist.

   Output

   Saved
   

5. Export the settings

   Manage Jenkins > Configuration as Code > View Configuration
   

   What it does: Copy the security parts into jenkins.yaml in Git.

   Output

   jenkins:
     securityRealm: ...
   

6. Start a new Jenkins from YAML

   docker run -d --name j26b -p 8082:8080 -e CASC_JENKINS_CONFIG=/casc/jenkins.yaml -v $PWD/casc:/casc:ro jenkins-hardened:lts
   

   What it does: The new Jenkins is hardened from the first start.

   Output

   Configuration as Code: configuration applied
   

7. Add the weekly check job

   New Item > harden-check > Pipeline > Build periodically H 6 * * 1
   

   What it does: Runs every Monday morning and fails on drift.

   Output

   Saved
   

Example 1: The twelve-line hardening checklist

Stack. Markdown

Goal. The twelve-line hardening checklist.

checklist.md

#  Check                                   Good result                         Fix
1  Login page before dashboard              /api/json gives 403 anonymous       Security realm on, anonymous read off
2  Self signup off                          /signup gives 404                   Untick Allow users to sign up
3  HTTPS only                               http redirects to https             nginx 301 + certificate
4  Port 8080 private                        no answer from outside              Bind 127.0.0.1, security group
5  SSO or LDAP with groups                  whoAmI shows groups                 Realm LDAP or OIDC
6  Role-based access                        access report matches plan          Role Strategy + JCasC
7  Admins two or three only                 admin role lists 2-3 groups/users   Remove extra admins
8  Built-in executors 0                     numExecutors 0                      Configure Built-In Node
9  CSRF on                                  crumbIssuer answers                 Default Crumb Issuer
10 Plugins current, no warnings             Updates page has no red             Monthly patch after backup
11 Script approvals reviewed                no risky signatures                 Remove, use steps or library
12 Backup tested                            restore drill done this quarter     Nightly backup + restore test

What each line does

• Check — What you look at.
• Good result — The exact thing that means PASS.
• Fix — The one action that turns FAIL into PASS.
• Lines 1-4 — Network and login.
• Lines 5-7 — Who has access.
• Lines 8-12 — Builds, plugins, scripts and recovery.

Green output

Checklist reviewed on 2026-10-25 by ravi
PASS 11 / 12
FAIL 12 Backup tested -> restore drill booked for Friday

Expected result. One page anyone can read. Print it before every demo and audit.

Example 2: An automatic checker for the lines you can test from outside

Stack. bash, curl, jq

Goal. An automatic checker for the lines you can test from outside.

harden-check.sh

#!/usr/bin/env bash
set -uo pipefail
J="${1:-http://localhost:8080}"
fails=0
ok()  { echo "PASS $1"; }
bad() { echo "FAIL $1"; fails=$((fails+1)); }
code() { curl -s -o /dev/null -w "%{http_code}" "$@"; }
[ "$(code "$J/api/json")" = "403" ] && ok "anonymous blocked" || bad "anonymous blocked"
[ "$(code "$J/signup")" = "404" ]   && ok "signup off"        || bad "signup off"
[ "$(code "$J/script")" = "403" ]   && ok "script console closed" || bad "script console closed"
n=$(curl -s -u "$ADMIN" "$J/computer/(built-in)/api/json" | jq .numExecutors)
[ "$n" = "0" ] && ok "built-in executors 0" || bad "built-in executors $n"
c=$(code -u "$ADMIN" "$J/crumbIssuer/api/json")
[ "$c" = "200" ] && ok "csrf on" || bad "csrf on ($c)"
w=$(curl -s -u "$ADMIN" "$J/pluginManager/api/json?depth=1&tree=plugins[hasUpdate]" | jq "[.plugins[]|select(.hasUpdate)]|length")
[ "$w" = "0" ] && ok "plugins current" || bad "plugins with updates: $w"
echo "fails=$fails"
exit $fails

What each line does

• fails=0 ... exit $fails — Exit code is the number of FAIL lines. Zero means all good, so Jenkins marks the job green.
• code "$J/api/json" — Anonymous call. Must be 403.
• $ADMIN — Admin user:token for the checks that need login. Read from a credential in Jenkins.
• numExecutors — Must be 0 on the built-in node.
• crumbIssuer 200 — CSRF protection is on.
• hasUpdate — Counts plugins with a newer version.

Green output

PASS anonymous blocked
PASS signup off
PASS script console closed
PASS built-in executors 0
PASS csrf on
FAIL plugins with updates: 2
fails=1

Expected result. One FAIL tells you exactly what to patch this week.

Example 3: Run the checker every week as a Jenkins job, with the admin token from credentials

Stack. Jenkins Pipeline, Mailer plugin

Goal. Run the checker every week as a Jenkins job, with the admin token from credentials.

Jenkinsfile (harden-check job)

pipeline {
  agent { label 'linux' }
  triggers { cron('H 6 * * 1') }
  options { timeout(time: 10, unit: 'MINUTES') }
  stages {
    stage('Check') {
      steps {
        withCredentials([usernameColonPassword(credentialsId: 'audit-bot', variable: 'ADMIN')]) {
          sh 'bash harden-check.sh https://jenkins.shop-lab.in'
        }
      }
    }
  }
  post {
    failure { mail to: 'platform@shoplab.in', subject: 'Jenkins hardening drift', body: "See ${env.BUILD_URL}console" }
  }
}

What each line does

• cron('H 6 * * 1') — Every Monday around 6 a.m.
• usernameColonPassword audit-bot — A read-mostly service user with a token, stored as a Jenkins credential.
• sh 'bash harden-check.sh ...' — Runs the checker on an agent.
• timeout 10 minutes — Stops the job if a call hangs.
• post failure mail — Emails the platform team when any line fails.

Green output

Started by timer
Running on linux-1
+ bash harden-check.sh https://jenkins.shop-lab.in
PASS anonymous blocked
PASS signup off
PASS script console closed
PASS built-in executors 0
PASS csrf on
PASS plugins current
fails=0
Finished: SUCCESS

Expected result. A green job every Monday is your proof for the auditor. A red one means drift, and the team gets an email.

Verify: did it work?

1. Run the checker

   bash harden-check.sh http://localhost:8080; echo "exit=$?"
   

   What it does: Exit 0 means all PASS.

   Output

   fails=0
   exit=0
   

2. New Jenkins from YAML

   bash harden-check.sh http://localhost:8082
   

   What it does: The fresh Jenkins from JCasC should pass too.

   Output

   fails=0
   

When it is red 1

Configuration as Code: Invalid configuration elements for type class hudson.model.Hudson

Cause. A key in jenkins.yaml is wrong or the plugin is missing.

Fix, step by step

1. Read the key name in the error.
2. Install the plugin that owns it.
3. Compare with View Configuration output.

When it is red 2

harden-check job: jq: command not found

Cause. The agent image has no jq.

Fix, step by step

1. Use an agent image with jq, or install it in the Dockerfile.

When it is red 3

Weekly job red: plugins with updates

Cause. A new plugin version came out.

Fix, step by step

1. Read the changelog and advisories.
2. Back up, update on a test Jenkins, then production.
3. Rerun the job.

Cleanup

1. Stop the second Jenkins

   docker rm -f j26b
   

   What it does: Lab cleanup.

   Output

   j26b
   

2. Keep checklist in Git

   git add checklist.md harden-check.sh casc/jenkins.yaml && git commit -qm "jenkins hardening"
   

   What it does: Your hardening is now reviewed and versioned.

   Output

   (committed)
   

Practice tasks

Homework: try at home

Recap

• Join every lesson into one twelve-line checklist.
• Automate the checks with curl and run them weekly.
• Keep settings in JCasC in Git.
• Keep two or three admins and a break-glass account in a vault.
• Test backups with a real restore.

FAQ

Is twelve lines enough?

It is a strong start. Add lines for your company, like audit logs or network rules.

Does JCasC store secrets?

Only references like ${VAR}. Real values come from a secret store.

Who owns the checklist?

The platform team, with reviews from security.

Should developers see the report?

Yes, it builds trust. Hide only secret values.

What about Kubernetes Jenkins?

Same checklist, plus pod security and network policies for agents.

Interview questions

1. How do you harden Jenkins?

   Login with SSO, no anonymous access, RBAC with folders, HTTPS, builds on agents, CSRF on, sandbox and reviewed approvals, plugin updates, JCasC, backups, and a weekly automatic check.

2. How do you prove Jenkins is secure to an auditor?

   A checklist with automatic checks, job history showing green runs, and JCasC in Git with review history.

3. What is configuration drift?

   Real settings changing away from the agreed settings over time.

4. Why use JCasC for security settings?

   Repeatable, reviewed and quick to restore.

5. What is in your Jenkins backup?

   jenkins_home without workspaces, including config, jobs, credentials and secrets keys, with a tested restore.

Samajla ka?

Ravi Sir's Cards for this lesson

Revision cardInterview card

मराठी cards