Ravindra BagaleCourses & study guides मराठी

Chapter 32: Jenkins Governance, Troubleshooting and Expert Projects

32.2 Will not start

Why: the problem

Suppose we are at TCS and it is Monday 9 a.m. Over the weekend someone updated plugins and restarted Jenkins. Now the page shows "Jenkins is starting" forever, or the service died. 200 developers cannot build. Panic makes people reinstall Jenkins, which can lose jobs. A calm engineer reads the log first. Jenkins almost always says why it will not start: a wrong Java version, a broken plugin, a port already used, or a full disk.

What we will build

You will make Jenkins fail to start in a lab in four common ways, read the exact log line, and fix each one. You will do it on a VM with systemd, in Docker, and on Kubernetes with the Helm chart.

What: every term in plain English

  • systemd — The Linux service manager. systemctl starts and stops services; journalctl shows their logs.
  • jenkins.log / journal — Where Jenkins writes start-up messages. On systemd hosts, use journalctl -u jenkins.
  • Port in use — Another program already listens on 8080, so Jenkins cannot.
  • Java version — Jenkins LTS 2.463 and newer need Java 17 or 21.
  • Plugin dependency — A plugin needs another plugin or a newer Jenkins core.
  • Safe restart — Restart after running builds finish: http://jenkins/safeRestart.
  • CrashLoopBackOff — Kubernetes status when a container keeps failing at start.
Check the service state Read the last log lines Check Java Check the port Check disk Start again after the fix Open the UI
Will not start: status, log, Java, port, disk, plugins, then restart

How: step by step

  1. Check the service state

    sudo systemctl status jenkins --no-pager
    

    What it does: Is it running, failed, or starting?

    Output

    ● jenkins.service - Jenkins Continuous Integration Server
         Active: failed (Result: exit-code) since Mon 2026-10-26 08:52:10 IST
    
  2. Read the last log lines

    sudo journalctl -u jenkins -n 50 --no-pager
    

    What it does: The real reason is near the end.

    Output

    jenkins[1422]: Running with Java 11 from /usr/lib/jvm/java-11-openjdk-amd64, which is older than the minimum required version (Java 17).
    
  3. Check Java

    java -version 2>&1 | head -1
    

    What it does: Which Java runs Jenkins.

    Output

    openjdk version "11.0.24" 2024-07-16
    
  4. Check the port

    sudo ss -ltnp | grep 8080
    

    What it does: Who holds 8080?

    Output

    LISTEN 0 100 *:8080 *:* users:(("java",pid=981,fd=7))
    
  5. Check disk

    df -h /var/lib/jenkins
    

    What it does: A full disk stops start-up.

    Output

    /dev/nvme1n1  50G  50G  0  100% /var/lib/jenkins
    
  6. Start again after the fix

    sudo systemctl restart jenkins && sudo journalctl -u jenkins -f | grep -m1 "fully up"
    

    What it does: Wait for the ready line.

    Output

    Jenkins is fully up and running
    
  7. Open the UI

    curl -s -o /dev/null -w "%{http_code}\n" http://localhost:8080/login
    

    What it does: 200 means the login page works.

    Output

    200
    

Example 1: VM: wrong Java after an OS update

Stack. Ubuntu 22.04, Jenkins LTS apt package

Goal. VM: wrong Java after an OS update. Install Java 17 and point Jenkins at it.

terminal

sudo apt-get install -y openjdk-17-jre-headless
sudo update-alternatives --list java
sudo systemctl edit jenkins
# in the editor add:
# [Service]
# Environment="JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64"
sudo systemctl daemon-reload
sudo systemctl restart jenkins
sudo journalctl -u jenkins -n 5 --no-pager

What each line does

  • openjdk-17-jre-headless — Java 17 without desktop parts.
  • update-alternatives --list java — Shows all installed Java versions.
  • systemctl edit jenkins — Creates an override file, so package updates do not undo your change.
  • Environment="JAVA_HOME=..." — Tells the Jenkins service which Java to use.
  • daemon-reload — systemd reads the new override.

Green output

/usr/lib/jvm/java-11-openjdk-amd64/bin/java
/usr/lib/jvm/java-17-openjdk-amd64/bin/java
Oct 26 09:04:12 ci-vm jenkins[2210]: Running from: /usr/share/java/jenkins.war
Oct 26 09:04:41 ci-vm jenkins[2210]: Jenkins is fully up and running

Expected result. Jenkins starts with Java 17.

Example 2: Docker: a broken plugin after an update

Stack. Docker, Jenkins LTS

Goal. Docker: a broken plugin after an update. Start Jenkins with that plugin disabled, then fix it.

terminal

docker logs jenkins 2>&1 | grep -m3 -E "SEVERE|Failed Loading"
# disable the bad plugin by adding a .disabled file in the volume
docker run --rm -v jenkins_home:/data alpine:3.20 touch /data/plugins/blueocean-pipeline-api-impl.jpi.disabled
docker restart jenkins
docker logs -f jenkins 2>&1 | grep -m1 "fully up"

What each line does

  • grep -E "SEVERE|Failed Loading" — The lines that name the broken plugin.
  • touch ... .jpi.disabled — Jenkins skips any plugin with a .disabled file next to it.
  • alpine with -v jenkins_home:/data — Edit the volume without starting Jenkins.
  • docker restart jenkins — Try again with the plugin off.

Green output

SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Blue Ocean Pipeline implementation v1.27.16 (blueocean-pipeline-api-impl)
java.io.IOException: Failed to load: Blue Ocean Pipeline implementation (blueocean-pipeline-api-impl 1.27.16)
 - Update required: Pipeline: Graph Analysis Plugin (pipeline-graph-analysis 202.va_d268e64deb_3) to be updated to 216.vfd8b_ece330ca_ or higher
blueocean-pipeline-api-impl
Jenkins is fully up and running

Expected result. Jenkins starts. Then in Plugins > Updates, update pipeline-graph-analysis, remove the .disabled file and restart.

Example 3: Kubernetes: the Jenkins Pod is in CrashLoopBackOff because the PVC is full

Stack. Kubernetes EKS, Jenkins Helm chart, gp3

Goal. Kubernetes: the Jenkins Pod is in CrashLoopBackOff because the PVC is full.

terminal

kubectl -n jenkins get pods
kubectl -n jenkins logs jenkins-0 -c jenkins --previous | tail -3
kubectl -n jenkins get pvc jenkins
kubectl -n jenkins patch pvc jenkins -p '{"spec":{"resources":{"requests":{"storage":"100Gi"}}}}'
kubectl -n jenkins delete pod jenkins-0
kubectl -n jenkins get pods -w

What each line does

  • logs --previous — Logs of the last crashed container, not the new one.
  • -c jenkins — The main container. The chart also has init and sidecar containers.
  • patch pvc ... 100Gi — Grow the disk. The StorageClass must allow volume expansion, like gp3 on EKS.
  • delete pod jenkins-0 — The StatefulSet makes a new Pod that mounts the bigger disk.

Green output

NAME        READY   STATUS             RESTARTS   AGE
jenkins-0   1/2     CrashLoopBackOff   6          9m
java.io.IOException: No space left on device
NAME      STATUS   VOLUME     CAPACITY   STORAGECLASS
jenkins   Bound    pvc-7c1e   50Gi       gp3
persistentvolumeclaim/jenkins patched
jenkins-0   2/2     Running            0          58s

Expected result. The Pod runs on a 100 Gi disk. Next lesson shows how to stop the disk filling up again.

Verify: did it work?

  1. Service is active

    systemctl is-active jenkins
    

    What it does: One word answer.

    Output

    active
    
  2. Login page answers

    curl -s -o /dev/null -w "%{http_code}\n" http://localhost:8080/login
    

    What it does: 200 is good.

    Output

    200
    
  3. No SEVERE lines since start

    sudo journalctl -u jenkins --since "10 min ago" | grep -c SEVERE
    

    What it does: Should be 0.

    Output

    0
    

When it is red 1

java.net.BindException: Address already in use

Cause. Another process uses port 8080, often an old Jenkins or Tomcat.

Fix, step by step

  1. sudo ss -ltnp | grep 8080 to find the pid.
  2. Stop that program, or change JENKINS_PORT in systemctl edit jenkins.

When it is red 2

Running with Java 11 ... older than the minimum required version (Java 17)

Cause. The OS default Java is too old.

Fix, step by step

  1. Install Java 17 or 21.
  2. Set JAVA_HOME in a systemd override.

When it is red 3

java.io.IOException: No space left on device

Cause. JENKINS_HOME disk is 100 percent full.

Fix, step by step

  1. df -h to confirm.
  2. Delete old builds and workspaces (next lesson).
  3. Grow the disk.

When it is red 4

AccessDeniedException: /var/lib/jenkins/config.xml

Cause. Files are owned by root after a manual copy.

Fix, step by step

  1. sudo chown -R jenkins:jenkins /var/lib/jenkins
  2. Restart.

Cleanup

  1. Remove the systemd override if you set a test port

    sudo systemctl revert jenkins
    

    What it does: Back to package defaults.

    Output

    Removed /etc/systemd/system/jenkins.service.d/override.conf.
    
  2. Delete .disabled files after fixing plugins

    find /var/lib/jenkins/plugins -name "*.disabled" -delete
    

    What it does: Turn plugins back on.

    Output

    (no output)
    

Ravindra Bagale's Tip

Reinstall karnyaadhi log vacha. Uttar log madhech asta!

Practice tasks

Easy Read a real start log

Why: Know the normal lines.

What to do: Start Jenkins and save the log.

How:
  1. journalctl -u jenkins -n 100 > start-ok.log

Check: You can point to the fully up line.

Medium Port clash on purpose

Why: See the error by face.

What to do: Run a second Jenkins on 8080.

How:
  1. docker run -d -p 8080:8080 jenkins/jenkins:lts-jdk17
  2. Read docker logs.

Check: You see the bind error.

Hard Plugin pin list

Why: Upgrades must be repeatable.

What to do: Export plugins with versions to plugins.txt.

How:
  1. jenkins-plugin-cli --list > plugins.txt

Check: You can rebuild the same plugin set.

Easy Say Jenkins start-up failures in two lines

Why: If you can say it simply, you understand it. Juniors and interviewers both want the short version first.

What to do: Write two short lines about Jenkins start-up failures for a fresher who knows only Linux and Git basics.

How:
  1. Write the problem it solves.
  2. Write the one Jenkinsfile step or command you use for it.
  3. Read it aloud to a friend.

Check: A fresher understands it without asking a question.

Easy Start a fresh lab Jenkins

Why: Practice needs a throwaway Jenkins. If you break it, you delete it and start again.

What to do: Run Jenkins LTS in Docker with a named volume.

How:
  1. docker volume create jenkins_home
  2. docker run -d --name jenkins -p 8080:8080 -v jenkins_home:/var/jenkins_home jenkins/jenkins:lts-jdk17
  3. docker logs jenkins 2>&1 | grep -A2 "initialAdminPassword"

Check: The Jenkins login page opens on localhost:8080.

Medium Do it on the second setup

Why: A release idea must work on a VM, on Docker and on Kubernetes. Only the deploy step should change.

What to do: Repeat this lesson on a different target than the one you used first.

How:
  1. If you used Docker, use kind or minikube now.
  2. Change only the deploy commands.
  3. Keep the same Jenkinsfile stages.

Check: Same result, and the stage names did not change.

Medium Break it on purpose

Why: You must know the red output by face. In a real outage there is no time to read slowly.

What to do: Cause one error from the "When it is red" part of this lesson and fix it.

How:
  1. Follow the error section.
  2. Copy the exact message into your notes.
  3. Write the fix in one line.

Check: You fixed it using only the steps, without searching the web.

Medium Write a release runbook page

Why: At 2 a.m. nobody remembers flags. A short page saves the on-call person.

What to do: Write a one-page runbook for this lesson.

How:
  1. Title, when to use, exact commands, how to check, how to undo.

Check: A teammate can follow it alone.

Hard Explain it like an interview

Why: Interviewers want real stories, not definitions.

What to do: Record a 60 second answer about this lesson.

How:
  1. Say the problem, the Jenkinsfile step, and one error you hit.

Check: Your answer has a real command and a real error message.

Hard Draw the flow top to bottom

Why: A drawing shows the order of work and who is waiting on whom.

What to do: Draw this lesson as boxes from top to bottom.

How:
  1. Start at git push, go through the stages, end at the check in production.

Check: Each box matches a real step you ran.

Homework: try at home

Try at home

At home, run Jenkins in Docker, then start a second container on the same port and read the error. Then disable a plugin with a .disabled file and see it greyed out in the plugin page.

Recap

  • Read the log before you touch anything.
  • Common causes: Java version, plugin, port, disk, ownership.
  • Disable a bad plugin with a .disabled file.
  • On Kubernetes use logs --previous and check the PVC.
  • Never reinstall Jenkins as a first step.

FAQ

Should I update all plugins at once?

No. Update in a test Jenkins first, a few at a time, and back up JENKINS_HOME.

Where is the log on Docker?

docker logs jenkins.

Is there a safe mode for plugins?

Not a full one. Disable suspect plugins with .disabled files, start, then turn them back on one by one.

Why stuck on "Jenkins is getting ready to work"?

Usually slow plugin loading or a full disk. Read the log, it moves on or shows SEVERE.

Does a restart lose builds?

Running builds stop. Use safeRestart to wait for them.

Interview questions

  1. Jenkins does not start after a plugin update. What do you do?

    Read journalctl or docker logs, find the SEVERE plugin line, disable it with .disabled, start, then update its dependency.

  2. How do you troubleshoot a Jenkins Pod in CrashLoopBackOff?

    kubectl describe pod, kubectl logs --previous, check PVC space and probes.

  3. How do you change Java for the Jenkins service?

    Install the right JDK and set JAVA_HOME in a systemd override.

  4. How do you avoid broken upgrades?

    Pin plugins in plugins.txt, test in a staging controller, back up first.

  5. What does safeRestart do?

    Waits for running builds, then restarts.

Quick quiz

Q1. First command when Jenkins will not start?

journalctl -u jenkins -n 50 (or docker logs).

Q2. Which Java does current Jenkins LTS need?

Java 17 or 21.

Q3. How do you disable a plugin without the UI?

Create PLUGIN.jpi.disabled in the plugins folder.

Q4. How do you see logs of a crashed Pod?

kubectl logs POD --previous.

Q5. Why use systemctl edit?

Overrides survive package updates.

Samajla ka?

Ravi Sir's Cards for this lesson

मराठी cards