Chapter 22: Jenkins Language Projects — Java to Python

22.2 Shared Pipeline job reminder

By Ravindra Bagale · Last updated: 25 October 2026

Why: the problem

Think of the Flipkart platform team. Ten product teams want CI. If each team clicks a Freestyle job by hand, every job looks different, nobody can review the changes, and when Jenkins dies the jobs are gone. So the team uses one pattern for every language: a Pipeline job that reads the Jenkinsfile from the repo. This is called Pipeline script from SCM. SCM means source control, like Git. The job only knows three things: the repo URL, the branch, and the path of the Jenkinsfile. Everything else lives in Git, where it is reviewed like code. In this lesson you create that job once, by hand and then as code, and reuse it for Java, Python and every other language in this chapter.

What we will build

You will push a small repo with a Jenkinsfile to a local Git server, create the Pipeline job from SCM, run it, copy the job for a second language by changing only the repo URL, and add a poll trigger so pushes start builds.

What: every term in plain English

• Pipeline job — A Jenkins job type that runs a Jenkinsfile.
• Pipeline script from SCM — Jenkins reads the Jenkinsfile from Git, not from a text box.
• Repository URL — Where Jenkins clones from.
• Branch Specifier — Which branch to build, like */main.
• Script Path — Where the Jenkinsfile is inside the repo. Usually Jenkinsfile.
• Credentials — A saved token or SSH key Jenkins uses for private repos.
• Poll SCM — Jenkins checks Git on a schedule and builds when there is a new commit.
• Webhook — Git tells Jenkins about a push at once. Faster than polling.

How: step by step

1. Make a bare Git server

   mkdir -p ~/devops-lab/ch22/git && git init -q --bare -b main ~/devops-lab/ch22/git/badge-python.git
   

   What it does: A local remote for the lab.

   Output

   (no output)
   

2. Clone it

   cd ~/devops-lab/ch22 && git clone -q git/badge-python.git && cd badge-python
   

   What it does: Working copy.

   Output

   (no output)
   

3. Add a Jenkinsfile

   printf "pipeline {\n agent any\n stages { stage(\x27Hi\x27) { steps { sh \x27echo hello from SCM\x27 } } }\n}\n" > Jenkinsfile && cat Jenkinsfile
   

   What it does: A tiny pipeline.

   Output

   pipeline {
    agent any
    stages { stage('Hi') { steps { sh 'echo hello from SCM' } } }
   }
   

4. Push

   git add . && git commit -qm "Add Jenkinsfile" && git push -q origin main
   

   What it does: Now Jenkins can read it.

   Output

   (no output)
   

5. Create the job in the UI

   New Item, name badge-python-pipeline, Pipeline, OK
   

   What it does: Choose Pipeline, not Freestyle.

   Output

   Job created
   

6. Point it at Git

   Definition: Pipeline script from SCM; SCM: Git; URL: file:///home/ravi/devops-lab/ch22/git/badge-python.git; Branch: */main; Script Path: Jenkinsfile
   

   What it does: Only these fields matter.

   Output

   Saved
   

7. Build Now

   Click Build Now, then Console Output
   

   What it does: First run.

   Output

   hello from SCM
   Finished: SUCCESS
   

Example 1: The same job as code with Job DSL

Stack. Jenkins Job DSL

Goal. The same job as code with Job DSL.

jobs.groovy

['java', 'python', 'dotnet', 'php'].each { lang ->
  pipelineJob("badge-${lang}-pipeline") {
    definition {
      cpsScm {
        scm {
          git {
            remote { url("file:///home/ravi/devops-lab/ch22/git/badge-${lang}.git") }
            branch('*/main')
          }
        }
        scriptPath('Jenkinsfile')
      }
    }
    triggers { scm('H/5 * * * *') }
  }
}

What each line does

• each { lang -> — Loop over four languages.
• pipelineJob — Make a Pipeline job.
• cpsScm — Pipeline script from SCM.
• scriptPath — Path of the Jenkinsfile.
• scm H/5 — Poll Git about every 5 minutes.

Green output

Processing provided DSL script
Added items:
    GeneratedJob{name='badge-java-pipeline'}
    GeneratedJob{name='badge-python-pipeline'}
    GeneratedJob{name='badge-dotnet-pipeline'}
    GeneratedJob{name='badge-php-pipeline'}
Finished: SUCCESS

Expected result. Four identical jobs from ten lines. A new language is one word in the list.

Example 2: Start a build from the command line

Stack. Jenkins REST

Goal. Start a build from the command line.

trigger.sh

JENKINS=http://localhost:8080
USER=ravi
TOKEN="$(cat ~/.jenkins-token)"
curl -s -X POST -u "$USER:$TOKEN" "$JENKINS/job/badge-python-pipeline/build" -o /dev/null -w "%{http_code}\n"
sleep 8
curl -s -u "$USER:$TOKEN" "$JENKINS/job/badge-python-pipeline/lastBuild/api/json?tree=number,result"

What each line does

• ~/.jenkins-token — Your API token in a file only you can read (chmod 600). Never paste it in the script.
• /build — Queue a build.
• 201 — Queued.
• lastBuild/api/json — Ask for the result.

Green output

201
{"_class":"org.jenkinsci.plugins.workflow.job.WorkflowRun","number":2,"result":"SUCCESS"}

Expected result. Scripts and release tools can start any language job the same way.

Example 3: Poll SCM so a push starts a build

Stack. Jenkins

Goal. Poll SCM so a push starts a build.

Jenkinsfile

pipeline {
  agent any
  triggers { pollSCM('H/2 * * * *') }
  stages {
    stage('Hi') { steps { sh 'echo "commit $(git rev-parse --short HEAD)"' } }
  }
}

push.sh

git commit -qam "Poll every 2 minutes" && git push -q
echo "change" >> README.md && git add . && git commit -qm "Trigger" && git push -q

What each line does

• pollSCM — Jenkins asks Git for new commits on a schedule.
• H/2 — About every 2 minutes, spread out with H.
• git rev-parse --short — Print which commit was built.

Green output

Started by an SCM change
commit 7c1d2e9
Finished: SUCCESS

Expected result. On a real server use a webhook from GitLab or Bitbucket; polling is fine for a local lab.

Example 4: A Jenkinsfile in a sub folder of a monorepo

Stack. Jenkins

Goal. A Jenkinsfile in a sub folder of a monorepo.

tree

shop/
  services/badge-java/Jenkinsfile
  services/badge-python/Jenkinsfile

job setting

Script Path: services/badge-python/Jenkinsfile

Jenkinsfile

pipeline {
  agent any
  stages {
    stage('Test') {
      steps { dir('services/badge-python') { sh 'python3 -m pytest -q' } }
    }
  }
}

What each line does

• Script Path — Can be any path, not only the root.
• dir() — Run the commands inside that folder.

Green output

3 passed in 0.04s
Finished: SUCCESS

Expected result. One repo, many services, one job each.

Verify: did it work?

1. Job exists

   curl -s -u "ravi:$(cat ~/.jenkins-token)" http://localhost:8080/job/badge-python-pipeline/api/json?tree=name
   

   What it does: Name returned.

   Output

   {"name":"badge-python-pipeline"}
   

2. Last result

   curl -s -u "ravi:$(cat ~/.jenkins-token)" "http://localhost:8080/job/badge-python-pipeline/lastBuild/api/json?tree=result"
   

   What it does: Green.

   Output

   {"result":"SUCCESS"}
   

3. Remote has the file

   git -C ~/devops-lab/ch22/badge-python ls-files
   

   What it does: Jenkinsfile is tracked.

   Output

   Jenkinsfile
   

When it is red 1

ERROR: Unable to find Jenkinsfile from git file:///...

Cause. Script Path is wrong, or the file was never pushed.

Fix, step by step

1. git ls-files on your clone.
2. Fix Script Path; it is case sensitive.

When it is red 2

fatal: '/home/ravi/devops-lab/ch22/git/badge-python.git' does not appear to be a git repository

Cause. Jenkins runs as the jenkins user and cannot read your home folder.

Fix, step by step

1. Use a real Git server like GitLab for the lab.
2. Or put the bare repo in a folder the jenkins user can read.

When it is red 3

Checking out a local repository is not allowed

Cause. Newer Git plugin blocks file:// URLs by default.

Fix, step by step

1. Prefer a real Git server.
2. For a throwaway lab only: start Jenkins with -Dhudson.plugins.git.GitSCM.ALLOW_LOCAL_CHECKOUT=true.

When it is red 4

HTTP ERROR 403 No valid crumb was included in the request

Cause. You called the API without a token.

Fix, step by step

1. Use user:API-token in curl, not your password.

Cleanup

1. Delete the test job

   Job page, Delete Pipeline
   

   What it does: Keep only jobs you need.

   Output

   Deleted
   

Practice tasks

Homework: try at home

Recap

• One job pattern for every language.
• The Jenkinsfile lives in Git.
• Change only the repo URL.
• Use webhooks on real servers.
• Use API tokens, never passwords.

FAQ

Why not paste the pipeline into the job text box?

It is not in Git, so no review and no history.

Can one repo have many Jenkinsfiles?

Yes. Set a different Script Path per job.

What is a Multibranch Pipeline?

A job that finds every branch with a Jenkinsfile. Chapter 24 covers it.

Is Job DSL required?

No, but it helps when you have many jobs.

Do I need credentials for public repos?

No. Private repos need a token or SSH key saved in Jenkins.

Why H in cron?

It spreads jobs so they do not all start at the same minute.

Interview questions

1. Explain Pipeline script from SCM.

   The job stores only the repo, branch and path; the pipeline itself is versioned in Git.

2. How do you create many similar jobs?

   Job DSL, Configuration as Code, or Multibranch / Organization folders.

3. Polling versus webhooks?

   Polling asks on a schedule; webhooks push events instantly and save load.

4. How do you trigger a job from a script?

   POST to /job/NAME/build with user and API token.

5. Where do you keep Jenkins API tokens?

   In a protected file or a secret manager, never in code.

Samajla ka?

Ravi Sir's Cards for this lesson

Revision cardInterview card

मराठी cards