Chapter 56: Docker CI/CD with Jenkins and ECR, Deploy to Amazon ECS, Troubleshooting, Best Practices and Interview Questions

56.11 Integration tests with Testcontainers

By Ravindra Bagale · Last updated: 25 October 2026

Why: the problem

Suppose we are at Freshworks and our order service passes every unit test, then fails in staging because a SQL query behaves differently on real Postgres than on the in-memory H2 or SQLite we used in tests. Mocks and fake databases lie. The fix is to test against the real thing. But a shared test database is slow, flaky and full of other people's data. Testcontainers is a library, for Java, Python, Go, Node.js, .NET and more, that starts real Docker containers from inside our test code: Postgres, Kafka, Redis, LocalStack, anything. Each test run gets a fresh, throwaway container on a random port, and the library waits until it is ready and removes it at the end. Our tests become real integration tests that run the same on a laptop and in CI. In this lesson we write Python and Java tests against real Postgres and Redis, wire them into CI, and learn the speed tricks.

What: every term in plain English

• Testcontainers — Library that controls Docker from tests to start real dependencies.
• Module — Ready-made container class: PostgresContainer, KafkaContainer, RedisContainer.
• Random host port — Mapped port chosen by Docker; tests ask the container for it.
• Wait strategy — How the library decides the container is ready: log line, port, HTTP.
• Ryuk — Helper container that removes test containers even if the test crashes.
• Fixture — pytest setup shared by tests; scope=session reuses one container.
• Reusable containers — Opt-in mode that keeps containers between runs on a laptop.
• Testcontainers Cloud — Docker's hosted option to run test containers remotely.

How: step by step

1. Install the library. pip or Maven dependency.
2. Start a container in a fixture. with PostgresContainer(...).
3. Get the URL. get_connection_url().
4. Run migrations and tests. Real SQL, real behaviour.
5. Clean up automatically. Context manager and Ryuk.
6. Run in CI. Any runner with Docker.
7. Speed up. Session scope, pinned small images, pre-pull.

Example 1: Python: test a repository against real Postgres

Goal. Python: test a repository against real Postgres.

app/repo.py

import psycopg2

def save_order(conn, customer, amount):
    with conn.cursor() as cur:
        cur.execute(
            "INSERT INTO orders(customer, amount) VALUES (%s, %s) "
            "ON CONFLICT (customer) DO UPDATE SET amount = orders.amount + EXCLUDED.amount "
            "RETURNING amount",
            (customer, amount),
        )
        conn.commit()
        return cur.fetchone()[0]

tests/test_repo.py

import psycopg2, pytest
from testcontainers.postgres import PostgresContainer
from app.repo import save_order

@pytest.fixture(scope="session")
def pg():
    with PostgresContainer("postgres:16-alpine") as c:
        yield c

@pytest.fixture
def conn(pg):
    url = pg.get_connection_url().replace("postgresql+psycopg2", "postgresql")
    conn = psycopg2.connect(url)
    with conn.cursor() as cur:
        cur.execute("DROP TABLE IF EXISTS orders")
        cur.execute("CREATE TABLE orders(customer text PRIMARY KEY, amount numeric NOT NULL)")
    conn.commit()
    yield conn
    conn.close()

def test_upsert_adds_amounts(conn):
    assert save_order(conn, "asha", 100) == 100
    assert save_order(conn, "asha", 50) == 150

Commands

pytest -q tests/test_repo.py
docker ps --format '{{.Image}} {{.Ports}}'

What each line does

• ON CONFLICT ... DO UPDATE — Postgres-specific SQL that SQLite or H2 may treat differently.
• PostgresContainer('postgres:16-alpine') — Pinned, small image, same major version as production.
• scope='session' — One container for the whole test run: fast.
• get_connection_url() — Includes the random host port.
• Fresh table per test — Isolation without a new container.
• docker ps during run — Shows the test Postgres and the Ryuk container.

Green output

.                                                     [100%]
1 passed in 4.21s
postgres:16-alpine 0.0.0.0:32771->5432/tcp
testcontainers/ryuk:0.8.1 0.0.0.0:32770->8080/tcp

Expected result. The test ran against real Postgres in about 4 seconds and the containers were removed afterwards.

Example 2: Java with JUnit 5: Postgres and Redis together

Goal. Java with JUnit 5: Postgres and Redis together.

build.gradle.kts (excerpt)

dependencies {
    testImplementation("org.testcontainers:postgresql:1.20.3")
    testImplementation("org.testcontainers:junit-jupiter:1.20.3")
    testImplementation("org.junit.jupiter:junit-jupiter:5.11.3")
    testImplementation("redis.clients:jedis:5.2.0")
}
tasks.test { useJUnitPlatform() }

OrderCacheIT.java

@Testcontainers
class OrderCacheIT {
  @Container
  static PostgreSQLContainer pg = new PostgreSQLContainer("postgres:16-alpine");

  @Container
  static GenericContainer redis = new GenericContainer(DockerImageName.parse("redis:7.4-alpine"))
      .withExposedPorts(6379)
      .waitingFor(Wait.forLogMessage(".*Ready to accept connections.*", 1));

  @Test
  void cachesOrderAfterFirstRead() throws Exception {
    var repo = new OrderRepo(pg.getJdbcUrl(), pg.getUsername(), pg.getPassword());
    var cache = new OrderCache(redis.getHost(), redis.getMappedPort(6379), repo);
    repo.insert("ord-1", 499);
    assertEquals(499, cache.get("ord-1"));
    repo.updateAmount("ord-1", 999);
    assertEquals(499, cache.get("ord-1"));
  }
}

Commands

./gradlew test --tests OrderCacheIT

What each line does

• @Testcontainers / @Container — JUnit starts and stops containers around the class.
• static — One container per class, not per test.
• getJdbcUrl() — Ready JDBC URL with the random port.
• withExposedPorts(6379) — Redis port; getMappedPort gives the host port.
• waitingFor log message — Test starts only when Redis is ready.
• Second read is cached — Real Redis behaviour, not a mock.

Green output

OrderCacheIT > cachesOrderAfterFirstRead() PASSED
BUILD SUCCESSFUL in 9s

Expected result. Two real dependencies, no shared environment, and the same test runs on every laptop and CI agent.

Example 3: Run in CI and make it fast

Goal. Run in CI and make it fast.

ci.yml (GitLab CI)

integration-tests:
  image: python:3.12-slim
  services:
    - name: docker:27-dind
      alias: docker
  variables:
    DOCKER_HOST: tcp://docker:2375
    DOCKER_TLS_CERTDIR: ""
    TESTCONTAINERS_HOST_OVERRIDE: docker
  before_script:
    - pip install -q "testcontainers[postgres]" pytest psycopg2-binary
  script:
    - pytest -q tests/

Jenkinsfile (stage)

stage('Integration tests') {
  agent { label 'docker' }
  steps {
    sh 'docker pull -q postgres:16-alpine'
    sh 'pip install -q "testcontainers[postgres]" pytest psycopg2-binary'
    sh 'pytest -q tests/ --junitxml=reports/it.xml'
  }
  post { always { junit 'reports/it.xml' } }
}

~/.testcontainers.properties (laptop only)

testcontainers.reuse.enable=true

What each line does

• docker:27-dind service — GitLab runners without a host socket use DinD (see 52-12).
• TESTCONTAINERS_HOST_OVERRIDE — Tells the library the containers are reachable at host docker.
• Jenkins docker agent — Agent with Docker access runs the same tests.
• docker pull -q first — Pre-pull so the test timer does not include the download.
• junit report — Results show in Jenkins.
• reuse.enable — Keeps containers between local runs; never in CI.

Green output

$ pytest -q tests/
....                                                  [100%]
4 passed in 7.92s
Job succeeded

Expected result. The same tests run in GitLab and Jenkins. Session scope, small pinned images and pre-pulling keep the run under 10 seconds.

Every command, step by step (plain docker and Compose)

1. Is Docker reachable?

   docker info --format '{{.ServerVersion}}'
   

   What it does: Testcontainers needs a Docker API.

   27.3.1
   

2. Run tests

   pytest -q tests/
   

   What it does: Starts containers as needed.

   4 passed
   

3. See test containers

   docker ps --filter label=org.testcontainers=true
   

   What it does: Labelled containers.

   postgres:16-alpine ...
   

4. Clean leftovers

   docker rm -f $(docker ps -aq --filter label=org.testcontainers=true)
   

   What it does: After a hard crash.

   (ids)
   

5. Disable Ryuk (restricted CI)

   export TESTCONTAINERS_RYUK_DISABLED=true
   

   What it does: Only if Ryuk cannot run; clean up yourself.

   (set)
   

6. Remote Docker

   export DOCKER_HOST=tcp://docker:2375
   

   What it does: Point at DinD or a remote daemon.

   (set)
   

7. Pre-pull

   docker pull -q postgres:16-alpine redis:7.4-alpine
   

   What it does: Faster first test.

   docker.io/library/postgres:16-alpine
   

When it is red 1

docker.errors.DockerException: Error while fetching server API version: Connection refused

Cause. The test process cannot reach Docker.

Fix, step by step

1. Start Docker or set DOCKER_HOST.
2. In CI, give the job Docker access (socket or DinD).

When it is red 2

Timed out waiting for container port to open (localhost ports: [32771] should be listening)

Cause. Container slow to start, or the host name is wrong in DinD.

Fix, step by step

1. Set TESTCONTAINERS_HOST_OVERRIDE=docker in DinD.
2. Use a log wait strategy; pre-pull images.

When it is red 3

Could not find a valid Docker environment

Cause. Java library could not detect Docker.

Fix, step by step

1. Check docker info works for the same user.
2. Set DOCKER_HOST, or use Testcontainers Cloud.

Practice tasks

pip install 'testcontainers[postgres]' pytest psycopg2-binary

pytest -q tests/test_repo.py

@pytest.fixture(scope='session')

DROP TABLE IF EXISTS orders

from testcontainers.redis import RedisContainer

./gradlew test --tests OrderCacheIT

Wait.forLogMessage(".*Ready.*", 1)

TESTCONTAINERS_HOST_OVERRIDE: docker

docker ps --filter label=org.testcontainers=true

docker pull -q postgres:16-alpine

Recap

• Test against real dependencies, not fakes.
• Testcontainers starts throwaway containers from test code.
• Use random ports via the library API.
• Ryuk cleans up even after crashes.
• CI needs a Docker API: socket, DinD or a cloud runner.
• Session scope, pinned small images and pre-pulling keep tests fast.

FAQ

Q. Is Testcontainers only for Java?

No: Java, Python, Go, Node.js, .NET, Rust and more.

Q. Can I test AWS code?

Yes, with the LocalStack module for S3, SQS, DynamoDB and others.

Q. Kafka?

KafkaContainer or the Redpanda module start a real broker.

Q. Does it work with Podman?

Yes, with the Podman socket set as DOCKER_HOST and Ryuk settings adjusted.

Q. Unit tests or integration tests?

Keep fast unit tests; use Testcontainers for code that talks to databases, queues and APIs.

Q. Where are the docs?

testcontainers.com and the module pages for each language.

Q. How is this different from docker compose up before tests?

Compose gives one shared environment that must be started and cleaned by hand; Testcontainers gives each run its own fresh containers, controlled by the test code, with random ports and automatic clean-up.

Interview questions

1. What is Testcontainers?

   Model answer: A library that starts real dependencies in Docker from test code with random ports, wait strategies and automatic clean-up.

2. Why prefer it over mocks or in-memory DBs?

   Model answer: Real engine behaviour: SQL dialect, constraints, transactions.

3. How do you run Testcontainers in CI?

   Model answer: Give the job a Docker API (host socket, DinD, or Testcontainers Cloud) and pre-pull images.

4. What is Ryuk?

   Model answer: A sidecar that removes containers, networks and volumes after the test session.

5. How do you keep tests fast?

   Model answer: Share containers per session or class, small pinned images, reuse locally.

6. How do you avoid port clashes?

   Model answer: Never fix host ports; use the mapped random port from the API.

Samajla ka?

Ravi Sir's Cards for this lesson

Revision cardInterview card

मराठी cards