CI Extension Lab: Mocks, Coverage, and Quality Gates

This lab extends the Python and Java pipelines you built in the earlier session. You will keep both repositories, add mocks and coverage, and make the pipelines enforce quality gates in GitHub Actions and Jenkins.

Starting point and what changes

Before you begin

You need the two repositories from the earlier session, both with passing pipelines:

What this extension adds

Jenkins: install the Coverage plugin once

Both Jenkins jobs now use the recordCoverage step. Install the plugin one time for the whole Jenkins server:

1. Open Jenkins at http://localhost:8080, go to Manage Jenkins, then Plugins, then Available plugins.
2. Search for Coverage, select it, and click Install. Restart Jenkins if asked.

Part 1: Python extension python-ci-demo

Work in the python-ci-demo repository. Commands run from its root with the virtual environment active.

Step 1.1 Activate the environment and confirm the baseline

Goal: start from a known, passing state.

Do

cd python-ci-demo
source .venv/bin/activate          # Windows PowerShell: .venv\Scripts\Activate.ps1
git checkout main && git pull
pytest -v

Explanation

• .venv is the virtual environment from the earlier session. Activating it makes python and pytest refer to the pinned versions.
• Pulling first makes sure you extend the latest code, including any changes made in the earlier session.

Step 1.2 Create the converter with an injected dependency

Goal: add code whose external rate service can be replaced by a mock in tests.

Do

from typing import Callable

RateProvider = Callable[[str, str], float]


class CurrencyConverter:
    def __init__(self, rate_provider: RateProvider) -> None:
        self._rate_provider = rate_provider

    def convert(self, amount: float, src: str, dst: str) -> float:
        if amount < 0:
            raise ValueError("amount must be non-negative")
        if src == dst:
            return amount
        rate = self._rate_provider(src, dst)
        return round(amount * rate, 2)

Explanation

• The converter does not fetch rates itself. It receives a function in its constructor. In production that function calls a real service. In tests it is a mock.
• There are three paths: negative input is rejected, identical currencies return early without calling the provider, and everything else asks the provider. Each path needs a test.

Step 1.3 Write tests with mocks

Goal: test every path without any network access.

Do

from unittest.mock import Mock

import pytest

from app.converter import CurrencyConverter


def test_convert_uses_provider_rate():
    provider = Mock(return_value=1.1)
    converter = CurrencyConverter(provider)

    assert converter.convert(10, "EUR", "USD") == 11.0
    provider.assert_called_once_with("EUR", "USD")


def test_same_currency_skips_provider():
    provider = Mock()

    assert CurrencyConverter(provider).convert(5, "USD", "USD") == 5
    provider.assert_not_called()


def test_negative_amount_rejected():
    converter = CurrencyConverter(Mock())

    with pytest.raises(ValueError):
        converter.convert(-1, "EUR", "USD")


def test_provider_failure_propagates():
    provider = Mock(side_effect=ConnectionError("rate service timeout"))

    with pytest.raises(ConnectionError):
        CurrencyConverter(provider).convert(1, "EUR", "USD")

Explanation

Step 1.4 Patch a function imported into a module

Goal: test run() without depending on the real add function.

Do

from unittest.mock import patch

import app.main


def test_run_prints_result_from_add(capsys):
    with patch("app.main.add", return_value=99):
        app.main.run()

    assert "2 + 3 = 99" in capsys.readouterr().out

Explanation

• patch replaces a name only inside the with block, then restores it.
• Patch where the name is used. app.main imported add, so the patch target is app.main.add. Patching app.calculator.add would not affect run().
• capsys captures what the function prints so the test can check it.

Step 1.5 Exclude the script guard from coverage

Goal: keep the coverage number about logic, not about the script wrapper.

Do

from app.calculator import add, divide


def run() -> None:
    print(f"2 + 3 = {add(2, 3)}")
    print(f"10 / 4 = {divide(10, 4)}")


if __name__ == "__main__":  # pragma: no cover
    run()

Explanation

• The guard runs only when the file is executed directly. Tests import the module, so this line can never be covered.
• # pragma: no cover is a marker for coverage.py. Use it only for code that cannot run under test.

Step 1.6 Run the tests with the coverage gate

Goal: measure coverage and make the command fail below 90%.

Do

pytest -v --cov=app --cov-branch --cov-report=term-missing --cov-report=html:reports/htmlcov --cov-fail-under=90
echo "exit code: $?"

Explanation

Step 1.7 Open the HTML report

open reports/htmlcov/index.html       # Windows: start reports\htmlcov\index.html

Click calculator.py and converter.py. Lines are green when they ran. Yellow means a branch ran only one way. Red means the line never ran.

Step 1.8 Update the GitHub Actions workflow

Goal: enforce the same gate in CI and show coverage on the job page.

Do

name: Python CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.11", "3.12"]

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
          cache: pip

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt

      - name: Lint
        run: flake8 app tests

      - name: Test with coverage gate
        run: |
          pytest --junitxml=reports/junit.xml \
            --cov=app --cov-branch \
            --cov-report=term-missing \
            --cov-report=xml:reports/coverage.xml \
            --cov-report=html:reports/htmlcov \
            --cov-fail-under=90

      - name: Coverage summary in job page
        if: always()
        run: |
          {
            echo "### Coverage (Python ${{ matrix.python-version }})"
            echo '```'
            python -m coverage report
            echo '```'
          } >> "$GITHUB_STEP_SUMMARY"

      - name: Upload reports
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: python-reports-${{ matrix.python-version }}
          path: reports/

  build:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Package application
        run: tar czf app-${{ github.sha }}.tar.gz app requirements.txt

      - name: Upload package
        uses: actions/upload-artifact@v4
        with:
          name: app-package
          path: app-${{ github.sha }}.tar.gz

Explanation: what changed from the earlier workflow

Step 1.9 Update the Jenkinsfile

Goal: run the same gate in Jenkins and record it with the Coverage plugin.

Do

pipeline {
    agent any

    options {
        timestamps()
        disableConcurrentBuilds()
        buildDiscarder(logRotator(numToKeepStr: '10'))
    }

    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }

        stage('Lint and Test') {
            steps {
                sh '''
                    python3 -m venv .venv
                    . .venv/bin/activate
                    pip install -r requirements.txt
                    flake8 app tests
                    mkdir -p reports
                    pytest --junitxml=reports/junit.xml --cov=app --cov-branch --cov-report=term-missing --cov-report=xml:reports/coverage.xml --cov-report=html:reports/htmlcov --cov-fail-under=90
                '''
            }
            post {
                always {
                    junit testResults: 'reports/junit.xml', allowEmptyResults: true
                    recordCoverage(
                        tools: [[parser: 'COBERTURA', pattern: 'reports/coverage.xml']],
                        id: 'python',
                        name: 'Python coverage',
                        qualityGates: [[threshold: 90.0, metric: 'LINE', baseline: 'PROJECT', criticality: 'FAILURE']]
                    )
                    archiveArtifacts artifacts: 'reports/htmlcov/**', allowEmptyArchive: true
                }
            }
        }

        stage('Package') {
            steps {
                sh 'tar czf app-${BUILD_NUMBER}.tar.gz app requirements.txt'
                archiveArtifacts artifacts: 'app-*.tar.gz', fingerprint: true
            }
        }
    }

    post {
        always {
            cleanWs()
        }
    }
}

Explanation

• Lint and Test combines the earlier Lint and Test stages into one stage, so the coverage gate and the lint step share the same virtual environment.
• junit in post runs even when the tests fail, so the failure details are still visible on the build page.
• recordCoverage reads the Cobertura XML and applies a line-coverage threshold of 90%. With criticality: 'FAILURE', a value below 90% marks the build red.
• baseline: 'PROJECT' compares against the whole project. The Coverage plugin also provides a trend chart, so you can see whether coverage is rising or falling across builds.

Step 1.10 Commit, push, and observe both runs

Goal: see the Python pipelines run with the new gate.

Do

git checkout -b ext/python-coverage
git add . && git commit -m "Add converter with mocks, coverage gate, and CI updates"
git push -u origin ext/python-coverage

Open a pull request into main on GitHub. Then, in Jenkins, open the python-ci-demo job and click Build Now after merging, or wait for the next poll.

1. GitHub: open the Actions run and read the Coverage summary at the bottom of the run page.
2. Jenkins: open the build and then Python coverage in the left menu. Check the quality gate result and the per-file table.

Part 2: Java extension java-ci-demo

Work in the java-ci-demo repository. Commands run from its root.

Step 2.1 Confirm the baseline and create a branch

cd ../java-ci-demo
git checkout main && git pull
git checkout -b ext/java-mocks-coverage
mvn -B verify

Step 2.2 Create the repository interface

Goal: define the dependency that the service will receive, so tests can replace it.

Do

package com.example.calc;

public interface InventoryRepository {
    int stockOf(String sku);
}

Explanation

• An interface states what the dependency does without saying how. A database class, an in-memory class, and a Mockito mock can all implement or stand in for it.
• The same idea as the Python RateProvider type alias. The dependency is a named contract.

Step 2.3 Create the service with constructor injection

Goal: write logic that depends only on the interface.

Do

package com.example.calc;

public final class OrderService {

    private final InventoryRepository inventory;

    public OrderService(InventoryRepository inventory) {
        this.inventory = inventory;
    }

    public boolean canFulfil(String sku, int quantity) {
        if (quantity <= 0) {
            throw new IllegalArgumentException("quantity must be positive");
        }
        return inventory.stockOf(sku) >= quantity;
    }
}

Explanation

• The repository arrives through the constructor. The service never creates one, which is what makes the mock possible.
• Validation runs before the repository is touched. A test can then prove that invalid input never calls the repository.

Step 2.4 Write Mockito tests

Goal: test each path, including the interaction checks.

Do

package com.example.calc;

import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.verifyNoInteractions;
import static org.mockito.Mockito.when;

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;

@ExtendWith(MockitoExtension.class)
class OrderServiceTest {

    @Mock
    private InventoryRepository inventory;

    @InjectMocks
    private OrderService service;

    @Test
    void canFulfilWhenStockIsSufficient() {
        when(inventory.stockOf("ABC")).thenReturn(10);

        assertTrue(service.canFulfil("ABC", 5));
        verify(inventory).stockOf("ABC");
    }

    @Test
    void cannotFulfilWhenStockIsShort() {
        when(inventory.stockOf("ABC")).thenReturn(2);

        assertFalse(service.canFulfil("ABC", 5));
    }

    @Test
    void rejectsNonPositiveQuantityWithoutTouchingInventory() {
        assertThrows(IllegalArgumentException.class, () -> service.canFulfil("ABC", 0));

        verifyNoInteractions(inventory);
    }
}

Explanation

Step 2.5 Add Mockito and the coverage rule to pom.xml

Goal: make Mockito available and make the build fail when coverage is too low.

Do

(a) Inside <dependencies>, add after the JUnit dependency:

<dependency>
  <groupId>org.mockito</groupId>
  <artifactId>mockito-junit-jupiter</artifactId>
  <version>5.11.0</version>
  <scope>test</scope>
</dependency>

(b) Inside the jacoco-maven-plugin element, add a plugin-level <configuration> before <executions>:

<configuration>
  <excludes>
    <exclude>com/example/calc/App.class</exclude>
  </excludes>
</configuration>

(c) Inside the same plugin's <executions>, add a check execution after the report execution:

<execution>
  <id>check</id>
  <goals><goal>check</goal></goals>
  <configuration>
    <rules>
      <rule>
        <element>BUNDLE</element>
        <limits>
          <limit>
            <counter>LINE</counter>
            <value>COVEREDRATIO</value>
            <minimum>0.80</minimum>
          </limit>
          <limit>
            <counter>BRANCH</counter>
            <value>COVEREDRATIO</value>
            <minimum>0.70</minimum>
          </limit>
        </limits>
      </rule>
    </rules>
  </configuration>
</execution>

Explanation

• (a) mockito-junit-jupiter brings in Mockito and the JUnit 5 extension. The test scope keeps it out of the jar.
• (b) excludes removes App.class from coverage. The entry point only prints, and the jar smoke test in Step 2.7 covers it instead.
• (c) The check goal runs during verify. BUNDLE applies the rule to the whole project. COVEREDRATIO with minimum means the covered fraction must be at least that value. Line must be at least 80% and branch at least 70%.

Step 2.6 Run the build with the gate

mvn -B verify
echo "exit code: $?"

Step 2.7 Smoke-test the jar and read the JaCoCo report

java -jar target/calc-ci-demo-1.0.0.jar
open target/site/jacoco/index.html       # Windows: start target\site\jacoco\index.html

Explanation

• The jar run prints 2 + 3 = 5 and 10 / 4 = 2.5. That covers App.main, which the coverage rule excludes.
• In the JaCoCo report, click OrderService to see which lines and branches ran. Every line should be green.

Step 2.8 Update the GitHub Actions workflow

Goal: run lint and verify in CI, comment coverage on pull requests, and keep the jar build.

Do

name: Java CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

permissions:
  contents: read
  pull-requests: write

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        java-version: ["17", "21"]

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up JDK ${{ matrix.java-version }}
        uses: actions/setup-java@v4
        with:
          distribution: temurin
          java-version: ${{ matrix.java-version }}
          cache: maven

      - name: Lint (compile with warnings as errors)
        run: mvn -B compile

      - name: Test and JaCoCo gate
        run: mvn -B verify

      - name: Coverage comment on pull request
        if: github.event_name == 'pull_request' && always()
        uses: madrapps/jacoco-report@v1.6.1
        with:
          paths: ${{ github.workspace }}/target/site/jacoco/jacoco.xml
          token: ${{ secrets.GITHUB_TOKEN }}
          min-coverage-overall: 80
          min-coverage-changed-files: 80
          title: Java coverage

      - name: Upload reports
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: java-reports-${{ matrix.java-version }}
          path: |
            target/surefire-reports/
            target/site/jacoco/

  build:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up JDK 17
        uses: actions/setup-java@v4
        with:
          distribution: temurin
          java-version: "17"
          cache: maven

      - name: Package application
        run: mvn -B -DskipTests package

      - name: Upload jar
        uses: actions/upload-artifact@v4
        with:
          name: calc-jar
          path: target/*.jar

Explanation: what changed

Step 2.9 Update the Jenkinsfile

Goal: record JaCoCo coverage in Jenkins with its own gate.

Do

pipeline {
    agent any

    options {
        timestamps()
        disableConcurrentBuilds()
        buildDiscarder(logRotator(numToKeepStr: '10'))
    }

    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }

        stage('Lint') {
            steps {
                sh 'mvn -B compile'
            }
        }

        stage('Test') {
            steps {
                sh 'mvn -B verify'
            }
            post {
                always {
                    junit testResults: 'target/surefire-reports/*.xml', allowEmptyResults: true
                    recordCoverage(
                        tools: [[parser: 'JACOCO', pattern: 'target/site/jacoco/jacoco.xml']],
                        id: 'java',
                        name: 'Java coverage',
                        qualityGates: [[threshold: 80.0, metric: 'LINE', baseline: 'PROJECT', criticality: 'FAILURE']]
                    )
                    archiveArtifacts artifacts: 'target/site/jacoco/**', allowEmptyArchive: true
                }
            }
        }

        stage('Package') {
            steps {
                sh 'mvn -B -DskipTests package'
                archiveArtifacts artifacts: 'target/*.jar', fingerprint: true
            }
        }
    }

    post {
        always {
            cleanWs()
        }
    }
}

Explanation

• Lint and Test are separate stages so the Jenkins stage view shows which one failed.
• recordCoverage with JACOCO reads the same jacoco.xml that the Maven rule reads. Maven fails first when the rule is broken, but the Jenkins gate is a second, visible check with a trend chart.
• junit reads surefire XML, so the Test Result page lists each test method.

Step 2.10 Commit, push, and observe both runs

git add . && git commit -m "Add OrderService with Mockito tests, JaCoCo gate, and CI updates"
git push -u origin ext/java-mocks-coverage

Open a pull request into main. Check the GitHub run and the coverage comment, then build in Jenkins with Build Now.

Part 3: Reading the coverage reports

Step 3.1 Read the Python terminal table

The term-missing report looks like this:

Name                 Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------
app/calculator.py        7      0      4      0   100%
app/converter.py        10      0      6      0   100%
app/main.py              4      0      0      0   100%
----------------------------------------------------------------
TOTAL                   21      0     10      0   100%

How to read it

• Stmts is the number of statements. Miss is how many did not run.
• Branch counts decision outcomes. BrPart counts decisions where only one outcome ran.
• Missing lists the uncovered line numbers. 12 means line 12 never ran. 19->21 means the jump from line 19 to 21 never happened, which is usually an untested if outcome.
• TOTAL combines statements and branches, which is the number --cov-fail-under checks.

Step 3.2 Read the JaCoCo summary

Open target/site/jacoco/index.html. The top table shows Missed and Covered for Instructions, Branches, Lines, Methods, and Classes. The Maven rule uses the Lines and Branches rows. In the class view, green is fully covered, yellow means some branches were missed (a diamond shows how many), and red means the code never ran.

Step 3.3 Connect the report to the pipeline

Part 4: Breaking the quality gates

Work on a new branch so main stays green. Make one change at a time, read the failure, then restore.

git checkout -b lab/break-gates

Exercise 4.1: Python coverage gate

Goal: see the coverage gate stop the Python pipeline.

1. Change: delete test_negative_amount_rejected from tests/test_converter.py.
2. Run:

pytest --cov=app --cov-branch --cov-report=term-missing --cov-fail-under=90
echo "exit code: $?"

3. Expected: converter.py lists the raise line under Missing, and the last line reads FAIL Required test coverage of 90% not reached. The exit code is 1.
4. Why: the branch if amount < 0 never took its true path, so the coverage total dropped below the gate.
5. Push and check that the GitHub test job turns red and the Jenkins Lint and Test stage fails. Then restore the test and push again.

Exercise 4.2: Java JaCoCo rule

Goal: see the Maven rule stop the Java build.

1. Change: delete rejectsNonPositiveQuantityWithoutTouchingInventory from OrderServiceTest.java.
2. Run: mvn -B verify
3. Expected: a warning similar to Rule violated for bundle calc-ci-demo: branches covered ratio is 0.50, but expected minimum is 0.70, then BUILD FAILURE.
4. Find the cause: open target/site/jacoco/index.html, click OrderService, and look for the yellow line on if (quantity <= 0).
5. Restore the test and confirm BUILD SUCCESS.

Exercise 4.3: Mockito strict stubs

Goal: understand why an unused stub is a failure.

1. Change: in cannotFulfilWhenStockIsShort, delete the assertFalse(...) line and keep the when(...) line.
2. Run: mvn -B test
3. Expected: UnnecessaryStubbingException, even though no assertion fails.
4. Why: the test no longer checks its result, so it would pass without testing anything. Restore the assertion.

Exercise 4.4: Java lint gate

Goal: see -Werror fail the build on a warning.

1. Change: add this line inside main in App.java:

java.util.List raw = new java.util.ArrayList();

2. Run: mvn -B compile
3. Expected: a rawtypes warning, then warnings found and -Werror specified. The tests never run.
4. Restore by deleting the line.

Part 5: Wrap-up and troubleshooting

Final checklist

Troubleshooting

Next steps

1. Raise each threshold by one or two points as you add tests. This is called ratcheting.
2. Add a mutation testing tool (mutmut for Python, PIT for Java) to check that tests fail when code is changed.
3. Protect main in GitHub: require the Python and Java checks to pass before a merge.
4. Move coverage gating to changed lines only, so legacy code does not block new work.