Guides for SE student projects »

Using Codecov

Codecov is an online service for generating test coverage reports.

Setting up Codecov with GitHub actions

Scenario: You have forked a project that already had support for using Codecov in the GitHub Actions CI pipeline. Now you want to set up your fork to work with Codecov as well.

Given below are the steps for achieving the above, recommended to be done by someone who has admin access to the GitHub organization that contains the fork.

  1. Sign up for a Codecov account using your GitHub account.
  2. Install the Codecov integration on GitHub::
    • Go to this page and click on the green button.
    • In the next page, select your GitHub organization
    • In the next page, choose the Only select repositories and choose your repo (you can also use the All repositories option instead). After that, click the .
  3. Sync your repo list in Codecov: Login to Codecov using your GitHub account. Ensure your organization is listed in the dropdown in the home page. If not, use the re-syncing link to sync Codecov data with GitHub (and refresh the page afterward).
    If the organization still fails to show up in the dropdown, proceed to the next step anyway (it's likely the syncing is done but the UI is not showing it).
  4. Configure Codecov for the specific fork:
    • Go to https://app.codecov.io/gh/{YOUR_ORG} (e.g., https://app.codecov.io/gh/my-team-org).
    • Click the option for your fork, and follow the instructions for GitHub Actions.
      • You can skip the Step 1: Output a Coverage report file in your CI, as it is already set up in your repo.
      • But you still need to do Step 2: Select an upload token to add as a secret on GitHub. In that step, choose the Repository Token option
      • Proceed to Step 3: add token as repository secret, and configure the CODECOV_TOKEN secret as instructed there.
      • You can skip steps 4 and 5.
  5. Check for coverage status: Go to your repo and re-run the most recent CI run (What to do if there are no CI runs in the repo?). If you set up Codecov correctly, this CI run should upload the coverage data to Codecov. Go back to your Codecov home page, choose the org from the dropdown and click on the repo name (ignore the no data label, if any). In the next page, you should see the code coverage details. Here is an example:
  6. Display the Codecov badge: If your fork has a badge showing the Codecov status (e.g, codecov), get the Markdown code for the Codecov badge provided in https://app.codecov.io/gh/{YOUR_ORG}/{YOUR_FORK}/config/badge (e.g., https://app.codecov.io/gh/se-edu/addressbook-level3/config/badge) and update the appropriate page in your fork.

You can control if CI still passes even if Codecov task fails using the line
fail_ci_if_error: true or fail_ci_if_error: false in .github/workflows/gradle.yml, under the section related to Codecov.