WTF Blog

Flutter Test Coverage via Gitlab CI/CD


Quite often you need to write tests for the project and maintain good test coverage. Test coverage is a measure used in software testing. It shows the percentage of the source code of the program that was executed during the testing process.




How to check test coverage?


The following command is used to run all tests and check the coverage:

flutter test --coverage

The coverage folder is generated as a result of the command execution. It contains the lcov.info file (more information about lcov here). This file contains all the information related to the test coverage. The result of the calculation will also be shown on the console.

It is better to add the coverage folder to .gitignore

But in this case, the generated files are also taken when calculating the coverage (you can see it in the lcov.info file). It’s better to exclude them, because tests will not be written for these files. To avoid the generated files from getting into the calculation, you can use the following command:

lcov --remove coverage/lcov.info '*.g.dart' '*.part.dart'
'*/generated/*' -o coverage/lcov.info

With this command, generated files are removed from lcov.info and are not counted anymore.

Depending on the specific project, unsupported for tests files can be different. That is why you should view the entire project and add such files to this command.

Visualization of results


For convenient viewing the results of calculating the coverage percentage, it is better to generate an HTML file, which visualize all the information:

genhtml -o coverage coverage/lcov.info

This command generates everything needed and adds it to the coverage folder. To view the generated HTML file, you should open index.html or use the command open coverage/index.html .

Let’s combine all the code in one file scripts/test_coverage.sh:

# Run test coverage check
flutter test --coverage
 
# Remove from created file (coverage/lcov.info) information about generated files and other unused files
lcov --remove coverage/lcov.info '*.g.dart' '*.part.dart'
'*/generated/*' -o coverage/lcov.info
# Generate HTML file for visualization of results
genhtml -o coverage coverage/lcov.info
# Open HTML file with results
open coverage/index.html


·   ·   ·



How to add test coverage to CI/CD?


Checking test coverage in CI/CD allows you to automate the verification process for a merge request, which is very convenient.


Add script


⚠️ Before creating a merge request, you need to run the earlier created script (without visualization) to make sure that the coverage is sufficient. Otherwise, the merge request won’t pass the pipeline if there is such a check.

First you need to add a script to the project that will determine whether the merge request will lower the current coverage percentage. Add the coverage_check.sh file to the scripts folder. And add the following code to the file:

# Minimum value for coverage percent
min_coverage=20
 
# Check if the coverage is good. Test will be passed if coverage value more or equal min_coverage
coverage_check=$(flutter pub run test_cov_console --pass=$min_coverage)
# If coverage_check was PASSED exit 0
if [ "$coverage_check" == "PASSED" ]
then
    echo "Good coverage"
    exit 0
# else exit with error
else
    echo "Coverage less than $min_coverage"
    exit 1
fi

You should add test_cov_console library to the project. It is that which allows you to check compliance with the minimum percentage.

dev_dependencies:
     test_cov_console: ^0.2.2

Setup .gitlab-ci.yaml


At the very beginning of the file is the stages tag, which is responsible for this. Let’s create new stage for test coverage:

stages:
    - test_coverage

Next, we will describe what exactly will happen at the test coverage stage. To do this, add the following:

test_coverage:
    stage: test_coverage
    tags:
        - project_name
    script:
        - flutter packages get
    # Generate files
        - flutter packages pub run build_runner build --delete-conflicting-outputs
        - flutter test --coverage
        - lcov --remove coverage/lcov.info '*.g.dart' '*.part.dart'
'*/generated/*' -o coverage/lcov.info
        - ./scripts/coverage_check.sh
    artifacts:
        paths:
            - coverage
    only:
        - merge_requests
        - dev

  • stage — the name of the stage to which the actions will relate
  • tags — list of tags that are used to select the runner
  • script — commands to be executed
  • artifacts — list of files and directories to be attached to the task in case of success
  • only — controls when the job is done

That’s it! If you wish you can add test coverage badge according to the official guideline.