If the automated tests are failing, or if a new module is being built for the first time, it is useful to be able to check modules interactively before they make their way into an official compose. (For details on the regular testing of modules and their contents, see the Automated Testing section below)
The interactive testing guide below assumes that fedpkg module-build is
succeeding - if that isn’t the case, then the module and package build logs
should provide details as to what is going wrong.
The most common problems that are encountered despite a successful build and passing integration tests are dependency resolution issues when attempting to install a module. The resolving-dependency-issues section provides more guidance on dealing with such situations.
The original Fedora 26 Boltron prototype lives on as a set of test container images, which build atop the Fedora modular composes, but include some additional utililties to help out with testing modules prior to their release in Fedora.
Two variants of the image are provided, a boltron-27 image with module
definitions from the Fedora 27 Modular Server release, and a
boltron-bikeshed with the latest module definitions from the module build
service that have passed their self-tests in Taskotron:
$ docker run --rm -it jamesantill/boltron-27 bash
$ docker run --rm -it jamesantill/boltron-bikeshed bash
See the README in the image source repository for more information on
the available utilities.
Module releases may take a day or more to become available in the default image, so the Boltron image provides a helper script to download built modules directly from the Module Build Service and incorporate them into the currently running image. For example:
# /LOCAL.sh postgresql:9.6:20171018083530
This will download the binary artifacts for that particular build of the PostreSQL 9.6 stream, set up a local repository for them (including the module metadata), enable that repository, and then install the module with its default profile.
A non-default install profile can be requested by appending the profile
name to the build identifer, separated by a /:
# /LOCAL.sh postgresql:9.6:20171018083530/client
The list of currently build modules and their identifiers is available at http://modularity.fedorainfracloud.org/modularity/mbs/.
As described in the Continuous Integration page on the Fedora wiki, Fedora is currently working to improve the level of automated test coverage for the entire distribution.
To provide feedback to package and module maintainers in a timely fashion, tests should be generally be executed at the first level where all the pieces necessary to run them are available.
Independently of the modular build process, individual packages may
define their own automated tests, both as part of the %check section in
their RPM spec file, as well as in the tests/ subdirectory of their
respective
dist-git RPM repositories
Modules may also define their own individual automated tests, in the tests/
subdirectory of their respective
dist-git module repositories
The Meta Test Family project is used to define test cases for whole modules and their resulting RPMs and container images.
When a module is built in Fedora’s infrastructure and MTF tests are available in its repository, they will be run afterwards by Taskotron.
Finally, a base set of automated tests check that the core set of modules defined in Fedora’s system profile can all be installed and updated.
These tests are written as Ansible playbooks, and can be found in the compose-tests repository. In addition to checking that the default set of available modules works, these tests also ensure that the actual system profile defined in the Pungi configuration matches the expected one defined in the F27 content tracking repository.
These tests are executed in the CentOS CI infrastructure:
Issues specific to the Fedora Modular release are tracked using the fedora-modular-release component in Bugzilla.
In addition to automated testing of the overall modular compose, there is also lower level integration testing of DNF’s module management support: https://github.com/rpm-software-management/ci-dnf-stack/tree/modularity
If a module or modular compose test failure appears to be related to an error in DNF, then it should be reported against the dnf component of the Fedora project in Bugzilla and then linked from the Modular DNF tracker BZ.