From 593490dd7da1a143bbac50f064133615fff09414 Mon Sep 17 00:00:00 2001 From: Lukas Puehringer Date: Mon, 16 Sep 2019 18:42:46 +0200 Subject: [PATCH 1/2] Add sslib master tox build + flesh out test docs Add a tox build that runs tests against securesystemslib's tip of development, i.e. master branch, to ease preparation of tuf for a new securesystmeslib release. The tox build is run on travis but is allowed to fail. This commit also fleshes out the testing section of the contribution documentation. Signed-off-by: Lukas Puehringer --- .travis.yml | 6 ++++++ docs/CONTRIBUTORS.rst | 44 ++++++++++++++++++++++++++++++++++++++++--- tox.ini | 13 +++++++++++++ 3 files changed, 60 insertions(+), 3 deletions(-) diff --git a/.travis.yml b/.travis.yml index 9649873b..369d2838 100644 --- a/.travis.yml +++ b/.travis.yml @@ -12,6 +12,12 @@ matrix: env: TOXENV=py35 - python: "3.6" env: TOXENV=py36 + - python: "3.6" + env: TOXENV=with-sslib-master + + allow_failures: + - python: "3.6" + env: TOXENV=with-sslib-master install: - pip install tox coveralls diff --git a/docs/CONTRIBUTORS.rst b/docs/CONTRIBUTORS.rst index dc946abb..c2afa86a 100644 --- a/docs/CONTRIBUTORS.rst +++ b/docs/CONTRIBUTORS.rst @@ -99,6 +99,7 @@ To work on the TUF project, it's best to perform a development install. optional cryptographic support, the testing/linting dependencies, etc. With a development installation, modifications to the code in the current directory will affect the installed version of TUF. + :: $ pip install -r dev-requirements.txt @@ -107,9 +108,46 @@ To work on the TUF project, it's best to perform a development install. Testing ======= -The Update Framework's unit tests can be executed by invoking -`tox `_. All supported Python versions are -tested, but must already be installed locally. +The Update Framework's unit test suite can be executed by invoking the test +aggregation script inside the *tests* subdirectory. ``tuf`` and its +dependencies must already be installed (see above). +:: + + $ cd tests + $ python aggregate_tests.py + + +To run the tests, measuring code coverage, the script can be run with the +``coverage`` tool. +:: + + $ coverage run aggregate_tests.py && coverage report + + +To develop and test ``tuf`` with above commands alongside its in-house dependency +`securesystemslib `_, +it is recommended to first make an editable install of ``securesystemslib`` (in +a *venv*), and then install ``tuf`` in editable mode too (in the same *venv*). +:: + + $ cd path/to/securesystemslib + $ pip install -r dev-requirements.txt + $ cd path/to/tuf + $ pip install -r dev-requirements.txt + + +With `tox `_ the test suite can be executed in a +separate *venv* for each supported Python version. While the supported +Python versions must already be available, ``tox`` will install ``tuf`` and its +dependencies anew in each environment. :: $ tox + + +An additional non-default ``tox`` environment is available and can be used to +test ``tuf`` against the tip of development of ``securesystemslib`` on GitHub, +to e.g. prepare the former for a new release of the latter. +:: + + $ tox -e with-sslib-master diff --git a/tox.ini b/tox.ini index 0fe7e078..23cc775e 100644 --- a/tox.ini +++ b/tox.ini @@ -26,3 +26,16 @@ deps = --editable {toxinidir} install_command = pip install --pre {opts} {packages} + + +# Develop test env to run tests against securesystemslib's master branch +# Must to be invoked explicitly with, e.g. `tox -e with-sslib-master` +[testenv:with-sslib-master] +deps = + --editable git+http://github.com/secure-systems-lab/securesystemslib.git@master#egg=securesystemslib[crypto,pynacl] + -r{toxinidir}/ci-requirements.txt + --editable {toxinidir} + +commands = + coverage run aggregate_tests.py + coverage report -m From 86257f2729f17fa8f2c2a746c53bf906d0dc36cb Mon Sep 17 00:00:00 2001 From: Lukas Puehringer Date: Wed, 18 Sep 2019 10:18:13 +0200 Subject: [PATCH 2/2] Add extra installation instruction to contrib doc Add hint to install `coverage` before using it. This should be installed via dev-requirements.txt, however it does not seem to fit in there, because dev-requirements.txt pins all its dependencies which does not seem to make sense for a development tool. Maybe a hierarchy of requirements.txt similar to https://github.com/in-toto/in-toto/commit/06a28987dc8092afa1060ee51cd3e19a6f324855 could be established. Signed-off-by: Lukas Puehringer --- docs/CONTRIBUTORS.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/CONTRIBUTORS.rst b/docs/CONTRIBUTORS.rst index c2afa86a..e7423c2e 100644 --- a/docs/CONTRIBUTORS.rst +++ b/docs/CONTRIBUTORS.rst @@ -117,8 +117,9 @@ dependencies must already be installed (see above). $ python aggregate_tests.py -To run the tests, measuring code coverage, the script can be run with the -``coverage`` tool. +To run the tests and measure their code coverage, the aggregation script can be +invoked with the ``coverage`` tool (requires installation of ``coverage``, e.g. +via PyPI). :: $ coverage run aggregate_tests.py && coverage report