Commit graph

4711 commits

Author SHA1 Message Date
Teodora Sechkova
6fa5d3ddd2
Add TestTargetFileSearch class
Extend test_updater_delegation_graphs.py with tests
for targets metadata search.
- create a new test class TestTargetFileSearch which creates
  a single repository and pefrorms multiple file searches in
  subtests.
- group the common functionality in a base class TestDelegations.
- extend the data classes to accomodate for target_files.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-12-20 19:04:08 +02:00
Teodora Sechkova
4fc2c19ba4
Modify delegations traversal loop condition
Reduce the number of variables in the while loop by using
len(visited_role_names) instead of number_of_delegations.
Include equality in the comparison with config.max_delegations
to account for visiting "targets". Shorten the commit message.

Add max number of delegations test case.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-12-20 19:04:06 +02:00
Jussi Kukkonen
0f1fc6e33c
Merge pull request #1689 from sechkova/test_delegated_roles
Test delegation graphs
2021-12-16 11:20:20 +02:00
Jussi Kukkonen
3823fd6353
Merge pull request #1724 from kairoaraujo/doc_warnings_to_errors
turn doc build warnings into errors
2021-12-16 11:20:04 +02:00
Kairo de Araujo
7dc5940827 update CONTRIBUTORS.rst related to tox
The mention of testing using multiple Python versions by default was
removed.

Signed-off-by: Kairo de Araujo <kdearaujo@vmware.com>
2021-12-16 10:04:58 +01:00
Kairo de Araujo
4187494de9 Simplify the tox envlist and add docs
Simplified ``tox`` environ list not to expect that developer has
all multiple Python version, but instead run with the python version
available.

Also, it adds docs build to environ list.

Running ``tox`` will run the lint, docs, and py.
The CI covers the multiple supported Python versions, and the
developers still can use tox -e py{version}

Signed-off-by: Kairo de Araujo <kdearaujo@vmware.com>
2021-12-16 10:04:58 +01:00
Kairo de Araujo
9955c1af55 Add the fail on warning to the readthedocs conf
CI uses the integrated webhooks from readthedocs.

Signed-off-by: Kairo de Araujo <kdearaujo@vmware.com>
2021-12-16 10:04:58 +01:00
Kairo de Araujo
d7ab1043d9 turn doc build warnings into errors
This commit adds to the docs tox session the flag ``-W``, which
turns the warnings into errors.

The CI will fail once it gets errors.

Signed-off-by: Kairo de Araujo <kdearaujo@vmware.com>
2021-12-16 10:04:58 +01:00
lukpueh
febe6c3dc3
Merge pull request #1732 from jku/release-0.20.0
Release 0.20.0
2021-12-16 09:16:43 +01:00
Lukas Puehringer
97d5395e79 Prepare v0.20.0
This will be the final release of python-tuf that includes the
legacy implementation code. Please see the [*1.0.0
announcement*](1.0.0-ANNOUNCEMENT.md) page for more details about
the next release and the deprecation of the legacy implementation,
including migration instructions.

Co-authored-by: Jussi Kukkonen <jkukkonen@vmware.com>
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-12-15 16:36:50 +02:00
Jussi Kukkonen
1ea497abae README: Update descriptions of the provided APIs
* ngclient and Metadata API are production ready: start with them, then
  mention legacy
* link to API reference instead of code
* link to examples instead of (legacy) getting started document

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-12-15 16:36:50 +02:00
Jussi Kukkonen
89b532db79 Move 1.0.0 announcement to docs
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-12-15 15:54:45 +02:00
lukpueh
0285bf9996
Merge pull request #1731 from jku/ngclient-arg-rename
ngclient: Rename constructor arg
2021-12-15 14:31:58 +01:00
Jussi Kukkonen
2a6e773323 ngclient: Rename constructor arg
metadata_dir matches metadata_base_url better.

This is an API break for anyone using named arguments.

Fixes #1638

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-12-15 13:22:24 +02:00
Teodora Sechkova
2562aff00b
Fix linter errors in tests
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-12-15 12:14:20 +02:00
Teodora Sechkova
26a5f81977
Use fetch_tracker in test_graph_traversal
Replace the use of "patch" in test_graph_traversal
with the newly added fetch_tracker from RepositorySimulator.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-12-15 12:14:18 +02:00
Teodora Sechkova
1ba93015c3
Add --dump option to tests
Extend updater tests with the option to dump repository
metadata locally.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-12-15 12:12:34 +02:00
Teodora Sechkova
db0e0d6de4
Add test_updater_deleagation_graphs.py
Add tests creating delegations graphs with different complexity
and successfully updating the delegated roles metadata.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-12-15 12:12:32 +02:00
Jussi Kukkonen
551a10ca05
Merge pull request #1702 from MVrachev/dump-dirs
Sanitize repo simulator role names when writing
2021-12-15 11:23:06 +02:00
Jussi Kukkonen
f75b58c20f
Merge pull request #1726 from MVrachev/dateutil-remove
Metadata API: remove dateutil requirement
2021-12-14 12:26:36 +02:00
Martin Vrachev
181b1a7e6d Metadata API: remove dateutil requirement
I added "dateutil" as a possible argument type for
`Metadata.bump_expiration()` as we are already testing for this and
implying it should be supported.
The problem is that "dateutil" is not added as a nontest requirement
and after a discussion, we decided we don't want to add it as well.

That's why we decided to remove "dateutil" mentions from the code
and not confuse our users we support it.

We will create a separate issue discussing the validity of
`Metadata.bump_expiration()`.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-12-14 11:23:40 +02:00
Martin Vrachev
3a00db02a3 Sanitize repo simulator role names when writing
Sanitize the delegated rolenames in the repo simulator when dumping in
a directory the same way the ngclient does.
That's necessary because when testing fishy role names leads to an
error:
"PermissionError: [Errno 13] Permission denied: '/.json'"

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-12-13 14:20:38 +02:00
Teodora Sechkova
74dddf02c3
Improve clean up in subtests
Use a "try" block to catch exceptions during failing
subtests and always execute the subtest clean up code.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-12-10 16:04:51 +02:00
Teodora Sechkova
c77b47ee82
Modify RepositorySimulator.add_delegation
Reduce the number of function arguments and use
DelegatedRole instead.

When adding a list of delegations to the repository,
move the Targets creation inside the loop to create
a separate Targets object for each delegation.

Create a new Metadata obgect only for delegated roles
which do not exist yet in the repository.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-12-10 16:04:48 +02:00
Teodora Sechkova
568580701c
tests: move _cleanup_dir to utils.py
Make the method _cleanup_dir public and move it
to tests/utils.py.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-12-10 15:59:51 +02:00
Teodora Sechkova
bdf1cbbbb5
Merge pull request #1715 from jku/tests-sim-updater-bootstrap
tests: Refactor client bootstrap
2021-12-10 13:06:53 +02:00
lukpueh
bc21225811
Merge pull request #1706 from lukpueh/1.0.0-announcement
doc: announce 1.0.0 stable release
2021-12-10 11:07:41 +01:00
Lukas Puehringer
d06ca61570 doc: announce 1.0.0 stable release
* Add a document that announces a stable "modern
  implementation"-only 1.0.0 release, with the following contents:
  - the scheduled release date
  - contents of release (metadata API, ngclient, no legacy code)
  - legacy code deprecation note (adr 2)
  - note about lack of repository tool (adr 10)
  - migration instructions

* Add "important notice" to head of main README, pointing to above
  document.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-12-10 10:59:55 +01:00
lukpueh
03cf3339ef
Merge pull request #1720 from lukpueh/add-missing-test-type-hint
test: add missing return type hint in test method
2021-12-10 10:59:05 +01:00
Lukas Puehringer
d8b23f1d55 test: add missing return type hint in test method
Following parallel merges of #1700 (added new test method),
and #1710 (started running mypy on tests), ci/cd fails in the
develop branch. This is fixed in this patch.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-12-10 10:48:36 +01:00
lukpueh
8209189a44
Merge pull request #1700 from lukpueh/repo-examples-hbd
doc:  basic hash bin delegation repo example + test
2021-12-10 10:33:04 +01:00
lukpueh
002c82855d
Merge pull request #1710 from MVrachev/start-linting
Start linting the test
2021-12-10 10:28:07 +01:00
Lukas Puehringer
ef388daa3a doc: clarify math in hash bin delegation example
Tries to clarify the introductory text in the hash bin delegation
example.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-12-10 10:13:42 +01:00
Martin Vrachev
28602e4a91 Metadata API: change bump_expiration delta type
Change the delta argument type from the tuf/api/metadata.py module
in Signed.bump_expiration() to include relativedelta as
this provides an easier interface for the callers.
We are already testing for that inside test/api line 338.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-12-09 16:55:33 +02:00
Martin Vrachev
811fa2756f Mypy: disable attr-defined warning
Mypy warns us when we assign a not defined variable to an object, but
that is something that we are warned for from pylint (seach for
"pylint: disable=no-member" in test_updater_key_rotations.py
and you will find an example where we have to disable it).
We don't want to have two linters checking for the same thing
as we can end up disabling two warnings that are actually the same
on a single line.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-12-09 16:44:25 +02:00
Martin Vrachev
a37df2eab4 Simulator: make _fetch_{metadata,target} public
Make _fetch_metadata and _fetch_taget public by renaming them to
fetch_metadata and fetch_target.
This will allow the removal of multiple pylint disables because of
"accessing private members".

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-12-09 16:44:25 +02:00
Martin Vrachev
d5d0b94f62 Add logging configuration in tests/test_examples
Logging configuration is part from each test file begging from commit
03b15fb4be
We should that as well for tests/test_examples.py

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-12-09 16:44:25 +02:00
Martin Vrachev
8e102b2084 Missing mypy stubs in for dateutil.relativedelta
Mypy reported a couple of errors for missing stubs on GitHub
https://github.com/theupdateframework/python-tuf/runs/4431787287?check_suite_focus=true

The error noted that:
"tests/test_api.py:19: error: Library stubs not installed for
"dateutil.relativedelta" (or incompatible with Python 3.10)"
In order to resolve it I added "types-python-dateutil" into
requirements-test.txt

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-12-09 16:44:25 +02:00
Martin Vrachev
2f689e70fb Start linting test files testing the new code
Exclude regexs/globs are needed to exclude the test files testing
the old code.
After we remove those files we will be able to remove the exclude
regex/globs.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-12-09 16:44:25 +02:00
Martin Vrachev
22fe1e69e4 Rename old test files by adding old suffix
Rename test files testing the old code by adding an "old" suffix.
This is done, so we can easily exclude them from linting.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-12-09 16:44:25 +02:00
Martin Vrachev
5aa547d6b6 Address more mypy warnings on tests files
All of the changes are made manual.
The target files are only those who test the new code.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-12-09 16:44:24 +02:00
Martin Vrachev
8d73330f86 Address more pylint warnings on tests files
All of the changes are made manual.
The target files are only those who test the new code.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-12-09 16:13:14 +02:00
Martin Vrachev
a728717eda Fix more black warnings on test files
The changes are automatic linting fixes from black.
The target files are only those who test the new code.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-12-09 15:52:40 +02:00
Jussi Kukkonen
b34d1b494e tests: Refactor client bootstrap
Setting up the client (when using RepositorySimulator) can be a bit
cleaner.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-12-09 09:45:45 +02:00
Jussi Kukkonen
d3b877b38a
Merge pull request #1705 from kairoaraujo/issue#1682/repositorysimulator-fetch-tracker
Implemented fetch_tracker to RepositorySimulator
2021-12-08 15:39:00 +02:00
lukpueh
b2d8572598
Merge pull request #1693 from jku/add-repo-lib-design-adr
ADR: Add New repository library design
2021-12-08 10:28:11 +01:00
Jussi Kukkonen
942e6d25ab
Merge pull request #1691 from MVrachev/key-rotations
Tests: add tests for non-root key rotations
2021-12-08 11:07:59 +02:00
Kairo de Araujo
35cbc3e21e Implemented fetch_tracker to RepositorySimulator
This commit implements a feature in Repository Simulator to
track the fetch calls to the metadata and targets. This feature was
mentioned in PR #1666 that generated issue #1682.
This commit adds RepositorySimulator.fetch_tracker. It also changes
the tests/test_updater_consistent_snapshot.py to use the
fetch_tracker instead of using mock.

It implements a dataclass that stores the calls to fetch metadata
(_fetch_metadata) in fetch_tracker.metadata and targets
(_fetch_targets) in fetch_tracker.targets.

The fetch calls for metadata, and targets are stored as lists.

Signed-off-by: Kairo de Araujo <kdearaujo@vmware.com>
2021-12-08 09:57:38 +01:00
Jussi Kukkonen
9ff5bfcff8
Merge pull request #1695 from ivanayov/repository_simulator_inconsistent_metafile_hashes
Fix inconsistent behaviour of compute_metafile_hashes_length
2021-12-07 14:08:22 +02:00
Jussi Kukkonen
f6ede42eae Fix issues from review
In addition to multiple smaller review fixes:
* Explain how the proposed library is minimal: more specific
  functionality may be added as we get more experience
* Explain what a concrete Repository implementation must implement
  (details are obviously subject to change but this is what the
  current prototype requires)

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-12-07 14:01:25 +02:00