Commit graph

4611 commits

Author SHA1 Message Date
Martin Vrachev
0d4d7f820c Add two more invalid serialization tests
Those tests are about missing "keys" and "roles" attributes in
Targets.Delegations.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-24 18:29:32 +02:00
Martin Vrachev
1f77b107df Fix annotations in test_metadata_serialization
In test_metadata_serialization.py "test_case_data" is actually a string
when the decorator calls the actual test functions.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-24 18:29:32 +02:00
Martin Vrachev
e18b6ba506 Configure mypy to show error codes
By configuring mypy to show error codes when we get a warning by mypy
we will receive an error code as well.
Those error codes are useful when you want to disable specific mypy
warning for a line with:

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-24 18:29:32 +02:00
Jussi Kukkonen
a24c4e95be
Merge pull request #1593 from sechkova/avoid_reloading_targets
ngclient: Avoid loading targets metadata twice
2021-11-24 14:15:19 +02:00
Jussi Kukkonen
8e6485b066
Merge pull request #1688 from MVrachev/delegationrole-attributes-validation
Delegations: paths & path_hash_prefixes validation
2021-11-24 14:12:02 +02:00
Teodora Sechkova
4db4737504
Avoid loading targets metadata twice
When traversing the delegations tree looking for targets,
avoid re-loading already verified targets metadata.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-11-23 12:21:49 +02:00
Jussi Kukkonen
acb201d6cc
Merge pull request #1680 from ivanayov/ivanayov/ngclient_loaded_metadata
Update ngclient to return loaded metadata
2021-11-22 16:12:42 +02:00
Jussi Kukkonen
fb645f9592
Merge pull request #1683 from ivanayov/ivanayov/ngclient_skip_visited_roles
Update ngclient to skip visited nodes on delegation tree traversal
2021-11-22 16:03:54 +02:00
Jussi Kukkonen
747ec19934
Merge pull request #1668 from MVrachev/validate-rest-of-metadata-api
Metadata API: make Root roles a Mapping
2021-11-22 16:02:09 +02:00
Martin Vrachev
2adddf7402 Delegations: paths & path_hash_prefixes validation
Add sanity types checks on Targets delegation paths and
path_hash_prefixes making sure that they are strings.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-22 14:43:43 +02:00
Ivana Atanasova
9c2bf6e784 Update ngclient to return loaded metadata
This changes `TrustedMetadataSet` to return new trusted Metadata
on successful calls of the `update_<role>` functions and also
changes `Updater._load_targets` to return loaded metadata as well

Signed-off-by: Ivana Atanasova <iyovcheva@iyovcheva-a02.vmware.com>
2021-11-22 14:36:12 +02:00
Ivana Atanasova
aff96c31d1 Update ngclient to skip visited nodes on delegation tree traversal
This change edits the ngclient `Updater` to traverse the delegation
tree on nodes, instead of edges in order to skip already visited
nodes.

For more detailed clarification, please review
theupdateframework/specification#177

Fixes #1528

Signed-off-by: Ivana Atanasova <iyovcheva@iyovcheva-a02.vmware.com>
2021-11-18 19:18:33 +02:00
Jussi Kukkonen
1b5df4ccfe
Merge pull request #1666 from sechkova/reposim-consistent-snapshot
RepositorySimulator: add non-consistent snapshot support
2021-11-18 14:03:17 +02:00
Teodora Sechkova
da1d975db0
Remove test_refresh_on_consistent_targets
Consistent snapshot and consistent targets are now
extensively tested in test_updater_consistent_snapshot.py.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-11-18 12:40:57 +02:00
Teodora Sechkova
a72fa62906
ngtests: Add consistent_snapshot tests
Add tests for ngclient.Updater toggling
'consitent_snapshot' and 'prefix_targets_with_hash'.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-11-18 11:14:12 +02:00
Teodora Sechkova
54969e8c76
RepositorySimulator: add non-consistent snapshot support
Extend URL partitioning to support serving non-versioned
metadata and non-prefixed target files.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-11-18 11:13:10 +02:00
Jussi Kukkonen
bb15ecfb99
Merge pull request #1677 from jku/advertize-annotations
Advertize type annotations with py.typed
2021-11-17 10:51:09 +02:00
Teodora Sechkova
67ff0424e7
Merge pull request #1654 from jku/make-refresh-optional
ngclient: Implicitly call refresh()
2021-11-17 10:50:34 +02:00
Jussi Kukkonen
6744f6a9c7
Merge pull request #1652 from jku/limit-github-token-visibility
GH actions: limit GitHub token visibility
2021-11-17 10:06:31 +02:00
Jussi Kukkonen
ec285a448c
Merge pull request #1674 from MVrachev/pylint-more-changes
Tests: address new pylint warnings
2021-11-16 09:57:40 +02:00
Martin Vrachev
29f936b76d Tests: address new pylint warnings
After the recent changes there are a couple of new pylint warnings that
appeared.
They are caused by the new test file that was added
test_updater_top_level_update.py and the limit of public functions was
reached in the TestMetadata class in test_api.py
The warnings should be addressed before enabling all of the linters
on the tests files.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-12 17:10:06 +02:00
Jussi Kukkonen
7951ed2b86 Advertize type annotations with py.typed
Existence of py.typed in the top-level package tells users of the
packages that the packages are annotated.

This should fix mypy errors like this in downstream projects:
  error: Skipping analyzing "tuf.api.metadata":
  found module but no type hints or library stubs

Fixes #1633

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-11-11 22:34:22 +02:00
Jussi Kukkonen
fa7990c354
Merge pull request #1670 from MVrachev/assert-raises-fix
Tests: self.assertRaises -> with self.assertRaises
2021-11-11 18:51:51 +02:00
Jussi Kukkonen
45f69a2639
Merge pull request #1649 from MVrachev/clarify-root-rotations-test
Clarify key rotations test cases by using keywords
2021-11-11 18:51:25 +02:00
Jussi Kukkonen
41b20df767
Merge pull request #1664 from MVrachev/apply-pylint
New implementation tests: address pylint warnings
2021-11-11 16:10:26 +02:00
Martin Vrachev
d4187f3186 Address pylint warnings on tests for the new code
Address or disable pylint warnings raised on all test files inside
the "tests/" directory testing the code of the new implementation.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-11 15:39:56 +02:00
Martin Vrachev
74c657dc9b Pylint config: add _ as a good variable name
_ is often used when a function returns multiple values and you need
a sub-portion of them. Then, those values that are unnecessary can be
named _.
Currently, pylint warns us that this is not a good variable name, so
fix that.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-11 15:37:49 +02:00
Martin Vrachev
de27f09a44 Pylint: disable duplicate-code for the new code
Pylint reported a couple of warnings flagged as "duplicate-code".
We were truly duplicating code - one of the examples was when we
imported the same objects from tuf/api/metadata.py:
MetaFile, Role, Root, Snapshot, TargetFile, Targets, and Timestamp
in two separate modules.
So, I thought we do want to be repetitive here and include that code at
both modules. The problem is that besides importing the above
classes the modules imported other classes from tuf.api.metadata.py
and there was no way to disable this check.
I searched and found out that this is a known problem:
https://github.com/PyCQA/pylint/issues/214.

That's why the only solution I see is to disable this warning
temporarily and hoping that one day when this issue is fixed we will
remember to turn it on again.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-11 15:37:49 +02:00
Martin Vrachev
2042a54dcf disable black for part of key rotations tests
In order to make the `RootVersion` test cases understandable it's
better if we use keywords and don't split the different RootVersions
into multiple lines.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-11 15:31:04 +02:00
Jussi Kukkonen
5a9b3fa963
Merge pull request #1671 from MVrachev/split-test
Tests: test_api split test_sign_verify()
2021-11-11 10:32:12 +02:00
Martin Vrachev
8c6e157519 Tests: test_api split test_sign_verify()
test_sign_verify() is testing too many cases and after the recent
pylint warning about the usage of too many local variables it became
clear it's time to split this test function.

I decided to split it logically as half of the function was about
failures connected with verify.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-10 15:55:34 +02:00
Martin Vrachev
3c80c5bcd0 Tests: self.assertRaises -> with self.assertRaises
Change the instances of "self.assertRaises" to "with self.assertRaises"
as using "self.assertRaises" can lead to long statements separated to
multiline expressions as pointed out by Jussi here:
https://github.com/theupdateframework/python-tuf/pull/1658#discussion_r741725382

On another hand "with self.assertRaises()" looks a lot better:
589ed9e0d4/tests/test_api.py (L131)

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-10 15:27:03 +02:00
Jussi Kukkonen
0088ebd444
Merge pull request #1636 from sechkova/ng-tests-metadata-update
ngclient: top-level-roles update tests
2021-11-10 10:09:15 +02:00
Teodora Sechkova
d66c3baf27
RepoSim: remove metadata version check
Except for 'root' role, RepositorySimulator does not
keep previous metadata versions, it always serves the latest
one. The metadata version check during fetch serves mostly
for informative purposes and removing it allows generating test
metadata with mismatching version.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-11-09 16:22:38 +02:00
Teodora Sechkova
8418d5267f
ngtests: Add asserts for expected version
Define _assert_version_equals for checking if the
local metadata file's version is as expected.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-11-09 14:18:24 +02:00
Teodora Sechkova
e51642a290
ngtests: Fix formatiing and linter issues
Fix formatting and some potential linter and typing
errors.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-11-09 14:18:17 +02:00
Teodora Sechkova
8a2c7857ac
ngtests: Add addtional asserts for files on disk
Extend the TestRefresh cases with additional checks
for expected metadata files and their content written
on the file system.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-11-09 14:18:05 +02:00
Martin Vrachev
29da5da4bb Metadata API: make root roles Mapping
Mark roles as a Mapping to indicate that users should not add or remove
values from the dictionary during the lifetime of the Root object)

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-08 19:30:49 +02:00
Teodora Sechkova
954331c8af
ngtests: Add top-level-roles update tests
Add ngclient/updater tests following the top-level-roles metadata
update from the specification (Detailed client workflow)
using RepositorySimulator.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-11-08 17:47:15 +02:00
Martin Vrachev
beb8087bf5 Clarify key rotations test cases by using keywords
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-05 16:46:33 +02:00
Jussi Kukkonen
8ae944ccb2
Merge pull request #1659 from MVrachev/disable-pylint-format
pylintc for new code: disable format checker
2021-11-05 16:19:22 +02:00
Martin Vrachev
b137fbcc3f plyintrc: remove redundant format section
The "FORMAT" section in pylint is no longer needed after the format
checker is disabled.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-05 15:07:39 +02:00
lukpueh
d70c3b32f2
Merge pull request #1663 from jku/gh-actions-set-permissions
github: explicitly set workflow permissions
2021-11-04 10:52:45 +01:00
Jussi Kukkonen
e073fea819 github: explicitly set workflow permissions
* current workflow only needs to read git content
* if the workflow in the future does need write access, it's good to
  see permissions explicitly changing

For context: "pull_request" runs never have write access anyway, so this
significantly changes only the "push" runs that happen when branches are
merged to develop.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-11-04 11:39:05 +02:00
Jussi Kukkonen
ce4a60eb26
Merge pull request #1658 from MVrachev/apply-linters
New implementation tests: apply black and isort automatic linting fixes
2021-11-04 09:32:56 +02:00
Jussi Kukkonen
26213ae576 tests: Add test for implicit refresh()
This is a minimal test case to show the basic functionality:
I'd like to update most of the tests to use the implicit method
but would like to do that when there's less churn in these tests.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-11-03 14:57:23 +02:00
Jussi Kukkonen
ec74499fc3 ngclient: Implicitly call refresh()
Refresh can still be done explicitly (so client has more control over
when data is downloaded) but there's no reason we can't do it
automatically as well: Call refresh() from get_targetinfo() if
top-level targets is not loaded.

Update API documentation accordingly.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-11-03 14:57:23 +02:00
Martin Vrachev
a7766ac53a pylintc for new code: disable format checker
By default pylint does format checks:
https://pylint.pycqa.org/en/latest/technical_reference/features.html?highlight=format#format-checker

The problem is we also use black and isort who have format checkers as
well. This makes pylint format checks obsolete.

Also, it's possible that you would want to disable a warning and you
can end up in the situation where you will have to disable it for
two tools altogether.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-02 19:45:18 +02:00
Martin Vrachev
6fe36a00c4 Rename & simplify a couple of tests in test_api.py
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-02 19:31:46 +02:00
Martin Vrachev
2e9ef79762 Apply isort on the tests of the new code
All of the changes included are a result of applying isort
on our tests on the new code.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-02 19:31:25 +02:00