Commit graph

1129 commits

Author SHA1 Message Date
Ivana Atanasova
00589f08e3 Apply top-level rolenames constants in tests
This applies the use of constants of top-level rolenames in the
tests instead of the previously hardcoded strings.
Fixes #1648

Signed-off-by: Ivana Atanasova <iyovcheva@iyovcheva-a02.vmware.com>
2021-12-02 12:25:48 +02:00
Jussi Kukkonen
bca1e67adc
Merge pull request #1686 from ivanayov/ivanayov/remove_url_normalisation
Remove URL normalisation
2021-11-30 10:02:45 +02:00
Jussi Kukkonen
2de883aedc
Merge pull request #1690 from kairoaraujo/issue#1558-delegate-roles-name-not-top-level-roles
Prevents delegate role name as top-level role name
2021-11-29 15:52:58 +02:00
lukpueh
51248db173
Merge pull request #1685 from lukpueh/repo-examples
Docs: Add repository tutorial based on metadata API
2021-11-29 13:31:16 +01:00
Lukas Puehringer
a1531d8c50 test: add test module for basic repo example
Adds new test module that executes the basic repo example
Python script and checks that it created certain (metadata)
files.

The test module is tailored for testing similar example scripts.

Co-authored-by: Joshua Lock <jlock@vmware.com>
Co-authored-by: Jussi Kukkonen <jku@goto.fi>

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-11-29 13:25:26 +01:00
Ivana Atanasova
aa591920a2 Remove URL normalisation
As a target path is a URL path it's not correct to consider it as
interchangeable with a filepath within every operation system. The
unquote is also removed as the ngclient cannot assume correctly
which encoding is intended and which not

Fixes #1483

Signed-off-by: Ivana Atanasova <iyovcheva@iyovcheva-a02.vmware.com>
2021-11-25 15:42:28 +02:00
Martin Vrachev
e2deff3148 Address mypy warnings
This commit includes manual fixes for a lot of mypy warnings.
When there were warnings that we are calling non-annotated function
in annotated context I decided to add annotations instead of ignoring
those warnings.
That's how I end up adding annotations in the whole tests/utils.py
module.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-24 20:50:10 +02:00
Martin Vrachev
0d4d7f820c Add two more invalid serialization tests
Those tests are about missing "keys" and "roles" attributes in
Targets.Delegations.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-24 18:29:32 +02:00
Martin Vrachev
1f77b107df Fix annotations in test_metadata_serialization
In test_metadata_serialization.py "test_case_data" is actually a string
when the decorator calls the actual test functions.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-24 18:29:32 +02:00
Kairo de Araujo
b6c482e1ee Ccheck empty strings, remove comprehensive lists
- Add the check for empty strings in the Delegate Role name
- Remove the comprehensive lists to make the code more readable
- Remove the test for empty file name from
``test_updater_with_simulator``

Signed-off-by: Kairo de Araujo <kdearaujo@vmware.com>
2021-11-24 17:12:45 +01:00
Jussi Kukkonen
a24c4e95be
Merge pull request #1593 from sechkova/avoid_reloading_targets
ngclient: Avoid loading targets metadata twice
2021-11-24 14:15:19 +02:00
Jussi Kukkonen
8e6485b066
Merge pull request #1688 from MVrachev/delegationrole-attributes-validation
Delegations: paths & path_hash_prefixes validation
2021-11-24 14:12:02 +02:00
Kairo de Araujo
597fc88036 Fix reuse datasets/tests, better keyids, standards
- Reuse the dataset and the existing tests
- Fix the keyids in the tests datasets to be aligned
- Fix the ``ValueError`` message aligned to the existent messages

Signed-off-by: Kairo de Araujo <kdearaujo@vmware.com>
2021-11-23 13:58:26 +01:00
Teodora Sechkova
4db4737504
Avoid loading targets metadata twice
When traversing the delegations tree looking for targets,
avoid re-loading already verified targets metadata.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-11-23 12:21:49 +02:00
Kairo de Araujo
f5bf6b0784 Prevents delegate role name as top-level role name
This commit adds the validation in the ``metadata.Delegations``
to prevent that one of the delegate role names given is a top-level
role name.

A ``ValueError`` will be raised if one of the roles names in the
list given to as delegated contains the role name as one of the
top-level roles.

Signed-off-by: Kairo de Araujo <kdearaujo@vmware.com>
2021-11-22 16:33:06 +01:00
Martin Vrachev
2adddf7402 Delegations: paths & path_hash_prefixes validation
Add sanity types checks on Targets delegation paths and
path_hash_prefixes making sure that they are strings.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-22 14:43:43 +02:00
Ivana Atanasova
9c2bf6e784 Update ngclient to return loaded metadata
This changes `TrustedMetadataSet` to return new trusted Metadata
on successful calls of the `update_<role>` functions and also
changes `Updater._load_targets` to return loaded metadata as well

Signed-off-by: Ivana Atanasova <iyovcheva@iyovcheva-a02.vmware.com>
2021-11-22 14:36:12 +02:00
Jussi Kukkonen
1b5df4ccfe
Merge pull request #1666 from sechkova/reposim-consistent-snapshot
RepositorySimulator: add non-consistent snapshot support
2021-11-18 14:03:17 +02:00
Teodora Sechkova
da1d975db0
Remove test_refresh_on_consistent_targets
Consistent snapshot and consistent targets are now
extensively tested in test_updater_consistent_snapshot.py.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-11-18 12:40:57 +02:00
Teodora Sechkova
a72fa62906
ngtests: Add consistent_snapshot tests
Add tests for ngclient.Updater toggling
'consitent_snapshot' and 'prefix_targets_with_hash'.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-11-18 11:14:12 +02:00
Teodora Sechkova
54969e8c76
RepositorySimulator: add non-consistent snapshot support
Extend URL partitioning to support serving non-versioned
metadata and non-prefixed target files.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-11-18 11:13:10 +02:00
Teodora Sechkova
67ff0424e7
Merge pull request #1654 from jku/make-refresh-optional
ngclient: Implicitly call refresh()
2021-11-17 10:50:34 +02:00
Martin Vrachev
29f936b76d Tests: address new pylint warnings
After the recent changes there are a couple of new pylint warnings that
appeared.
They are caused by the new test file that was added
test_updater_top_level_update.py and the limit of public functions was
reached in the TestMetadata class in test_api.py
The warnings should be addressed before enabling all of the linters
on the tests files.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-12 17:10:06 +02:00
Jussi Kukkonen
fa7990c354
Merge pull request #1670 from MVrachev/assert-raises-fix
Tests: self.assertRaises -> with self.assertRaises
2021-11-11 18:51:51 +02:00
Jussi Kukkonen
45f69a2639
Merge pull request #1649 from MVrachev/clarify-root-rotations-test
Clarify key rotations test cases by using keywords
2021-11-11 18:51:25 +02:00
Martin Vrachev
d4187f3186 Address pylint warnings on tests for the new code
Address or disable pylint warnings raised on all test files inside
the "tests/" directory testing the code of the new implementation.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-11 15:39:56 +02:00
Martin Vrachev
2042a54dcf disable black for part of key rotations tests
In order to make the `RootVersion` test cases understandable it's
better if we use keywords and don't split the different RootVersions
into multiple lines.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-11 15:31:04 +02:00
Martin Vrachev
8c6e157519 Tests: test_api split test_sign_verify()
test_sign_verify() is testing too many cases and after the recent
pylint warning about the usage of too many local variables it became
clear it's time to split this test function.

I decided to split it logically as half of the function was about
failures connected with verify.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-10 15:55:34 +02:00
Martin Vrachev
3c80c5bcd0 Tests: self.assertRaises -> with self.assertRaises
Change the instances of "self.assertRaises" to "with self.assertRaises"
as using "self.assertRaises" can lead to long statements separated to
multiline expressions as pointed out by Jussi here:
https://github.com/theupdateframework/python-tuf/pull/1658#discussion_r741725382

On another hand "with self.assertRaises()" looks a lot better:
589ed9e0d4/tests/test_api.py (L131)

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-10 15:27:03 +02:00
Teodora Sechkova
d66c3baf27
RepoSim: remove metadata version check
Except for 'root' role, RepositorySimulator does not
keep previous metadata versions, it always serves the latest
one. The metadata version check during fetch serves mostly
for informative purposes and removing it allows generating test
metadata with mismatching version.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-11-09 16:22:38 +02:00
Teodora Sechkova
8418d5267f
ngtests: Add asserts for expected version
Define _assert_version_equals for checking if the
local metadata file's version is as expected.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-11-09 14:18:24 +02:00
Teodora Sechkova
e51642a290
ngtests: Fix formatiing and linter issues
Fix formatting and some potential linter and typing
errors.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-11-09 14:18:17 +02:00
Teodora Sechkova
8a2c7857ac
ngtests: Add addtional asserts for files on disk
Extend the TestRefresh cases with additional checks
for expected metadata files and their content written
on the file system.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-11-09 14:18:05 +02:00
Teodora Sechkova
954331c8af
ngtests: Add top-level-roles update tests
Add ngclient/updater tests following the top-level-roles metadata
update from the specification (Detailed client workflow)
using RepositorySimulator.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-11-08 17:47:15 +02:00
Martin Vrachev
beb8087bf5 Clarify key rotations test cases by using keywords
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-05 16:46:33 +02:00
Jussi Kukkonen
26213ae576 tests: Add test for implicit refresh()
This is a minimal test case to show the basic functionality:
I'd like to update most of the tests to use the implicit method
but would like to do that when there's less churn in these tests.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-11-03 14:57:23 +02:00
Martin Vrachev
6fe36a00c4 Rename & simplify a couple of tests in test_api.py
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-02 19:31:46 +02:00
Martin Vrachev
2e9ef79762 Apply isort on the tests of the new code
All of the changes included are a result of applying isort
on our tests on the new code.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-02 19:31:25 +02:00
Martin Vrachev
c98b429643 Apply black on the tests of the new code
All of the changes included are a result of applying black
on our tests on the new code.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-02 19:30:35 +02:00
Jussi Kukkonen
e817473e3c tests: Add root key rotation tests
Add one test with 1 subtests for various root key rotation situations.

The test data definition format is a bit tricky but I tried to document
that in the test function docstring.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-10-27 19:01:23 +03:00
Jussi Kukkonen
ad80bd96c6 tests: Mark RepositorySimulator. create_key() static
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-10-27 18:57:12 +03:00
Jussi Kukkonen
fd40dfc094 tests: Refactor simulator signer handling
Store signers with their keyids so they are easier to remove.
The signers structure now looks like:
{
  "role1": {
    "keyidA": SSlibSigner,
    "keyidB": SSlibSigner,
  }
}

Add convenience method for adding a signer.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-10-27 18:57:08 +03:00
Jussi Kukkonen
7b8ff220b1
Merge pull request #1604 from jku/ngclient-api-polish
Ngclient api polish
2021-10-27 18:40:49 +03:00
Jussi Kukkonen
d519a413b0 ngclient: Rename get_one_valid_targetinfo()
This is slightly cosmetic but rename get_one_valid_targetinfo to
get_targetinfo:
* The function name is long without any reason: "one" and "valid" are
  always implicit
* shortening makes code (incl. our examples and tests) easier to read
* We're also already changing updater API (compared to legacy) so this
  alone does not break things -- it's also not a difficult "port".

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-10-27 09:55:57 +03:00
Jussi Kukkonen
9b761b8620 ngclient: Simplify caching
Remove updated_targets() as it doesn't fit the rest of the API.

In its stead add find_cached_target() which has a similar signature
as download_target(): both accept an optional local filepath as
argument and return full local filepath. In the
find_cached_target() case None is returned if the local file is not the
correct target file.

Updater constructor gets a new optional target_dir argument: This means
client can avoid giving a local filepath as an argument to
find_cached_target()/download_target() -- Updater will then generate a
filename within targets_dir.

A reasonable use pattern (when targets_dir is set in constructor):

    info = updater.get_one_valid_targetinfo("targetname")
    path = updater.find_cached_target(info)
    if path is None:
        path = updater.download_target(info)

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-10-27 09:55:57 +03:00
Jussi Kukkonen
cc9f3876c4 tests: Shorten variable names to reasonable length
Otherwise absolutely everything is split on multiple lines.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-10-27 09:55:57 +03:00
Martin Vrachev
4158272a7a Use TOP_LEVEL_ROLE_NAMES across TUF
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-10-25 15:58:56 +03:00
Martin Vrachev
9bc55ee568 Metadata API: validate root role names
Validate that root role names are 4 and that they are exactly
"root", "snapshot", "targets" and "timestamp" as described in
the spec:
https://theupdateframework.github.io/specification/latest/#root-role

Additionally, fix the valid_roots dataset, so each of the cases contains
the top metadata role names inside the roles dictionary.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-10-25 15:58:55 +03:00
Jussi Kukkonen
2206fc917e
Merge pull request #1631 from MVrachev/fix-test-error
Test metadata files: bump expiration date and resign
2021-10-25 11:02:34 +03:00
Martin Vrachev
d3d2f57f3a Test files: bump expiration date and resign
Our newly added metadata files in the
tests/repository_data/fishy_rolenames/metadata directory have an expiry
date until "2021-10-22T11:21:56Z" and today while running the tests on
develop branch I recived this error:
ExpiredMetadataError("Metadata X expired on Fri Oct 22 11:21:56 2021")
when running the tests in tests/test_updater.py file and more precisly
the TestUpdaterRolenames.test_unusual_rolenames() test.

That's why I decided to bump the expiration date to a random time in
the future (October 22-nd 2050) and I had to resign all of the metadata
files.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-10-23 18:39:22 +03:00