This is a new client library implementation using the Metadata API
(the only "old" file used is exceptions.py)
* Functional but largely still untested
* Requires work before it can replace tuf.client but should be good
enough to include in the repo
The major changes compared to current client so far are:
* Use of Metadata API
* Major simplification in the way downloads are handled due to removing
mirrors support
* Separating tracking of valid metadata into a separate component
* There's no MultiRepoUpdater
We do not expect other major changes (in the sense of moving large
amounts of code) but do plan to possibly improve the client API. The
API has already changed so this is not going to be a 1:1 compatible
implementation, but porting should not be difficult.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Start building the next-gen client: Copy existing components from the
current client.
All of these files have some changes compared to the already existing
copies (because ngclient uses the same linting rules as Metadata API).
* download.py is likely to see major changes in the future.
* requests_fetcher is likely to see some minor changes (like allowing
compression)
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Try to make errors derive from RepositoryError if they can be the
result of malicious or malfunctioning remote repository: The idea
here is that client can handle just RepositoryError instead of
individual errors (as it cannot do anything about any of them).
Also improve variable naming.
This is backwards compatible.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
There are no longer any files in TUF that came from Thandy, and
so the dual license is no longer required.
Signed-off-by: Marina Moore <mnm678@gmail.com>
We've been returning Signature objects since 49aa0fc167.
Also add a test case that does something with the returned signature.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
report an error whenever a function with type annotations calls a
function defined without annotations.
Also include exceptions.py in mypy checks.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
exceptions.py is used from metadata.py: Annotating it is required if we
want to enable "no_untyped_call" in mypy.
The only change is UnsignedMetadataError: I would have removed argument
"signable" altogether but figured this is not worth breaking API. So
now "signable" is still not used for anything but it is now optional.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
We're quite close to being able to enable only "strict = True" but not
quite there. In the mean time enable some useful individual settings.
disallow_untyped_defs:
report an error whenever code contains a function definition
without type annotations
warn_redundant_casts:
report an error whenever code uses an unnecessary cast
warn_unused_ignores:
report an error whenever code uses an unnecessary # type: ignore
comment
warn_unreachable:
report an error whenever code is determined to be unreachable or
redundant
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Explain the ways a delegation can happen: Do not try to cover the
complete process (specification should do that) but offer enough
details that the complexity is not completely hidden from the viewer.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Try to keep dostrings and comments to the point, avoid mentioning
details if they are not necessary or are likely to become outdated
and try to minimize number of comment lines.
Co-authored-by: Joshua Lock <jlock@vmware.com>
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
The idea of this commit is to separate (de)serialization testing outside
test_api.py and make sure we are testing from_dict/to_dict for all
possible valid data for all classes.
Jussi in his comment here:
https://github.com/theupdateframework/tuf/issues/1391#issuecomment-849390669
proposed using decorators when creating comprehensive testing
for metadata serialization.
The main problems he pointed out is that:
1) there is a lot of code needed to generate the data for each case
2) the test implementation scales badly when you want to add new
cases for your tests, then you would have to add code as well
3) the dictionary format is not visible - we are loading external files
and assuming they are not changed and valid
In this change, I am using a decorator with an argument that complicates
the implementation of the decorator and requires three nested functions,
but the advantages are that we are resolving the above three problems:
1) we don't need new code when adding a new test case
2) a small amount of hardcoded data is required for each new test
3) the dictionaries are all in the test module without the need of
creating new directories and copying data.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Securesystemslib digest() and digest_fileobject()
calls raise sslib specific exceptions that need to be
handled and re-raised as TUF exceptions.
Updated tests in test_api.py accordingly.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Changes tests/repository_data/keystore/root_key3* to be an ecdsa key,
created and encrypted with the generate_ecdsa_key and
encrypt_key methods of securesystemslib.keys.
The test_updater_root_rotation_integration.py test
tests both repotool and updater.
Signed-off-by: Velichka Atanasova <avelichka@vmware.com>
- valid length: greater than zero
- valid hashes: a non-empty dictionary of type Dict[str, str]
Checking the validity of hash algorithms is not part
of the metadata input validation and is done by
securesystemslib during hash verification.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Test unknown signature algorithm/scheme.
Also shorten the incorrect (but syntactically valid) signature a bit.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Aim to only raise UnsignedMetadataError from verify_signature().
Some of the situations could be things like UnsupportedAlgorithmError
-- where the underlying reason may be a missing dependency -- but it
seems impossible for a client to know whether it's that or whether it
is broken or malicious server side.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Clarify that we don't semantically validate "Key" instances during
initialization and that this is a responsibility of securesystemslib.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Probably there could be future API calls that modify "threshold"
to a new value, but the problem is we don't have a clear idea
if they would exist and what exactly they will do.
That's why it makes sense to validate against the potential problems
we can imagine - in this case, is passing a threshold below 1.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
In our discussion with Jussi we come to the conclusion that we want
to verify that all Key attributes contain values in the expected types,
but at the same time, we don't want to focus on validating the semantics
behind them.
The reason is that having a Key instance with invalid attributes is
possible and supported by the spec.
That's why we have a "threshold" for the roles meaning we can have up to
a certain number of invalid Keys until we satisfy
the required threshold.
Also, for deeper semantic validation it's better to be done in
securesystemslib which does the actual work with keys.
For context see: https://github.com/theupdateframework/tuf/issues/1438
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
According to point 2 in the semver specification:
"A normal version number MUST take the form X.Y.Z where X, Y, and Z are
non-negative integers...". See: https://semver.org/#spec-item-2
Also, even though version strings like "2.0.0-rc.2" or "1.0.0-beta" are
valid strings in semantic versioning format, in TUF we never needed
to add letters for our specification number.
That's why I validate that: spec_version is a . separated string
and when split it has a length of 3 and that each of the
three elements is a number.
The modules under the tuf/api folder in TUF are an alternative TUF
implementation. That's why they should use their own constant for
SPECIFICATION_VERSION in tuf/metadata/api.
This time, I used a list for the SPECIFICATION_VERSION constant in order
to retrieve major and minor versions easier.
I use the SPECIFICATION_VERSION to check that the given spec_version is
supported against the tuf code spec version.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
