make bootstrap required and explicit: callers must pass bootstrap=<root_bytes> or bootstrap=None.
also tighten docs, examples, and tests to reflect the explicit trust anchor choice.
Signed-off-by: 1seal <security@1seal.org>
securesystemslib.hash is a small wrapper around hashlib, which serves
two main purposes:
* provide helper function to hash a file
* translate custom hash algorithm name "blake2b-256" to "blake2b" with
(digest_size=32).
In preparation for the removal of securesystemslib.hash, this patch ports
above behavior to tuf and uses the builtin hashlib directly where
possible.
related secure-systems-lab/securesystemslib#943
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
This allows using some more nice annotations from 3.10
while still being compatible with even Python 3.8.
These are all annotation changes, should not modify any functionality.
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
* Remove exectuable flag from a couple of files
* Half of the test files have a shebang (but are
still not executable): remove the shebang
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
* API changes covered:
* keys and interface modules removed
* SSlibSigner removed
* CryptoSigner added: this replaces the removed functionality
* DSSE "signatures" container type changed
* Currently pins a securesystemslib main branch commit:
this shoudl be reverted before merging, when securesystemslib
has made a release
* tests/generated_data/generate_md.py was simplified
* Encrypted test keys in tests/repository_data/keystore were replaced
with the unencrypted PEM versions of the same keys
* The public test keys in tests/repository_data/keystore were removed
as they were not used anymore
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
New Securesystemslib Keys can now be instantiated in two ways:
* deserialize via Key.from_dict() as before
* generate new keys via implementation specific methods
Fix all cases where we call Key() or Key.from_securesystemslib_key()
and use SSlibKey methods instead. Fix related tests.
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
Add support for Targets using delegation with succinct_roles.
For that purpose, we needed a method that can add succinct_roles
information with its all corresponding bins to the target metadata
and self.md_delegates attribute in RepositorySimulator.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Here is the list of all breaking API changes:
1) The "role" and "key" arguments in "Root.add_key()" are in reverse
order - "key" becomes first and "role" second.
2) "Root.remove_key()" has been renamed to "Root.revoke_key()".
3) The "role" and "keyid" arguments in "Root.revoke_key()" are in
reverse order - "keyid" becomes first and "role" second.
4) The "role" and "key" arguments in "Targets.add_key()" are in reverse
order - "key" becomes first and "role" second.
5) "Targets.remove_key()" has been renamed to "Targets.revoke_key()".
6) The "role" and "keyid" arguments in "Targets.revoke_key()" are in
reverse order - "keyid" becomes first and "role" second.
7) In both methods "Targets.add_key()" and "Targets.revoke_key()" the
"role" argument becomes an optional with a default value of None.
Those changes are made in an effort to make those methods logical
for both cases when standard roles and succinct_roles are used.
The "Root" API change was done in order to preserve naming and argument
order consistency with "Targets" API.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
This commit contains 2 API changes in "Delegations" class from
tuf/api/metadata.py:
1. roles argment is made optional
2. unrecognized_fields argument becomes the 4-th rather than the 3-rd
as it used to be
In this commit, I add support for succinct_roles roles inside
Delegations class. This change is related to TAP 15 proposal.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
This change fixes the expired metadata tests to mock `datetime`
as previously they mocked `time` incorrectly, which did not affect
update methods, as they use `datetime.datetime.utcnow()` to
calculate now
Signed-off-by: Ivana Atanasova <iyovcheva@vmware.com>
I've not supported many renames but I'm suggesting this one:
FetcherHTTPError was created because we needed to signal 403/404
from the fetcher to updater. At that time the download error hierarchy
in general was not thought out.
Now we have a couple of different errors all derived from
DownloadError. I believe it does not make sense to point out "Fetcher"
in one of their names: DownloadHTTPError makes it clearer this is a
specific type of DownloadError.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Fetcher interface should only raise DownloadErrors,
regardless of the implementation.
* Make sure fetch() wraps non-DownloadError errors in a DownloadError
* Make the abstract function private _fetch()
* Try to be more consistent in doscstrings
This now makes the example client more sensible (when server does not
respond):
$ ./client_example.py download qwerty
...
Failed to download target qwerty: Failed to download url http://127.0.0.1:8000/metadata/2.root.json
(here the latter part of the error string comes from DownloadError
raised by FetcherInterface.fetch())
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
After we drop support for python3.6 we can relly that dictionaries
preserve the insertion order:
https://docs.python.org/3.7/whatsnew/3.7.html
This means we can replace the usage of OrderedDict with a standard
dictionaries.
Something we have to keep in mind is that even thought the insertion
order is preserved the equality comparison for normal dicts is
insensitive for normal dicts compared to OrderedDict
For example:
>>> OrderedDict([(1,1), (2,2)]) == OrderedDict([(2,2), (1,1)])
False
>>> dict([(1,1), (2,2)]) == dict([(2,2), (1,1)])
True
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Add tuf/api/exceptions.py for exceptions in the new code.
I copied the exceptions from tuf/exceptions.py with a few important
decisions:
1. I only added the exceptions that are used in the new code
2. I removed the general "Error" class as we can directly inherit
Exceptions
3. I tried grouping the exceptions by relevance
4. I removed the second argument "UnsignedMetadataError" as it's only
kept for backward compatibility and is not used
5. I tried following the new code style guidelines and linted the file
with our linters.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
This commit explicitly encodes role names. Mostly this encoding is already
happening in ``requests`` for what is not a URL.
The "/" in a role name will now be encoded.
Also, a slight change in the RepositorySimulator will align with the tests.
This commit partially covers issue #1634
Signed-off-by: Kairo de Araujo <kdearaujo@vmware.com>
Add a method to rotate roles keys into RepositorySimulator (only
top-level roles are supported for now). Rotation is used in four
places already and this refactoring makes the tests easier to
understand.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Sanitize the delegated rolenames in the repo simulator when dumping in
a directory the same way the ngclient does.
That's necessary because when testing fishy role names leads to an
error:
"PermissionError: [Errno 13] Permission denied: '/.json'"
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Reduce the number of function arguments and use
DelegatedRole instead.
When adding a list of delegations to the repository,
move the Targets creation inside the loop to create
a separate Targets object for each delegation.
Create a new Metadata obgect only for delegated roles
which do not exist yet in the repository.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Make _fetch_metadata and _fetch_taget public by renaming them to
fetch_metadata and fetch_target.
This will allow the removal of multiple pylint disables because of
"accessing private members".
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
This commit implements a feature in Repository Simulator to
track the fetch calls to the metadata and targets. This feature was
mentioned in PR #1666 that generated issue #1682.
This commit adds RepositorySimulator.fetch_tracker. It also changes
the tests/test_updater_consistent_snapshot.py to use the
fetch_tracker instead of using mock.
It implements a dataclass that stores the calls to fetch metadata
(_fetch_metadata) in fetch_tracker.metadata and targets
(_fetch_targets) in fetch_tracker.targets.
The fetch calls for metadata, and targets are stored as lists.
Signed-off-by: Kairo de Araujo <kdearaujo@vmware.com>
Previously when `compute_metafile_hashes_length` was set to `False`
`update_timestamp` did not set the hash and length values to `None`
as expected. This change fixes that, so they are not `None` when
`compute_metafile_hashes_length=True` and `None` when
`compute_metafile_hashes_length=False`
Signed-off-by: Ivana Atanasova <iyovcheva@vmware.com>
This applies the use of constants of top-level rolenames in the
tests instead of the previously hardcoded strings.
Fixes#1648
Signed-off-by: Ivana Atanasova <iyovcheva@iyovcheva-a02.vmware.com>
This commit includes manual fixes for a lot of mypy warnings.
When there were warnings that we are calling non-annotated function
in annotated context I decided to add annotations instead of ignoring
those warnings.
That's how I end up adding annotations in the whole tests/utils.py
module.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Address or disable pylint warnings raised on all test files inside
the "tests/" directory testing the code of the new implementation.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Except for 'root' role, RepositorySimulator does not
keep previous metadata versions, it always serves the latest
one. The metadata version check during fetch serves mostly
for informative purposes and removing it allows generating test
metadata with mismatching version.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Store signers with their keyids so they are easier to remove.
The signers structure now looks like:
{
"role1": {
"keyidA": SSlibSigner,
"keyidB": SSlibSigner,
}
}
Add convenience method for adding a signer.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Validate that root role names are 4 and that they are exactly
"root", "snapshot", "targets" and "timestamp" as described in
the spec:
https://theupdateframework.github.io/specification/latest/#root-role
Additionally, fix the valid_roots dataset, so each of the cases contains
the top metadata role names inside the roles dictionary.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Add support for adding delegated targets into RepositorySimulator.
Make the metadata URL parsing in RepositorySimulator more robust.
Add a test to make sure "../a" won't trick ngclient into creating the
metadata file outside the metadata cache.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Modify RepositorySimulator function delegates() to all_targets(), so
that all targets can be traversed and updated with one cycle when
calling update_snapshot() (which is the only use case for now for
delegates()).
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Add an option to calculate the hashes and length for timestamp/snapshot
meta.
This will help to cover more use cases with the repository simulator.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
The handling of consistent snapshot was not very clear: try to make
it more obvious what is supported and what is not.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
* Add very simple targets support into simulator
* Add documentation for the simulator
* Add an example targets test
This might need to be tweaked and/or extended as we add tests but the
implementation should give a good indication of how to extend it.
As an example, non-consistent targets are not yet supported, but
making fetch() check for the consistent_snapshot state and respond
accordingly should be easy.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
In Timestamp, the only valid "meta" value is the dictionary representing
meta information for the snapshot file. This makes the API unnecessarily
complicated and requires validation that only information about snapshot
is available inside "meta".
Together with the python-tuf maintainers, we decided that snapshot meta
information will not be represented by a "meta" dictionary but instead
by a MetaFile instance and with this it will diverge from the
specification.
Additionally, to prevent confusion, I will rename the "meta" attribute
to "snapshot_meta" as this attribute will be related only to meta
information about snapshot.
This decision is coherent with ADR9 and the rationale
behind it is to provide easier, safer, and direct access to the
snapshot meta information.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
