Here is the list of all breaking API changes:
1) The "role" and "key" arguments in "Root.add_key()" are in reverse
order - "key" becomes first and "role" second.
2) "Root.remove_key()" has been renamed to "Root.revoke_key()".
3) The "role" and "keyid" arguments in "Root.revoke_key()" are in
reverse order - "keyid" becomes first and "role" second.
4) The "role" and "key" arguments in "Targets.add_key()" are in reverse
order - "key" becomes first and "role" second.
5) "Targets.remove_key()" has been renamed to "Targets.revoke_key()".
6) The "role" and "keyid" arguments in "Targets.revoke_key()" are in
reverse order - "keyid" becomes first and "role" second.
7) In both methods "Targets.add_key()" and "Targets.revoke_key()" the
"role" argument becomes an optional with a default value of None.
Those changes are made in an effort to make those methods logical
for both cases when standard roles and succinct_roles are used.
The "Root" API change was done in order to preserve naming and argument
order consistency with "Targets" API.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
This commit contains 2 API changes in "Delegations" class from
tuf/api/metadata.py:
1. roles argment is made optional
2. unrecognized_fields argument becomes the 4-th rather than the 3-rd
as it used to be
In this commit, I add support for succinct_roles roles inside
Delegations class. This change is related to TAP 15 proposal.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Add two helper methods in SuccinctRoles.
Those methods proved useful in the testing code, but I believe they have
a potential value for production code as well.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Move the duplicating signatures tests from test_metadata_base function
in test_api.py into test_metadata_serialization.py.
This is a more logical place to store this test case as
test_metadata_base is actually focused on testing
Metadata.signed.is_expired.
That also is the reason why I renamed test_metadata_base to
test_metadata_signed_is_expired.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
If a securesystemslib.FormatError is raised inside
Key.from_securesystemslib_key() then reraise ValueError.
This is done so that our users don't have to import securesystemslib
in order to handle the error and because the securesystemslib error
itself is securesystemslib implementation-specific.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
UnsupportedAlgorithmError is a detailed securesystemslib exception
and there is no need for TUF to redefine it.
Moreover which hash "algorithms" are allowed is work for
securesystemslib not for TUF.
It's only used once inside "Targetfile.from_data()" and there it's used
to denote that there is a problem with the given argument.
That's why this error can be just replaced with "ValueError".
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Add tuf/api/exceptions.py for exceptions in the new code.
I copied the exceptions from tuf/exceptions.py with a few important
decisions:
1. I only added the exceptions that are used in the new code
2. I removed the general "Error" class as we can directly inherit
Exceptions
3. I tried grouping the exceptions by relevance
4. I removed the second argument "UnsignedMetadataError" as it's only
kept for backward compatibility and is not used
5. I tried following the new code style guidelines and linted the file
with our linters.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Remove `bump_version()` method, which is just an alias for "+= 1"
on the version attribute. For a slim low-level API it seems okay to
just directly access/modify the attribute.
The extra level of abstraction of "bumping a version" is more
appropriate for a repository library (see #1136).
This patch also removes a related unit test and updates another one
to directly do `(...).version +=`.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Remove `bump_expiration()` method, which is unlikely to be used as
is, i.e. bump to "current expiration date plus delta". A more
realistic use case is to bump to "now plus delta" (see #1727 for
details).
Moreover, bump_expiration can either way easily be replaced by a
one-liner expression using the 'datetime' module. A corresponding
code snippet is added to the `expires` property's docstring. Note:
`expires` became a property with a millisec-removing setter (for
spec conformance) in #1712, which further reduces the need for a
convenience bump_expiration method.
This patch also removes a related unit test and updates another
one.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Remove ambiguous, unspecific, opinionated and trivial 'update'
methods, which can be replaced by feasible one-liners that assign
values directly to the object attribute to be *updated*. (see #1627
for details).
Reasons to have these methods would be increased usability in terms of
- reduced work
- immediate feedback on invalid assignments
However, given above described issues, the reasons against the
methods as they are now seem to outweigh the reasons for them.
Furthermore, it seems easier to re-add similar methods, which
addressed these issues, after the upcoming 1.0.0 release than to
remove or modify them.
This patch also removes the corresponding tests as they become
irrelevant (there is no need to test object assignment). In the
case of the timestamp test, the removal also includes redundant
test logic, which is already tested in `test_metadata_base`.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
I added "dateutil" as a possible argument type for
`Metadata.bump_expiration()` as we are already testing for this and
implying it should be supported.
The problem is that "dateutil" is not added as a nontest requirement
and after a discussion, we decided we don't want to add it as well.
That's why we decided to remove "dateutil" mentions from the code
and not confuse our users we support it.
We will create a separate issue discussing the validity of
`Metadata.bump_expiration()`.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
The changes are automatic linting fixes from black.
The target files are only those who test the new code.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
This applies the use of constants of top-level rolenames in the
tests instead of the previously hardcoded strings.
Fixes#1648
Signed-off-by: Ivana Atanasova <iyovcheva@iyovcheva-a02.vmware.com>
This commit includes manual fixes for a lot of mypy warnings.
When there were warnings that we are calling non-annotated function
in annotated context I decided to add annotations instead of ignoring
those warnings.
That's how I end up adding annotations in the whole tests/utils.py
module.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
After the recent changes there are a couple of new pylint warnings that
appeared.
They are caused by the new test file that was added
test_updater_top_level_update.py and the limit of public functions was
reached in the TestMetadata class in test_api.py
The warnings should be addressed before enabling all of the linters
on the tests files.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Address or disable pylint warnings raised on all test files inside
the "tests/" directory testing the code of the new implementation.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
test_sign_verify() is testing too many cases and after the recent
pylint warning about the usage of too many local variables it became
clear it's time to split this test function.
I decided to split it logically as half of the function was about
failures connected with verify.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
This is an API change to the exceptions thrown in Root.add_key()
and Root.remove_key().
The reason for that change is that in my opinion the correct exceptions
in these cases should be "ValueError" instead of "KeyError" as
the problems are in the given values - role doesn't exist or
key is not used by a particular role.
Additionally, document the thrown exceptions in "Root.add_key" and
add a test which invokes that exception.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Root class has the functionality to add and remove keys for delegated
metadata (add_key()/remove_key()) but the other delegator Targets does
not.
It should provide the same/similar functionality.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Move the Delegation class serialization tests from "test_api.py"
to test_metadata_serialization.py module focused on serialization
testing.
Additionally, a test for empty keys and roles will be added in my
upcomming pr #1511.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
In Timestamp, the only valid "meta" value is the dictionary representing
meta information for the snapshot file. This makes the API unnecessarily
complicated and requires validation that only information about snapshot
is available inside "meta".
Together with the python-tuf maintainers, we decided that snapshot meta
information will not be represented by a "meta" dictionary but instead
by a MetaFile instance and with this it will diverge from the
specification.
Additionally, to prevent confusion, I will rename the "meta" attribute
to "snapshot_meta" as this attribute will be related only to meta
information about snapshot.
This decision is coherent with ADR9 and the rationale
behind it is to provide easier, safer, and direct access to the
snapshot meta information.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
This is a repository tooling use case but also helpful when testing.
It could be useful when we need to update the targets object.
Signed-off-by: Velichka Atanasova <avelichka@vmware.com>
Test basic cases of DelegatedRole.is_delegated_path()
This is not trying to be an extensive test of possible methods of delegation:
this gives us basic coverage for all code paths.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Make sure key is not removed from root.signed.keys on remove_key()
if another role is still using the key.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
According to the recently updated version of the specification the shell
style wildcard matching is glob-like (see https://github.com/theupdateframework/specification/pull/174),
and therefore a path separator in a path should not be matched by a
wildcard in the PATHPATTERN.
That's not what happens with `fnmatch.fnmatch()` which doesn't
see "/" separator as a special symbol.
For example: fnmatch.fnmatch("targets/foo.tgz", "*.tgz") will return
True which is not what glob-like implementation will do.
We should make sure that target_path and the pathpattern contain the
same number of directories and because each part of the pathpattern
could include a glob pattern we should check that fnmatch.fnmatch() is
true on each target and pathpattern directory fragment separated by "/".
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
After the addition of "path" argument in the TargetFile class the
filename argument in Targets.update() became redundant.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Currently, TargetFile instances do not contain the path relative URL of
the file they represent. The API itself does not need it but it could be
useful for users of the API.
As an example, the current client returns a dict for
get_one_valid_targetinfo(): that dict contains a filepath field and
a targetinfo field (essentially TargetFile).
We would like to keep a similar API, but avoid hand-crafted dicts.
It would be much nicer to return a TargetFile that would contain the
full "metadata" of the targetfile.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
The securesystemslib key dictionary representation includes
the private key in keyval. TUF key doesn't handle it in any way,
but considering that we allow unrecognized symbols in the format,
we should exclude the private key otherwise this could lead to
misuse.
A call to securesystemslib.keys.format_keyval_to_metadata
with the default private=False would do exactly that.
Signed-off-by: Velichka Atanasova <avelichka@vmware.com>
I was looking at how can we simplify or split test_api.py when I noticed
that the test cases covered by those two test functions are already
covered in the test_metadata_serialization.py module in the
"invalid_keys" and "invalid_roles" datasets.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
When we use Metadata, it is helpful if the specific signed type (and all of
the signed types attribute types are correctly annotated. Currently this is
not possible.
Making Metadata Generic with constraint T, where
T = TypeVar("T", "Root", "Timestamp", "Snapshot", "Targets")
allows these annotations. Using Generic annotations is completely
optional so all existing code still works -- the changes in test code
are done to make IDE annotations more useful in the test code, not
because they are required.
Examples:
md = Metadata[Root].from_bytes(data)
md:Metadata[Root] = Metadata.from_bytes(data)
In both examples md.signed is now statically typed as "Root" allowing IDE
annotations and static type checking by mypy.
Note that it's not possible to validate that "data" actually contains a
root metadata at runtime in these examples as the annotations are _not_
visible at runtime at all: new constructors would have to be added for that.
from_file() is now a class method like from_bytes() to make sure both
have the same definition of "T" when from_file() calls from_bytes():
This makes mypy happy.
Partially fixes#1433
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Metadata.to_bytes() is missing from the API and that is now becoming
annoying when writing the tests.
I think it makes sense to add:
it'll complete the serializing counterparts to from_bytes()/from_file().
We can also reuse to_bytes() in to_file() and that way ensure we don't
import the JSONSerializer locally twice.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
* Rename arguments so connection between the role name and the
metadata is stronger.
* Also add a comment on the list comprehension + next() trick.
* Add return value annotation
* Raise early if delegations is None to make the flow more obvious
(and modify test case so we have coverage for the new case)
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Make sure verify_delegate() succeeds when threshold is reached even if
some signatures fail to verify.
Make sure higher threshold (2/2) works.
Change error type for "Call is valid only on delegator metadata" error.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
The delegating Metadata (root or targets) verifies that the delegated
metadata is signed by required threshold of keys for the delegated
role.
Calling the function on non-delegator-metadata or giving a rolename
that is not actually delegated by the delegator is considered a
programming error and ValueError is raised.
If the threshold is not reached, UnsignedMetadataError is raised.
Tweak type annotation of Delegations.keys to match the one for
Root.keys (so they can be assigned to same local variable).
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
We have merged ADR 8 allowing for unrecognized fields and we have
added tests for that which are too specific and not scalable.
Now, I use table testing which we have used initially in https://github.com/theupdateframework/tuf/pull/1416
to test unrecognized fields support in a cleaner and much more readable
way.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
We've been returning Signature objects since 49aa0fc167.
Also add a test case that does something with the returned signature.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
