This may have been required by a linter at some point, but isn't
anymore: Not annotating makes the documentation look better.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Situation before
* constructor args are not documented
* object attributes are documented
* sphinx cannot show object attribute type annotations
* attribute docs take a lot of vertical space
Now:
* constructor args are documented
* sphinx can show annotated types of constructor args
* class docstring now explains the attributes are the same as
constructor args (and attributes are not explicitly documented)
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
This is an API change to the exceptions thrown in Root.add_key()
and Root.remove_key().
The reason for that change is that in my opinion the correct exceptions
in these cases should be "ValueError" instead of "KeyError" as
the problems are in the given values - role doesn't exist or
key is not used by a particular role.
Additionally, document the thrown exceptions in "Root.add_key" and
add a test which invokes that exception.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Root class has the functionality to add and remove keys for delegated
metadata (add_key()/remove_key()) but the other delegator Targets does
not.
It should provide the same/similar functionality.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
The spec does not say anything about role name uniqueness in a
delegations object, but I believe we cannot safely allow multiple roles
with the same role name in the roles array of a delegations object.
If we did then the roles could have different keyids, and then we would
end up in a situation where metadata may be both a valid delegation
and an invalid delegation at the same time, depending on how the role
gets chosen and that does not seem like the intention of the design.
There is an issue open in the specification with number 167 about
that issue.
Regardless of the Metadata API, I think we should enforce role name
uniqueness.
I chose to change the data structure containing roles to
OrderedDict, where keys are role names and values are DelegatedRole
instances.
This made sense to me as role names are the unique identifier of a role
and their order is important to the way they are traversed afterward.
Note: we can't use OrderedDict as type annotation until we drop support
for Python 3.6:
https://docs.python.org/3/library/typing.html#typing.OrderedDict
That's why I used quotes around "OrderedDict" annotation, because I
can't import it.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
In Timestamp, the only valid "meta" value is the dictionary representing
meta information for the snapshot file. This makes the API unnecessarily
complicated and requires validation that only information about snapshot
is available inside "meta".
Together with the python-tuf maintainers, we decided that snapshot meta
information will not be represented by a "meta" dictionary but instead
by a MetaFile instance and with this it will diverge from the
specification.
Additionally, to prevent confusion, I will rename the "meta" attribute
to "snapshot_meta" as this attribute will be related only to meta
information about snapshot.
This decision is coherent with ADR9 and the rationale
behind it is to provide easier, safer, and direct access to the
snapshot meta information.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Clarify the purpose of metadata API and that it's a low-level API
and as such it doesn't use concepts like "repository" or
"trusted collection of metadata" and don't implement the repository
logic or client updater workflow.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
This is a repository tooling use case but also helpful when testing.
It could be useful when we need to update the targets object.
Signed-off-by: Velichka Atanasova <avelichka@vmware.com>
The pylint warning W0703:broad-except was raised only
when six was used and python 2 was still supported.
The warning is no longer raised, the exceptions are
handled/raised correctly and the disabling can be removed.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
For targetpath: we don't want to support corner cases such as
file paths starting with separator.
Why this case should be threated specially than any other case where
you have multiple "/" for example "foo//bar/tar.gz"?
For pathpattern: it's recommended that the separator in the pathpattern
should be "/":
see https://theupdateframework.github.io/specification/latest/#targetpath
I believe it could lead to issues for a client implementation if it
supports arbitrary separators - every implementation needs to choose one
and stick with it.
Then, if we decide that "/" is our separator using lstrip on "os.sep" is
wrong, because the os separator from the server could be different that
the one used in the client.
Because of the above arguments, it makes sense to just remove
lstrip on os separators.
Additionally, document that the target_filepath and the DelegatedRole
paths are expected to be in their canonical forms and only "/" is
supported as target path separator.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
in the public API that we only support "/" as a
separator and don't handle corner cases such as leading separators
in either pathpattern or target_filepath.
By explicitly denoting the expected type of Metadata.signed
we help mypy understand our intentions and correctly figure
out types. This is entirely a typing feature and has no
runtime effect.
Modify the return type of Metadata.from_dict to match the
other factory methods (from_*).
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Needed in order to be compatible with the return type of
download_file (TemporaryFile is typed as IO[bytes]).
BinaryIO is a subclass of IO[bytes].
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
According to the recently updated version of the specification the shell
style wildcard matching is glob-like (see https://github.com/theupdateframework/specification/pull/174),
and therefore a path separator in a path should not be matched by a
wildcard in the PATHPATTERN.
That's not what happens with `fnmatch.fnmatch()` which doesn't
see "/" separator as a special symbol.
For example: fnmatch.fnmatch("targets/foo.tgz", "*.tgz") will return
True which is not what glob-like implementation will do.
We should make sure that target_path and the pathpattern contain the
same number of directories and because each part of the pathpattern
could include a glob pattern we should check that fnmatch.fnmatch() is
true on each target and pathpattern directory fragment separated by "/".
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
After the addition of "path" argument in the TargetFile class the
filename argument in Targets.update() became redundant.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Currently, TargetFile instances do not contain the path relative URL of
the file they represent. The API itself does not need it but it could be
useful for users of the API.
As an example, the current client returns a dict for
get_one_valid_targetinfo(): that dict contains a filepath field and
a targetinfo field (essentially TargetFile).
We would like to keep a similar API, but avoid hand-crafted dicts.
It would be much nicer to return a TargetFile that would contain the
full "metadata" of the targetfile.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
The securesystemslib key dictionary representation includes
the private key in keyval. TUF key doesn't handle it in any way,
but considering that we allow unrecognized symbols in the format,
we should exclude the private key otherwise this could lead to
misuse.
A call to securesystemslib.keys.format_keyval_to_metadata
with the default private=False would do exactly that.
Signed-off-by: Velichka Atanasova <avelichka@vmware.com>
Improve class DelegatedRole docstring in order to be rendered
correctly in the documentation built with sphinx and autodoc.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
When we use Metadata, it is helpful if the specific signed type (and all of
the signed types attribute types are correctly annotated. Currently this is
not possible.
Making Metadata Generic with constraint T, where
T = TypeVar("T", "Root", "Timestamp", "Snapshot", "Targets")
allows these annotations. Using Generic annotations is completely
optional so all existing code still works -- the changes in test code
are done to make IDE annotations more useful in the test code, not
because they are required.
Examples:
md = Metadata[Root].from_bytes(data)
md:Metadata[Root] = Metadata.from_bytes(data)
In both examples md.signed is now statically typed as "Root" allowing IDE
annotations and static type checking by mypy.
Note that it's not possible to validate that "data" actually contains a
root metadata at runtime in these examples as the annotations are _not_
visible at runtime at all: new constructors would have to be added for that.
from_file() is now a class method like from_bytes() to make sure both
have the same definition of "T" when from_file() calls from_bytes():
This makes mypy happy.
Partially fixes#1433
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
The specification does not state clearly what is the
behaviour when none of delegation's "paths" and
"path_hash_prefixes" is set. See #1497.
Until this issue is clarified, copy current
Updater which raises an error in such case.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Rename to DelegatedRole.is_delegated_path and
return a boolean flag instead of the role's name.
Minor comments and code style improvements.
Some code simplification.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Metadata.to_bytes() is missing from the API and that is now becoming
annoying when writing the tests.
I think it makes sense to add:
it'll complete the serializing counterparts to from_bytes()/from_file().
We can also reuse to_bytes() in to_file() and that way ensure we don't
import the JSONSerializer locally twice.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
We made Role.keyids a set because the keyids are supposed
to be unique and this still makes sense.
However, the data should also preserve order
(when deserialized and serialized) and currently, it does not.
This is fairly serious since writing signed data potentially modifies
the data (making the signature invalid).
The simplest solution (as proposed by Teodora) is to sort the
set during serialization and that would ensure the order of the items.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
* Rename arguments so connection between the role name and the
metadata is stronger.
* Also add a comment on the list comprehension + next() trick.
* Add return value annotation
* Raise early if delegations is None to make the flow more obvious
(and modify test case so we have coverage for the new case)
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
In pr https://github.com/theupdateframework/tuf/pull/1449 we introduced
Key validation and there decided to raise "ValueError" if one of
keyid, scheme, keyval or keytype is not a string.
That seems like a mistake given that in the python docs it's written:
"Passing arguments of the wrong type
(e.g. passing a list when an int is expected) should result
in a TypeError, but passing arguments with the wrong value
(e.g. a number outside expected boundaries) should result
in a ValueError."
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Currently, we require that the keyval attribute in the Key class
is a dictionary and has "public" as a key, otherwise, we throw
KeyError or ValueError.
This requirement is too strict given that in the spec for KEYVAL it's
only said that KEYVAL is:
"A dictionary containing the public portion of the key."
See: https://theupdateframework.github.io/specification/latest/index.html#keyval
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
There was an attempt at ensuring key content uniqueness in
verify_delegate() by making sure the values corresponding to "public"
keys in Key.keyval dictionaries are unique. This had two issues:
* it wasn't a security measure: it's not difficult to produce two
different "public" values of the same key content
* Spec does not actually guarantee the existence of "public" key in
the keyval dictionary (the three keys included in the spec just all
happen to have it)
Luckily the spec does require KEYIDs to be unique so we do not need to
do all this: Just count keyids of keys with verified signatures. Keep
building a Set of keyids as a belt-and-suspenders strategy: Role keyids
are currently guaranteed to be unique but we'd notice here if they
weren't.
Add a logger call for failed verifys: this might useful to figure out
which keys exactly are the issue when a delegate can not be verified.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Make sure verify_delegate() succeeds when threshold is reached even if
some signatures fail to verify.
Make sure higher threshold (2/2) works.
Change error type for "Call is valid only on delegator metadata" error.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
The delegating Metadata (root or targets) verifies that the delegated
metadata is signed by required threshold of keys for the delegated
role.
Calling the function on non-delegator-metadata or giving a rolename
that is not actually delegated by the delegator is considered a
programming error and ValueError is raised.
If the threshold is not reached, UnsignedMetadataError is raised.
Tweak type annotation of Delegations.keys to match the one for
Root.keys (so they can be assigned to same local variable).
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
We've been returning Signature objects since 49aa0fc167.
Also add a test case that does something with the returned signature.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Explain the ways a delegation can happen: Do not try to cover the
complete process (specification should do that) but offer enough
details that the complexity is not completely hidden from the viewer.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Try to keep dostrings and comments to the point, avoid mentioning
details if they are not necessary or are likely to become outdated
and try to minimize number of comment lines.
Co-authored-by: Joshua Lock <jlock@vmware.com>
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
