For targetpath: we don't want to support corner cases such as
file paths starting with separator.
Why this case should be threated specially than any other case where
you have multiple "/" for example "foo//bar/tar.gz"?
For pathpattern: it's recommended that the separator in the pathpattern
should be "/":
see https://theupdateframework.github.io/specification/latest/#targetpath
I believe it could lead to issues for a client implementation if it
supports arbitrary separators - every implementation needs to choose one
and stick with it.
Then, if we decide that "/" is our separator using lstrip on "os.sep" is
wrong, because the os separator from the server could be different that
the one used in the client.
Because of the above arguments, it makes sense to just remove
lstrip on os separators.
Additionally, document that the target_filepath and the DelegatedRole
paths are expected to be in their canonical forms and only "/" is
supported as target path separator.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
in the public API that we only support "/" as a
separator and don't handle corner cases such as leading separators
in either pathpattern or target_filepath.
By explicitly denoting the expected type of Metadata.signed
we help mypy understand our intentions and correctly figure
out types. This is entirely a typing feature and has no
runtime effect.
Modify the return type of Metadata.from_dict to match the
other factory methods (from_*).
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Needed in order to be compatible with the return type of
download_file (TemporaryFile is typed as IO[bytes]).
BinaryIO is a subclass of IO[bytes].
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
According to the recently updated version of the specification the shell
style wildcard matching is glob-like (see https://github.com/theupdateframework/specification/pull/174),
and therefore a path separator in a path should not be matched by a
wildcard in the PATHPATTERN.
That's not what happens with `fnmatch.fnmatch()` which doesn't
see "/" separator as a special symbol.
For example: fnmatch.fnmatch("targets/foo.tgz", "*.tgz") will return
True which is not what glob-like implementation will do.
We should make sure that target_path and the pathpattern contain the
same number of directories and because each part of the pathpattern
could include a glob pattern we should check that fnmatch.fnmatch() is
true on each target and pathpattern directory fragment separated by "/".
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
After the addition of "path" argument in the TargetFile class the
filename argument in Targets.update() became redundant.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Currently, TargetFile instances do not contain the path relative URL of
the file they represent. The API itself does not need it but it could be
useful for users of the API.
As an example, the current client returns a dict for
get_one_valid_targetinfo(): that dict contains a filepath field and
a targetinfo field (essentially TargetFile).
We would like to keep a similar API, but avoid hand-crafted dicts.
It would be much nicer to return a TargetFile that would contain the
full "metadata" of the targetfile.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
The securesystemslib key dictionary representation includes
the private key in keyval. TUF key doesn't handle it in any way,
but considering that we allow unrecognized symbols in the format,
we should exclude the private key otherwise this could lead to
misuse.
A call to securesystemslib.keys.format_keyval_to_metadata
with the default private=False would do exactly that.
Signed-off-by: Velichka Atanasova <avelichka@vmware.com>
Improve class DelegatedRole docstring in order to be rendered
correctly in the documentation built with sphinx and autodoc.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
When we use Metadata, it is helpful if the specific signed type (and all of
the signed types attribute types are correctly annotated. Currently this is
not possible.
Making Metadata Generic with constraint T, where
T = TypeVar("T", "Root", "Timestamp", "Snapshot", "Targets")
allows these annotations. Using Generic annotations is completely
optional so all existing code still works -- the changes in test code
are done to make IDE annotations more useful in the test code, not
because they are required.
Examples:
md = Metadata[Root].from_bytes(data)
md:Metadata[Root] = Metadata.from_bytes(data)
In both examples md.signed is now statically typed as "Root" allowing IDE
annotations and static type checking by mypy.
Note that it's not possible to validate that "data" actually contains a
root metadata at runtime in these examples as the annotations are _not_
visible at runtime at all: new constructors would have to be added for that.
from_file() is now a class method like from_bytes() to make sure both
have the same definition of "T" when from_file() calls from_bytes():
This makes mypy happy.
Partially fixes#1433
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
The specification does not state clearly what is the
behaviour when none of delegation's "paths" and
"path_hash_prefixes" is set. See #1497.
Until this issue is clarified, copy current
Updater which raises an error in such case.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Rename to DelegatedRole.is_delegated_path and
return a boolean flag instead of the role's name.
Minor comments and code style improvements.
Some code simplification.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Metadata.to_bytes() is missing from the API and that is now becoming
annoying when writing the tests.
I think it makes sense to add:
it'll complete the serializing counterparts to from_bytes()/from_file().
We can also reuse to_bytes() in to_file() and that way ensure we don't
import the JSONSerializer locally twice.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
We made Role.keyids a set because the keyids are supposed
to be unique and this still makes sense.
However, the data should also preserve order
(when deserialized and serialized) and currently, it does not.
This is fairly serious since writing signed data potentially modifies
the data (making the signature invalid).
The simplest solution (as proposed by Teodora) is to sort the
set during serialization and that would ensure the order of the items.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
* Rename arguments so connection between the role name and the
metadata is stronger.
* Also add a comment on the list comprehension + next() trick.
* Add return value annotation
* Raise early if delegations is None to make the flow more obvious
(and modify test case so we have coverage for the new case)
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
In pr https://github.com/theupdateframework/tuf/pull/1449 we introduced
Key validation and there decided to raise "ValueError" if one of
keyid, scheme, keyval or keytype is not a string.
That seems like a mistake given that in the python docs it's written:
"Passing arguments of the wrong type
(e.g. passing a list when an int is expected) should result
in a TypeError, but passing arguments with the wrong value
(e.g. a number outside expected boundaries) should result
in a ValueError."
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Currently, we require that the keyval attribute in the Key class
is a dictionary and has "public" as a key, otherwise, we throw
KeyError or ValueError.
This requirement is too strict given that in the spec for KEYVAL it's
only said that KEYVAL is:
"A dictionary containing the public portion of the key."
See: https://theupdateframework.github.io/specification/latest/index.html#keyval
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
There was an attempt at ensuring key content uniqueness in
verify_delegate() by making sure the values corresponding to "public"
keys in Key.keyval dictionaries are unique. This had two issues:
* it wasn't a security measure: it's not difficult to produce two
different "public" values of the same key content
* Spec does not actually guarantee the existence of "public" key in
the keyval dictionary (the three keys included in the spec just all
happen to have it)
Luckily the spec does require KEYIDs to be unique so we do not need to
do all this: Just count keyids of keys with verified signatures. Keep
building a Set of keyids as a belt-and-suspenders strategy: Role keyids
are currently guaranteed to be unique but we'd notice here if they
weren't.
Add a logger call for failed verifys: this might useful to figure out
which keys exactly are the issue when a delegate can not be verified.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Make sure verify_delegate() succeeds when threshold is reached even if
some signatures fail to verify.
Make sure higher threshold (2/2) works.
Change error type for "Call is valid only on delegator metadata" error.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
The delegating Metadata (root or targets) verifies that the delegated
metadata is signed by required threshold of keys for the delegated
role.
Calling the function on non-delegator-metadata or giving a rolename
that is not actually delegated by the delegator is considered a
programming error and ValueError is raised.
If the threshold is not reached, UnsignedMetadataError is raised.
Tweak type annotation of Delegations.keys to match the one for
Root.keys (so they can be assigned to same local variable).
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
We've been returning Signature objects since 49aa0fc167.
Also add a test case that does something with the returned signature.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Explain the ways a delegation can happen: Do not try to cover the
complete process (specification should do that) but offer enough
details that the complexity is not completely hidden from the viewer.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Try to keep dostrings and comments to the point, avoid mentioning
details if they are not necessary or are likely to become outdated
and try to minimize number of comment lines.
Co-authored-by: Joshua Lock <jlock@vmware.com>
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Securesystemslib digest() and digest_fileobject()
calls raise sslib specific exceptions that need to be
handled and re-raised as TUF exceptions.
Updated tests in test_api.py accordingly.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
- valid length: greater than zero
- valid hashes: a non-empty dictionary of type Dict[str, str]
Checking the validity of hash algorithms is not part
of the metadata input validation and is done by
securesystemslib during hash verification.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Aim to only raise UnsignedMetadataError from verify_signature().
Some of the situations could be things like UnsupportedAlgorithmError
-- where the underlying reason may be a missing dependency -- but it
seems impossible for a client to know whether it's that or whether it
is broken or malicious server side.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Clarify that we don't semantically validate "Key" instances during
initialization and that this is a responsibility of securesystemslib.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Probably there could be future API calls that modify "threshold"
to a new value, but the problem is we don't have a clear idea
if they would exist and what exactly they will do.
That's why it makes sense to validate against the potential problems
we can imagine - in this case, is passing a threshold below 1.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
In our discussion with Jussi we come to the conclusion that we want
to verify that all Key attributes contain values in the expected types,
but at the same time, we don't want to focus on validating the semantics
behind them.
The reason is that having a Key instance with invalid attributes is
possible and supported by the spec.
That's why we have a "threshold" for the roles meaning we can have up to
a certain number of invalid Keys until we satisfy
the required threshold.
Also, for deeper semantic validation it's better to be done in
securesystemslib which does the actual work with keys.
For context see: https://github.com/theupdateframework/tuf/issues/1438
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
