Generalize the decorator used in test_metadata_serialization.py and
move it inside tests/utils.py, so it can be reused in other similar
situations.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Move the Delegation class serialization tests from "test_api.py"
to test_metadata_serialization.py module focused on serialization
testing.
Additionally, a test for empty keys and roles will be added in my
upcomming pr #1511.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
The spec does not say anything about role name uniqueness in a
delegations object, but I believe we cannot safely allow multiple roles
with the same role name in the roles array of a delegations object.
If we did then the roles could have different keyids, and then we would
end up in a situation where metadata may be both a valid delegation
and an invalid delegation at the same time, depending on how the role
gets chosen and that does not seem like the intention of the design.
There is an issue open in the specification with number 167 about
that issue.
Regardless of the Metadata API, I think we should enforce role name
uniqueness.
I chose to change the data structure containing roles to
OrderedDict, where keys are role names and values are DelegatedRole
instances.
This made sense to me as role names are the unique identifier of a role
and their order is important to the way they are traversed afterward.
Note: we can't use OrderedDict as type annotation until we drop support
for Python 3.6:
https://docs.python.org/3/library/typing.html#typing.OrderedDict
That's why I used quotes around "OrderedDict" annotation, because I
can't import it.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Currently, TargetFile instances do not contain the path relative URL of
the file they represent. The API itself does not need it but it could be
useful for users of the API.
As an example, the current client returns a dict for
get_one_valid_targetinfo(): that dict contains a filepath field and
a targetinfo field (essentially TargetFile).
We would like to keep a similar API, but avoid hand-crafted dicts.
It would be much nicer to return a TargetFile that would contain the
full "metadata" of the targetfile.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Those tests are needed to cover use cases when syntatcticly as
standalone objects the metadata classes and their helper classes defined
in tuf/api/metadata.py are valid even if they cannot be verified.
An example where an object is valid, but cannot be verified is
if we have a Role instance with an empty list of "keyids".
This instance is valid and can be created, but cannot be verified
because there is a requirement that the threshold should be above
1, meaning that there should be at least 1 element inside the "keyids"
list to complete successful threshold verification.
The situation is the same for the rest of the tests I am adding to this
commit:
- Root object without keys
- Root object without roles
- DelegationRole object with empty "keyids"
- DelegationRole object with an empty list of "paths"
- DelegationRole object with an empty list of "path_hash_prefixes"
all of these objects can be instantiated, but cannot complete
successfully threshold verification.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Test metadata (de)serialization with input data containing containers
with zero or more elements.
Here is the status for the different use cases:
Root keys:
- many keys: added
Root roles:
- many roles: added
Root role keyids:
- many keids: already added in https://github.com/theupdateframework/tuf/pull/1481
MetaFile hashes:
- many hashes: already tested
- zero hashes: added. Testing as invalid test case.
Timestamp meta:
- zero elements: already tested
- many elements: added
Snapshot meta:
- zero items: added
- many items: added
Delegation keys:
- many keys: added
Delegation role keyids:
- many keyids: added
Delegation role paths:
- many paths: already tested
Delegation role path_hash_prefixes:
- many path_hash_path_prefixes: already tested
Delegation roles:
- zero roles: added
- multiple roles: added
Targets targets:
- zero items: already tested
- multiple items: added
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
The specification does not state clearly what is the
behaviour when none of delegation's "paths" and
"path_hash_prefixes" is set. See #1497.
Until this issue is clarified, copy current
Updater which raises an error in such case.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
We made Role.keyids a set because the keyids are supposed
to be unique and this still makes sense.
However, the data should also preserve order
(when deserialized and serialized) and currently, it does not.
This is fairly serious since writing signed data potentially modifies
the data (making the signature invalid).
The simplest solution (as proposed by Teodora) is to sort the
set during serialization and that would ensure the order of the items.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
A while ago we decided that it's best to research each of the individuals
attributes one by one and identify what level of validation it needs
compared to how we use it:
https://github.com/theupdateframework/tuf/pull/1366#issuecomment-829288790.
This work is ongoing and there are a couple of commits already merged
for this:
- 6c5d970799
- f20664d2fc
- 41afb1e134
We want to be able to test the attributes validation against known bad
values.
The way we want to do that is with table testing we have added
using decorators for our metadata classes defined in New API:
https://github.com/theupdateframework/tuf/pull/1416.
This gives us an easy way to add new cases for each of the attributes and
not depend on external files.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
We have merged ADR 8 allowing for unrecognized fields and we have
added tests for that which are too specific and not scalable.
Now, I use table testing which we have used initially in https://github.com/theupdateframework/tuf/pull/1416
to test unrecognized fields support in a cleaner and much more readable
way.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
The idea of this commit is to separate (de)serialization testing outside
test_api.py and make sure we are testing from_dict/to_dict for all
possible valid data for all classes.
Jussi in his comment here:
https://github.com/theupdateframework/tuf/issues/1391#issuecomment-849390669
proposed using decorators when creating comprehensive testing
for metadata serialization.
The main problems he pointed out is that:
1) there is a lot of code needed to generate the data for each case
2) the test implementation scales badly when you want to add new
cases for your tests, then you would have to add code as well
3) the dictionary format is not visible - we are loading external files
and assuming they are not changed and valid
In this change, I am using a decorator with an argument that complicates
the implementation of the decorator and requires three nested functions,
but the advantages are that we are resolving the above three problems:
1) we don't need new code when adding a new test case
2) a small amount of hardcoded data is required for each new test
3) the dictionaries are all in the test module without the need of
creating new directories and copying data.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
