Only "Updater" and "FetcherInterface" are considered
public classes of the client. Exposing them in __init__.py
makes usage and access simpler.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Separate public/private API. Keep modules
containing the piblic classes in the main client
directory and move the rest to _internal.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
The current client and the next-gen client should
coexist in the same repository during the ongoing
development of the latter.
Looking for a name which is client-related,
short, meeting PEP8 package names requirements.
Currently "ngclient" seems to fit in until a
better proposal comes.
Rename updater_rework.py to updater.py
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Mostly remove comments that provide little value after all the changes.
Also remove a unused variable from a test.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
We have tests which make sure we can use `Timestamp.update()` and
`Snapshot.update()` with MetaFile instance storing only version
(because length and hashes are optional).
Those tests were created to make sure that we are actually supporting
optional hashes and length when we call `update` for those classes, but
after we changed the `update()` signature to accept `MetaFile` instance
the tests are obsolete.
The reason is that length and hashes can be optional because of the
MetaFile implementation, no the update function itself and we have
other tests validating creating a MetaFie instance without hashes and
length.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Currently, when we call Targets/Snapshot/Timestamp.update() we are
passing all of the necessary values to create MetaFile/Targets File
respectively.
This is not needed, given that one of the reasons we have created
MetaFile and TargetFile is to make the API easier to use.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
In the top-level metadata classes, there are complex attributes such as
"meta" in Targets and Snapshot, "key" and "roles" in Root etc.
We want to represent those complex attributes with a class to allow
easier verification and support for metadata with unrecognized fields.
For more context read ADR 0004 and ADR 0008 in the docs/adr folder.
As written in the spec "targets" in "targets.json" has defined the
"custom" field serving the same purpose as "unrecognized_fields" in the
implementation.
That's why to conform against the spec and support "custom" and allow
"unrecognized_fields" everywhere where it's not sensitive we can define
custom as property which actually access data stored in
unrecognized_fields.
For context read ADR 8 in tuf/docs/adr.
Additionally, after adding the TargetFile class, when we create a
Targets an object we are now calling from dict twice - one for the main
Targets class and one for each of the complex attributes
TargetFile.from_dict() and Delegations.from_dict().
Given that the "from_dict" methods have the side effect of destroying
the given dictionary, we would need to start using deepcopy()
for our tests.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
In the top-level metadata classes, there are complex attributes such as
"meta" in Targets and Snapshot, "key" and "roles" in Root etc.
We want to represent those complex attributes with a class to allow
easier verification and support for metadata with unrecognized fields.
For more context read ADR 0004 and ADR 0008 in the docs/adr folder.
Additionally, after adding the MetaFile class, when we create an object
we are now calling from dict twice - one for the main class (Timestamp,
Snapshot) and one for the pacticular complex attribute -
MetaFile.from_dict(). Given that the "from_dict" methods have the
side effect of destroying the given dictionary, we would need to
start using deepcopy() for our tests.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Add test for a refresh with just a local root.json.
Remove unused code. Add docstrings for raised exceptions, add TODOs for
the missing exception handling.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Use the MetadataBundle to verify metadata validity.
* Updater now handles reading metadata files (from filesystem as
well as network
* Updater feeds bytes to MetadataBundle for verification
* Updater persists data on disk after it had been verified
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
NOTE: making consistent_snapshot optional requires using a default value
for the argument in __init__ in Root and thus consistent_snapshot should
be rearranged in the end.
Read more: https://github.com/theupdateframework/tuf/pull/1394#issuecomment-842134961
From chapter 7 in the spec (version 1.0.17)
"Finally, the root metadata should write the Boolean
"consistent_snapshot" attribute at the root level of its keys of
attributes.
If consistent snapshots are not written by the repository,
then the attribute may either be left unspecified or be set to the
False value. Otherwise, it must be set to the True value."
We want to make sure we support repositories
without consistent_snapshot set.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Remove file IO from MetadataBundle:
* This make the bundle API very clear and easy to understand
* This means caller must now read from and persist data to disk
but initial prototypes suggest this won't make Updater too
complex
This change is something we can still back out from if it turns out to
be the wrong decision: the file-persisting MetadataBundle has been tested
and works fine.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
The bundle should now raise
* derivatives of RepositoryError on failures that are likely a result
of server error or a malicious server
* RuntimeErrors if calls were made when they are not possible
* ValueErrors if arguments are invalid
last two are callers errors and avoidable.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Don't use the serialized format as that won't match any hashes in
"meta".
Add basic tests for updating metadata.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Use either "if X is not None:" or a try-except instead of a "if X:".
I believe Targets.from_dict() was not really broken with previous code
but it looks suspicious and did fail the added test with a strange
exception: I expect the from_dict() methods to mainly fail with
KeyErrors, ValueErrors or AttributeErrors if file format structure
is incorrect.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
A DelegatedRole with paths=[] fails to serialize correctly (paths is not
included in the output json).
Fix the issue, modify tests to notice a regression.
Fixes#1389
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Use either "if X is not None:" or a try-except instead of a "if X:".
I believe Targets.from_dict() was not really broken with previous code
but it looks suspicious and did fail the added test with a strange
exception: I expect the from_dict() methods to mainly fail with
KeyErrors, ValueErrors or AttributeErrors if file format structure
is incorrect.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
A DelegatedRole with paths=[] fails to serialize correctly (paths is not
included in the output json).
Fix the issue, modify tests to notice a regression.
Fixes#1389
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
After the implementation of a Key class representing
the public portion of a key, the method add_key() should
take an argument of type Key, instead of a dictionary.
Test cases are updated accordingly.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
In the top level metadata classes, there are complex attributes such as
"meta" in Targets and Snapshot, "key" and "roles" in Root etc.
We want to represent those complex attributes with a class to allow
easier verification and support for metadata with unrecognized fields.
For more context read ADR 0004 and ADR 0008 in the docs/adr folder.
DelegatedRole shares a couple of fields with the Role class and that's
why it inherits it.
I decided to use a separate Delegations class because I thought it will
make it easier to read, verify and add additional helper functions.
Also, I tried to make sure that I test each level of the delegations
representation for support of storing unrecognized fields.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
We should not do multiple lookups through data structures if one is
enough (here we have extra lookups on both roles and keyids).
Also in this case raising on missing key seems like the preferable
alternative so even a try-except is not needed.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
As per the specification (v1.0.1) length and hashes fields
in timestamp and snapshot metadata are optional.
We have implement this in the older API
(see https://github.com/theupdateframework/tuf/pull/1031) and we should
implement it in the new API.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Add a use case for the root class to be tested in test_generic_read
and test_read_write_read_compare tests in test_apy.py
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Verify that adding an already existing key to keyid for a particular
role in Root won't create duplicate key.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Updater now uses only a single url for metadata download.
Target files download use either a default url or an
optional one for each file passed by the caller.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
In the top level metadata classes, there are complex attributes such as
"meta" in Targets and Snapshot, "key" and "roles" in Root etc.
We want to represent those complex attributes with a class to allow
easier verification and support for metadata with unrecognized fields.
For more context read ADR 0004 and ADR 0008 in the docs/adr folder.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
In the top level metadata classes, there are complex attributes such as
"meta" in Targets and Snapshot, "key" and "roles" in Root etc.
We want to represent those complex attributes with a class to allow
easier verification and support for metadata with unrecognized fields.
For more context read ADR 0004 and ADR 0008 in the docs/adr folder.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
In order to support ADR 0008 we would want to accept unrecognized
fields in all metadata classes.
Input that contains unknown fields in the 'signed' dictionary should
successfully deserialize into a Metadata object, and that object should
successfully serialize with the unknown fields intact.
Also, we should test that we support unrecognized fields when adding
new classes or modifying existing ones to make sure we support
ADR 0008.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
This is essentially short-hand for
JSONDeserializer().deserialize(data)
but seems much easier for the API user so may be worth it.
Metadata.from_file() now uses Metadata.from_bytes() internally.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Checks metadata expiration against a reference time (a naive datetime in UTC).
If not provided, checks against the current UTC date and time.
Returns True if expiration time is less than the reference time.
Signed-off-by: Velichka Atanasova <avelichka@vmware.com>
These tests seem to try to remove temp files before the processes
using those files had stopped. This likely lead to an error (and
dangling temp files) on Windows, but Modified_Testcase hides the error
Make sure temp directories are removed as the last thing in teardown.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Call the parent (Modified_Testcase) tearDown as the last thing in
tearDown(). This is good practice anyway and in practice may prevent
bugs where the instance needs to cleanup something before
Modified_Testcase removes the temp dir.
In practice there does not seem to be visible bugs in these tests
(as the all have top level temp directory handling in tearDownClass())
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Make sure test server processes are killed before the temporary
directories are removed.
Let Modified_Testcase handle the top-level temporary directory.
Don't let Modified_testcase handle any subdirectories because:
* teardown will try to remove them in the wrong order
* removing the top level is enough
Fixes#1344
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
