Remove pylint disable logging-no-lazy, fix remaining non-lazy logging
(ngclient/updater.py still contains some but pylint does not notice
them: These will be fixed in issue #1400)
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Restore coverage back to 97% but omit ngclient
form the overall score until tests are implemented.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Only "Updater" and "FetcherInterface" are considered
public classes of the client. Exposing them in __init__.py
makes usage and access simpler.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Separate public/private API. Keep modules
containing the piblic classes in the main client
directory and move the rest to _internal.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
The current client and the next-gen client should
coexist in the same repository during the ongoing
development of the latter.
Looking for a name which is client-related,
short, meeting PEP8 package names requirements.
Currently "ngclient" seems to fit in until a
better proposal comes.
Rename updater_rework.py to updater.py
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Fixes incorrct access to MetaFile and TargetFile
after the merge of develop into experimetnal-client
branch.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Mostly remove comments that provide little value after all the changes.
Also remove a unused variable from a test.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
We have tests which make sure we can use `Timestamp.update()` and
`Snapshot.update()` with MetaFile instance storing only version
(because length and hashes are optional).
Those tests were created to make sure that we are actually supporting
optional hashes and length when we call `update` for those classes, but
after we changed the `update()` signature to accept `MetaFile` instance
the tests are obsolete.
The reason is that length and hashes can be optional because of the
MetaFile implementation, no the update function itself and we have
other tests validating creating a MetaFie instance without hashes and
length.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Currently, when we call Targets/Snapshot/Timestamp.update() we are
passing all of the necessary values to create MetaFile/Targets File
respectively.
This is not needed, given that one of the reasons we have created
MetaFile and TargetFile is to make the API easier to use.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Disable the "C0302: Too many lines in module" warning which warns for modules
with more 1000 lines, because all of the code here is logically connected
and currently, we are above 1000 lines by a small margin.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
In the top-level metadata classes, there are complex attributes such as
"meta" in Targets and Snapshot, "key" and "roles" in Root etc.
We want to represent those complex attributes with a class to allow
easier verification and support for metadata with unrecognized fields.
For more context read ADR 0004 and ADR 0008 in the docs/adr folder.
As written in the spec "targets" in "targets.json" has defined the
"custom" field serving the same purpose as "unrecognized_fields" in the
implementation.
That's why to conform against the spec and support "custom" and allow
"unrecognized_fields" everywhere where it's not sensitive we can define
custom as property which actually access data stored in
unrecognized_fields.
For context read ADR 8 in tuf/docs/adr.
Additionally, after adding the TargetFile class, when we create a
Targets an object we are now calling from dict twice - one for the main
Targets class and one for each of the complex attributes
TargetFile.from_dict() and Delegations.from_dict().
Given that the "from_dict" methods have the side effect of destroying
the given dictionary, we would need to start using deepcopy()
for our tests.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
In the top-level metadata classes, there are complex attributes such as
"meta" in Targets and Snapshot, "key" and "roles" in Root etc.
We want to represent those complex attributes with a class to allow
easier verification and support for metadata with unrecognized fields.
For more context read ADR 0004 and ADR 0008 in the docs/adr folder.
Additionally, after adding the MetaFile class, when we create an object
we are now calling from dict twice - one for the main class (Timestamp,
Snapshot) and one for the pacticular complex attribute -
MetaFile.from_dict(). Given that the "from_dict" methods have the
side effect of destroying the given dictionary, we would need to
start using deepcopy() for our tests.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Add test for a refresh with just a local root.json.
Remove unused code. Add docstrings for raised exceptions, add TODOs for
the missing exception handling.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Use the MetadataBundle to verify metadata validity.
* Updater now handles reading metadata files (from filesystem as
well as network
* Updater feeds bytes to MetadataBundle for verification
* Updater persists data on disk after it had been verified
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Document arguments and exceptions, improve prose in general. Remove
mention of local file deletion now that file IO is not done here.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
NOTE: making consistent_snapshot optional requires using a default value
for the argument in __init__ in Root and thus consistent_snapshot should
be rearranged in the end.
Read more: https://github.com/theupdateframework/tuf/pull/1394#issuecomment-842134961
From chapter 7 in the spec (version 1.0.17)
"Finally, the root metadata should write the Boolean
"consistent_snapshot" attribute at the root level of its keys of
attributes.
If consistent snapshots are not written by the repository,
then the attribute may either be left unspecified or be set to the
False value. Otherwise, it must be set to the True value."
We want to make sure we support repositories
without consistent_snapshot set.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Remove file IO from MetadataBundle:
* This make the bundle API very clear and easy to understand
* This means caller must now read from and persist data to disk
but initial prototypes suggest this won't make Updater too
complex
This change is something we can still back out from if it turns out to
be the wrong decision: the file-persisting MetadataBundle has been tested
and works fine.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
This is backwards-compatible and means that most (all?) errors resulting
from suspicious or broken metadata are now RepositoryErrors.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
The bundle should now raise
* derivatives of RepositoryError on failures that are likely a result
of server error or a malicious server
* RuntimeErrors if calls were made when they are not possible
* ValueErrors if arguments are invalid
last two are callers errors and avoidable.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Signed now has "type" attribute, use that.
Also remove another pylint disable that is no longer needed (logging is
now old style).
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Don't use the serialized format as that won't match any hashes in
"meta".
Add basic tests for updating metadata.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Use either "if X is not None:" or a try-except instead of a "if X:".
I believe Targets.from_dict() was not really broken with previous code
but it looks suspicious and did fail the added test with a strange
exception: I expect the from_dict() methods to mainly fail with
KeyErrors, ValueErrors or AttributeErrors if file format structure
is incorrect.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
A DelegatedRole with paths=[] fails to serialize correctly (paths is not
included in the output json).
Fix the issue, modify tests to notice a regression.
Fixes#1389
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Use either "if X is not None:" or a try-except instead of a "if X:".
I believe Targets.from_dict() was not really broken with previous code
but it looks suspicious and did fail the added test with a strange
exception: I expect the from_dict() methods to mainly fail with
KeyErrors, ValueErrors or AttributeErrors if file format structure
is incorrect.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
A DelegatedRole with paths=[] fails to serialize correctly (paths is not
included in the output json).
Fix the issue, modify tests to notice a regression.
Fixes#1389
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Lots of fixes, mostly obvious ones. The trickier ones and pylint
disables have comments added to explain them.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
* Loading root.json will fail just as descriptively
* As long as Bundle doesn't implement bootstrapping
the local repo, there's also no need to create
missing directories
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
