_fileinfo_has_changed() and _update_fileinfo() have been unused internal
methods since 2016. Remove them.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
The original commit 051b8229 handled the loading and saving metadata
cases but the legacy client actually checks for the files existence
in various other places:
* _update_versioninfo() never reads the file but operates differently
depending on whether the file exists or not
* _move_current_to_previous() that copies files around
* MultiRepoUpdater initialization: this only handle root.json so
is still correct
* _update_fileinfo() which is dead code
Fix the first two of these cases.
If an attacker manages to create arbitrary rolenames they could trick
the client into writing metadata files into unexpected locations:
To avoid directory traversal and writing files into unexpected
locations, encode the rolename before using it as filename.
If a client has delegated targets metadata with rolenames that have
percent-encoded characters in them, these metadata will now not be
found in local metadata cache and must be re-downloaded.
Note that this does not mean using rolenames that get encoded is
advisable (as forming the download URLs still has issues with them),
this just means the client will not do unsafe writes when it encounters
rolenames like this.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Remove use of six
Signed-off-by: Velichka Atanasova <avelichka@vmware.com>
Replace the use of dict.items(mydict) with mydict.items(), dict.keys(mydict) with mydict.keys() and dict.values(mydict) with mydict.values()
Signed-off-by: Velichka Atanasova <avelichka@vmware.com>
Replace 'import urllib' and 'import urllib.x' with 'from urllib import x' for vendor compatibility
Signed-off-by: Velichka Atanasova <avelichka@vmware.com>
The linter now understands our imports (yay), and complains a lot (boo):
* Remove really unused imports
* disable lints for tuf.log and securesystemslib imports: these imports
have logging side-effects (they set default loggers for tuf and
securesystemslib respectively) and I'm cautious about just removing
them
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.
Also fix import orders so tuf internal imports are last.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Make them compatible with vendoring, use
from securesystemslib import hash as sslib_hash
to have the same style as other securesystemslib imports (and to avoid
potential conflict with system hash()).
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Make the import compatible with vendoring tool and alias the import so
it does not clash with the local module. Fix all references
to the module in the code.
In one instance import a specific function to avoid a more complex
redirection in the code.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Make them compatible with vendoring, use
from securesystemslib import keys as sslib_keys
to have the same style as other securesystemslib imports.
Note that developer_tool already used a
from securesystemslib.keys import ...
for some functions so that style was used consistently there.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Make the import compatible with vendoring tool and alias the import so
it does not clash with the local module. Fix all references
to the module in the code.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Make the updater imports compatible with vendoring tool by importing
the Updater class directly (don't import the whole module to avoid the
clash with the obvious variable name 'updater').
Also update the example: This is not required in the clients but tuf
source code will be vendored and this import line (even though in a
comment) might trigger an error in future vendoring tool releases.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Make the import compatible with vendoring tool and alias the import so
it does not clash with the local module. Fix all references
to the module in the code.
Remove a related repo.py comment that was badly duplicated from module
docstring.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.
Remove one unused import.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Make sure mirrors is not used as variable name (so it can be used for
the module import name later).
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Remove the magic number, a whence value of 2 for file.seek(), and instead
use the io.SEEK_END constant from the io module.
Signed-off-by: Joshua Lock <jlock@vmware.com>
Ensure that the newly added files' docstrings adhere to the
recently adopted code style guideline (#1232).
Small code style improvements in comments and imports.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
* Move FetcherInterface to tuf/client/ directory: This way everything
inside that directory is clearly part of client API, and everything
outside _may_ be more of an implementation detail (settings is still
an unfortunate exception)
* Keep RequestsFetcher in tuf/ for same reasons: it's just the default
implementation, not explicitly part of client API
An even clearer division would be if we moved all the client specific
implementation details (download.py, mirrors.py, requests_fetcher.py)
to tuf/client/_internal/ but that's a larger change...
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
A custom error is required so that updater is able to special case
403 & 404 status codes.
Rewrite the test case a bit to be more readable.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Initialize Updater with an external implementation of
FetcherInterface. If not provided, tuf.fetcher.RequestsFetcher
is used.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
The specification, as of 1.0.16, describes an update expiration check as:
> The expiration timestamp in the trusted $ROLE metadata file MUST be
higher than the fixed update expiration time.
Having done some research into how other security providers are comparing
expiration equivalents (i.e. OpenSSL x509 certificate checking code, and
GnuPG expiration checks), and how other TUF implementations are performing
the same check (rust-tuf, go-tuf), we came to a consensus that the correct
way to implement expiration comparisons is:
expiration <= now
Where:
expiration: is the metadata's expiration datetime
now: is the current system time, or the fixed notion of time in the
detailed client workflow (introduced in 1.0.16 of the spec)
Fixes#1231
Signed-off-by: Joshua Lock <jlock@vmware.com>
If during updater.download_target() the download succeeds but a later
check fails (e.g. BadHashError), remember to close the tempfile.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Simplify the loop exit logic in _get_target_file() to simply return a
verified file_object, once we have it, rather than breaking from the loop
and then returning the file_object.
This converts a use of a try/except/else to a try/except and is a little
easier to read.
Signed-off-by: Joshua Lock <jlock@vmware.com>
When verifying newly downloaded root metadata with the keys listed in the
root metadata being verified, multiple signatures with the same keyid
should not be counted towards the threshold. A keyid should only count
once towards the threshold.
This fixes the _verify_root_self_signed() method introduced in PR #1101 to
ensure that keyids are only counted once when verifying a threshold of new
root signatures.
Signed-off-by: Joshua Lock <jlock@vmware.com>
We don't want to read the whole file in memory as it can be huge. Use
digest_fileobject() instead: This way Securesystemslib will read the
file in chunks.
Securesystemslib already takes care of seeking to beginning of file.
Fixes#1215
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Rather than read to the end of the file in order to determin its size, use
the whence value of seek() to move the file object's position to the end
of the file, then the tell() method of the file object to read the current
position in bytes.
Co-authored-by: Jussi Kukkonen <jkukkonen@vmware.com>
Signed-off-by: Joshua Lock <jlock@vmware.com>
The call stack and code for download_target() is more complex than
required:
* download_target() : builds target destination filepath, gets length
and hashes
* _get_target_file() : fixes filenames if consistent snapshots enabled,
defines verification callback
* _get_file() : iterates mirrors, tries to download files, verifies them
Remove the verification callback and collapse the call stack by a single
level to make the code easier to follow.
Signed-off-by: Joshua Lock <jlock@vmware.com>
Added ExpiredMetadataError to function documentation where it seems to
be missing.
Corrected the refresh() documentation: ExpiredMetadataError can only
happen when top level metadata does not need to be updated but is
expired. If the metadata gets updated and is expired, the result will
be a NoWorkingMirror with ExpiredMetadata inside it.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
The field confined_target_dirs from the MIRROR_SCHEMA is
a list of strings. Those strings define the accessible target
paths for that mirror. For one target to be available for that mirror,
its path should have as a prefix at least one of the strings defined
in confined_target_dirs.
That's why when confined_target_dirs is a list with one element empty
string (e.g. ['']) this means all targets files on that mirror are
available and if confined_target_dirs is empty list (e.g. []) this
would be interpreted as none of the target files is available.
This is a confusing API that could easily lead to mistakes.
That's why it's better we promote to not set confined_target_dirs
at all if a user wants targets to be available.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Our 'expires' strings are constrained by the ISO8601_DATETIME_SCHEMA
which matches regex '\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z'. This can be
parsed with just a datetime.strptime(): iso8601 module is not needed.
* Add formats.expiry_string_to_datetime() helper function
* Modify the 3 locations that used iso8601 and the api/metadata.py usage
of datetime.strptime()
* Remove related unnecessary logger setup
* Add the missing exception documentation to relevant functions (in many
cases the exception is rather unlikely as the schema has been verified
many times before this though...)
Fixes#1065
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
This allows clients to separate
a) missing local repository and
b) error while loading local repository
This is fully backwards-compliant: MissingLocalRepositoryError derives
from RepositoryError and every situation that now results in
MissingLocalRepositoryError used to result in a RepositoryError.
Fixes#1063
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
* move code to only create objects after potential raises
* Use 'with' when possible
* close manually if those did not help
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Don't use log level ERROR when we are only raising an exception for user
to handle (the issue is not a bug in TUF: TUF is working as expected in
all of these cases). Don't log at all if all the info is included in the
raised exception.
Also definitely don't log at error level when we don't know if this will
even be raised (e.g. a 404 is expected at least once on every
Updater.refresh()).
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
PR #1014 removed uses of keyid_hash_algorithms in favour of using the calculated
keyid values from the metadata. A few instances of this removal were
unintentionally reintroduced in PR #1016, when changing to explicitly passing
a list of hash algorithms rather than changing securesystemslib settings
values.
This change removes uneccessary uses of keyid_hash_algorithms.
Signed-off-by: Joshua Lock <jlock@vmware.com>
