make bootstrap required and explicit: callers must pass bootstrap=<root_bytes> or bootstrap=None.
also tighten docs, examples, and tests to reflect the explicit trust anchor choice.
Signed-off-by: 1seal <security@1seal.org>
We could just stop testing with 3.9... but I think this will lead to
unintentionally breaking 3.9 anyway sooner or later.
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
Application may have a "more secure" data store than the metadata cache
is: Allow application to bootstrap the Updater with this more secure
root. This means the Updater must also cache the subsequent root versions
(and not just the last one).
* Store versioned root metadata in local cache
* maintain a non versioned symlink to last known good root
* When loading root metadata, look in local cache too
* Add a 'bootstrap' argument to Updater: this allows
initializing the Updater with known good root metadata
instead of trusting the root.json in cache
Additional changes to current functionality:
* when using bootstrap argument, the initial root is written to cache.
This write happens every time Updater is initialized with bootstrap
* The "root.json" symlink is recreated at the end of every refresh()
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
urllib3 does not handle this but we do want to support proxy users.
The environment variable handling is slightly simplified from the
requests implementation.
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
This is still copy-paste in three different files but now at least
the function is the same in every location and not directly copied.
We really should have generic TestCase class...
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
test_updater_ng.py is a little archaic (as it uses the static test
repository content from ye olden days). This commit does not change that
but removes an extra file in client cache dir: it is now quite confusing
as it looks a bit like intermediate root caching but is just an unused
file.
This has the nice side effect that tests now longer need to workaround
this extra file.
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
This allows using some more nice annotations from 3.10
while still being compatible with even Python 3.8.
These are all annotation changes, should not modify any functionality.
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
* Remove exectuable flag from a couple of files
* Half of the test files have a shebang (but are
still not executable): remove the shebang
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
* API changes covered:
* keys and interface modules removed
* SSlibSigner removed
* CryptoSigner added: this replaces the removed functionality
* DSSE "signatures" container type changed
* Currently pins a securesystemslib main branch commit:
this shoudl be reverted before merging, when securesystemslib
has made a release
* tests/generated_data/generate_md.py was simplified
* Encrypted test keys in tests/repository_data/keystore were replaced
with the unencrypted PEM versions of the same keys
* The public test keys in tests/repository_data/keystore were removed
as they were not used anymore
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
Change TrustedMetadataSet to load and verify metadata using a
MetadataUnwrapper instance.
IMPORTANT NOTES:
* Requires changing the TrustedMetadataSet to store payloads only, which
is okay, because signatures are no longer needed, after being verified
at load time.
* Includes a minor re-ordering of validation steps. That is, the
version increment for root metadata is now checked after signature
verification. Preserving the order would require including the check in
the Unwrapper interface, which is feasible but does not seem correct wrt
separation of responsibility.
Changes are adopted in updater, tests and _localrepo.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
This test covers `targetinfo`, `target_path`, `target_base_url`,
`metadata_dir` and `filepath` input validation of the `Updater`
methods
Signed-off-by: Ivana Atanasova <iyovcheva@vmware.com>
Update new test modules to stop using unittest_toolbox, in
preparation for its removal in #1790.
The tools provided by unittest_toolbox can easily (in a more
obvious way) be replaced by using the standard library modules
`tempfile` and `random` (no more used) directly.
In the case of tempdir and -file creation/removal, skipping the use
of unittest_toolbox, which does this by default, also uncovers some
test cleanup failures, which would occur when temporary test
directories were removed while a test server hadn't released them.
(see `except OSError: pass` in unittest_toolbox's `tearDown`
method)
**Change details**
**test_fetcher_ng.py:**
- Stop implicitly creating (setUp) and removing (tearDown) tmp test
dirs. -Move now manual creation of an exemplary targets file to
setUpClass, as the same file is used by all tests. And remove it
explicitly in tearDownClass after killing the server (see note
about failure above). - Trigger URL parsing error with a hardcoded
invalid URL string instead of a random string.
**test_updater_ng.py**
- Stop implicitly creating (setUp) and removing (tearDown) tmp test
dirs.
- Explicitly create tmp test dirs in setUp, but don't remove
them in tearDown to avoid above mentioned failures. They will be
removed all at once when removing the tmp root test dir in
tearDownClass
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Define TESTS_DIR constant in tests/util.py as full path to the
parent directory of the util module. This may be used to reliably
read other files in tests dir, such es "repository_data" or
"simple_server", regardless of cwd.
This commit also replaces a couple of `getcwd() + "filename"` with
`TESTS_DIR + filename`, so that in the future (post #1790) we
should be able to invoke the tests from anywhere, not only from
within the tests directory as is now the case.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Add tuf/api/exceptions.py for exceptions in the new code.
I copied the exceptions from tuf/exceptions.py with a few important
decisions:
1. I only added the exceptions that are used in the new code
2. I removed the general "Error" class as we can directly inherit
Exceptions
3. I tried grouping the exceptions by relevance
4. I removed the second argument "UnsignedMetadataError" as it's only
kept for backward compatibility and is not used
5. I tried following the new code style guidelines and linted the file
with our linters.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
When calling updater._persist_metadata() there is a possibility that
writing the temporary file to storage can succeed, but moving it with
os.replace could fail with OSError.
Make sure we are removing the newly created temporary file in that case.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Remove `bump_version()` method, which is just an alias for "+= 1"
on the version attribute. For a slim low-level API it seems okay to
just directly access/modify the attribute.
The extra level of abstraction of "bumping a version" is more
appropriate for a repository library (see #1136).
This patch also removes a related unit test and updates another one
to directly do `(...).version +=`.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
metadata_dir matches metadata_base_url better.
This is an API break for anyone using named arguments.
Fixes#1638
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
The changes are automatic linting fixes from black.
The target files are only those who test the new code.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
This applies the use of constants of top-level rolenames in the
tests instead of the previously hardcoded strings.
Fixes#1648
Signed-off-by: Ivana Atanasova <iyovcheva@iyovcheva-a02.vmware.com>
As a target path is a URL path it's not correct to consider it as
interchangeable with a filepath within every operation system. The
unquote is also removed as the ngclient cannot assume correctly
which encoding is intended and which not
Fixes#1483
Signed-off-by: Ivana Atanasova <iyovcheva@iyovcheva-a02.vmware.com>
This commit includes manual fixes for a lot of mypy warnings.
When there were warnings that we are calling non-annotated function
in annotated context I decided to add annotations instead of ignoring
those warnings.
That's how I end up adding annotations in the whole tests/utils.py
module.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Consistent snapshot and consistent targets are now
extensively tested in test_updater_consistent_snapshot.py.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Address or disable pylint warnings raised on all test files inside
the "tests/" directory testing the code of the new implementation.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
This is a minimal test case to show the basic functionality:
I'd like to update most of the tests to use the implicit method
but would like to do that when there's less churn in these tests.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
This is slightly cosmetic but rename get_one_valid_targetinfo to
get_targetinfo:
* The function name is long without any reason: "one" and "valid" are
always implicit
* shortening makes code (incl. our examples and tests) easier to read
* We're also already changing updater API (compared to legacy) so this
alone does not break things -- it's also not a difficult "port".
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Remove updated_targets() as it doesn't fit the rest of the API.
In its stead add find_cached_target() which has a similar signature
as download_target(): both accept an optional local filepath as
argument and return full local filepath. In the
find_cached_target() case None is returned if the local file is not the
correct target file.
Updater constructor gets a new optional target_dir argument: This means
client can avoid giving a local filepath as an argument to
find_cached_target()/download_target() -- Updater will then generate a
filename within targets_dir.
A reasonable use pattern (when targets_dir is set in constructor):
info = updater.get_one_valid_targetinfo("targetname")
path = updater.find_cached_target(info)
if path is None:
path = updater.download_target(info)
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Do not add similar assertions for test_refresh_on_consistent_targets():
The test is broken and can't actually update metadata (#1573).
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Add tests covering missing branches of the Updater
code. Inlcude ngclient in the total coverage report.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
