Commit graph

1387 commits

Author SHA1 Message Date
Jussi Kukkonen
9f34e0f232
Merge pull request #2903 from 1seal/hardening/require-explicit-bootstrap
feat(ngclient): require explicit bootstrap argument
2026-02-19 17:55:40 +02:00
1seal
d5fa0b0594 address review feedback: remove redundant root.json writes, rename docs section
Signed-off-by: 1seal <security@1seal.org>
2026-02-19 14:46:40 +01:00
Jussi Kukkonen
927a598877 tests: Keep linter happy
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2026-02-17 12:15:12 +02:00
1seal
c49bdb9322 feat(ngclient): require explicit bootstrap argument
make bootstrap required and explicit: callers must pass bootstrap=<root_bytes> or bootstrap=None.

also tighten docs, examples, and tests to reflect the explicit trust anchor choice.

Signed-off-by: 1seal <security@1seal.org>
2026-01-25 11:58:09 +00:00
Jussi Kukkonen
0785c78b33 Make linter happy after python upgrade
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2026-01-08 13:08:53 +02:00
Jussi Kukkonen
8513f46c2b Bump minimum Python version to 3.10
We could just stop testing with 3.9... but I think this will lead to
unintentionally breaking 3.9 anyway sooner or later.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2026-01-08 13:08:47 +02:00
Jussi Kukkonen
ee50fea0c6 annotation fixes
* Start linting securesystemslib calls
  (this requires new securesystemslib)
* Fix various issues that suddenly popup

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-04-22 10:53:54 +03:00
Lukas Puehringer
6f50998c37 Add tests for custom blake hash
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2025-03-19 10:34:35 +01:00
Lukas Puehringer
57010fb0b1 Rename hash algo global in repo simulator
Remove the "default" prefix, because it's not a default but rather a
fixed value.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2025-03-19 09:28:01 +01:00
Lukas Puehringer
866409ffe9 Port securesystemslib.hash module
securesystemslib.hash is a small wrapper around hashlib, which serves
two main purposes:
* provide helper function to hash a file
* translate custom hash algorithm name "blake2b-256" to "blake2b" with
  (digest_size=32).

In preparation for the removal of securesystemslib.hash, this patch ports
above behavior to tuf and uses the builtin hashlib directly where
possible.

related secure-systems-lab/securesystemslib#943

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2025-03-18 16:39:46 +01:00
Dimitri Papadopoulos
4a28307270
Fix typos
Signed-off-by: Dimitri Papadopoulos <3234522+DimitriPapadopoulos@users.noreply.github.com>
2025-03-10 22:06:30 +01:00
Jussi Kukkonen
15933a93b6
ngclient: Create directories as needed (#2808) 2025-03-09 06:56:37 +00:00
Kairo Araujo
fee5148abd
Merge pull request #2789 from jku/handle-proxy-variables 2025-03-04 02:58:47 +01:00
Jussi Kukkonen
c4cd7935e3 tests: lint fixes
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 11:10:04 +02:00
Jussi Kukkonen
339b52394e tests: Add tests for caching intermediate roots
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 11:10:04 +02:00
Jussi Kukkonen
3798002345 tests: Use Updater bootstrap argument
Update test_updater_toplevel_update to use bootstrap argument by
default.

This still does not include tests for bootstrap feature specifically
but it should prove nothing has broken when the feature was added.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 11:10:04 +02:00
Jussi Kukkonen
4aa09ff7d5 tests: Fix test_load_metadata_from_cache for versioned roots
Expect (failing) call to open for "root_history/2.root.json" now that
the client stores versioned roots.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 11:10:04 +02:00
Jussi Kukkonen
cea1745cef Implement root bootstrapping
Application may have a "more secure" data store than the metadata cache
is: Allow application to bootstrap the Updater with this more secure
root. This means the Updater must also cache the subsequent root versions
(and not just the last one).

* Store versioned root metadata in local cache
* maintain a non versioned symlink to last known good root
* When loading root metadata, look in local cache too
* Add a 'bootstrap' argument to Updater: this allows
  initializing the Updater with known good root metadata
  instead of trusting the root.json in cache

Additional changes to current functionality:
* when using bootstrap argument, the initial root is written to cache.
  This write happens every time Updater is initialized with bootstrap
* The "root.json" symlink is recreated at the end of every refresh()

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 11:09:54 +02:00
Jussi Kukkonen
f35b237739 tests: Make tests cope with root history in local cache
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 11:00:50 +02:00
Jussi Kukkonen
265e772dba ProxyEnvironment: Handle no_proxy="*"
Add support for leading dots in no_proxy and "*" as a no_proxy value.

Both are supported in requests and based on
https://about.gitlab.com/blog/2021/01/27/we-need-to-talk-no-proxy/
both are somewhat common.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 10:56:23 +02:00
Jussi Kukkonen
80b629013e Use __future__ to make old python happy
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 10:56:23 +02:00
Jussi Kukkonen
5f9fefb80f tests: Add tests for ProxyEnvironment
This does not actually test using tuf through proxies: it only tests
that ProxyEnvironment creates the ProxyManagers that we expect to be
created based on the proxy environment variables.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 10:56:23 +02:00
Jussi Kukkonen
396ba079d6 ngclient: Add proxy environment variable handling
urllib3 does not handle this but we do want to support proxy users.

The environment variable handling is slightly simplified from the
requests implementation.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 10:56:23 +02:00
pakagronglb
acd7ed08d1 Update Python shebangs to explicitly use python3
Signed-off-by: pakagronglb <pakagronglebel@gmail.com>
2025-02-19 19:44:21 +07:00
pakagronglb
94639360ec Enable FA (future annotations) linting ruleset
Signed-off-by: pakagronglb <pakagronglebel@gmail.com>
2025-02-19 19:44:05 +07:00
NicholasTanz
2ac8bdc863 linting
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-02-12 00:12:18 -05:00
NicholasTanz
6318760cc1 swap invalid urls that are used in testing. (takes care of deprecation warning in #2776)
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-02-12 00:07:38 -05:00
Nicholas Tanzillo
326529bf99
Merge branch 'theupdateframework:develop' into switchUrlLib3 2025-01-29 16:38:21 -05:00
Jussi Kukkonen
2bb4ff6386 tests: Standardize cache file checking code
This is still copy-paste in three different files but now at least
the function is the same in every location and not directly copied.

We really should have generic TestCase class...

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-01-17 11:27:27 +02:00
Jussi Kukkonen
166434d844 tests: Remove unused test files
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-01-17 11:11:26 +02:00
Jussi Kukkonen
416c34c6fc tests: Remove unused file
test_updater_ng.py is a little archaic (as it uses the static test
repository content from ye olden days). This commit does not change that
but removes an extra file in client cache dir: it is now quite confusing
as it looks a bit like intermediate root caching but is just an unused
file.

This has the nice side effect that tests now longer need to workaround
this extra file.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-01-17 11:11:26 +02:00
NicholasTanz
f8b1dbd253 linting
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-09 23:59:13 -05:00
NicholasTanz
a48fca51f9 add retry error handling to _chunks()
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-09 23:56:06 -05:00
NicholasTanz
2aed81f019 change error handling to MaxRetryError in _fetch()
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-09 23:31:50 -05:00
NicholasTanz
21280302e7 utilize one pool manager
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-09 20:59:56 -05:00
NicholasTanz
0675f0ce3a create urllib3 fetcher, replace requestsFetcher with urllibFetcher in ngclient, replace requestsFecther with urllibFetcher in requestsFetcher unit tests.
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-06 02:17:09 -05:00
Jussi Kukkonen
ec81bfa0b1 tests: Simplify test data generation
We always want to either verify or generate new results:
don't have multiple arguments.

Also fix annotated types.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-12-11 10:11:55 +02:00
Jussi Kukkonen
31bb232ca3 tests: Remove various unneeded coverage workarounds
Tests now run from root dir so various coverage complications
can be removed.

Also remove the duplicate .coveragerc and rely on pyproject.toml

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-12-11 10:11:55 +02:00
Jussi Kukkonen
9946dc5277 tests: Make sure tests can execute from root source dir
"python -m unittest" now works in the root source dir too

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-12-11 10:11:55 +02:00
Jussi Kukkonen
28a031f039 tests: Remove aggregate_tests.py
This was essentially unused now (originally it was used to
randomize the test order).

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-12-11 10:11:49 +02:00
Jussi Kukkonen
caa4960691 tests: Fix return value of a test
We don't actually want to return anything here: just
make sure download_file() gets executed

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-12-10 20:30:49 +02:00
Jussi Kukkonen
1d81a04707 Use __future.annotations module
This allows using some more nice annotations from 3.10
while still being compatible with even Python 3.8.

These are all annotation changes, should not modify any functionality.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-11-29 12:41:09 +02:00
NicholasTanz
5c71f4f062 update python annotations
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2024-11-03 23:21:23 -05:00
Michał Górny
ee27bcccc1 tests: Use freezegun for time mocking to fix pypy3 compatibility
Use freezegun for time mocking instead of manually patching the datetime
module, as it provides a more streamlined solution that works both
on CPython and on PyPy.  Unfortunately, due to differences between
the C datetime extension used by CPython, and the pure Python version
of datetime (used by PyPy, and as a fallback on CPython), there does not
seem to be a trivial way to mock time that would work with both
versions.

Fixes #2708

Signed-off-by: Michał Górny <mgorny@gentoo.org>
2024-10-09 14:01:09 +02:00
Kapil Sharma
0ac86c67ad
repository: handle online key changes (#2650)
* repository: Handle online key change situations in do_snapshot() and do_timestamp():
  always create a new version if current version is not correctly signed
* remove expectedFailure marks from the related tests

Signed-off-by: h4l0gen <ks3913688@gmail.com>
Signed-off-by: Kapil Sharma <ks3913688@gmail.com>
2024-06-04 09:56:53 +03:00
Jussi Kukkonen
2b9cc1e462 tests: Add some initial tests for tuf.repository
These are pretty basic and do not test much about the content of the
repository... but it does check version numbers (and how many versions
have been published) in a couple of situations.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-06-03 10:12:35 +03:00
h4l0gen
d2f9f6dee6
changing useragent
Signed-off-by: h4l0gen <ks3913688@gmail.com>
2024-05-20 21:55:02 +05:30
Jussi Kukkonen
419bfe34ec linting: Enable all Ruff rulesets by default
The goal here is to have ruff enable new rulesets when new releases are
made without us having to o anything: we can then decide if we disable
or not.

* Enable a couple more rulesets (ERA, INP, T )
* Add a few individual ignores to tests and examples
* Default to enable all, disable the rulesets we don't want

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-05-07 15:39:53 +03:00
Jussi Kukkonen
80882dbe7d lint: Enable flake8-annotations
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-05-06 09:19:19 +03:00
Jussi Kukkonen
9a61be1bf4 lint: Enable flake8-executable
* Remove exectuable flag from a couple of files
* Half of the test files have a shebang (but are
  still not executable): remove the shebang

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-05-06 09:19:19 +03:00