Commit graph

2566 commits

Author SHA1 Message Date
Jussi Kukkonen
85ce3e87e1 Prepare 7.0 release
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2026-05-18 10:49:12 +03:00
Jussi Kukkonen
8ed4414946 Fix incorrect targetpath comparison on Windows
fnmatch() will do case-normalization if the OS expects it:
we do not want this as
* the path is more a URL than a file path
* results need to not change based on OS

Fix the result on Windows by using fnmatchcase()

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2026-04-21 12:27:16 +03:00
1seal
c49bdb9322 feat(ngclient): require explicit bootstrap argument
make bootstrap required and explicit: callers must pass bootstrap=<root_bytes> or bootstrap=None.

also tighten docs, examples, and tests to reflect the explicit trust anchor choice.

Signed-off-by: 1seal <security@1seal.org>
2026-01-25 11:58:09 +00:00
Jussi Kukkonen
7ecb67d83e api: make the zip() usage clearer
We manually enforce matching lengths but there's no harm in doing this
too.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2026-01-08 13:15:37 +02:00
Jussi Kukkonen
0785c78b33 Make linter happy after python upgrade
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2026-01-08 13:08:53 +02:00
Jussi Kukkonen
8513f46c2b Bump minimum Python version to 3.10
We could just stop testing with 3.9... but I think this will lead to
unintentionally breaking 3.9 anyway sooner or later.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2026-01-08 13:08:47 +02:00
Lukas Puehringer
f1afc98c95 lint: change base tuple order in SimpleEnvelope
Latest ruff complains about Generic[T] not being the last base class in
the bases tuple of Simple Envelope (generic-not-last-base-class
(PYI059)).

This commit applies the default fix by changing the order of the bases.
While this can change the MRO, there shouldn't be a change of behavior
given the used bases.

See https://docs.astral.sh/ruff/rules/generic-not-last-base-class/ for
details.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2025-09-16 09:10:50 +02:00
Jussi Kukkonen
88cc98420e lint fixes
Fixes for ruff 0.12
* Tweak some annotations
* Add __hash__() implementations to api classes: These really should be
  hashable
* My use of "super().__hash__()" is not very optimized but avoids some
  repetition

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-06-24 11:02:09 +03:00
Lukas Pühringer
500e8b9a8b
Merge pull request #2815 from lukpueh/port-sslib-hash
Port securesystemslib.hash module
2025-03-19 12:21:09 +01:00
Lukas Puehringer
75e83b36d0 docs: Remove reference to securesystemslib hash
Default hash sha256 is now defined locally.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2025-03-19 11:29:36 +01:00
Lukas Puehringer
535a18918b Refactor hash helpers
Consolidate interface of bytes hash and file hash helpers.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2025-03-19 10:07:58 +01:00
Jussi Kukkonen
f3eddc19ff lint: Accept ruff suggestions for cast()
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-03-18 18:20:11 +02:00
Lukas Puehringer
866409ffe9 Port securesystemslib.hash module
securesystemslib.hash is a small wrapper around hashlib, which serves
two main purposes:
* provide helper function to hash a file
* translate custom hash algorithm name "blake2b-256" to "blake2b" with
  (digest_size=32).

In preparation for the removal of securesystemslib.hash, this patch ports
above behavior to tuf and uses the builtin hashlib directly where
possible.

related secure-systems-lab/securesystemslib#943

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2025-03-18 16:39:46 +01:00
Jussi Kukkonen
44eed614f0 Prepare v6.0
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-03-11 11:41:10 +02:00
Dimitri Papadopoulos
4a28307270
Fix typos
Signed-off-by: Dimitri Papadopoulos <3234522+DimitriPapadopoulos@users.noreply.github.com>
2025-03-10 22:06:30 +01:00
Jussi Kukkonen
15933a93b6
ngclient: Create directories as needed (#2808) 2025-03-09 06:56:37 +00:00
Kairo Araujo
fee5148abd
Merge pull request #2789 from jku/handle-proxy-variables 2025-03-04 02:58:47 +01:00
Jussi Kukkonen
38e4eaba1f updater: Improve comments on bootstrap arg
This includes some minor example improvements

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 11:11:28 +02:00
Jussi Kukkonen
ab288304a6 updater: Update root.json symlink on initialize
When application initializes an Updater with bootstrap, it should be
considered the trusted version from that point onwards: Update the
symlink "root.json" already here (even if refresh is never called).
n that Updater instance).

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 11:10:04 +02:00
Jussi Kukkonen
8519bb43ed ngclient: Make sure non-versioned link in cache is up-to-date
Even if last root version from remote is not accepted (leading to an
exception in load_root()) we should update the symlink "root.json" in
local cache to point to last good version.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 11:10:04 +02:00
Jussi Kukkonen
cea1745cef Implement root bootstrapping
Application may have a "more secure" data store than the metadata cache
is: Allow application to bootstrap the Updater with this more secure
root. This means the Updater must also cache the subsequent root versions
(and not just the last one).

* Store versioned root metadata in local cache
* maintain a non versioned symlink to last known good root
* When loading root metadata, look in local cache too
* Add a 'bootstrap' argument to Updater: this allows
  initializing the Updater with known good root metadata
  instead of trusting the root.json in cache

Additional changes to current functionality:
* when using bootstrap argument, the initial root is written to cache.
  This write happens every time Updater is initialized with bootstrap
* The "root.json" symlink is recreated at the end of every refresh()

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 11:09:54 +02:00
Jussi Kukkonen
265e772dba ProxyEnvironment: Handle no_proxy="*"
Add support for leading dots in no_proxy and "*" as a no_proxy value.

Both are supported in requests and based on
https://about.gitlab.com/blog/2021/01/27/we-need-to-talk-no-proxy/
both are somewhat common.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 10:56:23 +02:00
Jussi Kukkonen
9a4e749def ngclient: Add docs on HTTP in general
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 10:56:23 +02:00
Jussi Kukkonen
80b629013e Use __future__ to make old python happy
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 10:56:23 +02:00
Jussi Kukkonen
396ba079d6 ngclient: Add proxy environment variable handling
urllib3 does not handle this but we do want to support proxy users.

The environment variable handling is slightly simplified from the
requests implementation.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 10:56:23 +02:00
Jussi Kukkonen
73273813f3
Merge pull request #2773 from jku/no-requests
More porting from  requests to urllib3
2025-02-20 08:51:42 +00:00
pakagronglb
acd7ed08d1 Update Python shebangs to explicitly use python3
Signed-off-by: pakagronglb <pakagronglebel@gmail.com>
2025-02-19 19:44:21 +07:00
pakagronglb
94639360ec Enable FA (future annotations) linting ruleset
Signed-off-by: pakagronglb <pakagronglebel@gmail.com>
2025-02-19 19:44:05 +07:00
Jussi Kukkonen
5acd3f7df7 ngclient: Add note about RequestsFetcher being deprecated
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-15 09:57:06 +02:00
Jussi Kukkonen
6ddc0838a2 Move fetchers around to make them public
RequestsFetcher should still be public (even if deprecated).

* We don't want to import RequestsFetcher in __init__
  (because that requires importing requests)
* but we do want RequestsFetcher to be importable publicly

Move both fetchers out of _internal: that was never the right place for
them anyway: they are public modules.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-14 17:42:45 +02:00
Jussi Kukkonen
140abd34dc Update requirements: drop requests
This also removes RequestsFetcher from tuf.ngclient.__init__.py:
Otherwise we can't drop the requests dependency.

This means RequestsFetcher is not currently public.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-14 17:42:43 +02:00
Jussi Kukkonen
cfee40aa96 More porting from from requests to urllib3
This is related to #2762 (that replaces RequestsFetcher with
Urllib3Fetcher) and takes care of the remaining requests use cases in
the code base.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-14 17:41:50 +02:00
Jussi Kukkonen
d42426f137
Merge pull request #2762 from NicholasTanz/switchUrlLib3
replace RequestsFetcher for Urllib3Fetcher
2025-02-14 16:10:38 +02:00
NicholasTanz
d67f126233 remove self.app_user_agent attribute, as it's not used outside of init
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-02-05 17:49:02 -05:00
Jussi Kukkonen
1a1312e1af dsse: Improve type checking
mypy rightly complains our types do not match (this only happen if you
enable type checks for securesystemslib):
* I think the annotation is actually wrong: Envelope does not know the
  contained type at this point.
* Likely SimpleEnvelope should not be generic: it does not relly know
  what it contains

I decided not to break the API here and just made the type cast
explicit (even though we don't really know that the cast is correct):
this silences mypy but has no other consequences.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-01-31 13:47:45 +02:00
NicholasTanz
86cc7ad3ee clarify urllib3 as requirement in pyproject.toml and add back in requestsFetcher as option.
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-30 21:29:08 -05:00
Jussi Kukkonen
bb62dded29 Add type check
mypy warns about this but we know that encode_canonical() cannot return
None if we don't set output_function argument.

ruff does not like assert so I added a "noqa" and a comment

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-01-30 18:20:39 +02:00
Nicholas Tanzillo
326529bf99
Merge branch 'theupdateframework:develop' into switchUrlLib3 2025-01-29 16:38:21 -05:00
Jussi Kukkonen
5b2c041da0 lint: Fixes from new ruff
The noqa comment was added manually to avoid
  A005 Module `json` shadows a Python standard-library module

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-01-14 10:40:21 +02:00
NicholasTanz
a48fca51f9 add retry error handling to _chunks()
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-09 23:56:06 -05:00
NicholasTanz
2aed81f019 change error handling to MaxRetryError in _fetch()
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-09 23:31:50 -05:00
NicholasTanz
21280302e7 utilize one pool manager
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-09 20:59:56 -05:00
NicholasTanz
18e42cea52 replacing RequestsFecther with Urllib3Fetcher in .rst
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-06 02:55:15 -05:00
NicholasTanz
031778fd8d more linting stuff
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-06 02:47:51 -05:00
NicholasTanz
20d825f041 fix line too long linting error
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-06 02:31:20 -05:00
NicholasTanz
0675f0ce3a create urllib3 fetcher, replace requestsFetcher with urllibFetcher in ngclient, replace requestsFecther with urllibFetcher in requestsFetcher unit tests.
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-06 02:17:09 -05:00
Jussi Kukkonen
fca3086b5d repository: Change RuntimeError to AssertionError
These are assertions that should happen in production:
something is wrong in an unrecoverable way.

This is not an API change since no-one should be catching these.
Making these AssertionErrors makes them skippable in coverage.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-11-29 13:19:54 +02:00
Jussi Kukkonen
687d4557ad Revert "refactor to use dict union, instead of unpacking"
This reverts commit eb6d82f324.

The change itself was fine but since the code is otherwise compatible
with python 3.8, let's revert this to be compatible for one more
release.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-11-29 12:51:35 +02:00
Jussi Kukkonen
1d81a04707 Use __future.annotations module
This allows using some more nice annotations from 3.10
while still being compatible with even Python 3.8.

These are all annotation changes, should not modify any functionality.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-11-29 12:41:09 +02:00
NicholasTanz
eb6d82f324 refactor to use dict union, instead of unpacking
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2024-11-03 23:30:09 -05:00