Commit graph

213 commits

Author SHA1 Message Date
1seal
c49bdb9322 feat(ngclient): require explicit bootstrap argument
make bootstrap required and explicit: callers must pass bootstrap=<root_bytes> or bootstrap=None.

also tighten docs, examples, and tests to reflect the explicit trust anchor choice.

Signed-off-by: 1seal <security@1seal.org>
2026-01-25 11:58:09 +00:00
Jussi Kukkonen
0785c78b33 Make linter happy after python upgrade
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2026-01-08 13:08:53 +02:00
Jussi Kukkonen
f3eddc19ff lint: Accept ruff suggestions for cast()
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-03-18 18:20:11 +02:00
Dimitri Papadopoulos
4a28307270
Fix typos
Signed-off-by: Dimitri Papadopoulos <3234522+DimitriPapadopoulos@users.noreply.github.com>
2025-03-10 22:06:30 +01:00
Jussi Kukkonen
15933a93b6
ngclient: Create directories as needed (#2808) 2025-03-09 06:56:37 +00:00
Kairo Araujo
fee5148abd
Merge pull request #2789 from jku/handle-proxy-variables 2025-03-04 02:58:47 +01:00
Jussi Kukkonen
38e4eaba1f updater: Improve comments on bootstrap arg
This includes some minor example improvements

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 11:11:28 +02:00
Jussi Kukkonen
ab288304a6 updater: Update root.json symlink on initialize
When application initializes an Updater with bootstrap, it should be
considered the trusted version from that point onwards: Update the
symlink "root.json" already here (even if refresh is never called).
n that Updater instance).

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 11:10:04 +02:00
Jussi Kukkonen
8519bb43ed ngclient: Make sure non-versioned link in cache is up-to-date
Even if last root version from remote is not accepted (leading to an
exception in load_root()) we should update the symlink "root.json" in
local cache to point to last good version.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 11:10:04 +02:00
Jussi Kukkonen
cea1745cef Implement root bootstrapping
Application may have a "more secure" data store than the metadata cache
is: Allow application to bootstrap the Updater with this more secure
root. This means the Updater must also cache the subsequent root versions
(and not just the last one).

* Store versioned root metadata in local cache
* maintain a non versioned symlink to last known good root
* When loading root metadata, look in local cache too
* Add a 'bootstrap' argument to Updater: this allows
  initializing the Updater with known good root metadata
  instead of trusting the root.json in cache

Additional changes to current functionality:
* when using bootstrap argument, the initial root is written to cache.
  This write happens every time Updater is initialized with bootstrap
* The "root.json" symlink is recreated at the end of every refresh()

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 11:09:54 +02:00
Jussi Kukkonen
265e772dba ProxyEnvironment: Handle no_proxy="*"
Add support for leading dots in no_proxy and "*" as a no_proxy value.

Both are supported in requests and based on
https://about.gitlab.com/blog/2021/01/27/we-need-to-talk-no-proxy/
both are somewhat common.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 10:56:23 +02:00
Jussi Kukkonen
9a4e749def ngclient: Add docs on HTTP in general
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 10:56:23 +02:00
Jussi Kukkonen
80b629013e Use __future__ to make old python happy
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 10:56:23 +02:00
Jussi Kukkonen
396ba079d6 ngclient: Add proxy environment variable handling
urllib3 does not handle this but we do want to support proxy users.

The environment variable handling is slightly simplified from the
requests implementation.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-20 10:56:23 +02:00
Jussi Kukkonen
73273813f3
Merge pull request #2773 from jku/no-requests
More porting from  requests to urllib3
2025-02-20 08:51:42 +00:00
pakagronglb
94639360ec Enable FA (future annotations) linting ruleset
Signed-off-by: pakagronglb <pakagronglebel@gmail.com>
2025-02-19 19:44:05 +07:00
Jussi Kukkonen
5acd3f7df7 ngclient: Add note about RequestsFetcher being deprecated
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-15 09:57:06 +02:00
Jussi Kukkonen
6ddc0838a2 Move fetchers around to make them public
RequestsFetcher should still be public (even if deprecated).

* We don't want to import RequestsFetcher in __init__
  (because that requires importing requests)
* but we do want RequestsFetcher to be importable publicly

Move both fetchers out of _internal: that was never the right place for
them anyway: they are public modules.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-14 17:42:45 +02:00
Jussi Kukkonen
140abd34dc Update requirements: drop requests
This also removes RequestsFetcher from tuf.ngclient.__init__.py:
Otherwise we can't drop the requests dependency.

This means RequestsFetcher is not currently public.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-02-14 17:42:43 +02:00
NicholasTanz
d67f126233 remove self.app_user_agent attribute, as it's not used outside of init
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-02-05 17:49:02 -05:00
NicholasTanz
86cc7ad3ee clarify urllib3 as requirement in pyproject.toml and add back in requestsFetcher as option.
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-30 21:29:08 -05:00
NicholasTanz
a48fca51f9 add retry error handling to _chunks()
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-09 23:56:06 -05:00
NicholasTanz
2aed81f019 change error handling to MaxRetryError in _fetch()
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-09 23:31:50 -05:00
NicholasTanz
21280302e7 utilize one pool manager
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-09 20:59:56 -05:00
NicholasTanz
18e42cea52 replacing RequestsFecther with Urllib3Fetcher in .rst
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-06 02:55:15 -05:00
NicholasTanz
031778fd8d more linting stuff
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-06 02:47:51 -05:00
NicholasTanz
20d825f041 fix line too long linting error
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-06 02:31:20 -05:00
NicholasTanz
0675f0ce3a create urllib3 fetcher, replace requestsFetcher with urllibFetcher in ngclient, replace requestsFecther with urllibFetcher in requestsFetcher unit tests.
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2025-01-06 02:17:09 -05:00
Jussi Kukkonen
1d81a04707 Use __future.annotations module
This allows using some more nice annotations from 3.10
while still being compatible with even Python 3.8.

These are all annotation changes, should not modify any functionality.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-11-29 12:41:09 +02:00
NicholasTanz
5c71f4f062 update python annotations
Signed-off-by: NicholasTanz <nicholastanzillo@gmail.com>
2024-11-03 23:21:23 -05:00
Jussi Kukkonen
6eaf405bd5 ngclient: Increase default max_root_rotations
this configuration variable controls how many root versions
the client will upgrade in a single refresh(). The idea is to prevent
a malicious repository from filling the disk with root versions.

We want a number that is high enough that a repository should not have
made that many roots in the time that clients take to update the "embedded"
root that the client shipped with ship with.

32 is small enough that a repository could reach it while clients with
v1 embedded in them are still in use. Let's bump to 256: this should be
plenty.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-07-17 11:50:43 +03:00
Kapil Sharma
83974c7cab
removing linting contraint arise from noqa:SLF001
Signed-off-by: Kapil Sharma <ks3913688@gmail.com>
2024-05-28 17:45:12 +05:30
h4l0gen
e63ba54906
made variable public
Signed-off-by: h4l0gen <ks3913688@gmail.com>
2024-05-28 17:35:14 +05:30
h4l0gen
d2f9f6dee6
changing useragent
Signed-off-by: h4l0gen <ks3913688@gmail.com>
2024-05-20 21:55:02 +05:30
Jussi Kukkonen
716cde704b lint: Remove unneeded noqa flags
Re-raising a blind exception is now ok.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-05-07 11:11:08 +02:00
Jussi Kukkonen
dc5194e9e9 lint: Enable flake8-self
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-05-06 09:19:19 +03:00
Jussi Kukkonen
460424a620 lint: Enable flake8-raise
I'm not sure I agree with not using the parens in
   raise SomeError
but being consistent is definitely better than not being consistent.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-05-06 09:19:19 +03:00
Jussi Kukkonen
38f309bbbf WIP: Update to new securesystemslib API
* API changes covered:
  * keys and interface modules removed
  * SSlibSigner removed
  * CryptoSigner added: this replaces the removed functionality
  * DSSE "signatures" container type changed
* Currently pins a securesystemslib main branch commit:
  this shoudl be reverted before merging, when securesystemslib
  has made a release
* tests/generated_data/generate_md.py was simplified
* Encrypted test keys in tests/repository_data/keystore were replaced
  with the unencrypted PEM versions of the same keys
* The public test keys in tests/repository_data/keystore were removed
  as they were not used anymore

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-04-25 14:27:54 +03:00
Jussi Kukkonen
f165c76fc2
Merge pull request #2613 from NicholasTanz/enableRulesets
Enable rulesets (BLE and RUF)
2024-04-22 16:19:55 +03:00
E3E
52601e2bd8 add RUF and BLE rulesets; ignore some broad exceptions (BLE001) and RUF012
Signed-off-by: E3E <ntanzill@purdue.edu>
2024-04-21 01:37:40 -04:00
Jussi Kukkonen
fe2068697c Support app-specific user-agents
* application user-agent can be set with UpdaterConfig object
* Setting will affect the default fetcher only
* the application user-agent will be prefixed to the ngclient
  default user-agent

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-04-19 17:55:29 +03:00
Jussi Kukkonen
009e1ddbf4 lint: Enable more ruff ulesets
Minor fixes were needed, the only possibly interesting one is
the one in RequestsFetcher (use "yield from").

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-04-02 11:02:48 +03:00
Jussi Kukkonen
1512653995 linter: Enable more rules
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-03-19 16:30:52 +02:00
Lukas Pühringer
e1b52e7fd8
Merge pull request #2580 from theupdateframework/dependabot/pip/test-and-lint-dependencies-c17666fe2e
build(deps): bump the test-and-lint-dependencies group with 1 update
2024-03-12 09:40:40 +01:00
Jussi Kukkonen
289950a17c updater: Minor improvements to error handling
In RequestsFetcher it makes sense to raise "from e" but in
updater the error we raise is not related to the original
error: use "from None".

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-03-08 15:00:42 +02:00
Jussi Kukkonen
884cae9660 lint: Update formatting for ruff 3.x
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-03-07 10:05:36 +02:00
Nurgaleev_Mansur_908
53ac05d5d3 change utcnow() to now(timezone.utc)
Signed-off-by: Nurgaleev_Mansur_908 <mansur0158@gmail.com>
2024-02-27 21:57:46 +03:00
Lukas Pühringer
1e8820930e
Update tuf/ngclient/_internal/trusted_metadata_set.py
Co-authored-by: Jussi Kukkonen <jku@goto.fi>
Signed-off-by: Lukas Pühringer <luk.puehringer@gmail.com>
2024-02-22 13:43:44 +01:00
Lukas Puehringer
8eb6f08a5c ngclient: replace internal wrapping interface
The internal wrapping interface to case handle deserialization and
verification of traditional metadata vs. simple envelopes inside
trusted metadata set might be a more complicated solution than
necessary.

This removes the interface and instead adds the methods of the interface
implementations as helpers to trusted metadata set, and updates it to
to call one or the other function based on the envelope type
configuration flag.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2024-02-21 17:01:13 +01:00
Lukas Puehringer
1897f9a652 ngclient: change envelope type config to flag
The flag allows adding other envelope types in the future (unlikely),
or parallel support (`METADATA & SIMPLE`) without breaking the API.

Internally, the flag is now just passed on to TrustedMetadataSet as
mandatory parameter. (Optional parameters make less sense when we
control all the invocations.)

This change requires updating all invocations of TrustedMetadataSet,
including the duplication of a test function.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2024-02-21 16:59:55 +01:00