Commit graph

6224 commits

Author SHA1 Message Date
Lukas Pühringer
efc6877adf
Merge pull request #1829 from lukpueh/add-logo-rtd
doc: render tuf logo and favicon on rtd
2022-02-08 09:34:58 +01:00
Jussi Kukkonen
a622276fee build: Remove pylintrc from MANIFEST
pylint config lives in pyproject.toml nowadays.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-08 09:34:28 +02:00
Jussi Kukkonen
891e1c76e8 build: Re-add setup.py
The Python build tools are fine without a setup.py but Dependabot
chokes: https://github.com/dependabot/dependabot-core/issues/4483

Add a setup.py to keep Dependabot happy.

Fixes #1828

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-08 09:12:28 +02:00
Jussi Kukkonen
a21036eaa1
Merge pull request #1830 from jku/pin-actions-hashes 2022-02-07 19:09:02 +02:00
Martin Vrachev
6db3f69b61 Add small missing tests
Add a test triggering the MetaFile version validation and a TargetFile
test accessing custom.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2022-02-07 17:07:48 +02:00
Jussi Kukkonen
ba911e07b2
Merge pull request #1825 from MVrachev/specification-version
Update supported specification version to 1.0.28
2022-02-07 16:44:26 +02:00
Jussi Kukkonen
92e49ad2a1 github: Pin actions hashes
This allows us to control when our workflows change.
Dependabot should now open PRs when the actions update.

This still leaves the actual OS image as a variable but Github does not
support pinning that: we'd have to start using our own containers (and
installing our own pythons, etc) to do that -- not worth the trouble.

Fixes #1826

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-07 15:32:23 +02:00
Martin Vrachev
5b2290cf42 Exceptions docs for __init__ and from_dict()
Document ValueError, KeyError and TypeError exceptions for __init__ and
from_dict() methods in Metadata API.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2022-02-07 15:19:03 +02:00
Lukas Puehringer
cf9af047a5 doc: render tuf logo and favicon on rtd
Configure docs to display
- tuf icon as favicon
- tuf horizontal logo (white) in navbar

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2022-02-07 12:47:44 +01:00
Lukas Pühringer
70c7358040
Merge pull request #1800 from jku/document-serialization-hash-issue
Metadata API: Document serialization "repro" issue
2022-02-07 11:28:02 +01:00
Lukas Pühringer
8a0bb880be
Merge pull request #1827 from jku/do-dependabots-job-for-dependabot
Do dependabots job
2022-02-07 09:56:30 +01:00
Lukas Pühringer
1c46eebfdc
Merge pull request #1824 from jku/revert-pip-cache-disable
Revert "github: disable pip caching temporarily"
2022-02-07 09:38:21 +01:00
Jussi Kukkonen
3f3b921337 Metadata API: Document serialization "repro" issue
It's not obvious to casual reader that reading metadata and then
writing it might not always produce the same file. It's also not
immediately obvious why this matters.

Document both concepts.

Fixes #1392

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-06 18:06:11 +02:00
dependabot[bot]
cd95ff06dc build(deps): bump cryptography from 35.0.0 to 36.0.1
Bumps [cryptography](https://github.com/pyca/cryptography) from 35.0.0 to 36.0.1.
- [Release notes](https://github.com/pyca/cryptography/releases)
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/35.0.0...36.0.1)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-05 18:34:29 +02:00
dependabot[bot]
e2ce7e884b build(deps): bump charset-normalizer from 2.0.7 to 2.0.11
Bumps [charset-normalizer](https://github.com/ousret/charset_normalizer) from 2.0.7 to 2.0.11.
- [Release notes](https://github.com/ousret/charset_normalizer/releases)
- [Changelog](https://github.com/Ousret/charset_normalizer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ousret/charset_normalizer/compare/2.0.7...2.0.11)

---
updated-dependencies:
- dependency-name: charset-normalizer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-05 18:33:04 +02:00
dependabot[bot]
59064d5a72 build(deps): bump urllib3 from 1.26.7 to 1.26.8
Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.7 to 1.26.8.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.7...1.26.8)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-05 18:32:08 +02:00
dependabot[bot]
a03381f193 build(deps): bump requests from 2.26.0 to 2.27.1
Bumps [requests](https://github.com/psf/requests) from 2.26.0 to 2.27.1.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.26.0...v2.27.1)

---
updated-dependencies:
- dependency-name: requests
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-05 18:31:25 +02:00
dependabot[bot]
7d4858817d build(deps): bump pynacl from 1.4.0 to 1.5.0
Bumps [pynacl](https://github.com/pyca/pynacl) from 1.4.0 to 1.5.0.
- [Release notes](https://github.com/pyca/pynacl/releases)
- [Changelog](https://github.com/pyca/pynacl/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/pynacl/compare/1.4.0...1.5.0)

---
updated-dependencies:
- dependency-name: pynacl
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-05 18:31:11 +02:00
Jussi Kukkonen
1a59b292f4 Revert "github: disable pip caching temporarily"
This reverts commit 55d6cb47da.

According to changelog setup-python v2.3.2 should include a workaround
for the issue.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-05 12:30:31 +02:00
Martin Vrachev
99a4d71b72 Update supported specification version to 1.0.28
I looked into all changes between our current version 1.0.19 and the
current version of the specification 1.0.28 and I agree with Jussi that
the only one not fully resolved is:
"8dafd00 (tag: v1.0.24) Clarify optional attributes" and more precisely
the changes from commit:
4dd279bc31

It doesn't make sense to have a target file without "paths" or
"path_hash_prefixes", so our `python-tuf requirement to have at least
one of them set makes sense.

Both with Jussi we agreed that we can easily loosen this requirement if
when solving https://github.com/theupdateframework/specification/issues/200
it's decided that both of them can be omitted,
but for now, we decided it's better to stick to our current requirement
to have one of them set.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2022-02-04 15:38:34 +02:00
lukpueh
31fd8d4f03
Merge pull request #1790 from lukpueh/rm-all
Rm all legacy
2022-02-04 14:01:53 +01:00
lukpueh
c25065fa74
Merge pull request #1823 from jku/fix-badge-link
README: Fix CI badge
2022-02-04 11:13:38 +01:00
Jussi Kukkonen
a8a7337504
Merge pull request #1810 from jku/fetcher-error-cleanup
ngclient: Make DownloadErrors more consistent
2022-02-04 11:30:21 +02:00
Jussi Kukkonen
537ce12c2b README: Fix CI badge
The URL was referring to an older workflow name which lead to a badge
that no longer updates based on CI results. This commit also changes to
using "actions/workflows/FILENAME/badge.svg" as current documentation
suggests.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-04 11:09:10 +02:00
Jussi Kukkonen
e6f363273f ngclient: Small refactor, avoid isinstance
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-04 11:03:12 +02:00
Jussi Kukkonen
6b079eefec ngclient: Add missing f to an f-string
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-04 11:03:12 +02:00
Jussi Kukkonen
6718620d60 fetcher: docstring fix
Make the dosctring match the similar argument in download_bytes()

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-04 11:03:12 +02:00
Jussi Kukkonen
17f2ddff02 exceptions: rename FetcherHTTPError
I've not supported many renames but I'm suggesting this one:
FetcherHTTPError was created because we needed to signal 403/404
from the fetcher to updater. At that time the download error hierarchy
in general was not thought out.

Now we have a couple of different errors all derived from
DownloadError. I believe it does not make sense to point out "Fetcher"
in one of their names: DownloadHTTPError makes it clearer this is a
specific type of DownloadError.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-04 11:03:12 +02:00
Jussi Kukkonen
4efd9496dc ngclient: Make DownloadErrors consistent
Fetcher interface should only raise DownloadErrors,
regardless of the implementation.
 * Make sure fetch() wraps non-DownloadError errors in a DownloadError
 * Make the abstract function private _fetch()
 * Try to be more consistent in doscstrings

This now makes the example client more sensible (when server does not
respond):
    $ ./client_example.py download qwerty
    ...
    Failed to download target qwerty: Failed to download url http://127.0.0.1:8000/metadata/2.root.json

(here the latter part of the error string comes from DownloadError
raised by FetcherInterface.fetch())

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-04 11:03:12 +02:00
lukpueh
67e2b24a6c
Merge pull request #1821 from jku/disable-pip-cache
github: disable pip caching temporarily
2022-02-04 09:51:16 +01:00
Jussi Kukkonen
55d6cb47da github: disable pip caching temporarily
setup-python fails on Windows currently
(https://github.com/actions/virtual-environments/issues/5009)
Disable caching to workaround the failure.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-04 09:19:25 +02:00
lukpueh
215073e250
Merge pull request #1766 from jku/tests-sim-add-key-rotation
tests: Refactor key rotation in simulator
2022-02-03 13:29:23 +01:00
lukpueh
397af1a945
Merge pull request #1803 from jku/maintainer-permissions-reminder
CI: Add yearly reminder issue to review maintainers
2022-02-03 12:59:42 +01:00
lukpueh
735368a705
Merge pull request #1805 from jku/remove-ngclient-todos
ngclient: Remove obsolete TODOs
2022-02-03 11:02:24 +01:00
lukpueh
f0043eea61
Merge pull request #1794 from jku/remove-todos
Remove TODOs about too-many-arguments
2022-02-03 10:48:48 +01:00
Lukas Puehringer
9816c40d77 doc: update deprecation info in repo_example docs
Rephrase deprecation info in repo_example modules doc headers
to reflect that the deprecation has happened.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2022-02-03 10:00:22 +01:00
Jussi Kukkonen
6c1dfce03f
Merge pull request #1814 from MVrachev/fix-black-linting 2022-02-01 13:10:51 +02:00
Martin Vrachev
a2f7b4588f Fix new space warning by black
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2022-02-01 12:29:50 +02:00
Jussi Kukkonen
d3ca8fd2fe ngclient: Remove obsolete TODOs
First TODO is now handled (or we have PRs already).
Second TODO is moved to issue #1804.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-01-28 09:53:03 +02:00
lukpueh
cb7bd6aff9
Merge pull request #1734 from MVrachev/securesystemslib-exceptions
Metadata API: Avoid raising securesystemslib exceptions
2022-01-27 16:58:10 +01:00
Martin Vrachev
9533c3f974 Metadata API: add exception tests
Add missing tests testing raising documented
exceptions for "Metadata.sign()",
"Metadata.to_file()" and "Metadata.from_file()".

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2022-01-27 17:34:00 +02:00
Martin Vrachev
896e552fd7 sign sslib.exceptions -> UnsignedMetadataError
Catch Metadata.sign() securesystemslib exceptions and instead throw
a more general UnsignedMetadataError exception.
We don't want to expose securesystemslib exceptions and it's better
to replace them with a more general exception that could be easily
handled.

As the signer is an argument implementing securesystemslib.signer.Signer
interface we don't know what exception will it throw.
That's why we need to catch all possible exceptions during signing and
raise UnsignedMetadataError.
That is the same reason why we should move the serialization outside
the "try" block, so a tuf.api.serialization.SerializationError can
propagate and warn the user that 'signed' cannot be serialized.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2022-01-27 17:31:02 +02:00
Martin Vrachev
b5fbfed194 Reexport sslib StorageError and change docs
Reexport securesystemslib StorageError, so that our users can catch
it without importing securesystemslib.
The securesystemslib StorageError makes sense in the context of
metadata API, because it supports different storage interfaces and
this exception is denoting all possible errors that could arrise
from using any kind of storage interface.

Additionally, I changed the places where we mention that StorageError
is thrown, so that our users will know they can directly import it
from tuf/api/exceptions.py instead of importing securesystemslib.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2022-01-27 17:05:57 +02:00
Martin Vrachev
0666520e62 Fix type annotation in test_api
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2022-01-27 17:05:57 +02:00
lukpueh
4de561728e
Merge pull request #1799 from MVrachev/rm-persist_temp_file
ngclient: simplify storing a downloaded file
2022-01-27 15:52:47 +01:00
Martin Vrachev
3fa0668f89 ngclient: simplify storing a downloaded file
Replace the usage of securesystemslib.util.persist_temp_file() with
shutil.copyfileobj() as file system abstraction is not used in the
client.
This way we prevent securesystemslib.exception.StorageError from
leaking through client API calls.

Note: with those changes we are no longer do fsync.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2022-01-27 16:15:11 +02:00
lukpueh
d95ead6280
Merge pull request #1787 from MVrachev/ngclient-exceptions
Review and document ngclient exceptions
2022-01-27 14:28:13 +01:00
Jussi Kukkonen
b0524e53dc CI: Add yearly reminder issue to review maintainers
This is easy to forget:
 * there are multiple different critical services
 * some permissions are not visible to everyone

but review is important as every maintainer account increases attack surface.
So let's remind ourselves once a year.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-01-27 15:03:37 +02:00
Martin Vrachev
9f7866db0a Additions to the client exceptions
I made a review on all files inside tuf/ngclient to see which of them
needs additions or changes in their function docstrings regarding
exceptions.

I didn't find any changes required inside the request_fetcher.py
and of course inside the config module.
Other than that multiple additions had to be made.

For trusted_metadata_set we had a discussion with Jussi that there is
no need to list each of the specific RepositoryErrors one by one as
this is an internal module and this will only create a bigger
maintenance burden.

For updater.py we had discussions with Jussi and Lukas that we want to
document only those exceptions that could be potentially handled.
This means there is no point in documenting each of the RepositoryErrors
or DownloadErrors separately.

Finally, I added a little documentation for download_bytes() inside
fetcher.py, as it's naming, suggests it's not an internal function.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2022-01-27 14:39:24 +02:00
Lukas Puehringer
ad6d2cad8a Update misc repo metadata after legacy code drop
- Update linter config to no longer distinguish between legacy
and new implementation. This requires addressing a linter warning
in an until now not linted module (tuf/__init__.py).

- Remove obsolete rules in MANIFEST.in (source distribution) and
tests/.coveragerc (test coverage).

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2022-01-26 15:25:40 +01:00