The URL was referring to an older workflow name which lead to a badge
that no longer updates based on CI results. This commit also changes to
using "actions/workflows/FILENAME/badge.svg" as current documentation
suggests.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
I've not supported many renames but I'm suggesting this one:
FetcherHTTPError was created because we needed to signal 403/404
from the fetcher to updater. At that time the download error hierarchy
in general was not thought out.
Now we have a couple of different errors all derived from
DownloadError. I believe it does not make sense to point out "Fetcher"
in one of their names: DownloadHTTPError makes it clearer this is a
specific type of DownloadError.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Fetcher interface should only raise DownloadErrors,
regardless of the implementation.
* Make sure fetch() wraps non-DownloadError errors in a DownloadError
* Make the abstract function private _fetch()
* Try to be more consistent in doscstrings
This now makes the example client more sensible (when server does not
respond):
$ ./client_example.py download qwerty
...
Failed to download target qwerty: Failed to download url http://127.0.0.1:8000/metadata/2.root.json
(here the latter part of the error string comes from DownloadError
raised by FetcherInterface.fetch())
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Rephrase deprecation info in repo_example modules doc headers
to reflect that the deprecation has happened.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Catch Metadata.sign() securesystemslib exceptions and instead throw
a more general UnsignedMetadataError exception.
We don't want to expose securesystemslib exceptions and it's better
to replace them with a more general exception that could be easily
handled.
As the signer is an argument implementing securesystemslib.signer.Signer
interface we don't know what exception will it throw.
That's why we need to catch all possible exceptions during signing and
raise UnsignedMetadataError.
That is the same reason why we should move the serialization outside
the "try" block, so a tuf.api.serialization.SerializationError can
propagate and warn the user that 'signed' cannot be serialized.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Reexport securesystemslib StorageError, so that our users can catch
it without importing securesystemslib.
The securesystemslib StorageError makes sense in the context of
metadata API, because it supports different storage interfaces and
this exception is denoting all possible errors that could arrise
from using any kind of storage interface.
Additionally, I changed the places where we mention that StorageError
is thrown, so that our users will know they can directly import it
from tuf/api/exceptions.py instead of importing securesystemslib.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Replace the usage of securesystemslib.util.persist_temp_file() with
shutil.copyfileobj() as file system abstraction is not used in the
client.
This way we prevent securesystemslib.exception.StorageError from
leaking through client API calls.
Note: with those changes we are no longer do fsync.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
This is easy to forget:
* there are multiple different critical services
* some permissions are not visible to everyone
but review is important as every maintainer account increases attack surface.
So let's remind ourselves once a year.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
I made a review on all files inside tuf/ngclient to see which of them
needs additions or changes in their function docstrings regarding
exceptions.
I didn't find any changes required inside the request_fetcher.py
and of course inside the config module.
Other than that multiple additions had to be made.
For trusted_metadata_set we had a discussion with Jussi that there is
no need to list each of the specific RepositoryErrors one by one as
this is an internal module and this will only create a bigger
maintenance burden.
For updater.py we had discussions with Jussi and Lukas that we want to
document only those exceptions that could be potentially handled.
This means there is no point in documenting each of the RepositoryErrors
or DownloadErrors separately.
Finally, I added a little documentation for download_bytes() inside
fetcher.py, as it's naming, suggests it's not an internal function.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
- Update linter config to no longer distinguish between legacy
and new implementation. This requires addressing a linter warning
in an until now not linted module (tuf/__init__.py).
- Remove obsolete rules in MANIFEST.in (source distribution) and
tests/.coveragerc (test coverage).
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Remove legacy client, repository/developer tool, command line
tools, and underlying libraries and utilities.
See docs/1.0.0-ANNOUNCEMENT.md for details about their replacement,
deprecation strategy and migration instructions.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Remove tests for legacy client, repository/developer tool and
command line tools, which will be removed in subsequent commits.
This commits also removes obsolete test tooling:
- Regarding simple_https_server + test certificates -- http/https
is no longer handled by tuf client directly but transparently by
the underlying requests module used by the default fetcher
implementation.
- For details about unittest_toolbox see #1792
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Old tests are not touched as they will be removed in subsequent
commits, along with the custom log module.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Remove metadata generation scripts based on legacy
repository/developer tools, which will be removed in subsequent
commits.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Remove documentation for legacy client, repository/developer tool
and command line tools, which will be removed in subsequent
commits.
See #1797 and #1798 for replacing ATTACKS.md and QUICKSTART.md.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
This commit tries to deal with two interests:
* metadata is highly repetitive and compressible: allowing compression
would be good
* there may be broken web servers (see
404838abcc/src/pip/_internal/download.py (L842))
that have problems with compression on already compressed target files
We can make things better for that first interest while we have no real
data for the second interest -- our current workarounds to avoid
compression are based on hearsay, not testing.
Now that individual fetchers are possible I suggest we simplify
ngclient and allow compression. As an example the pip Fetcher
could still use the pip response chunking code with all their
workarounds -- pip certainly has better capability to maintain
a mountain of workarounds and also has endless amounts of real-world
testing compared to python-tuf.
Details:
* Stop modifying Accept-Encoding (Requests default includes gzip)
* Don't use response.raw in RequestsFetcher as there is no need:
This was a workaround for false "Content-encoding: gzip" inserted by
a broken server -- and the workaround was only possible because we
knew we never asked for compression
* Fix issue in test_session_get_timeout(): it's not mocking the error
that requests really raises in this case
Fixes#1251
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
This commit simplifies the contributor's doc.
It adds the reference to the Secure Systems Lab Development Guidelines,
gives more evidence to the tox usage, shares information about the tests,
linting, and coverage, and creates a session about submitting
the contributions highlighted by the DCO.
Fixes#1709
Signed-off-by: Kairo de Araujo <kdearaujo@vmware.com>
The lint warning about argument count is useful in general but in these
two cases we want to break the rule: remove TODOs.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Update new test modules to stop using unittest_toolbox, in
preparation for its removal in #1790.
The tools provided by unittest_toolbox can easily (in a more
obvious way) be replaced by using the standard library modules
`tempfile` and `random` (no more used) directly.
In the case of tempdir and -file creation/removal, skipping the use
of unittest_toolbox, which does this by default, also uncovers some
test cleanup failures, which would occur when temporary test
directories were removed while a test server hadn't released them.
(see `except OSError: pass` in unittest_toolbox's `tearDown`
method)
**Change details**
**test_fetcher_ng.py:**
- Stop implicitly creating (setUp) and removing (tearDown) tmp test
dirs. -Move now manual creation of an exemplary targets file to
setUpClass, as the same file is used by all tests. And remove it
explicitly in tearDownClass after killing the server (see note
about failure above). - Trigger URL parsing error with a hardcoded
invalid URL string instead of a random string.
**test_updater_ng.py**
- Stop implicitly creating (setUp) and removing (tearDown) tmp test
dirs.
- Explicitly create tmp test dirs in setUp, but don't remove
them in tearDown to avoid above mentioned failures. They will be
removed all at once when removing the tmp root test dir in
tearDownClass
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Define TESTS_DIR constant in tests/util.py as full path to the
parent directory of the util module. This may be used to reliably
read other files in tests dir, such es "repository_data" or
"simple_server", regardless of cwd.
This commit also replaces a couple of `getcwd() + "filename"` with
`TESTS_DIR + filename`, so that in the future (post #1790) we
should be able to invoke the tests from anywhere, not only from
within the tests directory as is now the case.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
We no longer need to catch LengthOrHashMismatchError and reraise a
RepositoryError as LengthOrHashMismatchError is changed to inherit
RepositoryError.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
