Remove legacy client, repository/developer tool, command line
tools, and underlying libraries and utilities.
See docs/1.0.0-ANNOUNCEMENT.md for details about their replacement,
deprecation strategy and migration instructions.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Remove tests for legacy client, repository/developer tool and
command line tools, which will be removed in subsequent commits.
This commits also removes obsolete test tooling:
- Regarding simple_https_server + test certificates -- http/https
is no longer handled by tuf client directly but transparently by
the underlying requests module used by the default fetcher
implementation.
- For details about unittest_toolbox see #1792
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Old tests are not touched as they will be removed in subsequent
commits, along with the custom log module.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Remove metadata generation scripts based on legacy
repository/developer tools, which will be removed in subsequent
commits.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Remove documentation for legacy client, repository/developer tool
and command line tools, which will be removed in subsequent
commits.
See #1797 and #1798 for replacing ATTACKS.md and QUICKSTART.md.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Most of the test_invalid_signed_serialization subtests are currently
failing because "_type": "signed" and then the test tries to deserialize
them as Snapshot (which fails a type check).
Correct the type to "snapshot" so that we can fail in the correct places
during serialization instead.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
All TUF implementations used to use "1.0" as the spec version and most
of them have never modified that value since.
Accept two-part spec_version for legacy compatibility: it is strictly
speaking against the current spec (which requires semver) but there
should be no harm in doing this and it allows us to deserialize
metadata generated by e.g. go-tuf.
Fixes#1751
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
This commit tries to deal with two interests:
* metadata is highly repetitive and compressible: allowing compression
would be good
* there may be broken web servers (see
404838abcc/src/pip/_internal/download.py (L842))
that have problems with compression on already compressed target files
We can make things better for that first interest while we have no real
data for the second interest -- our current workarounds to avoid
compression are based on hearsay, not testing.
Now that individual fetchers are possible I suggest we simplify
ngclient and allow compression. As an example the pip Fetcher
could still use the pip response chunking code with all their
workarounds -- pip certainly has better capability to maintain
a mountain of workarounds and also has endless amounts of real-world
testing compared to python-tuf.
Details:
* Stop modifying Accept-Encoding (Requests default includes gzip)
* Don't use response.raw in RequestsFetcher as there is no need:
This was a workaround for false "Content-encoding: gzip" inserted by
a broken server -- and the workaround was only possible because we
knew we never asked for compression
* Fix issue in test_session_get_timeout(): it's not mocking the error
that requests really raises in this case
Fixes#1251
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
This commit simplifies the contributor's doc.
It adds the reference to the Secure Systems Lab Development Guidelines,
gives more evidence to the tox usage, shares information about the tests,
linting, and coverage, and creates a session about submitting
the contributions highlighted by the DCO.
Fixes#1709
Signed-off-by: Kairo de Araujo <kdearaujo@vmware.com>
The lint warning about argument count is useful in general but in these
two cases we want to break the rule: remove TODOs.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Update new test modules to stop using unittest_toolbox, in
preparation for its removal in #1790.
The tools provided by unittest_toolbox can easily (in a more
obvious way) be replaced by using the standard library modules
`tempfile` and `random` (no more used) directly.
In the case of tempdir and -file creation/removal, skipping the use
of unittest_toolbox, which does this by default, also uncovers some
test cleanup failures, which would occur when temporary test
directories were removed while a test server hadn't released them.
(see `except OSError: pass` in unittest_toolbox's `tearDown`
method)
**Change details**
**test_fetcher_ng.py:**
- Stop implicitly creating (setUp) and removing (tearDown) tmp test
dirs. -Move now manual creation of an exemplary targets file to
setUpClass, as the same file is used by all tests. And remove it
explicitly in tearDownClass after killing the server (see note
about failure above). - Trigger URL parsing error with a hardcoded
invalid URL string instead of a random string.
**test_updater_ng.py**
- Stop implicitly creating (setUp) and removing (tearDown) tmp test
dirs.
- Explicitly create tmp test dirs in setUp, but don't remove
them in tearDown to avoid above mentioned failures. They will be
removed all at once when removing the tmp root test dir in
tearDownClass
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Define TESTS_DIR constant in tests/util.py as full path to the
parent directory of the util module. This may be used to reliably
read other files in tests dir, such es "repository_data" or
"simple_server", regardless of cwd.
This commit also replaces a couple of `getcwd() + "filename"` with
`TESTS_DIR + filename`, so that in the future (post #1790) we
should be able to invoke the tests from anywhere, not only from
within the tests directory as is now the case.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
We no longer need to catch LengthOrHashMismatchError and reraise a
RepositoryError as LengthOrHashMismatchError is changed to inherit
RepositoryError.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
After we drop support for python3.6 we can relly that dictionaries
preserve the insertion order:
https://docs.python.org/3.7/whatsnew/3.7.html
This means we can replace the usage of OrderedDict with a standard
dictionaries.
Something we have to keep in mind is that even thought the insertion
order is preserved the equality comparison for normal dicts is
insensitive for normal dicts compared to OrderedDict
For example:
>>> OrderedDict([(1,1), (2,2)]) == OrderedDict([(2,2), (1,1)])
False
>>> dict([(1,1), (2,2)]) == dict([(2,2), (1,1)])
True
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
SerializationError and DeserializationError are both errors coming
from the repository side looking from the clients point of view.
That's why it makes sense to make them repository errors.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
LengthOrHashMismatchError is a thrown when there are problems with
metadata verification or problems from the repository side when looking
it from the user's perspective.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
UnsupportedAlgorithmError is a detailed securesystemslib exception
and there is no need for TUF to redefine it.
Moreover which hash "algorithms" are allowed is work for
securesystemslib not for TUF.
It's only used once inside "Targetfile.from_data()" and there it's used
to denote that there is a problem with the given argument.
That's why this error can be just replaced with "ValueError".
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
ReplayedMetadataError is a subset of
BadVersionNumberError and in a discussion with
Jussi we realized that ReplayedMetadataError can
be replaced by BadVersionNumberError with a
good message.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
URLParsingError is a specific download error and
is not clear what benefit it provides.
It's used only once in the new code and the
message says everything you need to know about
the exception.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Make FetcherHTTPError a DownloadError as the
error itself denotes an error happening during
the download process.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Stop linting tuf/exceptions.py with mypy as we are going to use
tuf/api/exceptions.py for exceptions in the new code.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Add tuf/api/exceptions.py for exceptions in the new code.
I copied the exceptions from tuf/exceptions.py with a few important
decisions:
1. I only added the exceptions that are used in the new code
2. I removed the general "Error" class as we can directly inherit
Exceptions
3. I tried grouping the exceptions by relevance
4. I removed the second argument "UnsignedMetadataError" as it's only
kept for backward compatibility and is not used
5. I tried following the new code style guidelines and linted the file
with our linters.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
The code examples now have a simple README.md. This commit simplifies
the README.md examples to the example folder instead of listing each
example.
Signed-off-by: Kairo de Araujo <kdearaujo@vmware.com>
This commit adds to the RTD the links references to source code
examples.
The examples are added to TUF ngclient Updater, Metadata and API
reference.
includes a seed for examples/README.md
Signed-off-by: Kairo de Araujo <kdearaujo@vmware.com>
This commit fixes the relatives links on README.md, adding absolute
links.
The PyPI renders the README.md as an HTML page on the landing page
of the python-tuf package. If the links are not absolute links to
the files in git repository, they will fail.
Fixes: #1739
Signed-off-by: Kairo de Araujo <kdearaujo@vmware.com>
This commit explicitly encodes role names. Mostly this encoding is already
happening in ``requests`` for what is not a URL.
The "/" in a role name will now be encoded.
Also, a slight change in the RepositorySimulator will align with the tests.
This commit partially covers issue #1634
Signed-off-by: Kairo de Araujo <kdearaujo@vmware.com>
Remove old doc/tuf-spec* documents, which are merely pointers to
the theupdateframework/specification repo (created in late 2017).
They were likely kept in place to avoid 404s of old links, but the
up-to-date TUF specification location should be discoverable enough
to get rid of the pointers.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
