* Move FetcherInterface to tuf/client/ directory: This way everything
inside that directory is clearly part of client API, and everything
outside _may_ be more of an implementation detail (settings is still
an unfortunate exception)
* Keep RequestsFetcher in tuf/ for same reasons: it's just the default
implementation, not explicitly part of client API
An even clearer division would be if we moved all the client specific
implementation details (download.py, mirrors.py, requests_fetcher.py)
to tuf/client/_internal/ but that's a larger change...
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
A custom error is required so that updater is able to special case
403 & 404 status codes.
Rewrite the test case a bit to be more readable.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Add test cases to test_fetcher and test_download that
decrease default chunk size and download data in more
than one chunk.
Small code-style improvements.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Use '>=' as the defensive version of the equality check.
Add a comment describing the need of a chunks() generator.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
On Windows (Github Actions) the lookup for 'localhost' takes 1 second.
This is because:
- Windows retries connect() with a timeout
- the machine has IPv6 and IPv4 but Testserver only binds the port on IPv4
- the test clients connect to 'localhost'
Since socketserver.TCPServer does not seem to support IPv6 before 3.8,
just replace 'localhost' with '127.0.0.1' in client-side URLs.
See #1257
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
- Update RequestsFetcher.fetch to return a generator object.
- Update _download_file to skip connection time when calculating
average download speed.
- Write chunk to temp file before exiting the fetcher loop
on error.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Initialize Updater with an external implementation of
FetcherInterface. If not provided, tuf.fetcher.RequestsFetcher
is used.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Abstract the network IO. Move the network operations from
tuf.download to the RequestsFercher class which is TUF's
implementation of the abstract FetcherInterface.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
The new class FetcherInterface defines an interface for
abstract network download which can be implemented for a
variety of network libraries and configurations.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Dependabot pushes to main repository and ends up triggering two builds
every time (one for PR, one for push): limit the rule for build-on-push
to apply to develop branch only.
If release branches are used later on they should be added to list here.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
On Python3 bump cryptography from 3.3.1 to 3.4.5.
On python2 bump from 3.3.1 to 3.3.2 (3.3-branch is the last branch
with python2 support).
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
If you follow the instructions we provide for our contributors in
docs/CONTRIBUTORS.rst your sys.path (used to search for imports)
will put securesystemlib project directory first and tuf directory
second.
This creates a problem with imports from tuf modules because we can
import the wrong file or on relative imports (as currently we
use in the tests when we import utils), the imports cannot be resolved.
If we change the installation order, then tuf directory will be the
first in the import resolution path and those problems will be fixed.
PS: I want to express my gratitude towards Jussi who helped me find
this problem.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
The Dependabot status badge no longer works, since having
migrated from stand-alone to GitHub native in #1258.
The issue is tracked upstream in dependabot/dependabot-core#1912.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
A recent commit reinstates publishing of coverage data to
coveralls.io. This commit re-adds the corresponding badge which
was temporarily removed in #1242.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Prior to this commit our GitHub workflow would set up one Python
version only for each build, which means that the commands to run
the tests and publish coverage (tox and coveralls) were run with
the same Python version as tox runs the tests in.
Given that the coveralls CLI tool dropped py2 a couple of releases
ago, this commit sets up an additional service py3 to run coveralls
(and tox) on when building for py2.
To prevent tox from using the wrong Python version to run the tests
on, this commit changes the toxenv value from the generic 'py'
(uses default python on path) to 'py27'.
For convenience and readability we use the environment variable
TOXENV instead of the tox -e option.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
New releases of the transitive (via 'requests') dependency 'idna'
break Python 2.7 builds. To fix this we configure dependabot to not
bump 'idna' in requirements-pinned.txt, which lists and
auto-updates all immediate and transitive dependencies for CI/CD
testing.
An alternative would be to add and restrict 'idna' in
'requirements.txt' but this is less preferable because
'requirements.txt' should only have direct dependencies.
For consulted dependabot config docs see:
https://docs.github.com/en/free-pro-team@latest/github/administering-a-repository/enabling-and-disabling-version-updates#disabling-dependabot-version-updates
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
The newly add GitHub workflow, which replaces Travis and Appveyor
does not support publishing coverage (coveralls) and license
(fossa) data yet (formerly done by Travis).
This commit removes the corresponding badges from README. It shall
be reverted once coveralls and fossa are re-enabled.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Replace mentions of travis/appveyor with GitHub Actions in
governance (contribution) and readme (badges) documents.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Configure workflow to run all tox environments, where each 'py' env
runs on linux, macos and windows, and sslib master and lint builds
run only Linux/Python3.x only.
The workflow also configures pip caching.
TODO: Adopt publishing of coverage (coveralls) and license (fossa)
data from .travis.yml.
Co-authored-by: Jussi Kukkonen <jkukkonen@vmware.com>
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
