Commit graph

6224 commits

Author SHA1 Message Date
dependabot-preview[bot]
73241f4376
build(deps): bump urllib3 from 1.25.10 to 1.25.11
Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.25.10 to 1.25.11.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/master/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.25.10...1.25.11)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-20 10:24:11 +00:00
Martin Vrachev
3c4bc77cd7 Add test if confined_target_dirs is ['']
Even though we don't want to promote the usage of [''] as a value
for confined_target_dirs, it's good to test against because we
don't want to introduce a breaking change for our users.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2020-10-16 16:10:05 +03:00
Martin Vrachev
946d11bca4 Make confined_target_dirs optional field
The field confined_target_dirs from the MIRROR_SCHEMA  is
a list of strings. Those strings define the accessible target
paths for that mirror. For one target to be available for that mirror,
its path should have as a prefix at least one of the strings defined
in confined_target_dirs.

That's why when confined_target_dirs is a list with one element empty
string (e.g. ['']) this means all targets files on that mirror are
available and if confined_target_dirs is empty list (e.g. []) this
would be interpreted as none of the target files is available.

This is a confusing API that could easily lead to mistakes.
That's why it's better we promote to not set confined_target_dirs
at all if a user wants targets to be available.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2020-10-16 15:35:24 +03:00
lukpueh
a64a334cd4
Merge pull request #1177 from joshuagl/ww/tuf-api-package
tuf/api: Expose tuf.api as a package (take 2)
2020-10-15 17:55:00 +02:00
Lukas Puehringer
54963c43fc ci: skip fossa and coverage for lint jobs
Authored-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-10-15 15:50:12 +01:00
Joshua Lock
a4d851de14 Switch sslib master env to Python 3.8
Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-10-15 14:35:14 +01:00
Joshua Lock
d36d3194a7 Move linting to a separate tox env
We don't need to lint the code with every version of Python, instead add
an extra tox env which lints once with the latest supported Python version

Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-10-15 14:35:14 +01:00
Joshua Lock
a53d4ec475 Disable an instance of too-many-arguments error
The Targets constructor takes seven arguments, which violates pylints
default value of five for max-arguments:

R0913: Too many arguments (7/5) (too-many-arguments)

As this feels like a coding style decision that should be made and
documented disable that test for only the Targets constructor until
a coding style decision has been made and documented as a decision
record.

Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-10-15 14:35:14 +01:00
Joshua Lock
f205e98851 Remove else after raise usage in api.metadata
Using an else after a raise results in a refactor message from pylint:

R1720: Unnecessary "elif" after "raise" (no-else-raise)

This is because the raise will exit the block, and pylint suggests that
explicit if's, rather than an if-elif-else, are clearer style. Update the
style of Metadata.verify() to match pylint expectations.

Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-10-15 14:35:14 +01:00
Joshua Lock
fea52b54ee Remove use of single letter variable
A single letter variable name of 'f' causes pylint to throw a coding style
convention warning:

C0103: Variable name "f" doesn't conform to snake_case naming style
(invalid-name)

Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-10-15 14:35:14 +01:00
Joshua Lock
3877667ff4 Remove unused import from tuf.api.metadata
The logging module is not used in metadata, therefore remove it

Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-10-15 14:35:14 +01:00
Joshua Lock
f91ce395e5 Add minimal pylintrc for new code in tuf/api
Add a minimal pylintrc to lint for new code being developed in tuf/api and
update the tox configuration to ignore tuf/api with the default pylintrc
and run an extra invocation of pylint for just the modules in tuf/api.

Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-10-15 14:35:14 +01:00
William Woodruff
05b1609786 tuf/api: Expose tuf.api as a package
Signed-off-by: William Woodruff <william@trailofbits.com>
2020-10-15 14:35:14 +01:00
Joshua Lock
ba7c6922c0
Merge pull request #1152 from MVrachev/fix-pr-template
Fix automatic issue closing
2020-10-15 10:39:04 +01:00
Joshua Lock
f4520317b1
Merge pull request #1163 from joshuagl/joshuagl/nits
Update release process and mark unit tests as executable
2020-10-15 10:38:26 +01:00
Joshua Lock
1d9c6ac63b
Merge pull request #1153 from jku/allow-no-metadata-or-targets-in-client-config
mirrors: Make targets_path and metadata_path optional
2020-10-15 10:37:09 +01:00
Joshua Lock
b57aa5857b
Merge pull request #1173 from jku/add-missing-local-repository-error
Add MissingLocalRepositoryError
2020-10-15 10:35:39 +01:00
Joshua Lock
a7b74de454
Merge pull request #1170 from jku/filter-warnings
tests: Filter useless warnings
2020-10-14 15:54:05 +01:00
lukpueh
c3923613eb
Merge pull request #1176 from jku/remove-iso8601
Remove iso8601 dependency
2020-10-14 10:01:33 +02:00
Jussi Kukkonen
2f69986e2b Remove iso8601 dependency
Our 'expires' strings are constrained by the ISO8601_DATETIME_SCHEMA
which matches regex '\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z'. This can be
parsed with just a datetime.strptime(): iso8601 module is not needed.

* Add formats.expiry_string_to_datetime() helper function
* Modify the 3 locations that used iso8601 and the api/metadata.py usage
  of datetime.strptime()
* Remove related unnecessary logger setup
* Add the missing exception documentation to relevant functions (in many
  cases the exception is rather unlikely as the schema has been verified
  many times before this though...)

Fixes #1065

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2020-10-13 20:06:14 +03:00
Jussi Kukkonen
f8606d9645 sig: Remove iso8601 logger mentions
sig does not use iso8601: no need to silence its logging

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2020-10-13 12:11:37 +03:00
Jussi Kukkonen
7d73958a2a Add MissingLocalRepositoryError
This allows clients to separate
a) missing local repository and
b) error while loading local repository

This is fully backwards-compliant: MissingLocalRepositoryError derives
from RepositoryError and every situation that now results in
MissingLocalRepositoryError used to result in a RepositoryError.

Fixes #1063

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2020-10-12 15:23:15 +03:00
Jussi Kukkonen
060ef1dfd5 tests: Filter useless warnings
Filter out:
 * DeprecationWarnings for updater module when we are on purpose
   testing deprecated methods from updater
 * SubjectAltNameWarning for connections to our test server

These warnings are visible with e.g.
  python3 test_updater.py

The large change in test_download.py is just indentation into with-block.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2020-10-09 12:10:49 +03:00
Martin Vrachev
a4d6329b23 Fix automatic issue closing
This pr aims to fix the issue that there are occurrences
when a pr is merged, but the issues that it mentions
that it fixes are not automatically closed by GitHub.

According to their documentation:
https://docs.github.com/en/free-pro-team@latest/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword
"Fixes" is a keyword that should fix this.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2020-10-07 13:11:12 +03:00
Trishank Karthik Kuppusamy
baae9bd1a2
Merge pull request #1162 from jku/1160-Add-LICENSE-to-MANIFEST-in
MANIFEST.in: Include LICENSE files
2020-10-05 14:38:07 -04:00
Joshua Lock
0db58df8aa Update release docs
* Strongly encourage the reader to use Python 3 for creating release
  artefacts
* Recommend cleaning the tuf directory before creating the release

Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-10-05 11:22:08 +01:00
Jussi Kukkonen
f3ab08d73f MANIFEST.in: Include LICENSE files
This seems to be a common way to handle license files. Also, vendoring
tool fails to handle tuf without a LICENSE file.

Fixes #1160

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2020-10-05 13:13:58 +03:00
Joshua Lock
38a86393c2 Make unit test files executable
It's convenient to be able to run unit test scripts directly, rather than
having to pass them as arguments to Python. This is already possible for
several of our unit tests, make it possible for all by setting the execute
bit.

Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-10-05 10:36:31 +01:00
Joshua Lock
d51dbb8639
Merge pull request #1100 from sechkova/issue-1050
Raise an error on loading/writing unsigned metadata
2020-10-02 14:37:01 +01:00
Teodora Sechkova
bebf19ac20
Update test_updater_root_rotation_integration
Modify test_root_rotation_missing_keys to not use an empty signing
keys list.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-10-02 16:05:12 +03:00
Teodora Sechkova
27388833e8
Update test_repository_lib and test_formats
Modify test cases which use unsigned metadata.
Update test_sign_metadata to check for empty key list.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-10-02 16:05:12 +03:00
Teodora Sechkova
e0086d64a5
Log warning on UnsignedMetadataError in repository_lib
Except the UnsignedMetadataError generated by
check_signable_object_format() and log a warning.

When creating metadata objects on the repository side of TUF,
a use case may exist where  metadata is generated unsigned on
one machine and signed on another.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-10-02 16:05:11 +03:00
Teodora Sechkova
d3c612bb53
Raise error on empty 'signatures' list
Modify  check_signable_object_format() to raise an error
if signable has an empty 'signatures' list.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-10-02 16:05:07 +03:00
Joshua Lock
461663f363
Merge pull request #1151 from MVrachev/testing-doc
Add doc for the possible cmd args while testing
2020-10-01 21:58:11 +01:00
Martin Vrachev
cce6b7bb59 Add doc for the possible cmd args while testing
This information could be useful for newcomers
to better understand how to debug TUF.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2020-10-01 15:20:59 +03:00
Joshua Lock
fb2eaa5f23
Merge pull request #1104 from MVrachev/silence-log-with-temp-files
Log subproceses stdout and stderr in temp files
2020-10-01 12:40:23 +01:00
Martin Vrachev
e2ccfdb213 Reorder the tearDownClass cleanup
Fixes an issue where rmtree tries to access and consequently remove
a temp folder where the server has opened a file already.
This results in error:
"PermissionError: [WinError 32] The process cannot access the file
because it is being used by another process"

For reference read:
https://github.com/theupdateframework/tuf/issues/1119

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2020-10-01 12:22:19 +03:00
Martin Vrachev
2693620ee8 Make TimeoutError message more comprehensive
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2020-10-01 12:22:19 +03:00
Martin Vrachev
7f885d2160 Remove redundant comments from test classes
I don't see a need to leave a comment about what setupClass,
tearDownClass, setup and tearDown functions do.
There is documentation that describes that.
Additionally, the links referenced in the comments are from
Python 2 is deprecated.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2020-10-01 12:22:19 +03:00
Martin Vrachev
02c67d1980 Remove a not used function
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2020-10-01 12:22:19 +03:00
Martin Vrachev
5b44dd8808 Remove unneceserry checks in server files
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2020-10-01 12:22:19 +03:00
Martin Vrachev
6f02646408 Log subproceses stdout and stderr in temp files
Logging the stdout and stderr from the test subprocesses into
temporary files clean the console from unnecessary messages from
the server-side such as "code 404, message File not found" or
"GET" queries.

I have decided to create TestServerProcess class that will handle
the server subprocess creation and redirection to a temporary file
object. That way that code can be reused in more than 10 files.

Also, I have cleaned some parts of the unit test to make them more
readable and efficient with the new abstraction.

The unit tests are executed in sequential order and that's why
we can reuse one temporary file object for multiple tests.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2020-10-01 12:22:08 +03:00
lukpueh
e06e8e1afc
Merge pull request #1078 from joshuagl/joshuagl/existing-fileinfo
Minor error message clarifications and greater testing for generating targets metadata with existing fileinfo
2020-09-29 17:19:24 +02:00
Jussi Kukkonen
7a94f7856a mirrors: Make targets_path and metadata_path optional
Now clients can leave out targets_path or metadata_path if the
client knows the mirror does not have that type of targets.

This is backwards compatible: old mirror configs continue to work.

Fixes #1079

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2020-09-25 10:01:44 +03:00
Martin Vrachev
45d031957f Remove redundant "url" class member from tests
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2020-09-23 17:17:41 +03:00
Joshua Lock
fdccb8dc0b
Merge pull request #1149 from theupdateframework/dependabot/pip/cryptography-3.1.1
build(deps): bump cryptography from 3.1 to 3.1.1
2020-09-23 13:00:55 +01:00
dependabot-preview[bot]
ec90e1cd75
build(deps): bump cryptography from 3.1 to 3.1.1
Bumps [cryptography](https://github.com/pyca/cryptography) from 3.1 to 3.1.1.
- [Release notes](https://github.com/pyca/cryptography/releases)
- [Changelog](https://github.com/pyca/cryptography/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/3.1...3.1.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-09-23 10:30:35 +00:00
Joshua Lock
505f3545d5
Merge pull request #1145 from jku/silence-errors-to-be-raised
Silence errors to be raised
2020-09-22 13:42:27 +01:00
Joshua Lock
687506fa61
Merge pull request #1147 from jku/close-all-the-things
Close file objects, requests.Responses
2020-09-22 13:03:21 +01:00
Jussi Kukkonen
fb9d8e7d64 Download: Fix documentation
* Remove dead urls: trust reader to find Requests documentation on
  their own
* Mention that "stream" means that downloading the body is deferred

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2020-09-22 14:22:46 +03:00