Commit graph

4110 commits

Author SHA1 Message Date
dependabot[bot]
fd515dfd3e
build(deps): bump cryptography from 3.4.6 to 3.4.7
Bumps [cryptography](https://github.com/pyca/cryptography) from 3.4.6 to 3.4.7.
- [Release notes](https://github.com/pyca/cryptography/releases)
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/3.4.6...3.4.7)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-26 10:04:15 +00:00
Joshua Lock
1d234798d6
Merge pull request #1321 from sechkova/gitattr
Add .gitattributes file
2021-03-25 10:20:44 +00:00
lukpueh
974de44ce9
Merge pull request #1323 from MVrachev/fix-version-comment
New API: Fix exception message for version
2021-03-24 15:07:16 +01:00
lukpueh
2eca15e59b
Merge pull request #1322 from MVrachev/extra-version-setup
Remove additional version settup in Signed
2021-03-24 14:55:35 +01:00
Martin Vrachev
cbc814ffa8 New API: Fix exception message for version
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-03-24 15:55:29 +02:00
Martin Vrachev
901b7f4491 Remove additional version settup in Signed
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-03-24 15:38:11 +02:00
Teodora Sechkova
98ddbdd397
Add .gitattributes file
For compatibility with Windows systems, declare repository_data
files to always have LF line endings on checkout.

A trailing "/**" matches everything inside, with infinite depth.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-03-23 18:16:32 +02:00
lukpueh
b5c8ba007e
Merge pull request #1261 from jku/vendoring-compatible-imports
Vendoring-compatible imports
2021-03-19 17:22:15 +01:00
Jussi Kukkonen
ab56344a53 metadata: Make isort happy and bundle imports
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 17:10:45 +02:00
Jussi Kukkonen
30ab838575 Make requests_fetcher import vendoring compatible
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 17:01:45 +02:00
Jussi Kukkonen
7dcfb12f61 requests_fetcher: Move 'tuf' import from download
requests_fetcher uses tuf.__version__ for user-agent, move the import to
the correct file.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
d0e5bd2311 imports: Fix urllib3 exception import
Make it compatible with vendoring: import the exception only to avoid
having to rename the module locally.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
d5b6f91f6f imports: Remove unused imports
The linter now understands our imports (yay), and complains a lot (boo):
* Remove really unused imports
* disable lints for tuf.log and securesystemslib imports: these imports
  have logging side-effects (they set default loggers for tuf and
  securesystemslib respectively) and I'm cautious about just removing
  them

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
0aabb82a80 imports: Move six imports to 3rd party section
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
538623b6eb imports: Make 'keydb' imports vendoring-compatible
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.

Also fix import orders so tuf internal imports are last.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
996b2a0f91 imports: Fix securesystemslib.settings imports
Make the import compatible with vendoring tool and alias the import so
it does not clash with the local module. Fix all references
to the module in the code.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
dd134a43c8 imports: Fix securesystemslib.hash imports
Make them compatible with vendoring, use
  from securesystemslib import hash as sslib_hash
to have the same style as other securesystemslib imports (and to avoid
potential conflict with system hash()).

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
4b66c173d8 imports: Fix securesystemslib.storage imports
Make them compatible with vendoring, use
  from securesystemslib import storage as sslib_storage
to have the same style as other securesystemslib imports.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
46ebfd0461 imports: Fix securesystemslib.util imports
Make the import compatible with vendoring tool and alias the import so
it does not clash with the local module. Fix all references
to the module in the code.

In one instance import a specific function to avoid a more complex
redirection in the code.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
a7c2b8d7d8 imports: Fix securesystemslib.interface imports
Make them compatible with vendoring, use
  from securesystemslib import interface as sslib_interface
to have the same style as other securesystemslib imports.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
f702fdfd0c imports: Fix securesystemslib.keys imports
Make them compatible with vendoring, use
  from securesystemslib import keys as sslib_keys
to have the same style as other securesystemslib imports.

Note that developer_tool already used a
   from securesystemslib.keys import ...
for some functions so that style was used consistently there.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
79385cc10f imports: Fix securesystemslib.formats imports
Make the import compatible with vendoring tool and alias the import so
it does not clash with the local module. Fix all references
to the module in the code.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
fe3daccdb5 imports: Make 'updater' import vendoring-compatible
Make the updater imports compatible with vendoring tool by importing
the Updater class directly (don't import the whole module to avoid the
clash with the obvious variable name 'updater').

Also update the example: This is not required in the clients but tuf
source code will be vendored and this import line (even though in a
comment) might trigger an error in future vendoring tool releases.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
6faed27e0a imports: Fix securesystemslib.exceptions imports
Make the import compatible with vendoring tool and alias the import so
it does not clash with the local module. Fix all references
to the module in the code.

Remove a related repo.py comment that was badly duplicated from module
docstring.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
9d7047ffc9 imports: Make 'sig' imports vendoring-compatible
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.

Remove one unused import.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
c66c61f1d0 imports: Make 'settings' imports vendoring-compatible
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
9550b1470d imports: Make 'roledb' imports vendoring-compatible
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
02046c0f49 imports: Make 'mirrors' imports vendoring compatible
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
696b92902e updater: rename mirrors variables
Make sure mirrors is not used as variable name (so it can be used for
the module import name later).

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
07b3aed03c imports: Make 'log' imports vendoring-compatible
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:50 +02:00
Jussi Kukkonen
4575637efd imports: Make 'formats' imports vendoring-compatible
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:56:47 +02:00
Jussi Kukkonen
4b078b0975 imports: Make 'exceptions' imports vendoring-compatible
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:54:39 +02:00
Jussi Kukkonen
8dc07ccd9a imports: Make 'download' imports vendoring-compatible
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-03-19 16:53:42 +02:00
lukpueh
c2b1f0e0aa
Merge pull request #1314 from lukpueh/update-api-linting
Update linter and add auto-formatter config
2021-03-19 15:40:16 +01:00
Lukas Puehringer
f9bf52f8cd Add contributor instructions for auto-formatters
Add cli snippet to run black and isort on the command line and
pointers to editor and pre-commit configuration to
docs/CONTRIBUTORS.rst.

Also add .pre-commit-config.yaml to .gitignore for independent
pre-commit configuration.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-17 14:13:43 +01:00
Lukas Puehringer
38ef45f542 Revert "Add basic pre-commit config..."
This reverts commit "Add basic pre-commit configuration for
tuf/api/*" (44aea45fd3) in order to
reduce maintenance burdern:

- pre-commit really is a package manager, thus the packages (git
hooks) pulled in via pre-commit would need to be kept up-to-date
and securely so (sic!).

- pre-commit requires contributors to opt-in via "pre-commit
install" regardless, so we might as well ask contributors to add
and tend to the corresponding configuration file on their own.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-17 12:19:57 +01:00
Lukas Puehringer
be0cef067c Manually standardize quotes where black cannot
Black standardizes single to double quotes where feasible.
However, it doesn't seem to change double to single quotes nor adds
escape characters, as a consequence it skips standardization on
strings with mixed quotes.

Unfortunately, pylint's quote consistency check also doesn't detect
this, so the onus will remain on the reviewer in these cases.

**Unrelated changes**:
The commit still enables pylint's "check-quote-consistency" just in
case it can detect something the black doesn't.

The commit also fixes a syntax inconsistency in pylintrc.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-17 11:57:45 +01:00
Joshua Lock
28f96bfc80
Merge pull request #1289 from jku/ci-remove-pip-workaround
ci: Remove workaround for pip resolver issue
2021-03-17 10:09:07 +00:00
Joshua Lock
abd30d9006
Merge pull request #1315 from theupdateframework/dependabot/pip/urllib3-1.26.4
Bump urllib3 from 1.26.3 to 1.26.4
2021-03-16 12:16:02 +00:00
dependabot[bot]
9af247fb02
Bump urllib3 from 1.26.3 to 1.26.4
Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.3 to 1.26.4.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/1.26.4/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.3...1.26.4)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-16 10:04:47 +00:00
Lukas Puehringer
44aea45fd3 Add basic pre-commit configuration for tuf/api/*
Add optional pre-commit configuration to install and run
auto-formatters when committing new code to tuf/api/*.
Auto-formatters include:
- trailing-whitespace
- end-of-file-fixer
- black
- isort

This commit also adds pre-commit to the dev dependencies
and updates the contributor instructions accordingly.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-12 18:41:24 +01:00
Lukas Puehringer
5a626ca4b4 Require black and isort in tuf/api/* via tox
Configure lint build in tox.ini to check if code in tuf/api/* is
formatted according to black and isort style rules:
https://black.readthedocs.io/en/stable/the_black_code_style.html
https://pycqa.github.io/isort/

In addition to our new style guide (#1128) and corresponding linter
configuration, requiring auto-formatting should help to further
reduce reviewing effort. The auto-formatter black was chosen for
the following reasons:
- It seems to be the most popular formatter in the Python ecosystem
- It is well documented including integration instructions with
  most of the tools we use (git, GitHub Actions, pylint, a range of
  editors, pyproject.toml #1161)
- It checks that the reformatted code produces a valid AST that is
  equivalent to the original
- It has almost no ways of customization, which means no
  customization effort required, and more (cross-project) style
  uniformity, lowering contribution barriers
- It converts single to double quotes, where reasonable, which is
  exactly what we recommend
- The style choices it makes seem generally reasonable and don't
 conflict with our style guide, except for favoring hanging over
 aligned indentation, which is the opposite of what we recommend.
 But we are willing to update the adapt our style guide.

Auto-format pre-commit configuration will be added in a subsequent
commit.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-12 18:41:13 +01:00
Lukas Puehringer
4648fbfadd Update tuf/api/pylintrc for new code
The updated pylintrc is based on the Google Python Style Guide
pylint configuration at
https://google.github.io/styleguide/pylintrc with the following
differences:
- We don't list defaults which are applied anyway.
- We don't configure checks that seem unrelated to the code style
  guide.
- We don't disable any checks that are not in conflict with the
  current code or code style guide.

This has the advantage of a minimal configuration file which should
be easy to maintain and extend as required, e.g. if conflicting
code is added, or linting time becomes too long, due to unnecessary
checks.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-12 12:27:59 +01:00
Lukas Puehringer
42a797b4a1 Re-format tuf/api/* using black and isort
Use black and isort to reformat new code in tuf/api/*, like so:

```
black --line-length 80 api
isort --line-length 80 --profile black api
```

Besides downsizing the default line length to fit our Code Style
Guide no extra configuration is required.

Unified format according to black and isort will be enforced by
CI/CD in a future commit.

**Changes include:**
- Use double quotes instead of single quotes where feasible
- Re-wrap and re-indent long lines such as dict literals, function
  signatures and function calls, using hanging indent
  This will require an update in our Code Style Guide, which the
  benefits of using black seem worth.
  https://github.com/secure-systems-lab/code-style-guidelines/blob/master/python.md#indentation-and-line-continuation
- Update vertical and horizontal spacing
- Sort and wrap imports

See black and isort docs for details:
https://black.readthedocs.io/en/stable/the_black_code_style.html
https://pycqa.github.io/isort/docs/configuration/black_compatibility/

NOTE: If desired I can split commits by change and/or configure git
for this repo to ignore the corresponding revision(s) in git-blame.
https://github.com/psf/black#migrating-your-code-style-without-ruining-git-blame

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-12 11:51:55 +01:00
Jussi Kukkonen
2c360d0538
Merge pull request #1302 from sechkova/unclosed_socked
Fix unclosed socket warning caused by RequestsFetcher
2021-03-12 10:17:41 +02:00
lukpueh
477abc8151
Merge pull request #1298 from timgates42/bugfix_typo_exception
docs: fix simple typo, excepion -> exception
2021-03-11 13:39:44 +01:00
Teodora Sechkova
237e74b122
Fix unclosed socket warning caused by RequestsFetcher
Close the Response object when an HTTP error is raised.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-03-11 11:33:00 +02:00
Tim Gates
f0b8b1e18c
docs: fix simple typo, excepion -> exception
There is a small typo in tests/test_roledb.py.

Closes #1262

Signed-off-by: Tim Gates <tim.gates@iress.com>
2021-03-11 20:16:57 +11:00
lukpueh
65005cfe11
Merge pull request #1272 from MVrachev/signer-interface
Make new api compatible with the Signing interface
2021-03-10 15:46:20 +01:00
Martin Vrachev
49aa0fc167 Make new API compatible with the Signing interface
In the securesystemslib pr https://github.com/secure-systems-lab/securesystemslib/pull/319
I added a new Signer interface with the purpose of supporting multiple
signing implementations.
Additionally, I added the SSlibSigner implementation of that interface
which implements the signing operation for rsa, ed25519 and ecdsa
schemes.
With this commit, I integrate the SSlibSigner into the new API in tuf.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-03-10 10:57:45 +01:00