Commit graph

4569 commits

Author SHA1 Message Date
Martin Vrachev
beb8087bf5 Clarify key rotations test cases by using keywords
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-05 16:46:33 +02:00
Jussi Kukkonen
8ae944ccb2
Merge pull request #1659 from MVrachev/disable-pylint-format
pylintc for new code: disable format checker
2021-11-05 16:19:22 +02:00
Martin Vrachev
b137fbcc3f plyintrc: remove redundant format section
The "FORMAT" section in pylint is no longer needed after the format
checker is disabled.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-05 15:07:39 +02:00
lukpueh
d70c3b32f2
Merge pull request #1663 from jku/gh-actions-set-permissions
github: explicitly set workflow permissions
2021-11-04 10:52:45 +01:00
Jussi Kukkonen
e073fea819 github: explicitly set workflow permissions
* current workflow only needs to read git content
* if the workflow in the future does need write access, it's good to
  see permissions explicitly changing

For context: "pull_request" runs never have write access anyway, so this
significantly changes only the "push" runs that happen when branches are
merged to develop.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-11-04 11:39:05 +02:00
Jussi Kukkonen
ce4a60eb26
Merge pull request #1658 from MVrachev/apply-linters
New implementation tests: apply black and isort automatic linting fixes
2021-11-04 09:32:56 +02:00
Martin Vrachev
a7766ac53a pylintc for new code: disable format checker
By default pylint does format checks:
https://pylint.pycqa.org/en/latest/technical_reference/features.html?highlight=format#format-checker

The problem is we also use black and isort who have format checkers as
well. This makes pylint format checks obsolete.

Also, it's possible that you would want to disable a warning and you
can end up in the situation where you will have to disable it for
two tools altogether.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-02 19:45:18 +02:00
Martin Vrachev
6fe36a00c4 Rename & simplify a couple of tests in test_api.py
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-02 19:31:46 +02:00
Martin Vrachev
2e9ef79762 Apply isort on the tests of the new code
All of the changes included are a result of applying isort
on our tests on the new code.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-02 19:31:25 +02:00
Martin Vrachev
c98b429643 Apply black on the tests of the new code
All of the changes included are a result of applying black
on our tests on the new code.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-11-02 19:30:35 +02:00
Jussi Kukkonen
589ed9e0d4
Merge pull request #1635 from jku/key-rotation-tests
tests: Add ngclient root key rotation tests
2021-10-27 19:07:06 +03:00
Jussi Kukkonen
e817473e3c tests: Add root key rotation tests
Add one test with 1 subtests for various root key rotation situations.

The test data definition format is a bit tricky but I tried to document
that in the test function docstring.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-10-27 19:01:23 +03:00
Jussi Kukkonen
ad80bd96c6 tests: Mark RepositorySimulator. create_key() static
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-10-27 18:57:12 +03:00
Jussi Kukkonen
fd40dfc094 tests: Refactor simulator signer handling
Store signers with their keyids so they are easier to remove.
The signers structure now looks like:
{
  "role1": {
    "keyidA": SSlibSigner,
    "keyidB": SSlibSigner,
  }
}

Add convenience method for adding a signer.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-10-27 18:57:08 +03:00
Jussi Kukkonen
7b8ff220b1
Merge pull request #1604 from jku/ngclient-api-polish
Ngclient api polish
2021-10-27 18:40:49 +03:00
Jussi Kukkonen
1d115b57b6
Merge pull request #1630 from MVrachev/validate-role
Metadata API: validate root role names
2021-10-27 18:39:58 +03:00
Jussi Kukkonen
6aaa1ead59 ngclient: Refactor target path generation
Also tweak the docstrings: the "caching" target_dir usage is
presented in the module doc example: there should be no need for
additional comments in the methods themselves as long as the argument
docs are readable.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-10-27 10:19:00 +03:00
Jussi Kukkonen
d519a413b0 ngclient: Rename get_one_valid_targetinfo()
This is slightly cosmetic but rename get_one_valid_targetinfo to
get_targetinfo:
* The function name is long without any reason: "one" and "valid" are
  always implicit
* shortening makes code (incl. our examples and tests) easier to read
* We're also already changing updater API (compared to legacy) so this
  alone does not break things -- it's also not a difficult "port".

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-10-27 09:55:57 +03:00
Jussi Kukkonen
9b761b8620 ngclient: Simplify caching
Remove updated_targets() as it doesn't fit the rest of the API.

In its stead add find_cached_target() which has a similar signature
as download_target(): both accept an optional local filepath as
argument and return full local filepath. In the
find_cached_target() case None is returned if the local file is not the
correct target file.

Updater constructor gets a new optional target_dir argument: This means
client can avoid giving a local filepath as an argument to
find_cached_target()/download_target() -- Updater will then generate a
filename within targets_dir.

A reasonable use pattern (when targets_dir is set in constructor):

    info = updater.get_one_valid_targetinfo("targetname")
    path = updater.find_cached_target(info)
    if path is None:
        path = updater.download_target(info)

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-10-27 09:55:57 +03:00
Jussi Kukkonen
cc9f3876c4 tests: Shorten variable names to reasonable length
Otherwise absolutely everything is split on multiple lines.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-10-27 09:55:57 +03:00
Martin Vrachev
4158272a7a Use TOP_LEVEL_ROLE_NAMES across TUF
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-10-25 15:58:56 +03:00
Martin Vrachev
9bc55ee568 Metadata API: validate root role names
Validate that root role names are 4 and that they are exactly
"root", "snapshot", "targets" and "timestamp" as described in
the spec:
https://theupdateframework.github.io/specification/latest/#root-role

Additionally, fix the valid_roots dataset, so each of the cases contains
the top metadata role names inside the roles dictionary.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-10-25 15:58:55 +03:00
Martin Vrachev
59b5498918 Add TOP_LEVEL_ROLE_NAMES constant
This constant can be used across tuf without defining it each time.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-10-25 15:32:43 +03:00
Jussi Kukkonen
2206fc917e
Merge pull request #1631 from MVrachev/fix-test-error
Test metadata files: bump expiration date and resign
2021-10-25 11:02:34 +03:00
Martin Vrachev
d3d2f57f3a Test files: bump expiration date and resign
Our newly added metadata files in the
tests/repository_data/fishy_rolenames/metadata directory have an expiry
date until "2021-10-22T11:21:56Z" and today while running the tests on
develop branch I recived this error:
ExpiredMetadataError("Metadata X expired on Fri Oct 22 11:21:56 2021")
when running the tests in tests/test_updater.py file and more precisly
the TestUpdaterRolenames.test_unusual_rolenames() test.

That's why I decided to bump the expiration date to a random time in
the future (October 22-nd 2050) and I had to resign all of the metadata
files.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-10-23 18:39:22 +03:00
Joshua Lock
6d18d53ec0
Merge pull request #1626 from joshuagl/joshuagl/build
Modernise packaging configuration
2021-10-21 20:01:55 +01:00
Joshua Lock
de1a3af019 build: more intentional about what's included in sdist
Our sdist has typically included everything from git apart from the CI
related files (.github/*, .fossa.yml, .readthedocs.yaml). Update our
MANIFEST.in and the check-manifest section of setup.cfg to be explicit
about this.

Signed-off-by: Joshua Lock <jlock@vmware.com>
2021-10-21 13:30:46 +01:00
Joshua Lock
352f6c2c56 Add vscode project directory to gitignore
Signed-off-by: Joshua Lock <jlock@vmware.com>
2021-10-21 13:30:46 +01:00
Joshua Lock
ada35c9e8f build: add pyproject.toml to list build tools
Per PEP 517 the pyproject.toml file lists tools required when building the
project with build: https://pypa-build.readthedocs.io/en/stable/index.html

Signed-off-by: Joshua Lock <jlock@vmware.com>
2021-10-21 13:30:46 +01:00
Joshua Lock
52ad17a710 build: update check-manifest options
Signed-off-by: Joshua Lock <jlock@vmware.com>
2021-10-21 13:30:46 +01:00
Joshua Lock
ac0ea24ca9 Remove references to setup.py
We now use a static setup.cfg instead, update __init__.py and
docs/RELEASE.md to point there.

Signed-off-by: Joshua Lock <jlock@vmware.com>
2021-10-21 13:30:46 +01:00
Joshua Lock
38ea974674 build: convert to static setuptools metadata
This is preferred and removes the crutch of invoking setup.py as a CLI
tool for building dists.

Signed-off-by: Joshua Lock <jlock@vmware.com>
2021-10-21 13:30:46 +01:00
Joshua Lock
d3e34acd98 build: add docs to project_urls
Add a 'Documentation' entry to project_urls pointing to our stable docs
on readthedocs.io. This will result in a 'Documentation' entry under the
'Project links' section on PyPI.

Signed-off-by: Joshua Lock <jlock@vmware.com>
2021-10-21 13:28:08 +01:00
Joshua Lock
36242adc74 build: recommend using build, not setup.py
Invoking setup.py directly is deprecated, see:
https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html

Therefore:
* remove the executable bit from setup.py's permissions
* remove the shebang entry from setup.py
* update the comments in setup.py to recommend using build to create dists
  and pip to install them

Signed-off-by: Joshua Lock <jlock@vmware.com>
2021-10-21 13:28:08 +01:00
Joshua Lock
4fd35434bd build: update MANIFEST.in to match sdist
Update the MANIFEST.in to be explicit about what we choose to ship in our
sdist. This _does not_ result in any additional files being included in
our sdist, but does remove warnings from build.

Signed-off-by: Joshua Lock <jlock@vmware.com>
2021-10-21 13:28:08 +01:00
Joshua Lock
08decea2d0 Remove unused .gitmodules directory
Signed-off-by: Joshua Lock <jlock@vmware.com>
2021-10-21 13:28:07 +01:00
Joshua Lock
2ab518b3f0 build: cleanup setup.cfg
* List license files in a new metadata section
* Remove .travis.yml from check-manifest section's ignore entry

Signed-off-by: Joshua Lock <jlock@vmware.com>
2021-10-21 13:28:07 +01:00
Joshua Lock
2ed51167f6 build: add build and release deps to requirements-dev
build, twine and wheel packages should all be installed in order to be
able to build and release python-tuf -- add those dependencies to
requirements-dev.txt

Signed-off-by: Joshua Lock <jlock@vmware.com>
2021-10-21 13:28:07 +01:00
Joshua Lock
9e113d6135 docs/RELEASE: use build for building dists
This is the PyPA recommend tool for building Python packages:
https://packaging.python.org/guides/distributing-packages-using-setuptools/

Furthermore, direct invocation of setup.py is long ago deprecated:
https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html

Signed-off-by: Joshua Lock <jlock@vmware.com>
2021-10-21 13:28:07 +01:00
Jussi Kukkonen
69eb29fc80
Merge pull request #1628 from MVrachev/add-python3.10
Add python3.10 and fix test failures on python3.10
2021-10-21 14:40:43 +03:00
Martin Vrachev
2e94e39275 Use quotes for python version for github workflows
Fix GitHub workflow failures by using quotes for python versions.
It seems that adding `3.10` as a number is transformed then to `3.1`
which as a result is translated to Python version 3.1 instead of Python
version 3.10.
This seems to work for other projects as well:
https://github.com/MasoniteFramework/masonite4/blob/master/.github/workflows/pythontest.yml
https://github.com/python-pillow/Pillow/blob/main/.github/workflows/test-windows.yml
https://github.com/PyGithub/PyGithub/blob/master/.github/workflows/ci.yml

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-10-21 14:32:05 +03:00
Martin Vrachev
6ff852ad0f Add support for python 3.10
Python 3.10 is released on October 4-th 2021 and it seems
logical to add support for it as it doesn't require any major effort
from the project.

For reference read:
https://www.python.org/downloads/release/python-3100/

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-10-21 14:32:04 +03:00
Martin Vrachev
4c81340610 Replace depricated ssl function and fix CI errors
When I tried adding support for Python3.10 we had CI errors due to
test failures: https://github.com/theupdateframework/python-tuf/pull/1610/checks?check_run_id=3861875325
The problem comes from the fact that we start a subprocess
executing simple_https_server.py, but then we fail to communicate the
message we expect from the server process to the main process actually
running the test. We expect our custom message to be the first line
printed from the server process, but instead, a deprecation warning is
printed first about the usage of ssl.wrap_socket(). Our custom message
is printed second.
As of Python 3.7 this function has been deprecated:
https://docs.python.org/3/library/ssl.html#ssl.wrap_socket and for
whatever the reason we didn't get a warning when using it before.

My fix does what is suggested in the warning and replaces the usage of
ssl.wrap_socket() by instantiating a ssl.SSLContext object and then
calling SSLContext.wrap_socket().
This removes the warning.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-10-21 14:15:29 +03:00
lukpueh
0cf6ba2258
Merge pull request #1620 from lukpueh/misc-metadata-api-docs
Add missing method args docs in metadata API
2021-10-20 15:51:07 +02:00
Jussi Kukkonen
4aef2b2ae6
Merge pull request #1611 from theupdateframework/dependabot/pip/charset-normalizer-2.0.7
build(deps): bump charset-normalizer from 2.0.6 to 2.0.7
2021-10-20 12:04:39 +03:00
dependabot[bot]
9864e8ef5e
build(deps): bump charset-normalizer from 2.0.6 to 2.0.7
Bumps [charset-normalizer](https://github.com/ousret/charset_normalizer) from 2.0.6 to 2.0.7.
- [Release notes](https://github.com/ousret/charset_normalizer/releases)
- [Commits](https://github.com/ousret/charset_normalizer/compare/2.0.6...2.0.7)

---
updated-dependencies:
- dependency-name: charset-normalizer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-20 08:40:11 +00:00
Jussi Kukkonen
6839e81edc
Merge pull request #1609 from theupdateframework/dependabot/pip/certifi-2021.10.8
build(deps): bump certifi from 2021.5.30 to 2021.10.8
2021-10-20 11:39:43 +03:00
dependabot[bot]
b642a44ce1
build(deps): bump certifi from 2021.5.30 to 2021.10.8
Bumps [certifi](https://github.com/certifi/python-certifi) from 2021.5.30 to 2021.10.8.
- [Release notes](https://github.com/certifi/python-certifi/releases)
- [Commits](https://github.com/certifi/python-certifi/compare/2021.05.30...2021.10.08)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-20 08:25:45 +00:00
Jussi Kukkonen
761349919b
Merge pull request #1621 from theupdateframework/dependabot/pip/cffi-1.15.0
build(deps): bump cffi from 1.14.6 to 1.15.0
2021-10-20 11:23:45 +03:00
Jussi Kukkonen
d19d40353d
Merge pull request #1616 from theupdateframework/dependabot/pip/idna-3.3
build(deps): bump idna from 3.2 to 3.3
2021-10-20 11:23:33 +03:00