Commit graph

35 commits

Author SHA1 Message Date
Jussi Kukkonen
7d57ab65d2 workflows: Simplify testing
* Don't try to handle sslib main test within the matrix
* Put it in a separate workflow
* Include the new workflow in CI but not in CD
* Bonus: Make cache-dependency-path more complete

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-04-16 10:32:08 +03:00
Jussi Kukkonen
53521bfda0 workflows: Set top-level permissions
This changes very little but it does mean any jobs added in future have to
be explicit about the permissions they need. This also makes OSSF scorecard
happier.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2022-10-30 12:56:22 +02:00
Lukas Puehringer
db471a5fd5 Refactor ci/cd workflows
Prior to this change, ci triggered cd, depending on the event that
triggered ci. Due to the vague information about that event
available to cd, the workflow pipeline was a bit brittle.

This change disassociates ci and cd workflows to allow for an
independent configuration of trigger events.

The test jobs, which used to be defined in ci, are now in a
separate workflow file _test.yml that can be included in both ci
and cd workflows.

**Changes in ci**
- Only defines trigger events and permissions, the "meat" of ci is
  defined in the called _test.yml now.
- No longer triggers on tag pushes, this was only needed for cd.

**Changes in cd**
- Now triggers directly on tag pushes instead of (cd)-workflow_run.
- Calls _test.yml, and require successful run before build/release.
  (`needs: test` replaces `if: ...`)
- Changes variable names about pushed tag that triggered the event.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2022-04-26 10:36:58 +02:00
Lukas Puehringer
38b774e0eb Refactor ci/cd workflows (WIP)
This is an intermediate commit for easier review. See subsequent
commit for details.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2022-04-26 10:36:58 +02:00
dependabot[bot]
68fd8a1cc6
build(deps): bump actions/checkout from 3.0.0 to 3.0.2
Bumps [actions/checkout](https://github.com/actions/checkout) from 3.0.0 to 3.0.2.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...2541b1294d2704b0964813337f33b291d3f8596b)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-04-22 10:19:38 +00:00
Lukas Pühringer
72424a958b
Merge pull request #1946 from lukpueh/auto-release
Add GH workflow to build and release on GH and PyPI
2022-04-21 13:03:25 +02:00
Lukas Puehringer
b99d0432a7 build: minor updates in CI/CD workflow files
- polish code comments
- wrap long lines

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2022-04-20 16:02:25 +02:00
dependabot[bot]
4d54629293
build(deps): bump actions/setup-python from 3.1.1 to 3.1.2
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 3.1.1 to 3.1.2.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](21c0493ecf...98f2ad02fd)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-04-20 06:58:22 +00:00
dependabot[bot]
65d1b87a2f
build(deps): bump actions/checkout from 3.0.0 to 3.0.1
Bumps [actions/checkout](https://github.com/actions/checkout) from 3.0.0 to 3.0.1.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](a12a3943b4...dcd71f6466)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-04-15 10:16:40 +00:00
dependabot[bot]
156e535dcf
build(deps): bump actions/setup-python from 3.1.0 to 3.1.1
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 3.1.0 to 3.1.1.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](9c644ca2ab...21c0493ecf)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-04-07 10:19:18 +00:00
Lukas Puehringer
a1a71c11a1 build: update CI/CD workflow to run in series
- Change CI workflow to also run on push to (release) tag
- Change CD workflow to run on successful CI run, and only if a
  (release) tag push triggered the CI

NOTE: Unfortunately the setup is not very robust
      (see code comment in cd.yml)

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2022-04-07 12:15:39 +02:00
dependabot[bot]
b0a73e41c6
build(deps): bump actions/setup-python from 3.0.0 to 3.1.0
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 3.0.0 to 3.1.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](0ebf233433...9c644ca2ab)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-04-04 10:21:57 +00:00
dependabot[bot]
38b5e07f62
build(deps): bump actions/checkout from 2.4.0 to 3
Bumps [actions/checkout](https://github.com/actions/checkout) from 2.4.0 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](ec3a7ce113...a12a3943b4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-03-02 10:21:30 +00:00
dependabot[bot]
311120a192
build(deps): bump actions/setup-python from 2.3.2 to 3
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2.3.2 to 3.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](7f80679172...0ebf233433)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-03-01 10:21:10 +00:00
Jussi Kukkonen
92e49ad2a1 github: Pin actions hashes
This allows us to control when our workflows change.
Dependabot should now open PRs when the actions update.

This still leaves the actual OS image as a variable but Github does not
support pinning that: we'd have to start using our own containers (and
installing our own pythons, etc) to do that -- not worth the trouble.

Fixes #1826

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-07 15:32:23 +02:00
Jussi Kukkonen
1a59b292f4 Revert "github: disable pip caching temporarily"
This reverts commit 55d6cb47da.

According to changelog setup-python v2.3.2 should include a workaround
for the issue.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-05 12:30:31 +02:00
Jussi Kukkonen
55d6cb47da github: disable pip caching temporarily
setup-python fails on Windows currently
(https://github.com/actions/virtual-environments/issues/5009)
Disable caching to workaround the failure.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-04 09:19:25 +02:00
Martin Vrachev
0f59f4b749 Drop support for python version 3.6
Python version 3.6 was supported until December 23-rd 2021 meaning its
end of life has expired before more than 20 days.
Dropping support for python version 3.6 will allow us to remove
OrderedDicts.

After a quick check I saw that Warehouse target python version 3.8.2:
- their docker file: https://github.com/pypa/warehouse/blob/main/Dockerfile#L47
- https://github.com/pypa/warehouse/blob/main/.python-version
- last pr updating pr version: https://github.com/pypa/warehouse/pull/7828
Pip supports python version 3.7+ as well. They dropped python 3.6 a
couple of months ago:
https://github.com/pypa/pip/pull/10641

This means it shouldn't cause headache to our users if we drop python
version 3.6 too.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2022-01-19 17:11:18 +02:00
Jussi Kukkonen
f7006f5df0 CI: Use builtin package cache support
actions/setup-python now supports pip cache: use that instead of
handling cache locations manually.

Cache invalidates when any requirements file changes (same as before):
this is a bit over cautious but probably harder to break.

Fixes #1692

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-12-09 13:03:18 +02:00
Jussi Kukkonen
6744f6a9c7
Merge pull request #1652 from jku/limit-github-token-visibility
GH actions: limit GitHub token visibility
2021-11-17 10:06:31 +02:00
Jussi Kukkonen
e073fea819 github: explicitly set workflow permissions
* current workflow only needs to read git content
* if the workflow in the future does need write access, it's good to
  see permissions explicitly changing

For context: "pull_request" runs never have write access anyway, so this
significantly changes only the "push" runs that happen when branches are
merged to develop.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-11-04 11:39:05 +02:00
Jussi Kukkonen
15e84dfb2e GH actions: limit GitHub token visibility
Token should be visible to only the code that actually needs it.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-11-01 09:47:50 +02:00
Martin Vrachev
2e94e39275 Use quotes for python version for github workflows
Fix GitHub workflow failures by using quotes for python versions.
It seems that adding `3.10` as a number is transformed then to `3.1`
which as a result is translated to Python version 3.1 instead of Python
version 3.10.
This seems to work for other projects as well:
https://github.com/MasoniteFramework/masonite4/blob/master/.github/workflows/pythontest.yml
https://github.com/python-pillow/Pillow/blob/main/.github/workflows/test-windows.yml
https://github.com/PyGithub/PyGithub/blob/master/.github/workflows/ci.yml

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-10-21 14:32:05 +03:00
Martin Vrachev
6ff852ad0f Add support for python 3.10
Python 3.10 is released on October 4-th 2021 and it seems
logical to add support for it as it doesn't require any major effort
from the project.

For reference read:
https://www.python.org/downloads/release/python-3100/

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-10-21 14:32:04 +03:00
Jussi Kukkonen
65fc968b7f CI: Do not require coveralls-fin to succeed
We already do not require individual build uploads to succeed: let's
also not require the final step to succeed.

The immediate context for this is that coveralls has been down for
three days now.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-09-20 12:51:32 +03:00
Jussi Kukkonen
4110a1cf9c GitHub workflow: Use Python 3.x for lint
pylint 2.7 supports Python 3.9. This issue might reappear with next
Python release but let's deal with that if it happens.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-09-17 19:34:08 +03:00
Jussi Kukkonen
b59679c374 GitHub CI workflow: tweak names
Currently the github UI dropdown for checks looks useless since
checks are named "Run TUF tests and...".

Tweak the workflow and job names to hopefully fit the actual
step name in the UI.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-09-17 19:34:08 +03:00
samuelgregorovic
112575d987 updated docs and comments
Updated/removed documented commands and comments which were referencing Python2. Also updated links to documentation referencing Python2 docs (unchanged where needed)

Signed-off-by: Samuel Gregorovic <samuelgregorovic@gmail.com>
Signed-off-by: samuelgregorovic <samuelgregorovic@gmail.com>
2021-07-14 10:37:08 +03:00
Teodora Sechkova
785350b28e CI: Allow failure when publishing on coveralls
A failure during publishing of the coverage results
on coveralls should not fail the whole build job.
Allow the step to fail.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-04-29 15:31:56 +03:00
Joshua Lock
16bd3c2358 Remove Python 2.7 from GitHub CI configuration
- Drop Python 2.7 from GitHub Actions workflows. Note: There is likely
  additional cleanup that can be done to the workflow now we no longer
  care about supporting Python 2.7.
- No longer tell dependabot to ignore idna updates.

Signed-off-by: Joshua Lock <jlock@vmware.com>
2021-03-03 09:37:21 +00:00
Jussi Kukkonen
7c5416d5c3 CI: Limit build-on-push to develop branch only
Dependabot pushes to main repository and ends up triggering two builds
every time (one for PR, one for push): limit the rule for build-on-push
to apply to develop branch only.

If release branches are used later on they should be added to list here.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-02-16 15:40:35 +02:00
Lukas Puehringer
d97c2872db Re-add coveralls.io badge
A recent commit reinstates publishing of coverage data to
coveralls.io. This commit re-adds the corresponding badge which
was temporarily removed in #1242.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-01-13 13:45:03 +01:00
Lukas Puehringer
8bb0187a69 Use py3 coveralls to publish coverage on py2
Prior to this commit our GitHub workflow would set up one Python
version only for each build, which means that the commands to run
the tests and publish coverage (tox and coveralls) were run with
the same Python version as tox runs the tests in.

Given that the coveralls CLI tool dropped py2 a couple of releases
ago, this commit sets up an additional service py3 to run coveralls
(and tox) on when building for py2.

To prevent tox from using the wrong Python version to run the tests
on, this commit changes the toxenv value from the generic 'py'
(uses default python on path) to 'py27'.

For convenience and readability we use the environment variable
TOXENV instead of the tox -e option.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-01-13 13:43:46 +01:00
Lukas Puehringer
32452c4142 Configure GitHub workflow to publish coverage
Re-add coverage publishing on coveralls.io, formerly performed by
Travis CI (prior to #1242), using the coveralls cli tool according
to the documentation:
https://coveralls-python.readthedocs.io/en/latest/usage/configuration.html#github-actions-gotcha

**Considered alternatives:**
- Official coveralls GitHub action, which does not seem to work
  well for Python:
  https://github.com/coverallsapp/github-action/issues/4
  https://github.com/coverallsapp/github-action/issues/30

- Inofficial fork of that action, which seems to work better
  but had issues finding the coverage data in the tests folder,
  or the covered code respectively.
  https://github.com/AndreMiras/coveralls-python-action

Besides aforementioned issues of these actions the use of cli tools
from curated package managers seems slightly preferable over
actions from the GitHub Marketplace (see #1246).

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-01-13 13:43:39 +01:00
Lukas Puehringer
36b8d43bd8 Add basic GitHub workflow to run tests and linters
Configure workflow to run all tox environments, where each 'py' env
runs on linux, macos and windows, and sslib master and lint builds
run only Linux/Python3.x only.

The workflow also configures pip caching.

TODO: Adopt publishing of coverage (coveralls) and license (fossa)
data from .travis.yml.

Co-authored-by: Jussi Kukkonen <jkukkonen@vmware.com>
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2020-12-17 10:28:31 +01:00