Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Use "from tuf import <module>" instead of "import tuf.<module>": this
makes it possible for vendoring tool to vendor tuf. Fix all references
to <module> in the code.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Add cli snippet to run black and isort on the command line and
pointers to editor and pre-commit configuration to
docs/CONTRIBUTORS.rst.
Also add .pre-commit-config.yaml to .gitignore for independent
pre-commit configuration.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
This reverts commit "Add basic pre-commit configuration for
tuf/api/*" (44aea45fd3) in order to
reduce maintenance burdern:
- pre-commit really is a package manager, thus the packages (git
hooks) pulled in via pre-commit would need to be kept up-to-date
and securely so (sic!).
- pre-commit requires contributors to opt-in via "pre-commit
install" regardless, so we might as well ask contributors to add
and tend to the corresponding configuration file on their own.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Black standardizes single to double quotes where feasible.
However, it doesn't seem to change double to single quotes nor adds
escape characters, as a consequence it skips standardization on
strings with mixed quotes.
Unfortunately, pylint's quote consistency check also doesn't detect
this, so the onus will remain on the reviewer in these cases.
**Unrelated changes**:
The commit still enables pylint's "check-quote-consistency" just in
case it can detect something the black doesn't.
The commit also fixes a syntax inconsistency in pylintrc.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Add optional pre-commit configuration to install and run
auto-formatters when committing new code to tuf/api/*.
Auto-formatters include:
- trailing-whitespace
- end-of-file-fixer
- black
- isort
This commit also adds pre-commit to the dev dependencies
and updates the contributor instructions accordingly.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Configure lint build in tox.ini to check if code in tuf/api/* is
formatted according to black and isort style rules:
https://black.readthedocs.io/en/stable/the_black_code_style.htmlhttps://pycqa.github.io/isort/
In addition to our new style guide (#1128) and corresponding linter
configuration, requiring auto-formatting should help to further
reduce reviewing effort. The auto-formatter black was chosen for
the following reasons:
- It seems to be the most popular formatter in the Python ecosystem
- It is well documented including integration instructions with
most of the tools we use (git, GitHub Actions, pylint, a range of
editors, pyproject.toml #1161)
- It checks that the reformatted code produces a valid AST that is
equivalent to the original
- It has almost no ways of customization, which means no
customization effort required, and more (cross-project) style
uniformity, lowering contribution barriers
- It converts single to double quotes, where reasonable, which is
exactly what we recommend
- The style choices it makes seem generally reasonable and don't
conflict with our style guide, except for favoring hanging over
aligned indentation, which is the opposite of what we recommend.
But we are willing to update the adapt our style guide.
Auto-format pre-commit configuration will be added in a subsequent
commit.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
The updated pylintrc is based on the Google Python Style Guide
pylint configuration at
https://google.github.io/styleguide/pylintrc with the following
differences:
- We don't list defaults which are applied anyway.
- We don't configure checks that seem unrelated to the code style
guide.
- We don't disable any checks that are not in conflict with the
current code or code style guide.
This has the advantage of a minimal configuration file which should
be easy to maintain and extend as required, e.g. if conflicting
code is added, or linting time becomes too long, due to unnecessary
checks.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
In the securesystemslib pr https://github.com/secure-systems-lab/securesystemslib/pull/319
I added a new Signer interface with the purpose of supporting multiple
signing implementations.
Additionally, I added the SSlibSigner implementation of that interface
which implements the signing operation for rsa, ed25519 and ecdsa
schemes.
With this commit, I integrate the SSlibSigner into the new API in tuf.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Call an instance method and a static method that are only defined
in a parent class from child instances using self (instance) and
cls (static) instead of super().
While this doesn't make a practical difference, the new syntax is
probably less confusing to the reader.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
- Make class docstrings wording consistent.
- Emphasize that we use the OLPC Canonical JSON specification.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
tuf.api is not designed for Python 2 compatibility. This commit
removes the following stray compatibility constructs in its
serialization subpackage:
- '__metaclass__ = abc.ABCMeta'
- six.raise_from
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Prior to this commit the (abstract) 'Signed' base class implemented
from/to_dict methods, to be used by any subclass in addition to
or instead of a custom from/to_dict method. The design led to some
confusion, especially in 'Signed.from_dict' factories, which
instantiated subclass objects when called on a subclass, which
didn't implement its own 'from_dict' method.
This commit demystifies the design, by implementing from/to_dict
on all 'Signed' subclasses, and moving common from/to_dict tasks
to helper functions in the 'Signed' class.
The newly gained clarity and explicitness comes at the cost of
slightly more lines of code.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Clarify that the TUF metadata class model is not bound to a JSON
wireline format by:
- re-wording module, class and method docstrings and code comments
to add details about custom and default serialization and the
purpose of from/to_dict methods, and
- removing the 'JsonDict' type annotation -- instead we use
generic Mapping[str, Any] for method arguments and strict
Dict[str, Any] as return value as suggested in
https://docs.python.org/3/library/typing.html#typing.Dict
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
- Rename _dict to json_dict to avoid wrong semantics of leading
underscore. (leading underscore was initially chosen to avoid name
shadowing)
- Rename 'serializer' argument of type 'SignedSerializer' to
'signed_serializer', to distinguish from 'serializer' argument of
type 'MetadataSerializer'.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Revert an earlier commit that moved to/from_dict metadata class
model methods to a util module of the serialization sub-package.
We keep to/from_dict methods on the metadata classes because:
- It seems **idiomatic** (see e.g. 3rd-party libaries such as attrs,
pydantic, marshmallow, or built-ins that provide default or
customizable dict representation for higher-level objects).
The idiomatic choice should make usage more intuitive.
- It feels better **structured** when each method is encapsulated
within the corresponding class, which in turn should make
maintaining/modifying/extending the class model easier.
- It allows us to remove function-scope imports (see subsequent
commit).
Caveat:
Now that "the meat" of the sub-packaged JSON serializer is
implemented on the class, it might make it harder to create a
non-dict based serializer by copy-paste-amending the JSON
serializer.
However, the benefits from above seem to outweigh the disadvantage.
See option 5 of ADR0006 for further details (#1270).
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Add tuf.api.serialization.util module with functions to
convert between TUF metadata class model and the corresponding
dictionary representation. These functions replace the
corresponding to/from_dict classmethods.
Configure api/pylintrc to exempt '_type' from protected member
access warning, because the underscore prefix here is only used to
avoid name shadowing.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Re-raise all errors that happen during de/serialization as custom
De/SerializationError.
Whilelist 'e', which is idiomatic for error, in api/pylintrc, and
inline exempt broad-except, which are okay if re-raised.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Add serializer.json module with implementations to serialize and
deserialize TUF role metadata to and from the JSON wireline format
for transportation, and to serialize the 'signed' part of TUF role
metadata to the OLPC Canonical JSON format for signature generation
and verification.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Add sub-package with 3 abstract base classes to:
- serialize Metadata objects to bytes (transport)
- deserialize Metadata objects from bytes (transport)
- serialize Signed objects to bytes (signatures)
pylint notes:
- configure tox to use api/pylintrc
- configure api/pylintrc to allow classes without public methods
(default was 2)
Design considerations
---------------------
- Why not implement de/serialization on metadata classes?
-> See ADR0006.
- Why use separate classes for serialization and deserialization?
-> Some users might only need either one, e.g. client only needs
Deserializer. Maybe there are use cases where different
implementations are used to serialize and deserialize.
- Why use separate classes for Metadata- and Signed-Serialization?
-> They require different concrete types, i.e. Metadata and
Signed as parameters, and using these specific types seems to
make the interface stronger.
- Why are de/serialize methods not class/staticmethods?
-> In reality we only use classes to namespace and define a
type annotated interface, thus it would be enough to make the
methods classmethods. However, to keep the de/serialize
interface minimal, we move any custom format configuration to
the constructor. (See e.g. "compact" for JSONSerializer in
subsequent commit).
Naming considerations
---------------------
- Why de/serialize?
-> Implies byte stream as input or output to the function, which
is what our interface needs.
- Why not marshaling?
-> Synonym for serialize but implies transport, would be okay.
- Why not encoding?
-> Too abstract and too many connotations (character, a/v).
- Why not parse?
-> Too abstract and no good opposite terms (unparse, write,
dump?)
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Without this new pip will notice the conflict between
securesystemslib (git master)
securesystemslib[crypto,pynacl] >= 0.18
The former does not have the extras as it should.
Add the extras and also do not accept plain http for anything.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
While doing packaging works I noticed that
pypi file 'tuf.egg-info/requires.txt' was modified on build.
Let's sort dependencies alphabetically
to avoid lint to change sources files.
Signed-off-by: Philippe Coval <rzr@users.sf.net>
Remove the magic number, a whence value of 2 for file.seek(), and instead
use the io.SEEK_END constant from the io module.
Signed-off-by: Joshua Lock <jlock@vmware.com>
- Drop Python 2.7 from GitHub Actions workflows. Note: There is likely
additional cleanup that can be done to the workflow now we no longer
care about supporting Python 2.7.
- No longer tell dependabot to ignore idna updates.
Signed-off-by: Joshua Lock <jlock@vmware.com>
Remove references to, and handling of, Python 2.7 in our project scaffolding:
- updated python_requires in setup.py to state our intent to support
Python 3.6 and above (but not Python 4, yet)
- Drop no longer required dependencies in setup.py, and requirements-*.txt
(further refinement of requirements files will be handled in #1161)
- Remove Python 2.7 from our tox environments
Signed-off-by: Joshua Lock <jlock@vmware.com>
