Commit graph

31 commits

Author SHA1 Message Date
Jussi Kukkonen
88cc98420e lint fixes
Fixes for ruff 0.12
* Tweak some annotations
* Add __hash__() implementations to api classes: These really should be
  hashable
* My use of "super().__hash__()" is not very optimized but avoids some
  repetition

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-06-24 11:02:09 +03:00
pakagronglb
acd7ed08d1 Update Python shebangs to explicitly use python3
Signed-off-by: pakagronglb <pakagronglebel@gmail.com>
2025-02-19 19:44:21 +07:00
pakagronglb
94639360ec Enable FA (future annotations) linting ruleset
Signed-off-by: pakagronglb <pakagronglebel@gmail.com>
2025-02-19 19:44:05 +07:00
Jussi Kukkonen
bb62dded29 Add type check
mypy warns about this but we know that encode_canonical() cannot return
None if we don't set output_function argument.

ruff does not like assert so I added a "noqa" and a comment

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-01-30 18:20:39 +02:00
Jussi Kukkonen
5b2c041da0 lint: Fixes from new ruff
The noqa comment was added manually to avoid
  A005 Module `json` shadows a Python standard-library module

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2025-01-14 10:40:21 +02:00
Jussi Kukkonen
716cde704b lint: Remove unneeded noqa flags
Re-raising a blind exception is now ok.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>
2024-05-07 11:11:08 +02:00
E3E
52601e2bd8 add RUF and BLE rulesets; ignore some broad exceptions (BLE001) and RUF012
Signed-off-by: E3E <ntanzill@purdue.edu>
2024-04-21 01:37:40 -04:00
E3E
f156e21537 remove pylint and suppressed inline errors from pylint
Signed-off-by: E3E <ntanzill@purdue.edu>
2024-02-20 22:57:53 -05:00
Jussi Kukkonen
1d166f0b4e
Merge pull request #1876 from jku/more-details-on-verify-failure
Logging and error message improvements
2022-03-21 14:21:44 +02:00
Martin Vrachev
a17ceda4e5 Add "validation" arg in JSONSerializer
If the "validation" argument is set then when
serializing the metadata object will be validated.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2022-02-28 14:42:06 +02:00
Jussi Kukkonen
d4814e86d8 Metadata API: Add messages to serialization errors
We can't really add any details but this at least means
printing the error works.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2022-02-21 16:04:55 +02:00
Martin Vrachev
8327ea12d3 Make (De)SerializationError a RepositoryError
SerializationError and DeserializationError are both errors coming
from the repository side looking from the clients point of view.
That's why it makes sense to make them repository errors.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2022-01-19 16:52:19 +02:00
Jussi Kukkonen
8ed446c14b Metadata API: Stop annotating __init__() return value
This may have been required by a linter at some point, but isn't
anymore: Not annotating makes the documentation look better.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-09-23 17:57:19 +03:00
Jussi Kukkonen
ed520ee55d Metadata API: Improve serialization docs
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-09-23 17:52:55 +03:00
Teodora Sechkova
5d71aab9ec
Remove disable=broad-except
The pylint warning W0703:broad-except was raised only
when six was used and python 2 was still supported.

The warning is no longer raised, the exceptions are
handled/raised correctly and the disabling can be removed.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-09-01 17:41:32 +03:00
Jussi Kukkonen
1b9aec291c Metadata API: Use TYPE_CHECKING for cyclic import
The import is useful for mypy so it can check the types.
Add a pylint disable just like json.py does in the same situation.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-05-19 14:26:55 +03:00
Martin Vrachev
1712b71b55 Fix black docstring indentation errors
Black was updated from 20.8b1 to 21.4b0 requiring that one-line
docstring don't add additional space before the closing quotes.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-04-27 14:08:24 +03:00
Lukas Puehringer
42a797b4a1 Re-format tuf/api/* using black and isort
Use black and isort to reformat new code in tuf/api/*, like so:

```
black --line-length 80 api
isort --line-length 80 --profile black api
```

Besides downsizing the default line length to fit our Code Style
Guide no extra configuration is required.

Unified format according to black and isort will be enforced by
CI/CD in a future commit.

**Changes include:**
- Use double quotes instead of single quotes where feasible
- Re-wrap and re-indent long lines such as dict literals, function
  signatures and function calls, using hanging indent
  This will require an update in our Code Style Guide, which the
  benefits of using black seem worth.
  https://github.com/secure-systems-lab/code-style-guidelines/blob/master/python.md#indentation-and-line-continuation
- Update vertical and horizontal spacing
- Sort and wrap imports

See black and isort docs for details:
https://black.readthedocs.io/en/stable/the_black_code_style.html
https://pycqa.github.io/isort/docs/configuration/black_compatibility/

NOTE: If desired I can split commits by change and/or configure git
for this repo to ignore the corresponding revision(s) in git-blame.
https://github.com/psf/black#migrating-your-code-style-without-ruining-git-blame

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-12 11:51:55 +01:00
Lukas Puehringer
a53d68b91d Re-word api.serializer.json docstrings
- Make class docstrings wording consistent.
- Emphasize that we use the OLPC Canonical JSON specification.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-10 09:44:38 +01:00
Lukas Puehringer
bd94f6d8d1 Remove py2 compat from api.serialization package
tuf.api is not designed for Python 2 compatibility. This commit
removes the following stray compatibility constructs in its
serialization subpackage:

- '__metaclass__ = abc.ABCMeta'
- six.raise_from

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-10 09:44:38 +01:00
Lukas Puehringer
ab92ba257f Fix inconsistent returns in json serializers
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-10 09:44:38 +01:00
Lukas Puehringer
326d2af7c4 Fix blank lines in tuf.api as per styleguide
https://github.com/google/styleguide/blob/gh-pages/pyguide.md#35-blank-lines

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-10 09:44:38 +01:00
Lukas Puehringer
d823c8fc01 Rename a few variables in tuf.api
- Rename _dict to json_dict to avoid wrong semantics of leading
  underscore. (leading underscore was initially chosen to avoid name
  shadowing)

- Rename 'serializer' argument of type 'SignedSerializer' to
  'signed_serializer', to distinguish from 'serializer' argument of
  type 'MetadataSerializer'.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-10 09:44:30 +01:00
Lukas Puehringer
2b4085718b Re-word serialization cyclic import code comments
- Try to clarify purpose and remove unimportant TODO note
- Use pylint block-level control for shorter lines, see
  http://pylint.pycqa.org/en/latest/user_guide/message-control.html#block-disables

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-10 09:44:03 +01:00
Lukas Puehringer
2f57eb8ed7 Add SPDX style license and copyright boilerplate
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-04 16:26:34 +01:00
Lukas Puehringer
8e9afc96f9 Revert "Move to/from_dict metadata API methods..."
Revert an earlier commit that moved to/from_dict metadata class
model methods to a util module of the serialization sub-package.

We keep to/from_dict methods on the metadata classes because:
- It seems **idiomatic** (see e.g. 3rd-party libaries such as attrs,
pydantic, marshmallow, or built-ins that provide default or
customizable dict representation for higher-level objects).
The idiomatic choice should make usage more intuitive.
- It feels better **structured** when each method is encapsulated
within the corresponding class, which in turn should make
maintaining/modifying/extending the class model easier.
- It allows us to remove function-scope imports (see subsequent
commit).

Caveat:
Now that "the meat" of the sub-packaged JSON serializer is
implemented on the class, it might make it harder to create a
non-dict based serializer by copy-paste-amending the JSON
serializer.

However, the benefits from above seem to outweigh the disadvantage.

See option 5 of ADR0006 for further details (#1270).

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-04 12:46:16 +01:00
Lukas Puehringer
e1be085c3c Move to/from_dict metadata API methods to util
Add tuf.api.serialization.util module with functions to
convert between TUF metadata class model and the corresponding
dictionary representation. These functions replace the
corresponding to/from_dict classmethods.

Configure api/pylintrc to exempt '_type' from protected member
access warning, because the underscore prefix here is only used to
avoid name shadowing.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-04 12:33:28 +01:00
Lukas Puehringer
240fb547af Use custom errors in serializer.json sub-package
Re-raise all errors that happen during de/serialization as custom
De/SerializationError.

Whilelist 'e', which is idiomatic for error, in api/pylintrc, and
inline exempt broad-except, which are okay if re-raised.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-04 12:33:28 +01:00
Lukas Puehringer
499f1c858e Adopt serialization sub-package in metadata API
- Rename Metadata methods:
  - to_json_file -> to_file
  - from_json_file -> from_file
- Remove Metadata.from_json/to_json
- Remove Signed.to_canonical_bytes
- Accept optional de/serializer arguments:
  - from_file (default: JSONDeserializer)
  - to_file (default: JSONSerializer)
  - sign, verify (default: CanonicalJSONSerializer)
- inline disable pylint cyclic-import checks

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-04 12:33:18 +01:00
Lukas Puehringer
4a22b4a578 Add concrete de/serializer implementations
Add serializer.json module with implementations to serialize and
deserialize TUF role metadata to and from the JSON wireline format
for transportation, and to serialize the 'signed' part of TUF role
metadata to the OLPC Canonical JSON format for signature generation
and verification.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-04 10:51:37 +01:00
Lukas Puehringer
3d8cade471 Add metadata serialization sub-package
Add sub-package with 3 abstract base classes to:
- serialize Metadata objects to bytes (transport)
- deserialize Metadata objects from bytes (transport)
- serialize Signed objects to bytes (signatures)

pylint notes:
- configure tox to use api/pylintrc
- configure api/pylintrc to allow classes without public methods
  (default was 2)

Design considerations
---------------------
- Why not implement de/serialization on metadata classes?
  -> See ADR0006.

- Why use separate classes for serialization and deserialization?
  -> Some users might only need either one, e.g. client only needs
     Deserializer. Maybe there are use cases where different
     implementations are used to serialize and deserialize.

- Why use separate classes for Metadata- and Signed-Serialization?
  -> They require different concrete types, i.e. Metadata and
     Signed as parameters, and using these specific types seems to
     make the interface stronger.

- Why are de/serialize methods not class/staticmethods?
  -> In reality we only use classes to namespace and define a
     type annotated interface, thus it would be enough to make the
     methods classmethods. However, to keep the de/serialize
     interface minimal, we move any custom format configuration to
     the constructor. (See e.g. "compact" for JSONSerializer in
     subsequent commit).

Naming considerations
---------------------
- Why de/serialize?
  -> Implies byte stream as input or output to the function, which
     is what our interface needs.
- Why not marshaling?
  -> Synonym for serialize but implies transport, would be okay.
- Why not encoding?
  -> Too abstract and too many connotations (character, a/v).
- Why not parse?
  -> Too abstract and no good opposite terms (unparse, write,
     dump?)

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-03 15:09:48 +01:00