Commit graph

19 commits

Author SHA1 Message Date
Jussi Kukkonen
8ed446c14b Metadata API: Stop annotating __init__() return value
This may have been required by a linter at some point, but isn't
anymore: Not annotating makes the documentation look better.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-09-23 17:57:19 +03:00
Jussi Kukkonen
ed520ee55d Metadata API: Improve serialization docs
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-09-23 17:52:55 +03:00
Teodora Sechkova
5d71aab9ec
Remove disable=broad-except
The pylint warning W0703:broad-except was raised only
when six was used and python 2 was still supported.

The warning is no longer raised, the exceptions are
handled/raised correctly and the disabling can be removed.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2021-09-01 17:41:32 +03:00
Jussi Kukkonen
1b9aec291c Metadata API: Use TYPE_CHECKING for cyclic import
The import is useful for mypy so it can check the types.
Add a pylint disable just like json.py does in the same situation.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2021-05-19 14:26:55 +03:00
Martin Vrachev
1712b71b55 Fix black docstring indentation errors
Black was updated from 20.8b1 to 21.4b0 requiring that one-line
docstring don't add additional space before the closing quotes.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2021-04-27 14:08:24 +03:00
Lukas Puehringer
42a797b4a1 Re-format tuf/api/* using black and isort
Use black and isort to reformat new code in tuf/api/*, like so:

```
black --line-length 80 api
isort --line-length 80 --profile black api
```

Besides downsizing the default line length to fit our Code Style
Guide no extra configuration is required.

Unified format according to black and isort will be enforced by
CI/CD in a future commit.

**Changes include:**
- Use double quotes instead of single quotes where feasible
- Re-wrap and re-indent long lines such as dict literals, function
  signatures and function calls, using hanging indent
  This will require an update in our Code Style Guide, which the
  benefits of using black seem worth.
  https://github.com/secure-systems-lab/code-style-guidelines/blob/master/python.md#indentation-and-line-continuation
- Update vertical and horizontal spacing
- Sort and wrap imports

See black and isort docs for details:
https://black.readthedocs.io/en/stable/the_black_code_style.html
https://pycqa.github.io/isort/docs/configuration/black_compatibility/

NOTE: If desired I can split commits by change and/or configure git
for this repo to ignore the corresponding revision(s) in git-blame.
https://github.com/psf/black#migrating-your-code-style-without-ruining-git-blame

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-12 11:51:55 +01:00
Lukas Puehringer
a53d68b91d Re-word api.serializer.json docstrings
- Make class docstrings wording consistent.
- Emphasize that we use the OLPC Canonical JSON specification.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-10 09:44:38 +01:00
Lukas Puehringer
bd94f6d8d1 Remove py2 compat from api.serialization package
tuf.api is not designed for Python 2 compatibility. This commit
removes the following stray compatibility constructs in its
serialization subpackage:

- '__metaclass__ = abc.ABCMeta'
- six.raise_from

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-10 09:44:38 +01:00
Lukas Puehringer
ab92ba257f Fix inconsistent returns in json serializers
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-10 09:44:38 +01:00
Lukas Puehringer
326d2af7c4 Fix blank lines in tuf.api as per styleguide
https://github.com/google/styleguide/blob/gh-pages/pyguide.md#35-blank-lines

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-10 09:44:38 +01:00
Lukas Puehringer
d823c8fc01 Rename a few variables in tuf.api
- Rename _dict to json_dict to avoid wrong semantics of leading
  underscore. (leading underscore was initially chosen to avoid name
  shadowing)

- Rename 'serializer' argument of type 'SignedSerializer' to
  'signed_serializer', to distinguish from 'serializer' argument of
  type 'MetadataSerializer'.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-10 09:44:30 +01:00
Lukas Puehringer
2b4085718b Re-word serialization cyclic import code comments
- Try to clarify purpose and remove unimportant TODO note
- Use pylint block-level control for shorter lines, see
  http://pylint.pycqa.org/en/latest/user_guide/message-control.html#block-disables

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-10 09:44:03 +01:00
Lukas Puehringer
2f57eb8ed7 Add SPDX style license and copyright boilerplate
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-04 16:26:34 +01:00
Lukas Puehringer
8e9afc96f9 Revert "Move to/from_dict metadata API methods..."
Revert an earlier commit that moved to/from_dict metadata class
model methods to a util module of the serialization sub-package.

We keep to/from_dict methods on the metadata classes because:
- It seems **idiomatic** (see e.g. 3rd-party libaries such as attrs,
pydantic, marshmallow, or built-ins that provide default or
customizable dict representation for higher-level objects).
The idiomatic choice should make usage more intuitive.
- It feels better **structured** when each method is encapsulated
within the corresponding class, which in turn should make
maintaining/modifying/extending the class model easier.
- It allows us to remove function-scope imports (see subsequent
commit).

Caveat:
Now that "the meat" of the sub-packaged JSON serializer is
implemented on the class, it might make it harder to create a
non-dict based serializer by copy-paste-amending the JSON
serializer.

However, the benefits from above seem to outweigh the disadvantage.

See option 5 of ADR0006 for further details (#1270).

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-04 12:46:16 +01:00
Lukas Puehringer
e1be085c3c Move to/from_dict metadata API methods to util
Add tuf.api.serialization.util module with functions to
convert between TUF metadata class model and the corresponding
dictionary representation. These functions replace the
corresponding to/from_dict classmethods.

Configure api/pylintrc to exempt '_type' from protected member
access warning, because the underscore prefix here is only used to
avoid name shadowing.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-04 12:33:28 +01:00
Lukas Puehringer
240fb547af Use custom errors in serializer.json sub-package
Re-raise all errors that happen during de/serialization as custom
De/SerializationError.

Whilelist 'e', which is idiomatic for error, in api/pylintrc, and
inline exempt broad-except, which are okay if re-raised.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-04 12:33:28 +01:00
Lukas Puehringer
499f1c858e Adopt serialization sub-package in metadata API
- Rename Metadata methods:
  - to_json_file -> to_file
  - from_json_file -> from_file
- Remove Metadata.from_json/to_json
- Remove Signed.to_canonical_bytes
- Accept optional de/serializer arguments:
  - from_file (default: JSONDeserializer)
  - to_file (default: JSONSerializer)
  - sign, verify (default: CanonicalJSONSerializer)
- inline disable pylint cyclic-import checks

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-04 12:33:18 +01:00
Lukas Puehringer
4a22b4a578 Add concrete de/serializer implementations
Add serializer.json module with implementations to serialize and
deserialize TUF role metadata to and from the JSON wireline format
for transportation, and to serialize the 'signed' part of TUF role
metadata to the OLPC Canonical JSON format for signature generation
and verification.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-04 10:51:37 +01:00
Lukas Puehringer
3d8cade471 Add metadata serialization sub-package
Add sub-package with 3 abstract base classes to:
- serialize Metadata objects to bytes (transport)
- deserialize Metadata objects from bytes (transport)
- serialize Signed objects to bytes (signatures)

pylint notes:
- configure tox to use api/pylintrc
- configure api/pylintrc to allow classes without public methods
  (default was 2)

Design considerations
---------------------
- Why not implement de/serialization on metadata classes?
  -> See ADR0006.

- Why use separate classes for serialization and deserialization?
  -> Some users might only need either one, e.g. client only needs
     Deserializer. Maybe there are use cases where different
     implementations are used to serialize and deserialize.

- Why use separate classes for Metadata- and Signed-Serialization?
  -> They require different concrete types, i.e. Metadata and
     Signed as parameters, and using these specific types seems to
     make the interface stronger.

- Why are de/serialize methods not class/staticmethods?
  -> In reality we only use classes to namespace and define a
     type annotated interface, thus it would be enough to make the
     methods classmethods. However, to keep the de/serialize
     interface minimal, we move any custom format configuration to
     the constructor. (See e.g. "compact" for JSONSerializer in
     subsequent commit).

Naming considerations
---------------------
- Why de/serialize?
  -> Implies byte stream as input or output to the function, which
     is what our interface needs.
- Why not marshaling?
  -> Synonym for serialize but implies transport, would be okay.
- Why not encoding?
  -> Too abstract and too many connotations (character, a/v).
- Why not parse?
  -> Too abstract and no good opposite terms (unparse, write,
     dump?)

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2021-03-03 15:09:48 +01:00