Commit graph

3705 commits

Author SHA1 Message Date
Lukas Puehringer
3e249f5bdd Make Metadata a container class (WIP)
This commit performs restructuring on the recently added metadata
class model architecture, which shall be part of a new simple TUF
API.

The key change is that the Metadata class is now used as container
for inner TUF metadata (Root, Timestamp, Snapshot, Targets) instead
of serving as base class for these, that means we use 'composition'
instead of 'inheritance'. Still, in order to aggregate common
attributes of the inner Metadata (expires, version, spec_version),
we use a new baseclass 'Signed', which also corresponds to the
signed field of the outer metadata container.

Based on prior observations in TUF's sister project in-toto, this
architecture seems to more closely represent the metadata model as
it is defined in the specification (see in-toto/in-toto#98 and
in-toto/in-toto#142 for related discussions).

Note that the proposed changes require us to now access some
attributes/methods via the signed attribute of a Metadata object
and not directly on the Metadata object, but it would be possible
to add short-cuts. (see todo notes in doc header).

Further changes include:
 - Add minimal doc header with TODO notes

 - Make attributes that correspond to fields in TUF JSON metadata
public again. There doesn't seem to be a good reason to protect
them with leading underscores and use setters/getters instead, it
just adds more code.

 - Generally try to reduce code.

 - Remove keyring and consistent_snapshot attributes from metadata
   class. As discussed in #1060 they are a better fit for extra
   management code (also see #660)

- Remove sslib schema checks (see TODO notes about validation in
  doc header)

 - Drop usage of build_dict_conforming_to_schema, it seems a lot
   simpler and more explicit to just code this here.

 - ... same goes for make_metadata_fileinfo

 - Adapt tests accordingly

TODO: Document!!!
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2020-07-10 16:05:53 +02:00
Lukas Puehringer
565768efd9 Quick-fix programming errors in api.keys module
- Finalize min/max -> least/most refactor
- Comment out unclear input validation
- Use string literal for foward referencing type hint
(see https://www.python.org/dev/peps/pep-0484/#forward-references)

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
2020-07-10 15:26:08 +02:00
Trishank Karthik Kuppusamy
76cb560a46
minor edits
* classmethod for init RAMKey from file
* private class variables
* more typing for methods
* better names for arguments

Signed-off-by: Trishank Karthik Kuppusamy <trishank.kuppusamy@datadoghq.com>
2020-07-07 23:55:22 -04:00
Joshua Lock
42372872d2 Test Metadata.bump_expiration() with relativedelta
dateutil provides an interface which is much easier to reason about for
users, i.e. it provides an interface for year deltas which automatically
handles leap years. Add some tests to try and ensure that, even though it
uses standard library functionality, the metadata API can accept
dateutil.relativedelta and do the right thing.

Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-07 10:35:08 +01:00
Joshua Lock
54e1f9c03b tuf.api: drop use of dateutil
All of the functionality we need is available from the standard library
which reduces our dependency footprint. Having minimal dependencies is
especially important for update clients which often have to vendor their
dependencies.

However, dateutil.relativedelta is richer than timedelta and helps to
provide a clearer API. For example, with relativedelta it's possible
to specify a delta in years *and* dateutil will do the right thing for
leap years.

Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-07 10:35:08 +01:00
Joshua Lock
fd5732a024 tuf.api: treat all datetime's as UTC
We don't capture timezone information in the metadata, therefore we should
not capture it in the interfaces. Ensure we remove timezone information
from any datetime objects when they are assigned to the expiration
property of a Metadata object.

Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-07 10:33:40 +01:00
Teodora Sechkova
3c5e312e60 WIP added tuf.api keys tests
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-07-06 15:23:26 +01:00
Joshua Lock
34a0680947 More tests tuf.api and verify data!
Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-03 14:42:54 +01:00
Joshua Lock
0ecdfba229 tuf.api: shuffle up signed/signatures/signable interface
* Add signed_bytes property to return a canonicalised version of signed
* Make the signed property responsible for converting the object into the
  dict format expected of the metadata, requiring a per-class
  implementation
* Make signable a generic base-class property which returns a dict of
  signatures and signed

With these changes we can load and verify metadata with the low-level API!

Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-03 14:42:54 +01:00
Joshua Lock
1daefa456b tuf.api: instantiate objects from the JSON
Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-03 14:42:54 +01:00
Joshua Lock
3cc98ae2d5 tuf.api: make signed & signatures properties of Metadata
Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-03 14:42:36 +01:00
Joshua Lock
f8d8bb5462 tuf.api: convert keys.Algorithm to dict
Enum isn't available in Python 2.7 and accessing the enum members
as items, i.e. Algorithm['RSA'] throws KeyError when the member
is a method.
Work around both of these issues by converting to a dict.

Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-03 14:41:36 +01:00
Joshua Lock
eb93fe133e tuf.api: make expires a property
Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-03 14:41:31 +01:00
Teodora Sechkova
0ca471ed2a tuf.api: use StorageBackendInterface
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-07-02 15:05:51 +01:00
Teodora Sechkova
1fbff557ad tuf.api: add basic schema checks in read_from_json
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-07-02 15:03:49 +01:00
Teodora Sechkova
916055aa54 tuf.api: simplify metadata.Targets.signable()
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-07-02 10:32:25 +01:00
Teodora Sechkova
3e022aae32 Skip an optional keyword in the schema
Skip a keyword if it is optional in the schema and the value
passed in is set to None.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-07-02 10:32:19 +01:00
Teodora Sechkova
46977f977e tux.api: implement metadata.Targets.signable()
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-07-02 10:32:13 +01:00
Teodora Sechkova
db0f8a73a7 tuf.api: implement metadata.Targets.update()
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-07-02 10:32:04 +01:00
Teodora Sechkova
37a235f97c tuf.api: implement metadata.Targtes.read_from_json()
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-07-02 10:31:55 +01:00
Trishank Karthik Kuppusamy
f2861bfd41
much simpler keys
Signed-off-by: Trishank Karthik Kuppusamy <trishank.kuppusamy@datadoghq.com>
2020-07-01 18:39:35 -04:00
Joshua Lock
11d76e72dd tuf.api: WIP implement Timestamp
Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-01 17:17:33 +01:00
Joshua Lock
57c98d45ac WIP tests for tuf.api.metadata
Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-01 17:17:33 +01:00
Joshua Lock
bc1134f488 tuf.api: fix loading expiration
Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-01 17:17:33 +01:00
Joshua Lock
2758f48214 tuf.api: fix version check in metadata
Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-01 17:17:33 +01:00
Joshua Lock
d58a944c8b tuf.api: fix imports in metadata
Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-01 17:17:33 +01:00
Joshua Lock
eb9c56b52d tuf.api: fix missing ':' in keys
Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-01 17:17:33 +01:00
Joshua Lock
92f97a45e3 tuf.api: implement metadata.Snapshot
Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-01 17:17:33 +01:00
Joshua Lock
5ef60ca187 tuf.api: implement update_signatures()
Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-01 17:17:24 +01:00
Joshua Lock
721def4eb0 tuf.api: add helpers to bump version and expiration
Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-01 17:15:47 +01:00
Joshua Lock
2e3ceb7ff3 tuf.api: set consistent_snapshot during read_from_json
Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-01 17:15:47 +01:00
Trishank Karthik Kuppusamy
83d6d07943
WIP 2020-06-26 16:40:59 -04:00
lukpueh
5d16f91ca7
Merge pull request #1054 from jku/update-docs-on-crypto-details
Update docs on crypto details
2020-06-23 12:00:31 +02:00
Jussi Kukkonen
dc78d89f4f Update Tutorial on dependency installation
* Remove reference to deprecated settings
* Mention that the tutorial expects the dependencies and link to
  instructions

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2020-06-23 11:02:31 +03:00
lukpueh
017a5ff33b
Merge pull request #1056 from theupdateframework/dependabot/pip/certifi-2020.6.20
build(deps): bump certifi from 2020.4.5.2 to 2020.6.20
2020-06-23 09:47:08 +02:00
lukpueh
116e66e604
Merge pull request #1055 from theupdateframework/dependabot/pip/requests-2.24.0
build(deps): bump requests from 2.23.0 to 2.24.0
2020-06-22 19:02:08 +02:00
dependabot-preview[bot]
bc75c8c08c
build(deps): bump certifi from 2020.4.5.2 to 2020.6.20
Bumps [certifi](https://github.com/certifi/python-certifi) from 2020.4.5.2 to 2020.6.20.
- [Release notes](https://github.com/certifi/python-certifi/releases)
- [Commits](https://github.com/certifi/python-certifi/compare/2020.04.05.2...2020.06.20)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-06-22 11:38:01 +00:00
dependabot-preview[bot]
943ed41ada
build(deps): bump requests from 2.23.0 to 2.24.0
Bumps [requests](https://github.com/psf/requests) from 2.23.0 to 2.24.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/master/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.23.0...v2.24.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-06-18 10:32:51 +00:00
Jussi Kukkonen
179892c1e9 Update Tutorial on cryptographic keys
Lot of changes in 7 lines:
* PyCrypto is no longer an option: remove mention of it
* RSA-PSS wiki page now redirects to a fairly useless stub: replace it
  with the RFC (it's not light reading but better than nothing)
* Mention ECDSA
* Remove mention of json for RSA keys: that does not seem to be true

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2020-06-17 18:01:45 +03:00
Jussi Kukkonen
5a8f93529b Update comments about optional crypto dependencies
tools-extra does not exist in tuf anymore: mention the securesystemslib
extras instead.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2020-06-17 17:33:58 +03:00
lukpueh
8e6ca67f31
Merge pull request #1047 from theupdateframework/dependabot/pip/certifi-2020.4.5.2
build(deps): bump certifi from 2020.4.5.1 to 2020.4.5.2
2020-06-12 10:08:24 +02:00
lukpueh
d875dd4bd3
Merge pull request #1051 from jcstr/patch2-docs
Add python 3 use case
2020-06-11 10:07:32 +02:00
Jesús Castro
9badf8a51e
Add python 3 use case
This indication can be found on other documents.

Signed-off-by: Jesús Castro <x51v4n@gmail.com>
2020-06-10 06:30:23 -05:00
lukpueh
ff5afe441a
Merge pull request #1049 from sechkova/issue-1046
Load full target file info for delegated targets metadata
2020-06-09 16:34:31 +02:00
Teodora Sechkova
2553dff276
Update test_load_repository
Extend test_load_repository to check if targets file info is loaded
correctly.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-06-09 16:48:53 +03:00
Teodora Sechkova
88f6755153
Load full target file info for delegated targets
Fix load_repository to actually load the full targets file info from
file system for delegated targets.

Update _load_top_level_metadata to load targets and delegated targets
metadata in a consistent way.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-06-09 16:48:42 +03:00
dependabot-preview[bot]
a5e015f8f7
build(deps): bump certifi from 2020.4.5.1 to 2020.4.5.2
Bumps [certifi](https://github.com/certifi/python-certifi) from 2020.4.5.1 to 2020.4.5.2.
- [Release notes](https://github.com/certifi/python-certifi/releases)
- [Commits](https://github.com/certifi/python-certifi/compare/2020.04.05.1...2020.04.05.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-06-08 10:59:00 +00:00
Joshua Lock
5d40ffa3c4
Merge pull request #1034 from joshuagl/joshuagl/abstract-files-fixes
Fix and better test abstract files and directories support
2020-06-05 13:40:21 +01:00
lukpueh
95d08cc5b4
Merge pull request #1044 from jcstr/patch1
Remove unused imports
2020-06-05 09:42:53 +02:00
Jesús Castro
f4121e8f75
Remove unused imports
Those imports are marked as a non used libraries.

Signed-off-by: Jesús Castro <x51v4n@gmail.com>
2020-06-04 19:18:33 -05:00