Commit graph

3694 commits

Author SHA1 Message Date
Joshua Lock
284ff1cdbe
Merge pull request #1072 from MVrachev/fix-unit-test
Fix prefix file_prefix to support "" value
2020-07-10 11:25:28 +01:00
lukpueh
95827640b8
Merge pull request #1071 from joshuagl/joshuagl/issue1069
Fix consistent snapshot handling in `repository_lib.get_delegated_roles_metadata()`
2020-07-09 11:13:32 +02:00
lukpueh
1fe6f16c3d
Merge pull request #1073 from jku/remove-failing-content-length-code
Download: Remove Content-Length header handling
2020-07-09 08:54:03 +02:00
Jussi Kukkonen
338dc4aca7 Download: Remove Content-Length header handling
These functions currently only accomplish logging: there are no actual
effects from this code... except for failure to download if the header
happens to be missing:
 TypeError: '<' not supported between instances of 'NoneType' and 'int'

Fixes #1068.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2020-07-08 18:21:51 +03:00
Martin Vrachev
e650670f77 Fix prefix file_prefix to support "" value
If we run the tuf unit tests with latest, not yet released,
securesyslib changes we will see that one unit test is failing
because the file_prefix argument should confront the PATH_SCHEMA
(which requires a non-empty string) but an empty string value
is passed to it.

That happens, because in tuf/developer_tool.py create_new_project
function the "location_in_repository" arg has a default value of ''
and if not changed when creating a new object of type Project
on line 650 will cause an exception in the __init__ function
because of the file_prefix argument.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2020-07-07 16:24:14 +03:00
Joshua Lock
b309e1befa
Merge pull request #1064 from MVrachev/fix-failing-tests
Fix failing unit tests
2020-07-06 15:55:14 +01:00
Joshua Lock
cdb35560fd
Merge pull request #1066 from jku/fix-repo-py-key-decode
repo.py: Decode key bytes for SHA key as well
2020-07-06 15:30:05 +01:00
Joshua Lock
aa49ed9307 Fix consistent sshot in get_delegated_roles_metadata
Consistent snapshot handling was being turned for all files after the
root.json file had been processed because the internal state tracking
variable shared the same name as the method argument: consistent_snapshot.

Therefore once the root.json file was met in the file list and the
following condition evaluated to True:

metadata_role.endswith('root.json') or repo_consistent_snapshots == True

the consistent_snapshot variable, both the name of a method argument and
the internal to the loop state tracking variable, was assigned True and
all future condition checks evaluate to True.

Resolve by renaming the loop state tracking variable from
consistent_snapshot to consistent.

Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-07-06 14:01:24 +01:00
Jussi Kukkonen
e662f4fda0 repo.py: Decode key bytes for SHA key as well
key bytes were decoded as utf-8 for the JSON keys. Do the same for SHA
key bytes.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2020-07-03 11:45:12 +03:00
Martin Vrachev
853cdd30a4 Fix failing unit tests
Corresponding to the securesyslib changes made in this pr:
https://github.com/secure-systems-lab/securesystemslib/pull/244
which changed the securesyslib.formats.PATH schema to be of type
AnyNonemptystring.
This made the tuf unit tests to fail because there are to places
where functional arguments should comply with the
securesyslib.formats.PATH schema, but have a default value of
an empty string.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
2020-06-30 21:32:50 +03:00
lukpueh
de8649e12a
Merge pull request #1058 from joshuagl/joshuagl/protected2public
Make some helper functions for determining hashed bins part of repository_lib
2020-06-30 10:42:04 +02:00
Joshua Lock
ed45fb24c2 Move find_bin_for_hash() to repository_lib
Move repository_tool._find_bin_for_hash() and helper functions it uses to
non-protected functions in repository_lib.
_find_bin_for_hash() becomes find_bin_for_target_hash()

These functions will be useful to adopters using the WIP low-level API for
updating metadata files (see #1048)

Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-06-29 15:15:25 +01:00
lukpueh
49031b424b
Merge pull request #1052 from sechkova/issue-1045
Load correctly the delegated Targets objects hierarchy
2020-06-29 14:29:52 +02:00
Teodora Sechkova
6ae3ea6d7d
Add TOP_LEVEL_ROLES as a global variable
Add TOP_LEVEL_ROLES as a global variable in roledb.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-06-29 13:55:45 +03:00
Teodora Sechkova
a69208c1c7
Rename get_*_metadata_filenames functions
Rename repository_lib.get_metadata_filenames() and
get_delegations_filenames() to better match their
functionality and tuf terminology.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-06-29 13:43:20 +03:00
Teodora Sechkova
97eff9e1cb
Reference loaded delegated targets objects by top-level targets
Use the top-level targets object to reference already loaded
delegated targets instead of storing them in an additional
dictionary in load_repository().

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-06-29 13:33:21 +03:00
Teodora Sechkova
f1a6676084
Improve delegated roles loading in load_repository()
Replace the list used for the delegations graph traversal with
a deque and use a set to store already loaded roles and avoid
loops in case of cycles in the graph.
Improve comments and readability.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-06-29 13:19:54 +03:00
Joshua Lock
b18eb34830 Consolidate _get_hash() and get_target_hash()
Remove the non-public function _get_hash() from repository_tool in favour
of the public function get_target_hash() in repository_lib

Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-06-25 20:16:54 +01:00
Teodora Sechkova
da09a22861
Improve get_delegations_filenames performance and readability
Remove unnecessary list keeping track of loaded file names and
rewrite outdated comments.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-06-24 13:52:50 +03:00
Teodora Sechkova
73bff87b68
Update test_load_repository
Add a tests case checking if delegated Targets() objects
are loaded correctly.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-06-24 13:52:50 +03:00
Teodora Sechkova
e8d8e84a84
Load correctly the delegated Targets objects hierarchy
Update load_repository() function to load the delegations metadata
starting from 'targets' and traversing downwards the delegated
roles in order to load correctly the delegations hierarchy.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-06-24 13:52:49 +03:00
lukpueh
5d16f91ca7
Merge pull request #1054 from jku/update-docs-on-crypto-details
Update docs on crypto details
2020-06-23 12:00:31 +02:00
Jussi Kukkonen
dc78d89f4f Update Tutorial on dependency installation
* Remove reference to deprecated settings
* Mention that the tutorial expects the dependencies and link to
  instructions

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2020-06-23 11:02:31 +03:00
lukpueh
017a5ff33b
Merge pull request #1056 from theupdateframework/dependabot/pip/certifi-2020.6.20
build(deps): bump certifi from 2020.4.5.2 to 2020.6.20
2020-06-23 09:47:08 +02:00
lukpueh
116e66e604
Merge pull request #1055 from theupdateframework/dependabot/pip/requests-2.24.0
build(deps): bump requests from 2.23.0 to 2.24.0
2020-06-22 19:02:08 +02:00
dependabot-preview[bot]
bc75c8c08c
build(deps): bump certifi from 2020.4.5.2 to 2020.6.20
Bumps [certifi](https://github.com/certifi/python-certifi) from 2020.4.5.2 to 2020.6.20.
- [Release notes](https://github.com/certifi/python-certifi/releases)
- [Commits](https://github.com/certifi/python-certifi/compare/2020.04.05.2...2020.06.20)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-06-22 11:38:01 +00:00
dependabot-preview[bot]
943ed41ada
build(deps): bump requests from 2.23.0 to 2.24.0
Bumps [requests](https://github.com/psf/requests) from 2.23.0 to 2.24.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/master/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.23.0...v2.24.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-06-18 10:32:51 +00:00
Jussi Kukkonen
179892c1e9 Update Tutorial on cryptographic keys
Lot of changes in 7 lines:
* PyCrypto is no longer an option: remove mention of it
* RSA-PSS wiki page now redirects to a fairly useless stub: replace it
  with the RFC (it's not light reading but better than nothing)
* Mention ECDSA
* Remove mention of json for RSA keys: that does not seem to be true

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2020-06-17 18:01:45 +03:00
Jussi Kukkonen
5a8f93529b Update comments about optional crypto dependencies
tools-extra does not exist in tuf anymore: mention the securesystemslib
extras instead.

Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
2020-06-17 17:33:58 +03:00
lukpueh
8e6ca67f31
Merge pull request #1047 from theupdateframework/dependabot/pip/certifi-2020.4.5.2
build(deps): bump certifi from 2020.4.5.1 to 2020.4.5.2
2020-06-12 10:08:24 +02:00
lukpueh
d875dd4bd3
Merge pull request #1051 from jcstr/patch2-docs
Add python 3 use case
2020-06-11 10:07:32 +02:00
Jesús Castro
9badf8a51e
Add python 3 use case
This indication can be found on other documents.

Signed-off-by: Jesús Castro <x51v4n@gmail.com>
2020-06-10 06:30:23 -05:00
lukpueh
ff5afe441a
Merge pull request #1049 from sechkova/issue-1046
Load full target file info for delegated targets metadata
2020-06-09 16:34:31 +02:00
Teodora Sechkova
2553dff276
Update test_load_repository
Extend test_load_repository to check if targets file info is loaded
correctly.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-06-09 16:48:53 +03:00
Teodora Sechkova
88f6755153
Load full target file info for delegated targets
Fix load_repository to actually load the full targets file info from
file system for delegated targets.

Update _load_top_level_metadata to load targets and delegated targets
metadata in a consistent way.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
2020-06-09 16:48:42 +03:00
dependabot-preview[bot]
a5e015f8f7
build(deps): bump certifi from 2020.4.5.1 to 2020.4.5.2
Bumps [certifi](https://github.com/certifi/python-certifi) from 2020.4.5.1 to 2020.4.5.2.
- [Release notes](https://github.com/certifi/python-certifi/releases)
- [Commits](https://github.com/certifi/python-certifi/compare/2020.04.05.1...2020.04.05.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-06-08 10:59:00 +00:00
Joshua Lock
5d40ffa3c4
Merge pull request #1034 from joshuagl/joshuagl/abstract-files-fixes
Fix and better test abstract files and directories support
2020-06-05 13:40:21 +01:00
lukpueh
95d08cc5b4
Merge pull request #1044 from jcstr/patch1
Remove unused imports
2020-06-05 09:42:53 +02:00
Jesús Castro
f4121e8f75
Remove unused imports
Those imports are marked as a non used libraries.

Signed-off-by: Jesús Castro <x51v4n@gmail.com>
2020-06-04 19:18:33 -05:00
Joshua Lock
5e5c598769 Support abstract storage for timestamp metadata
This was erroneously absent in PR 1024, which added support for abstract
files and directories. Resolve by adding a storage_backend argument to
generate_timestamp_metadata() and using it so that the fileinfo (hashes
and length) for the snapshot file can be generated for a snapshot
metadata file on any supported storage.

Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-06-03 14:16:47 +01:00
Joshua Lock
d9ec10e894 Test abstract storage backend support
Add a class implementing StorageBackendInterface for testhing which
mutates filenames on put()/get(), such that trying to read the expected
file paths for TUF metadata from the local filesystem doesn't find the
files.

Use this class when creating a repository and writing metadata to test
abstract files and directories support for metadata writing.

Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-06-03 14:16:47 +01:00
Joshua Lock
05d5639502 Better document generate_targets_metadata()
Clarify, through the docstrings and code comments, the expected behaviour
of generate_targets_metadata() and the interactions of the
use_existing_fileinfo and write_consistent_targets parameters.

Signed-off-by: Joshua Lock <jlock@vmware.com>
2020-06-03 14:16:47 +01:00
Marina Moore
a354fc01c0
Merge pull request #1040 from trailofbits/ww/return-bin-name-when-delegating
tuf/repository_tool: Return delegated bin_name during modifications
2020-06-01 15:59:45 -07:00
William Woodruff
1e532e825a
tests: Fill in more returned role name use
Signed-off-by: William Woodruff <william@trailofbits.com>
2020-06-01 14:22:29 -04:00
William Woodruff
4327a980cd
tests: Use newly returned role name
Signed-off-by: William Woodruff <william@trailofbits.com>
2020-06-01 14:01:46 -04:00
William Woodruff
65fd02c4ab
tuf/repository_tool: Return delegated bin_name during modifications
This makes it easier for consumers of repository_tool to mark the
appropriate delegated bin as dirty when using delegated targets.

Signed-off-by: William Woodruff <william@trailofbits.com>
2020-06-01 13:46:43 -04:00
lukpueh
a4b52e7e0d
Merge pull request #1036 from theupdateframework/dependabot/pip/pynacl-1.4.0
build(deps): bump pynacl from 1.3.0 to 1.4.0
2020-06-01 14:05:00 +02:00
dependabot-preview[bot]
f01a31f2f9
build(deps): bump pynacl from 1.3.0 to 1.4.0
Bumps [pynacl](https://github.com/pyca/pynacl) from 1.3.0 to 1.4.0.
- [Release notes](https://github.com/pyca/pynacl/releases)
- [Changelog](https://github.com/pyca/pynacl/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/pyca/pynacl/compare/1.3.0...1.4.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-05-28 09:37:12 +00:00
lukpueh
e36080e673
Merge pull request #1035 from theupdateframework/dependabot/pip/six-1.15.0
build(deps): bump six from 1.14.0 to 1.15.0
2020-05-28 11:34:26 +02:00
lukpueh
580334e707
Merge pull request #1021 from MVrachev/patch-1
Fix typo in comment
2020-05-27 14:16:30 +02:00