Currently, we are importing the "utils" module in tests/utils
with "import utils".
This could become a problem when there is another module with
the same general name "utils" and could lead to import mistakes.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
In 'repository_lib._generate_and_write_metadata' sort the set of
signing key keyids alphabetically before passing them on to signing
functions, to make the order in which signatures are added
deterministic.
This is above all beneficial for testing.
This commit also adds an exemplary test for signatures on root
metadata using the repository_tool interface to setup all the state
that required to test _generate_and_write_metadata.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
I don't see a need to leave a comment about what setupClass,
tearDownClass, setup and tearDown functions do.
There is documentation that describes that.
Additionally, the links referenced in the comments are from
Python 2 is deprecated.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
all test_*.py files now accept zero or more '-v' to increase tuf
logging level. The default is now ERROR.
default: ERROR
"-v": ERROR, but unittest prints test names
"-vv": WARNING
"-vvv": INFO
"-vvvv": DEBUG
Example to run a single test with DEBUG level:
python3 test_updater.py -vvvv TestUpdater.test_4_refresh
Also make test_log.py restore the log level it modifies during test.
Fixes#1093
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
This separation and refactoring is part of the change to
make length and hashes optional for timestamp and snapshot roles.
It separates FILEINFO_SCHEMA into two separate schemas:
TARGETS_FILEINFO_SCHEMA and METADATA_FILEINFO_SCHEMA.
The distinction is needed because as of version 1.0.1 of the tuf
spec targets role has mandatory length and hashes, and
snapshot and timestamp roles have a mandatory version, and optional
length and hashes.
That's why targets can't share the same schemas
as timestamp and snapshot.
Because of that schema distinction, make_fileinfo had to be too
separated into make_targets_fileinfo and make_metadata_fileinfo.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Currently TestTimestamp creates custom databases but only clears the
default ones. This means next create_*db() call will fail meaning every
test after this one will fail (currently TestTimestamp happens to be
last but the effect can be seen by renaming it to TestATimestamp).
Also remove the clear_*db() calls from TestRepository::Setup(): they are
likely to be a workaround for a similar problem earlier (earlier test
failed to cleanup).
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Add a class implementing StorageBackendInterface for testhing which
mutates filenames on put()/get(), such that trying to read the expected
file paths for TUF metadata from the local filesystem doesn't find the
files.
Use this class when creating a repository and writing metadata to test
abstract files and directories support for metadata writing.
Signed-off-by: Joshua Lock <jlock@vmware.com>
The specification lists four fundamental roles: root, targets, snapshot
and timestamp. Loading a repository where those roles are not present
should not be supported, therefore convert debug messages on the absence
of metadata files for these fundamental roles into a RepositoryError
exception.
Signed-off-by: Joshua Lock <jlock@vmware.com>
Switch to using the new abstract files and directories support in
securesystemslib by taking an object which implements
securesystemslib.storage.StorageBackendInterface in the Repository
constructor, passed in by tuf.repository_tool.create_new_repository() and
tuf.repository_tool.load_repository()
The Updater class in tuf.client.updater does not specify a storage backend
and instead allows the functions in securesystemslib to perform the
default action of instantiating a LocalFilesystemBackend, that is the
updater does not currently support abstract filesystem backends and always
defaults to using local storage.
Finally we drop support for tuf.settings.CONSISTENT_METHOD as it's not as
clear how different copying modes should work when the details of the
underlying storage are abstracted away.
Signed-off-by: Joshua Lock <jlock@vmware.com>
Use a hard-coded unix separator ('/') so that an
exception is also raised for paths starting with '/'
when executing on Windows systems.
Update test_check_path to explicitly test invalid paths
starting with Windows style separator.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
- add a test for _check_path() method of Targets class.
- update all tests calling _check_path() respectively
- update test_tutorial
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
delegate_hashed_bins() has a number_of_bins parameter which defaults to
1024. add_target_to_bin() and remove_target_from_bin() both have a
number_of_bins parameter with no default. This means that in the
(somewhat) unlikely case that someone is using the default
number_of_bins when creating hashed bins they will need know what that
default value is and pass it to add_target_to_bin() and
remove_target_from_bin().
In order to be consistent and simpler to use define the default number
of bins as a module level constant and use it as the default value for
the number_of_bins argument for each of:
* delegate_hashed_bins()
* add_target_to_bin()
* remove_target_from_bin()
Signed-off-by: Joshua Lock <jlock@vmware.com>
Add some additional checks to test_add_target_to_bin to ensure the code
to add a target passing a fileinfo is tested.
Signed-off-by: Joshua Lock <jlock@vmware.com>
When testing delegate_hashed_bins to ensure that hash_path_prefixes
map to the generated name of the bin, also check to ensure that at least
one of the delegations contains one or more path_hash_prefixes.
Signed-off-by: Joshua Lock <jlock@vmware.com>
Test the newly added functionality to:
* add a target to the repository without access to the target file on disk
* write targets metadata without access to target files on disk, by using
the existing fileinfo data from the roledb
Signed-off-by: Joshua Lock <jlock@vmware.com>
Add an additional optional parameter to add_target() and
add_target_to_bin() which is a fileinfo object matching
tuf.formats.FILEINFO_OBJECT
This parameter and the custom parameter are mutually exclusive and
thus cannot be passed at the same time.
Signed-off-by: Joshua Lock <jlock@vmware.com>
The file isn't strictly needed on-disk at the time add_target() and
add_targets() are called and this duplicates the check for the file's
presence in write[_all]()
By removing this check we allow extra versatility in adding targets.
Signed-off-by: Joshua Lock <jlock@vmware.com>
Vastly simplify the implementation, using the _get_hash() and
_find_bin_for_hash() helpers added in earlier commits.
Furthermore, enable passing of the custom parameter to
add_target_to_bin() to better match add_target()
Signed-off-by: Joshua Lock <jlock@vmware.com>
Add a helper function to determine the name of a bin that a hashed
targetfile will be delegated to.
Based sketches by Lukas Puehringer in issues #994 & #995
Signed-off-by: Joshua Lock <jlock@vmware.com>
As we are adding and removing items from the hashed bins and checking
for their presence/absence it's simplest if we being with the hashed
bins initially empty.
If we pass a list of targets when we call delgate_hashed_bins() the
delegated roles have an initial set of targets delegated to them,
which complicates testing of adding then removing a target to a
delegated bin.
Signed-off-by: Joshua Lock <jlock@vmware.com>
Add test to ensure delegated bin names are consistent with the hash
prefixes that are delegated to the role.
This is an implicit assumption of the current implementation, the
testing of which will enable us to modify the code with greater
confidence.
Signed-off-by: Joshua Lock <jlock@vmware.com>
Replace hard-coded logger names with __name__. For the most part this just uses
the standard conventions to create the same logger hierarchy as existed before.
The only real difference is that loggers created for printing during tests are
no longer part of the 'tuf' hierarchy.
Signed-off-by: Joshua Lock <jlock@vmware.com>
roledb.get_dirty_roles(repo_name) returns the list representation
of the global _dirty_roles[repo_name] set. To make the return value
deterministic this commit sorts the list before returning it.
The commit also removes calls to sorted on the return value of
get_dirty_roles in test_roledb.py and test_repository_tool.py.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Removal of securesystemslib exceptions that are TUF-specific
occurs in securesystemslib PR #165https://github.com/secure-systems-lab/securesystemslib/pull/165
This commit adapts to those changes. Exceptions that are specific
to TUF should be in TUF and not in securesystemslib. This commit
uses those already-existing TUF exceptions instead of pointing to
securesystemslib exceptions that will be removed.
For example, securesystemslib has no notion of repositories, so
it's ridiculous to have a RepositoryError in securesystemslib and
ridiculous for TUF to use
securesystemslib.exceptions.RepositoryError.
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
The test for repository_tool.get_filepaths_in_directory now expects
absolute paths, and also now tests the *results* of the function's use when
the recursive flag is on.
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
