Elgato_dark/python-tuf
1
0
Fork 0
mirror of https://github.com/theupdateframework/python-tuf synced 2026-05-24 10:08:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge pull request #1790 from lukpueh/rm-all

Browse source 
Rm all legacy
This commit is contained in:
lukpueh 2022-02-04 14:01:53 +01:00 committed by GitHub
commit 31fd8d4f03
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
80 changed files with 16 additions and 32507 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
MANIFEST.in
View file

@ -10,6 +10,3 @@ graft tests
# Documentation
graft docs
recursive-include tuf *.md
# To remove
recursive-include tuf/scripts *.py

6
README.md
View file

@ -30,12 +30,6 @@ High-level support for implementing
[repository operations](https://theupdateframework.github.io/specification/latest/#repository-operations)
is planned but not yet provided: see [1.0.0 plans](https://github.com/theupdateframework/python-tuf/blob/develop/docs/1.0.0-ANNOUNCEMENT.md).
In addition to these APIs the project also provides a *legacy
implementation* with `tuf.client` implementing the client workflow and
`tuf.repository_tool` providing a high-level interface for repository
operations. The legacy implementation is going to be
[deprecated](https://github.com/theupdateframework/python-tuf/blob/develop/docs/1.0.0-ANNOUNCEMENT.md) in the near future.
The reference implementation strives to be a readable guide and demonstration
for those working on implementing TUF in their own languages, environments, or
update systems.

447
docs/CLI.md
View file

@ -1,447 +0,0 @@
# Command-Line Interface #
The TUF command-line interface (CLI) requires a full
[TUF installation](INSTALLATION.rst). Be sure to include the installation of
extra dependencies and C extensions (
```python3 -m pip install securesystemslib[crypto,pynacl]```).
The use of the CLI is documented with examples below.
----
# Basic Examples #
## Create a repository ##
Create a TUF repository in the current working directory. A cryptographic key
is created and set for each top-level role. The written Targets metadata does
not sign for any targets, nor does it delegate trust to any roles. The
`--init` call will also set up a client directory. By default, these
directories will be `./tufrepo` and `./tufclient`.
```Bash
$ repo.py --init
```
Optionally, the repository can be written to a specified location.
```Bash
$ repo.py --init --path </path/to/repo_dir>
```
The default top-level key files created with `--init` are saved to disk
encrypted, with a default password of 'pw'. Instead of using the default
password, the user can enter one on the command line for each top-level role.
These optional command-line options also work with other CLI actions (e.g.,
repo.py --add).
```Bash
$ repo.py --init [--targets_pw, --root_pw, --snapshot_pw, --timestamp_pw]
```
Create a bare TUF repository in the current working directory. A cryptographic
key is *not* created nor set for each top-level role.
```Bash
$ repo.py --init --bare
```
Create a TUF repository with [consistent
snapshots](https://github.com/theupdateframework/specification/blob/master/tuf-spec.md#7-consistent-snapshots)
enabled, where target filenames have their hash prepended (e.g.,
`<hash>.README.txt`), and metadata filenames have their version numbers
prepended (e.g., `<hash>.snapshot.json`).
```Bash
$ repo.py --init --consistent
```
## Add a target file ##
Copy a target file to the repo and add it to the Targets metadata (or the
Targets role specified in --role). More than one target file, or directory,
may be specified in --add. The --recursive option may be toggled to also
include files in subdirectories of a specified directory. The Snapshot
and Timestamp metadata are also updated and signed automatically, but this
behavior can be toggled off with --no_release.
```Bash
$ repo.py --add <foo.tar.gz> <bar.tar.gz>
$ repo.py --add </path/to/dir> [--recursive]
```
Similar to the --init case, the repository location can be chosen.
```Bash
$ repo.py --add <foo.tar.gz> --path </path/to/my_repo>
```
## Remove a target file ##
Remove a target file from the Targets metadata (or the Targets role specified
in --role). More than one target file or glob pattern may be specified in
--remove. The Snapshot and Timestamp metadata are also updated and signed
automatically, but this behavior can be toggled off with --no_release.
```Bash
$ repo.py --remove <glob_pattern> ...
```
Examples:
Remove all target files, that match `foo*.tgz,` from the Targets metadata.
```Bash
$ repo.py --remove "foo*.tgz"
```
Remove all target files from the `my_role` metadata.
```Bash
$ repo.py --remove "*" --role my_role --sign tufkeystore/my_role_key
```
## Generate key ##
Generate a cryptographic key. The generated key can later be used to sign
specific metadata with `--sign`. The supported key types are: `ecdsa`,
`ed25519`, and `rsa`. If a keytype is not given, an Ed25519 key is generated.
If adding a top-level key to a bare repo (i.e., repo.py --init --bare),
the filenames of the top-level keys must be "root_key," "targets_key,"
"snapshot_key," "timestamp_key." The filename can vary for any additional
top-level key.
```Bash
$ repo.py --key
$ repo.py --key <keytype>
$ repo.py --key <keytype> [--path </path/to/repo_dir> --pw [my_password],
--filename <key_filename>]
```
Instead of using a default password, the user can enter one on the command
line or be prompted for it via password masking.
```Bash
$ repo.py --key ecdsa --pw my_password
```
```Bash
$ repo.py --key rsa --pw
Enter a password for the RSA key (...):
Confirm:
```
## Sign metadata ##
Sign, with the specified key(s), the metadata of the role indicated in --role.
The Snapshot and Timestamp role are also automatically signed, if possible, but
this behavior can be disabled with --no_release.
```Bash
$ repo.py --sign </path/to/key> ... [--role <rolename>, --path </path/to/repo>]
```
For example, to sign the delegated `foo` metadata:
```Bash
$ repo.py --sign </path/to/foo_key> --role foo
```
## Trust keys ##
The Root role specifies the trusted keys of the top-level roles, including
itself. The --trust command-line option, in conjunction with --pubkeys and
--role, can be used to indicate the trusted keys of a role.
```Bash
$ repo.py --trust --pubkeys </path/to/foo_key.pub> --role <rolename>
```
For example:
```Bash
$ repo.py --init --bare
$ repo.py --trust --pubkeys tufkeystore/my_key.pub tufkeystore/my_key_too.pub
--role root
```
### Distrust keys ###
Conversely, the Root role can discontinue trust of specified key(s).
Example of how to discontinue trust of a key:
```Bash
$ repo.py --distrust --pubkeys tufkeystore/my_key_too.pub --role root
```
## Delegations ##
Delegate trust of target files from the Targets role (or the one specified in
--role) to some other role (--delegatee). --delegatee is trusted to sign for
target files that match the delegated glob pattern(s). The --delegate option
does not create metadata for the delegated role, rather it updates the
delegator's metadata to list the delegation to --delegatee. The Snapshot and
Timestamp metadata are also updated and signed automatically, but this behavior
can be toggled off with --no_release.
```Bash
$ repo.py --delegate <glob pattern> ... --delegatee <rolename> --pubkeys
</path/to/pubkey.pub> ... [--role <rolename> --terminating --threshold <X>
--sign </path/to/role_privkey>]
```
For example, to delegate trust of `foo*.gz` packages to the `foo` role:
```
$ repo.py --delegate "foo*.tgz" --delegatee foo --pubkeys tufkeystore/foo.pub
```
## Revocations ##
Revoke trust of target files from a delegated role (--delegatee). The
"targets" role performs the revocation if --role is not specified. The
--revoke option does not delete the metadata belonging to --delegatee, instead
it removes the delegation to it from the delegator's (or --role) metadata. The
Snapshot and Timestamp metadata are also updated and signed automatically, but
this behavior can be toggled off with --no_release.
```Bash
$ repo.py --revoke --delegatee <rolename> [--role <rolename>
--sign </path/to/role_privkey>]
```
## Verbosity ##
Set the verbosity of the logger (2, by default). The lower the number, the
greater the verbosity. Logger messages are saved to `tuf.log` in the current
working directory.
```Bash
$ repo.py --verbose <0-5>
```
## Clean ##
Delete the repo in the current working directory, or the one specified with
`--path`. Specifically, the `tufrepo`, `tufclient`, and `tufkeystore`
directories are deleted.
```Bash
$ repo.py --clean
$ repo.py --clean --path </path/to/dirty/repo>
```
----
# Further Examples #
## Basic Update Delivery ##
Steps:
(1) initialize a repo.
(2) delegate trust of target files to another role.
(3) add a trusted file to the delegated role.
(4) fetch the trusted file from the delegated role.
```Bash
Step (1)
$ repo.py --init
Step (2)
$ repo.py --key ed25519 --filename mykey
$ repo.py --delegate "README.*" --delegatee myrole --pubkeys tufkeystore/mykey.pub
$ repo.py --sign tufkeystore/mykey --role myrole
Enter a password for the encrypted key (tufkeystore/mykey):
$ echo "my readme text" > README.txt
Step (3)
$ repo.py --add README.txt --role myrole --sign tufkeystore/mykey
Enter a password for the encrypted key (tufkeystore/mykey):
```
Serve the repo
```Bash
$ python3 -m http.server 8001
```
```Bash
Step (4)
$ client.py --repo http://localhost:8001 README.txt
$ tree .
.
├── tuf.log
├── tufrepo
│   └── metadata
│   ├── current
│   │   ├── 1.root.json
│   │   ├── myrole.json
│   │   ├── root.json
│   │   ├── snapshot.json
│   │   ├── targets.json
│   │   └── timestamp.json
│   └── previous
│   ├── 1.root.json
│   ├── root.json
│   ├── snapshot.json
│   ├── targets.json
│   └── timestamp.json
└── tuftargets
└── README.txt
5 directories, 13 files
```
## Correcting a Key ##
The filename of the top-level keys must be "root_key," "targets_key,"
"snapshot_key," and "root_key." The filename can vary for any additional
top-level key.
Steps:
(1) initialize a repo containing default keys for the top-level roles.
(2) distrust the default key for the root role.
(3) create a new key and trust its use with the root role.
(4) sign the root metadata file.
```Bash
Step (1)
$ repo.py --init
Step (2)
$ repo.py --distrust --pubkeys tufkeystore/root_key.pub --role root
Step (3)
$ repo.py --key ed25519 --filename root_key
$ repo.py --trust --pubkeys tufkeystore/root_key.pub --role root
Step (4)
$ repo.py --sign tufkeystore/root_key --role root
Enter a password for the encrypted key (tufkeystore/root_key):
```
## More Update Delivery ##
Steps:
(1) create a bare repo.
(2) add keys to the top-level roles.
(3) delegate trust of particular target files to another role X, where role X
has a signature threshold 2 and is marked as a terminating delegation. The
keys for role X and Y should be created prior to performing the delegation.
(4) Delegate from role X to role Y.
(5) have role X sign for a file also signed by the Targets role, to demonstrate
the expected file that should be downloaded by the client.
(6) perform an update.
(7) halt the server, add README.txt to the Targets role, restart the server,
and fetch the Target's role README.txt.
(8) Add LICENSE to 'role_y' and demonstrate that the client must not fetch it
because 'role_x' is a terminating delegation (and hasn't signed for it).
```Bash
Steps (1) and (2)
$ repo.py --init --consistent --bare
$ repo.py --key ed25519 --filename root_key
$ repo.py --trust --pubkeys tufkeystore/root_key.pub --role root
$ repo.py --key ecdsa --filename targets_key
$ repo.py --trust --pubkeys tufkeystore/targets_key.pub --role targets
$ repo.py --key rsa --filename snapshot_key
$ repo.py --trust --pubkeys tufkeystore/snapshot_key.pub --role snapshot
$ repo.py --key ecdsa --filename timestamp_key
$ repo.py --trust --pubkeys tufkeystore/timestamp_key.pub --role timestamp
$ repo.py --sign tufkeystore/root_key --role root
Enter a password for the encrypted key (tufkeystore/root_key):
$ repo.py --sign tufkeystore/targets_key --role targets
Enter a password for the encrypted key (tufkeystore/targets_key):
```
```Bash
Steps (3) and (4)
$ repo.py --key ed25519 --filename key_x
$ repo.py --key ed25519 --filename key_x2
$ repo.py --delegate "README.*" "LICENSE" --delegatee role_x --pubkeys
tufkeystore/key_x.pub tufkeystore/key_x2.pub --threshold 2 --terminating
$ repo.py --sign tufkeystore/key_x tufkeystore/key_x2 --role role_x
$ repo.py --key ed25519 --filename key_y
$ repo.py --delegate "README.*" "LICENSE" --delegatee role_y --role role_x
--pubkeys tufkeystore/key_y.pub --sign tufkeystore/key_x tufkeystore/key_x2
$ repo.py --sign tufkeystore/key_y --role role_y
```
```Bash
Steps (5) and (6)
$ echo "role_x's readme" > README.txt
$ repo.py --add README.txt --role role_x --sign tufkeystore/key_x tufkeystore/key_x2
```
Serve the repo
```Bash
$ python3 -m http.server 8001
```
Fetch the role x's README.txt
```Bash
$ client.py --repo http://localhost:8001 README.txt
$ cat tuftargets/README.txt
role_x's readme
```
```Bash
Step (7)
halt the server...
$ echo "Target role's readme" > README.txt
$ repo.py --add README.txt
restart the server...
```
```Bash
$ rm -rf tuftargets/ tuf.log
$ client.py --repo http://localhost:8001 README.txt
$ cat tuftargets/README.txt
Target role's readme
```
```Bash
Step (8)
$ echo "role_y's license" > LICENSE
$ repo.py --add LICENSE --role role_y --sign tufkeystore/key_y
```
```Bash
$ rm -rf tuftargets/ tuf.log
$ client.py --repo http://localhost:8001 LICENSE
Error: 'LICENSE' not found.
```

10
docs/GETTING_STARTED.rst
View file

@ -1,10 +0,0 @@
Getting Started
---------------
- `Overview of TUF <https://theupdateframework.io/overview/>`_
- `Installation <INSTALLATION.rst>`_
- Beginner Tutorials (using the basic command-line interface):
- `Quickstart <QUICKSTART.md>`_
- `CLI Documentation and Examples <CLI.md>`_
- `Advanced Tutorial <TUTORIAL.md>`_
- `Guidelines for Contributors <CONTRIBUTORS.rst>`_

149
docs/QUICKSTART.md
View file

@ -1,149 +0,0 @@
# Quickstart #
In this quickstart tutorial, we'll use the basic TUF command-line interface
(CLI), which includes the `repo.py` script and the `client.py` script, to set
up a repository with an update and metadata about that update, then download
and verify that update as a client.
Unlike the underlying TUF modules that the CLI uses, the CLI itself is a bit
bare-bones. Using the CLI is the easiest way to familiarize yourself with
how TUF works, however. It will serve as a very basic update system.
----
**Step (0)** - Make sure TUF is installed.
Make sure that TUF is installed, along with some of the optional cryptographic
libraries and C extensions. Try this command to do that:
`python3 -m pip install securesystemslib[colors,crypto,pynacl] tuf`
If you run into errors during that pip command, please consult the more
detailed [TUF Installation Instructions](INSTALLATION.rst). (There are some
system libraries that you may need to install first.)
**Step (1)** - Create a basic repository and client.
The following command will set up a basic update repository and basic client
that knows about the repository. `tufrepo`, `tufkeystore`, and
`tufclient` directories will be created in the current directory.
```Bash
$ repo.py --init
```
Four sets of keys are created in the `tufkeystore` directory. Initial metadata
about the repository is created in the `tufrepo` directory, and also provided
to the client in the `tufclient` directory.
**Step (2)** - Add an update to the repository.
We'll create a target file that will later be delivered as an update to clients.
Metadata about that file will be created and signed, and added to the
repository's metadata.
```Bash
$ echo 'Test file' > testfile
$ repo.py --add testfile
$ tree tufrepo/
tufrepo/
├── metadata
│   ├── 1.root.json
│   ├── root.json
│   ├── snapshot.json
│   ├── targets.json
│   └── timestamp.json
├── metadata.staged
│   ├── 1.root.json
│   ├── root.json
│   ├── snapshot.json
│   ├── targets.json
│   └── timestamp.json
└── targets
└── testfile
3 directories, 11 files
```
The new file `testfile` is added to the repository, and metadata is updated in
the `tufrepo` directory. The Targets metadata (`targets.json`) now includes
the file size and hashes of the `testfile` target file, and this metadata is
signed by the Targets role's key, so that clients can verify that metadata
about `testfile` and then verify `testfile` itself.
**Step (3)** - Serve the repo.
We'll host a toy http server containing the `testfile` update and the
repository's metadata.
```Bash
$ cd "tufrepo/"
$ python3 -m http.server 8001
```
**Step (4)** - Obtain and verify the `testfile` update on a client.
The client can request the package `testfile` from the repository. TUF will
download and verify metadata from the repository as necessary to determine
what the trustworthy hashes and length of `testfile` are, then download
the target `testfile` from the repository and keep it only if it matches that
trustworthy metadata.
```Bash
$ cd "../tufclient/"
$ client.py --repo http://localhost:8001 testfile
$ tree
.
├── tufrepo
│   └── metadata
│   ├── current
│   │   ├── 1.root.json
│   │   ├── root.json
│   │   ├── snapshot.json
│   │   ├── targets.json
│   │   └── timestamp.json
│   └── previous
│   ├── 1.root.json
│   ├── root.json
│   ├── snapshot.json
│   ├── targets.json
│   └── timestamp.json
└── tuftargets
└── testfile
5 directories, 11 files
```
Now that a trustworthy update target has been obtained, an updater can proceed
however it normally would to install or use the update.
----
### Next Steps
TUF provides functionality for both ends of a software update system, the
**update provider** and the **update client**.
`repo.py` made use of `tuf.repository_tool`'s functionality for an update
provider, helping you produce and sign metadata about your updates.
`client.py` made use of `tuf.client.updater`'s client-side functionality,
performing download and the critical verification steps for metadata and the
update itself.
You can look at [CLI.md](CLI.md) to toy with the TUF CLI a bit more.
After that, try out using the underlying modules for a great deal more control.
The more detailed [Advanced Tutorial](TUTORIAL.md) shows you how to use the
underlying modules, `repository_tool` and `updater`.
Ultimately, a sophisticated update client will use or re-implement those
underlying modules. The TUF design is intended to play well with any update
workflow.
Please provide feedback or questions for this or other tutorials, or
TUF in general, by checking out
[our contact info](https://github.com/theupdateframework/python-tuf#contact), or
creating [issues](https://github.com/theupdateframework/python-tuf/issues) in this
repository!

696
docs/TUTORIAL.md
View file

@ -1,696 +0,0 @@
# Advanced Tutorial #
## Table of Contents ##
- [How to Create and Modify a TUF Repository](#how-to-create-and-modify-a-tuf-repository)
- [Overview](#overview)
- [Keys](#keys)
- [Create RSA Keys](#create-rsa-keys)
- [Import RSA Keys](#import-rsa-keys)
- [Create and Import Ed25519 Keys](#create-and-import-ed25519-keys)
- [Create Top-level Metadata](#create-top-level-metadata)
- [Create Root](#create-root)
- [Create Timestamp, Snapshot, Targets](#create-timestamp-snapshot-targets)
- [Targets](#targets)
- [Add Target Files](#add-target-files)
- [Remove Target Files](#remove-target-files)
- [Delegations](#delegations)
- [Revoke Delegated Role](#revoke-delegated-role)
- [Wrap-up](#wrap-up)
- [Delegate to Hashed Bins](#delegate-to-hashed-bins)
- [Consistent Snapshots](#consistent-snapshots)
- [How to Perform an Update](#how-to-perform-an-update)
## How to Create and Modify a TUF Repository ##
### Overview ###
A software update system must follow two steps to integrate The Update
Framework (TUF). First, it must add the framework to the client side of the
update system. The [tuf.client.updater](../tuf/client/README.md) module assists in
integrating TUF on the client side. Second, the software repository on the
server side must be modified to include a minimum of four top-level metadata
(root.json, targets.json, snapshot.json, and timestamp.json). No additional
software is required to convert a software repository to a TUF one. The
low-level repository tool that generates the required TUF metadata for a
software repository is the focus of this tutorial. There is also separate
document that [demonstrates how TUF protects against malicious
updates](../tuf/ATTACKS.md).
The [repository tool](../tuf/repository_tool.py) contains functions to generate
all of the files needed to populate and manage a TUF repository. The tool may
either be imported into a Python module, or used with the Python interpreter in
interactive mode.
A repository object that encapsulates the metadata files of the repository can
be created or loaded by the repository tool. Repository maintainers can modify
the repository object to manipulate the metadata files stored on the
repository. TUF clients use the metadata files to validate files requested and
downloaded. In addition to the repository object, where the majority of
changes are made, the repository tool provides functions to generate and
persist cryptographic keys. The framework utilizes cryptographic keys to sign
and verify metadata files.
To begin, cryptographic keys are generated with the repository tool. However,
before metadata files can be validated by clients and target files fetched in a
secure manner, public keys must be pinned to particular metadata roles and
metadata signed by role's private keys. After covering keys, the four required
top-level metadata are created next. Examples are given demonstrating the
expected work flow, where the metadata roles are created in a specific order,
keys imported and loaded, and metadata signed and written to disk. Lastly,
target files are added to the repository, and a custom delegation performed to
extend the default roles of the repository. By the end, a fully populated TUF
repository is generated that can be used by clients to securely download
updates.
### Keys ###
The repository tool supports multiple public-key algorithms, such as
[RSA](https://en.wikipedia.org/wiki/RSA_%28cryptosystem%29) and
[Ed25519](https://ed25519.cr.yp.to/), and multiple cryptography libraries.
Using [RSA-PSS](https://tools.ietf.org/html/rfc8017#section-8.1) or
[ECDSA](https://en.wikipedia.org/wiki/Elliptic_Curve_Digital_Signature_Algorithm)
signatures requires the [cryptography](https://cryptography.io/) library. If
generation of Ed25519 signatures is needed
[PyNaCl](https://github.com/pyca/pynacl) library should be installed. This
tutorial assumes both dependencies are installed: refer to
[Installation Instructions](INSTALLATION.rst#install-with-more-cryptographic-flexibility)
for details.
The Ed25519 and ECDSA keys are stored in JSON format and RSA keys are stored in PEM
format. Private keys are encrypted and passphrase-protected (strengthened with
PBKDF2-HMAC-SHA256.) Generating, importing, and loading cryptographic key
files can be done with functions available in the repository tool.
To start, a public and private RSA key pair is generated with the
`generate_and_write_rsa_keypair()` function. The keys generated next are
needed to sign the repository metadata files created in upcoming sub-sections.
Note: In the instructions below, lines that start with `>>>` denote commands
that should be entered by the reader, `#` begins the start of a comment, and
text without prepended symbols is the output of a command.
#### Create RSA Keys ####
```python
>>> from tuf.repository_tool import *
# Generate and write the first of two root keys for the TUF repository. The
# following function creates an RSA key pair, where the private key is saved to
# "root_key" and the public key to "root_key.pub" (both saved to the current
# working directory).
>>> generate_and_write_rsa_keypair(password="password", filepath="root_key", bits=2048)
# If the key length is unspecified, it defaults to 3072 bits. A length of less
# than 2048 bits raises an exception. A similar function is available to supply
# a password on the prompt. If an empty password is entered, the private key
# is saved unencrypted.
>>> generate_and_write_rsa_keypair_with_prompt(filepath="root_key2")
enter password to encrypt private key file '/path/to/root_key2'
(leave empty if key should not be encrypted):
Confirm:
```
The following four key files should now exist:
1. **root_key**
2. **root_key.pub**
3. **root_key2**
4. **root_key2.pub**
If a filepath is not given, the KEYID of the generated key is used as the
filename. The key files are written to the current working directory.
```python
# Continuing from the previous section . . .
>>> generate_and_write_rsa_keypair_with_prompt()
enter password to encrypt private key file '/path/to/KEYID'
(leave empty if key should not be encrypted):
Confirm:
```
### Import RSA Keys ###
```python
# Continuing from the previous section . . .
# Import an existing public key.
>>> public_root_key = import_rsa_publickey_from_file("root_key.pub")
# Import an existing private key. Importing a private key requires a password,
# whereas importing a public key does not.
>>> private_root_key = import_rsa_privatekey_from_file("root_key")
enter password to decrypt private key file '/path/to/root_key'
(leave empty if key not encrypted):
```
### Create and Import Ed25519 Keys ###
```Python
# Continuing from the previous section . . .
# The same generation and import functions as for rsa keys exist for ed25519
>>> generate_and_write_ed25519_keypair_with_prompt(filepath='ed25519_key')
enter password to encrypt private key file '/path/to/ed25519_key'
(leave empty if key should not be encrypted):
Confirm:
# Import the ed25519 public key just created . . .
>>> public_ed25519_key = import_ed25519_publickey_from_file('ed25519_key.pub')
# and its corresponding private key.
>>> private_ed25519_key = import_ed25519_privatekey_from_file('ed25519_key')
enter password to decrypt private key file '/path/to/ed25519_key'
(leave empty if key should not be encrypted):
```
Note: Methods are also available to generate and write keys from memory.
* generate_ed25519_key()
* generate_ecdsa_key()
* generate_rsa_key()
* import_ecdsakey_from_pem(pem)
* import_rsakey_from_pem(pem)
### Create Top-level Metadata ###
The [metadata document](METADATA.md) outlines the JSON files that must exist
on a TUF repository. The following sub-sections demonstrate the
`repository_tool.py` calls repository maintainers may issue to generate the
required roles. The top-level roles to be created are `root`, `timestamp`,
`snapshot`, and `target`.
We begin with `root`, the locus of trust that specifies the public keys of the
top-level roles, including itself.
#### Create Root ####
```python
# Continuing from the previous section . . .
# Create a new Repository object that holds the file path to the TUF repository
# and the four top-level role objects (Root, Targets, Snapshot, Timestamp).
# Metadata files are created when repository.writeall() or repository.write()
# are called. The repository directory is created if it does not exist. You
# may see log messages indicating any directories created.
>>> repository = create_new_repository("repository")
# The Repository instance, 'repository', initially contains top-level Metadata
# objects. Add one of the public keys, created in the previous section, to the
# root role. Metadata is considered valid if it is signed by the public key's
# corresponding private key.
>>> repository.root.add_verification_key(public_root_key)
# A role's verification key(s) (to be more precise, the verification key's
# keyid) may be queried. Other attributes include: signing_keys, version,
# signatures, expiration, threshold, and delegations (attribute available only
# to a Targets role).
>>> repository.root.keys
['b23514431a53676595922e955c2d547293da4a7917e3ca243a175e72bbf718df']
# Add a second public key to the root role. Although previously generated and
# saved to a file, the second public key must be imported before it can added
# to a role.
>>> public_root_key2 = import_rsa_publickey_from_file("root_key2.pub")
>>> repository.root.add_verification_key(public_root_key2)
# The threshold of each role defaults to 1. Maintainers may change the
# threshold value, but repository_tool.py validates thresholds and warns users.
# Set the threshold of the root role to 2, which means the root metadata file
# is considered valid if it's signed by at least two valid keys. We also load
# the second private key, which hasn't been imported yet.
>>> repository.root.threshold = 2
>>> private_root_key2 = import_rsa_privatekey_from_file("root_key2", password="password")
# Load the root signing keys to the repository, which writeall() or write()
# (write multiple roles, or a single role, to disk) use to sign the root
# metadata.
>>> repository.root.load_signing_key(private_root_key)
>>> repository.root.load_signing_key(private_root_key2)
# repository.status() shows missing verification and signing keys for the
# top-level roles, and whether signatures can be created (also see #955).
# This output shows that so far only the "root" role meets the key threshold and
# can successfully sign its metadata.
>>> repository.status()
'targets' role contains 0 / 1 public keys.
'snapshot' role contains 0 / 1 public keys.
'timestamp' role contains 0 / 1 public keys.
'root' role contains 2 / 2 signatures.
'targets' role contains 0 / 1 signatures.
# In the next section we update the other top-level roles and create a repository
# with valid metadata.
```
#### Create Timestamp, Snapshot, Targets
Now that `root.json` has been set, the other top-level roles may be created.
The signing keys added to these roles must correspond to the public keys
specified by the Root role.
On the client side, `root.json` must always exist. The other top-level roles,
created next, are requested by repository clients in (Root -> Timestamp ->
Snapshot -> Targets) order to ensure required metadata is downloaded in a
secure manner.
```python
# Continuing from the previous section . . .
# 'datetime' module needed to optionally set a role's expiration.
>>> import datetime
# Generate keys for the remaining top-level roles. The root keys have been set above.
>>> generate_and_write_rsa_keypair(password='password', filepath='targets_key')
>>> generate_and_write_rsa_keypair(password='password', filepath='snapshot_key')
>>> generate_and_write_rsa_keypair(password='password', filepath='timestamp_key')
# Add the verification keys of the remaining top-level roles.
>>> repository.targets.add_verification_key(import_rsa_publickey_from_file('targets_key.pub'))
>>> repository.snapshot.add_verification_key(import_rsa_publickey_from_file('snapshot_key.pub'))
>>> repository.timestamp.add_verification_key(import_rsa_publickey_from_file('timestamp_key.pub'))
# Import the signing keys of the remaining top-level roles.
>>> private_targets_key = import_rsa_privatekey_from_file('targets_key', password='password')
>>> private_snapshot_key = import_rsa_privatekey_from_file('snapshot_key', password='password')
>>> private_timestamp_key = import_rsa_privatekey_from_file('timestamp_key', password='password')
# Load the signing keys of the remaining roles so that valid signatures are
# generated when repository.writeall() is called.
>>> repository.targets.load_signing_key(private_targets_key)
>>> repository.snapshot.load_signing_key(private_snapshot_key)
>>> repository.timestamp.load_signing_key(private_timestamp_key)
# Optionally set the expiration date of the timestamp role. By default, roles
# are set to expire as follows: root(1 year), targets(3 months), snapshot(1
# week), timestamp(1 day).
>>> repository.timestamp.expiration = datetime.datetime(2080, 10, 28, 12, 8)
# Mark roles for metadata update (see #964, #958)
>>> repository.mark_dirty(['root', 'snapshot', 'targets', 'timestamp'])
# Write all metadata to "repository/metadata.staged/"
>>> repository.writeall()
```
### Targets ###
TUF makes it possible for clients to validate downloaded target files by
including a target file's length, hash(es), and filepath in metadata. The
filepaths are relative to a `targets/` directory on the software repository. A
TUF client can download a target file by first updating the latest copy of
metadata (and thus available targets), verifying that their length and hashes
are valid, and saving the target file(s) locally to complete the update
process.
In this section, the target files intended for clients are added to a
repository and listed in `targets.json` metadata.
#### Add Target Files ####
The repository maintainer adds target files to roles (e.g., `targets` and
`unclaimed`) by specifying their filepaths. The target files must exist at the
specified filepaths before the repository tool can generate and add their
(hash(es), length, and filepath) to metadata.
First, the actual target files are manually created and saved to the `targets/`
directory of the repository:
```Bash
# Create and save target files to the targets directory of the software
# repository.
$ cd repository/targets/
$ echo 'file1' > file1.txt
$ echo 'file2' > file2.txt
$ echo 'file3' > file3.txt
$ mkdir myproject; echo 'file4' > myproject/file4.txt
$ cd ../../
```
With the target files available on the `targets/` directory of the software
repository, the `add_targets()` method of a Targets role can be called to add
the target filepaths to metadata.
```python
# Continuing from the previous section . . .
# NOTE: If you exited the Python interactive interpreter above you need to
# re-import the repository_tool-functions and re-load the repository and
# signing keys.
>>> from tuf.repository_tool import *
# The 'os' module is needed to gather file attributes, which will be included
# in a custom field for some of the target files added to metadata.
>>> import os
# Load the repository created in the previous section. This repository so far
# contains metadata for the top-level roles, but no target paths are yet listed
# in targets metadata.
>>> repository = load_repository('repository')
# Create a list of all targets in the directory.
>>> list_of_targets = ['file1.txt', 'file2.txt', 'file3.txt']
# Add the list of target paths to the metadata of the top-level Targets role.
# Any target file paths that might already exist are NOT replaced, and
# add_targets() does not create or move target files on the file system. Any
# target paths added to a role must fall under the expected targets directory,
# otherwise an exception is raised. The targets added to a role should actually
# exist once writeall() or write() is called, so that the hash and size of
# these targets can be included in Targets metadata.
>>> repository.targets.add_targets(list_of_targets)
# Individual target files may also be added to roles, including custom data
# about the target. In the example below, file permissions of the target
# (octal number specifying file access for owner, group, others e.g., 0755) is
# added alongside the default fileinfo. All target objects in metadata include
# the target's filepath, hash, and length.
# Note: target path passed to add_target() method has to be relative
# to the targets directory or an exception is raised.
>>> target4_filepath = 'myproject/file4.txt'
>>> target4_abspath = os.path.abspath(os.path.join('repository', 'targets', target4_filepath))
>>> octal_file_permissions = oct(os.stat(target4_abspath).st_mode)[4:]
>>> custom_file_permissions = {'file_permissions': octal_file_permissions}
>>> repository.targets.add_target(target4_filepath, custom_file_permissions)
```
The private keys of roles affected by the changes above must now be imported and
loaded. `targets.json` must be signed because a target file was added to its
metadata. `snapshot.json` keys must be loaded and its metadata signed because
`targets.json` has changed. Similarly, since `snapshot.json` has changed, the
`timestamp.json` role must also be signed.
```Python
# Continuing from the previous section . . .
# The private key of the updated targets metadata must be re-loaded before it
# can be signed and written (Note the load_repository() call above).
>>> private_targets_key = import_rsa_privatekey_from_file('targets_key')
enter password to decrypt private key file '/path/to/targets_key'
(leave empty if key not encrypted):
>>> repository.targets.load_signing_key(private_targets_key)
# Due to the load_repository() and new versions of metadata, we must also load
# the private keys of Snapshot and Timestamp to generate a valid set of metadata.
>>> private_snapshot_key = import_rsa_privatekey_from_file('snapshot_key')
enter password to decrypt private key file '/path/to/snapshot_key'
(leave empty if key not encrypted):
>>> repository.snapshot.load_signing_key(private_snapshot_key)
>>> private_timestamp_key = import_rsa_privatekey_from_file('timestamp_key')
enter password to decrypt private key file '/path/to/timestamp_key'
(leave empty if key not encrypted):
>>> repository.timestamp.load_signing_key(private_timestamp_key)
# Mark roles for metadata update (see #964, #958)
>>> repository.mark_dirty(['snapshot', 'targets', 'timestamp'])
# Generate new versions of the modified top-level metadata (targets, snapshot,
# and timestamp).
>>> repository.writeall()
```
#### Remove Target Files ####
Target files previously added to roles may also be removed. Removing a target
file requires first removing the target from a role and then writing the
new metadata to disk.
```python
# Continuing from the previous section . . .
# Remove a target file listed in the "targets" metadata. The target file is
# not actually deleted from the file system.
>>> repository.targets.remove_target('myproject/file4.txt')
# Mark roles for metadata update (see #964, #958)
>>> repository.mark_dirty(['snapshot', 'targets', 'timestamp'])
>>> repository.writeall()
```
#### Excursion: Dump Metadata and Append Signature ####
The following two functions are intended for those that wish to independently
sign metadata. Repository maintainers can dump the portion of metadata that is
normally signed, sign it with an external signing tool, and append the
signature to already existing metadata.
First, the signable portion of metadata can be generated as follows:
```Python
>>> signable_content = dump_signable_metadata('repository/metadata.staged/timestamp.json')
```
Then, use a tool like securesystemslib to create a signature over the signable
portion. *Note, to make the signing key count towards the role's signature
threshold, it needs to be added to `root.json`, e.g. via
`repository.timestamp.add_verification_key(key)` (not shown in below snippet).*
```python
>>> from securesystemslib.formats import encode_canonical
>>> from securesystemslib.keys import create_signature
>>> private_ed25519_key = import_ed25519_privatekey_from_file('ed25519_key')
enter password to decrypt private key file '/path/to/ed25519_key'
>>> signature = create_signature(
... private_ed25519_key, encode_canonical(signable_content).encode())
```
Finally, append the signature to the metadata
```Python
>>> append_signature(signature, 'repository/metadata.staged/timestamp.json')
```
Note that the format of the signature is the format expected in metadata, which
is a dictionary that contains a KEYID, the signature itself, etc. See the
specification and [METADATA.md](METADATA.md) for a detailed example.
### Delegations ###
All of the target files available on the software repository created so far
have been added to one role (the top-level Targets role). However, what if
multiple developers are responsible for the files of a project? What if
responsibility separation is desired? Performing a delegation, where one role
delegates trust of some paths to another role, is an option for integrators
that require additional roles on top of the top-level roles available by
default.
In the next sub-section, the `unclaimed` role is delegated from the top-level
`targets` role. The `targets` role specifies the delegated role's public keys,
the paths it is trusted to provide, and its role name. <!--
TODO: Uncomment together with "Revoke Delegated Role" section below
Furthermore, the example
below demonstrates a nested delegation from `unclaimed` to `django`. Once a
role has delegated trust to another, the delegated role may independently add
targets and generate signed metadata.
-->
```python
# Continuing from the previous section . . .
# Generate a key for a new delegated role named "unclaimed".
>>> generate_and_write_rsa_keypair(password='password', filepath='unclaimed_key', bits=2048)
>>> public_unclaimed_key = import_rsa_publickey_from_file('unclaimed_key.pub')
# Make a delegation (delegate trust of 'myproject/*.txt' files) from "targets"
# to "unclaimed", where "unclaimed" initially contains zero targets.
>>> repository.targets.delegate('unclaimed', [public_unclaimed_key], ['myproject/*.txt'])
# Thereafter, we can access the delegated role by its name to e.g. add target
# files, just like we did with the top-level targets role.
>>> repository.targets("unclaimed").add_target("myproject/file4.txt")
# Load the private key of "unclaimed" so that unclaimed's metadata can be
# signed, and valid metadata created.
>>> private_unclaimed_key = import_rsa_privatekey_from_file('unclaimed_key', password='password')
>>> repository.targets("unclaimed").load_signing_key(private_unclaimed_key)
# Mark roles for metadata update (see #964, #958)
>>> repository.mark_dirty(['snapshot', 'targets','timestamp', 'unclaimed'])
>>> repository.writeall()
```
<!--
TODO: Integrate section with an updated delegation tutorial.
As it is now, it just messes up the state of the repository, i.e. marks
"unclaimed" as dirty, although there is nothing new to write.
#### Revoke Delegated Role ####
```python
# Continuing from the previous section . . .
# Create a delegated role that will be revoked in the next step...
>>> repository.targets('unclaimed').delegate("django", [public_unclaimed_key], ['bar*.tgz'])
# Revoke "django" and write the metadata of all remaining roles.
>>> repository.targets('unclaimed').revoke("django")
>>> repository.writeall()
```
-->
#### Wrap-up ####
In summary, the five steps a repository maintainer follows to create a TUF
repository are:
1. Create a directory for the software repository that holds the TUF metadata and the target files.
2. Create top-level roles (`root.json`, `snapshot.json`, `targets.json`, and `timestamp.json`.)
3. Add target files to the `targets` role.
4. Optionally, create delegated roles to distribute target files.
5. Write the changes.
The repository tool saves repository changes to a `metadata.staged` directory.
Repository maintainers may push finalized changes to the "live" repository by
copying the staged directory to its destination.
```Bash
# Copy the staged metadata directory changes to the live repository.
$ cp -r "repository/metadata.staged/" "repository/metadata/"
```
## Consistent Snapshots ##
The basic TUF repository we have generated above is adequate for repositories
that have some way of guaranteeing consistency of repository data. A community
software repository is one example where consistency of files and metadata can
become an issue. Repositories of this kind are continually updated by multiple
maintainers and software authors uploading their packages, increasing the
likelihood that a client downloading version X of a release unexpectedly
requests the target files of a version Y just released.
To guarantee consistency of metadata and target files, a repository may
optionally support multiple versions of `snapshot.json` simultaneously, where a
client with version 1 of `snapshot.json` can download `target_file.zip` and
another client with version 2 of `snapshot.json` can also download a different
`target_file.zip` (same file name, but different file digest.) If the
`consistent_snapshot` parameter of writeall() or write() are `True`, metadata
and target file names on the file system have their digests prepended (note:
target file names specified in metadata do not contain digests in their names.)
The repository maintainer is responsible for the duration of multiple versions
of metadata and target files available on a repository. Generating consistent
metadata and target files on the repository is enabled by setting the
`consistent_snapshot` argument of `writeall()` or `write()` . Note that
changing the consistent_snapshot setting involves writing a new version of
root.
<!--
TODO: Integrate section with an updated consistent snapshot tutorial.
As it is now, it just messes up the state of the repository, i.e. marks
"root" as dirty, although all other metadata needs to be re-written with
<VERSION> prefix and target files need to be re-written with <HASH> prefix in
their filenames.
```Python
# ----- Tutorial Section: Consistent Snapshots
>>> repository.root.load_signing_key(private_root_key)
>>> repository.root.load_signing_key(private_root_key2)
>>> repository.writeall(consistent_snapshot=True)
```
-->
## Delegate to Hashed Bins ##
Why use hashed bin delegations?
For software update systems with a large number of target files, delegating to
hashed bins (a special type of delegated role) might be an easier alternative
to manually performing the delegations. How many target files should each
delegated role contain? How will these delegations affect the number of
metadata that clients must additionally download in a typical update? Hashed
bin delegations are available to integrators that rather not deal with the
management of delegated roles and a great number of target files.
A large number of target files may be distributed to multiple hashed bins with
`delegate_hashed_bins()`. The metadata files of delegated roles will be nearly
equal in size (i.e., target file paths are uniformly distributed by calculating
the target filepath's digest and determining which bin it should reside in.)
The updater client will use "lazy bin walk" (visit and download the minimum
metadata required to find a target) to find a target file's hashed bin
destination. This method is intended for repositories with a large number of
target files, a way of easily distributing and managing the metadata that lists
the targets, and minimizing the number of metadata files (and size) downloaded
by the client.
The `delegate_hashed_bins()` method has the following form:
```Python
delegate_hashed_bins(list_of_targets, keys_of_hashed_bins, number_of_bins)
```
We next provide a complete example of retrieving target paths to add to hashed
bins, performing the hashed bin delegations, signing them, and delegating paths
to some role.
```Python
# Continuing from the previous section . . .
# Remove 'myproject/file4.txt' from unclaimed role and instead further delegate
# all targets in myproject/ to hashed bins.
>>> repository.targets('unclaimed').remove_target("myproject/file4.txt")
# Get a list of target paths for the hashed bins.
>>> targets = ['myproject/file4.txt']
# Delegate trust to 32 hashed bin roles. Each role is responsible for the set
# of target files, determined by the path hash prefix. TUF evenly distributes
# hexadecimal ranges over the chosen number of bins (see output).
# To initialize the bins we use one key, which TUF warns us about (see output).
# However, we can assign separate keys to each bin, with the method used in
# previous sections, accessing a bin by its hash prefix range name, e.g.:
# "repository.targets('00-07').add_verification_key('public_00-07_key')".
>>> repository.targets('unclaimed').delegate_hashed_bins(
... targets, [public_unclaimed_key], 32)
Creating hashed bin delegations.
1 total targets.
32 hashed bins.
256 total hash prefixes.
Each bin ranges over 8 hash prefixes.
Adding a verification key that has already been used. [repeated 32x]
# The hashed bin roles can also be accessed by iterating the "delegations"
# property of the delegating role, which we do here to load the signing key.
>>> for delegation in repository.targets('unclaimed').delegations:
... delegation.load_signing_key(private_unclaimed_key)
# Mark roles for metadata update (see #964, #958)
>>> repository.mark_dirty(['00-07', '08-0f', '10-17', '18-1f', '20-27', '28-2f',
... '30-37', '38-3f', '40-47', '48-4f', '50-57', '58-5f', '60-67', '68-6f',
... '70-77', '78-7f', '80-87', '88-8f', '90-97', '98-9f', 'a0-a7', 'a8-af',
... 'b0-b7', 'b8-bf', 'c0-c7', 'c8-cf', 'd0-d7', 'd8-df', 'e0-e7', 'e8-ef',
... 'f0-f7', 'f8-ff', 'snapshot', 'timestamp', 'unclaimed'])
>>> repository.writeall()
```
## How to Perform an Update ##
The following [repository tool](../tuf/repository_tool.py) function creates a directory
structure that a client downloading new software using TUF (via
[tuf/client/updater.py](../tuf/client/updater.py)) expects. The `root.json` metadata file must exist, and
also the directories that hold the metadata files downloaded from a repository.
Software updaters integrating TUF may use this directory to store TUF updates
saved on the client side.
```python
>>> from tuf.repository_tool import *
>>> create_tuf_client_directory("repository/", "client/tufrepo/")
```
`create_tuf_client_directory()` moves metadata from `repository/metadata` to
`client/` in this example. The repository in `repository/` may be the
repository example created earlier in this document.
## Test TUF Locally ##
Run the local TUF repository server.
```Bash
$ cd "repository/"; python3 -m http.server 8001
```
We next retrieve targets from the TUF repository and save them to `client/`.
The `client.py` script is available to download metadata and files from a
specified repository. In a different command-line prompt, where `tuf` is
installed . . .
```Bash
$ cd "client/"
$ ls
tufrepo/
$ client.py --repo http://localhost:8001 file1.txt
$ ls . tuftargets/
.:
tufrepo tuftargets
tuftargets/:
file1.txt
```

5
docs/conf.py
View file

@ -38,11 +38,6 @@
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ['GETTING_STARTED.rst']
# -- Options for HTML output -------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for

BIN
docs/images/repository_tool-diagram.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 90 KiB

11
examples/repo_example/basic_repo.py
View file

@ -1,13 +1,10 @@
"""
A TUF repository example using the low-level TUF Metadata API.
As 'repository_tool' and 'repository_lib' are being deprecated, repository
metadata must be created and maintained *manually* using the low-level
Metadata API. The example code in this file demonstrates how to
implement similar functionality to that of the legacy 'repository_tool'
and 'repository_lib' until a new repository implementation is available.
The example code in this file demonstrates how to *manually* create and
maintain repository metadata using the low-level Metadata API. It implements
similar functionality to that of the deprecated legacy 'repository_tool' and
'repository_lib'. (see ADR-0010 for details about repository library design)
Contents:
* creation of top-level metadata

9
examples/repo_example/hashed_bin_delegation.py
View file

@ -1,11 +1,10 @@
"""
A TUF hash bin delegation example using the low-level TUF Metadata API.
As 'repository_tool' and 'repository_lib' are being deprecated, hash bin
delegation interfaces are no longer available in this implementation. The
example code in this file demonstrates how to easily implement those
interfaces, and how to use them together with the TUF metadata API, to perform
hash bin delegation.
The example code in this file demonstrates how to *manually* perform hash bin
delegation using the low-level Metadata API. It implements similar
functionality to that of the deprecated legacy 'repository_tool' and
'repository_lib'. (see ADR-0010 for details about repository library design)
Contents:
- Re-usable hash bin delegation helpers

426
pylintrc
View file

@ -1,426 +0,0 @@
[MASTER]
# A comma-separated list of package or module names from where C extensions may
# be loaded. Extensions are loading into the active Python interpreter and may
# run arbitrary code
extension-pkg-whitelist=
# Add files or directories to the blacklist. They should be base names, not
# paths.
ignore=CVS
# Add files or directories matching the regex patterns to the blacklist. The
# regex matches against base names, not paths.
ignore-patterns=
# Python code to execute, usually for sys.path manipulation such as
# pygtk.require().
#init-hook=
# Use multiple processes to speed up Pylint.
jobs=1
# List of plugins (as comma separated values of python modules names) to load,
# usually to register additional checkers.
load-plugins=
# Pickle collected data for later comparisons.
persistent=yes
# Specify a configuration file.
#rcfile=
# Allow loading of arbitrary C extensions. Extensions are imported into the
# active Python interpreter and may run arbitrary code.
unsafe-load-any-extension=no
[MESSAGES CONTROL]
# Only show warnings with the listed confidence levels. Leave empty to show
# all. Valid levels: HIGH, INFERENCE, INFERENCE_FAILURE, UNDEFINED
confidence=
# Disable the message, report, category or checker with the given id(s). You
# can either give multiple identifiers separated by comma (,) or put this
# option multiple times (only on the command line, not in the configuration
# file where it should appear only once).You can also use "--disable=all" to
# disable everything first and then reenable specific checks. For example, if
# you want to run only the similarities checker, you can use "--disable=all
# --enable=similarities". If you want to run only the classes checker, but have
# no Warning level messages displayed, use"--disable=all --enable=classes
# --disable=W"
disable=parameter-unpacking, unpacking-in-except, long-suffix, old-ne-operator, old-octal-literal, import-star-module-level, raw-checker-failed, bad-inline-option, locally-disabled, locally-enabled, file-ignored, suppressed-message, useless-suppression, deprecated-pragma, apply-builtin, basestring-builtin, buffer-builtin, cmp-builtin, coerce-builtin, execfile-builtin, file-builtin, long-builtin, raw_input-builtin, reduce-builtin, standarderror-builtin, unicode-builtin, xrange-builtin, coerce-method, delslice-method, getslice-method, setslice-method, no-absolute-import, old-division, dict-iter-method, dict-view-method, next-method-called, metaclass-assignment, indexing-exception, raising-string, reload-builtin, oct-method, hex-method, nonzero-method, cmp-method, input-builtin, round-builtin, intern-builtin, unichr-builtin, map-builtin-not-iterating, zip-builtin-not-iterating, range-builtin-not-iterating, filter-builtin-not-iterating, using-cmp-argument, eq-without-hash, div-method, idiv-method, rdiv-method, exception-message-attribute, invalid-str-codec, sys-max-int, deprecated-str-translate-call, global-statement, broad-except, logging-not-lazy, C, R
# Enable the message, report, category or checker with the given id(s). You can
# either give multiple identifier separated by comma (,) or put this option
# multiple time (only on the command line, not in the configuration file where
# it should appear only once). See also the "--disable" option for examples.
enable=
[REPORTS]
# Python expression which should return a note less than 10 (10 is the highest
# note). You have access to the variables errors warning, statement which
# respectively contain the number of errors / warnings messages and the total
# number of statements analyzed. This is used by the global evaluation report
# (RP0004).
evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)
# Template used to display messages. This is a python new-style format string
# used to format the message information. See doc for all details
#msg-template=
# Set the output format. Available formats are text, parseable, colorized, json
# and msvs (visual studio).You can also give a reporter class, eg
# mypackage.mymodule.MyReporterClass.
#output-format=parseable
output-format=text
# Tells whether to display a full report or only the messages
reports=no
# Activate the evaluation score.
score=yes
[REFACTORING]
# Maximum number of nested blocks for function / method body
max-nested-blocks=5
[BASIC]
# Naming hint for argument names
argument-name-hint=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
# Regular expression matching correct argument names
argument-rgx=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
# Naming hint for attribute names
attr-name-hint=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
# Regular expression matching correct attribute names
attr-rgx=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
# Bad variable names which should always be refused, separated by a comma
bad-names=foo,bar,baz,toto,tutu,tata
# Naming hint for class attribute names
class-attribute-name-hint=([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$
# Regular expression matching correct class attribute names
class-attribute-rgx=([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$
# Naming hint for class names
class-name-hint=[A-Z_][a-zA-Z0-9]+$
# Regular expression matching correct class names
class-rgx=[A-Z_][a-zA-Z0-9]+$
# Naming hint for constant names
const-name-hint=(([A-Z_][A-Z0-9_]*)|(__.*__))$
# Regular expression matching correct constant names
const-rgx=(([A-Z_][A-Z0-9_]*)|(__.*__))$
# Minimum line length for functions/classes that require docstrings, shorter
# ones are exempt.
docstring-min-length=-1
# Naming hint for function names
function-name-hint=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
# Regular expression matching correct function names
function-rgx=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
# Good variable names which should always be accepted, separated by a comma
good-names=i,j,k,ex,Run,_
# Include a hint for the correct naming format with invalid-name
include-naming-hint=no
# Naming hint for inline iteration names
inlinevar-name-hint=[A-Za-z_][A-Za-z0-9_]*$
# Regular expression matching correct inline iteration names
inlinevar-rgx=[A-Za-z_][A-Za-z0-9_]*$
# Naming hint for method names
method-name-hint=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
# Regular expression matching correct method names
method-rgx=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
# Naming hint for module names
module-name-hint=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$
# Regular expression matching correct module names
module-rgx=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$
# Colon-delimited sets of names that determine each other's naming style when
# the name regexes allow several styles.
name-group=
# Regular expression which should only match function or class names that do
# not require a docstring.
no-docstring-rgx=^_
# List of decorators that produce properties, such as abc.abstractproperty. Add
# to this list to register other decorators that produce valid properties.
property-classes=abc.abstractproperty
# Naming hint for variable names
variable-name-hint=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
# Regular expression matching correct variable names
variable-rgx=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
[FORMAT]
# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
expected-line-ending-format=
# Regexp for a line that is allowed to be longer than the limit.
ignore-long-lines=^\s*(# )?<?https?://\S+>?$
# Number of spaces of indent required inside a hanging or continued line.
indent-after-paren=4
# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1
# tab).
indent-string=' '
# Maximum number of characters on a single line.
max-line-length=80
# Maximum number of lines in a module
max-module-lines=1000
# List of optional constructs for which whitespace checking is disabled. `dict-
# separator` is used to allow tabulation in dicts, etc.: {1 : 1,\n222: 2}.
# `trailing-comma` allows a space between comma and closing bracket: (a, ).
# `empty-line` allows space-only lines.
no-space-check=trailing-comma,dict-separator
# Allow the body of a class to be on the same line as the declaration if body
# contains single statement.
single-line-class-stmt=no
# Allow the body of an if to be on the same line as the test if there is no
# else.
single-line-if-stmt=no
[LOGGING]
# Logging modules to check that the string format arguments are in logging
# function parameter format
logging-modules=logging
[MISCELLANEOUS]
# List of note tags to take in consideration, separated by a comma.
notes=XXX,
[SIMILARITIES]
# Ignore comments when computing similarities.
ignore-comments=yes
# Ignore docstrings when computing similarities.
ignore-docstrings=yes
# Ignore imports when computing similarities.
ignore-imports=yes
# Minimum lines number of a similarity.
min-similarity-lines=4
[SPELLING]
# Spelling dictionary name. Available dictionaries: none. To make it working
# install python-enchant package.
spelling-dict=
# List of comma separated words that should not be checked.
spelling-ignore-words=
# A path to a file that contains private dictionary; one word per line.
spelling-private-dict-file=
# Tells whether to store unknown words to indicated private dictionary in
# --spelling-private-dict-file option instead of raising a message.
spelling-store-unknown-words=no
[TYPECHECK]
# List of decorators that produce context managers, such as
# contextlib.contextmanager. Add to this list to register other decorators that
# produce valid context managers.
contextmanager-decorators=contextlib.contextmanager
# List of members which are set dynamically and missed by pylint inference
# system, and so shouldn't trigger E1101 when accessed. Python regular
# expressions are accepted.
generated-members=
# Tells whether missing members accessed in mixin class should be ignored. A
# mixin class is detected if its name ends with "mixin" (case insensitive).
ignore-mixin-members=yes
# This flag controls whether pylint should warn about no-member and similar
# checks whenever an opaque object is returned when inferring. The inference
# can return multiple potential results while evaluating a Python object, but
# some branches might not be evaluated, which results in partial inference. In
# that case, it might be useful to still emit no-member and other checks for
# the rest of the inferred objects.
ignore-on-opaque-inference=yes
# List of class names for which member attributes should not be checked (useful
# for classes with dynamically set attributes). This supports the use of
# qualified names.
ignored-classes=optparse.Values,thread._local,_thread._local
# List of module names for which member attributes should not be checked
# (useful for modules/projects where namespaces are manipulated during runtime
# and thus existing member attributes cannot be deduced by static analysis. It
# supports qualified module names, as well as Unix pattern matching.
ignored-modules=
# Show a hint with possible names when a member name was not found. The aspect
# of finding the hint is based on edit distance.
missing-member-hint=yes
# The minimum edit distance a name should have in order to be considered a
# similar match for a missing member name.
missing-member-hint-distance=1
# The total number of similar names that should be taken in consideration when
# showing a hint for a missing member.
missing-member-max-choices=1
[VARIABLES]
# List of additional names supposed to be defined in builtins. Remember that
# you should avoid to define new builtins when possible.
additional-builtins=
# Tells whether unused global variables should be treated as a violation.
allow-global-unused-variables=yes
# List of strings which can identify a callback function by name. A callback
# name must start or end with one of those strings.
callbacks=cb_,_cb
# A regular expression matching the name of dummy variables (i.e. expectedly
# not used).
dummy-variables-rgx=_+$|(_[a-zA-Z0-9_]*[a-zA-Z0-9]+?$)|dummy|^ignored_|^unused_|junk
# Argument names that match this expression will be ignored. Default to name
# with leading underscore
ignored-argument-names=_.*|^ignored_|^unused_
# Tells whether we should check for unused import in __init__ files.
init-import=no
# List of qualified module names which can have objects that can redefine
# builtins.
redefining-builtins-modules=future.builtins
[CLASSES]
# List of method names used to declare (i.e. assign) instance attributes.
defining-attr-methods=__init__,__new__,setUp
# List of member names, which should be excluded from the protected access
# warning.
exclude-protected=_asdict, _fields, _replace, _source, _make, _generate_and_write_metadata, _delete_obsolete_metadata, _log_status_of_top_level_roles, _load_top_level_metadata, _strip_version_number, _delegated_roles, _remove_invalid_and_duplicate_signatures, _repository_name, _targets_directory
# List of valid names for the first argument in a class method.
valid-classmethod-first-arg=cls
# List of valid names for the first argument in a metaclass class method.
valid-metaclass-classmethod-first-arg=mcs
[DESIGN]
# Maximum number of arguments for function / method
max-args=5
# Maximum number of attributes for a class (see R0902).
max-attributes=7
# Maximum number of boolean expressions in a if statement
max-bool-expr=5
# Maximum number of branch for function / method body
max-branches=12
# Maximum number of locals for function / method body
max-locals=15
# Maximum number of parents for a class (see R0901).
max-parents=7
# Maximum number of public methods for a class (see R0904).
max-public-methods=20
# Maximum number of return / yield for function / method body
max-returns=6
# Maximum number of statements in function / method body
max-statements=50
# Minimum number of public methods for a class (see R0903).
min-public-methods=2
[IMPORTS]
# Allow wildcard imports from modules that define __all__.
allow-wildcard-with-all=no
# Analyse import fallback blocks. This can be used to support both Python 2 and
# 3 compatible code, which means that the block might have code that exists
# only in one or another interpreter, leading to false positives when analysed.
analyse-fallback-blocks=no
# Deprecated modules which should not be used, separated by a comma
deprecated-modules=regsub,TERMIOS,Bastion,rexec
# Create a graph of external dependencies in the given file (report RP0402 must
# not be disabled)
ext-import-graph=
# Create a graph of every (i.e. internal and external) dependencies in the
# given file (report RP0402 must not be disabled)
import-graph=
# Create a graph of internal dependencies in the given file (report RP0402 must
# not be disabled)
int-import-graph=
# Force import order to recognize a module as part of the standard
# compatibility libraries.
known-standard-library=
# Force import order to recognize a module as part of a third party library.
known-third-party=enchant
[EXCEPTIONS]
# Exceptions that will emit a warning when being caught. Defaults to
# "Exception"
overgeneral-exceptions=Exception

8
pyproject.toml
View file

@ -7,8 +7,6 @@ build-backend = "setuptools.build_meta"
# Read more here: https://black.readthedocs.io/en/stable/usage_and_configuration/the_basics.html#configuration-via-a-file
[tool.black]
line-length=80
# TODO: remove "excludes" after deleting old test files
exclude="tests/.*old.py"
# Isort section
# Read more here: https://pycqa.github.io/isort/docs/configuration/config_files.html
@ -16,8 +14,6 @@ exclude="tests/.*old.py"
profile="black"
line_length=80
known_first_party = ["tuf"]
# TODO: remove "skip_glob" after deleting old test files
skip_glob="*old.py"
# Pylint section
@ -59,8 +55,6 @@ module-rgx="^(_?[a-z][a-z0-9_]*|__init__)$"
no-docstring-rgx="(__.*__|main|test.*|.*test|.*Test)$"
variable-rgx="^[a-z][a-z0-9_]*$"
docstring-min-length=10
# TODO: remove "ignore-patterns" after deleting old test files
ignore-patterns=".*_old.py"
[tool.pylint.logging]
logging-format-style="old"
@ -83,8 +77,6 @@ disallow_untyped_defs = "True"
disallow_untyped_calls = "True"
show_error_codes = "True"
disable_error_code = ["attr-defined"]
# TODO: remove "exclude" after deleting old test files
exclude=".*_old.py"
[[tool.mypy.overrides]]
module = [

3
setup.cfg
View file

@ -33,9 +33,6 @@ license_files = LICENSE LICENSE-MIT
[options]
packages = find:
scripts =
tuf/scripts/repo.py
tuf/scripts/client.py
python_requires = ~=3.7
install_requires =
requests>=2.19.1

3
tests/.coveragerc
View file

@ -2,9 +2,6 @@
branch = True
omit =
# Command-line scripts.
*/tuf/scripts/client.py
*/tuf/scripts/repo.py
*/tests/*
*/site-packages/*

25
tests/fast_server_exit_old.py
View file

@ -1,25 +0,0 @@
#!/usr/bin/env python
# Copyright 2020, TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
fast_server_exit.py
<Author>
Martin Vrachev.
<Started>
October 29, 2020.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Used for tests in tests/test_utils.py.
"""
import sys
sys.exit(0)

163
tests/repository_data/generate_old.py
View file

@ -1,163 +0,0 @@
#!/usr/bin/env python
# Copyright 2014 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
generate.py
<Author>
Vladimir Diaz <vladimir.v.diaz@gmail.com>
<Started>
February 26, 2014.
<Copyright>
See LICENSE-MIT.txt OR LICENSE-APACHE.txt for licensing information.
<Purpose>
Provide a set of pre-generated key files and a basic repository that unit
tests can use in their test cases. The pre-generated files created by this
script should be copied by the unit tests as needed. The original versions
should be preserved. 'tuf/tests/repository_data/' will store the files
generated. 'generate.py' should not require re-execution if the
pre-generated repository files have already been created, unless they need to
change in some way.
"""
import shutil
import datetime
import optparse
import stat
from tuf.repository_tool import *
import securesystemslib
parser = optparse.OptionParser()
parser.add_option("-k","--keys", action='store_true', dest="should_generate_keys",
help="Generate a new set of keys", default=False)
parser.add_option("-d","--dry-run", action='store_true', dest="dry_run",
help="Do not write the files, just run", default=False)
(options, args) = parser.parse_args()
repository = create_new_repository('repository')
root_key_file = 'keystore/root_key'
targets_key_file = 'keystore/targets_key'
snapshot_key_file = 'keystore/snapshot_key'
timestamp_key_file = 'keystore/timestamp_key'
delegation_key_file = 'keystore/delegation_key'
if options.should_generate_keys and not options.dry_run:
# Generate and save the top-level role keys, including the delegated roles.
# The unit tests should only have to import the keys they need from these
# pre-generated key files.
# Generate public and private key files for the top-level roles, and two
# delegated roles (these number of keys should be sufficient for most of the
# unit tests). Unit tests may generate additional keys, if needed.
generate_and_write_rsa_keypair(password='password', filepath=root_key_file)
generate_and_write_ed25519_keypair(password='password', filepath=targets_key_file)
generate_and_write_ed25519_keypair(password='password', filepath=snapshot_key_file)
generate_and_write_ed25519_keypair(password='password', filepath=timestamp_key_file)
generate_and_write_ed25519_keypair(password='password', filepath=delegation_key_file)
# Import the public keys. These keys are needed so that metadata roles are
# assigned verification keys, which clients use to verify the signatures created
# by the corresponding private keys.
root_public = import_rsa_publickey_from_file(root_key_file + '.pub')
targets_public = import_ed25519_publickey_from_file(targets_key_file + '.pub')
snapshot_public = import_ed25519_publickey_from_file(snapshot_key_file + '.pub')
timestamp_public = import_ed25519_publickey_from_file(timestamp_key_file + '.pub')
delegation_public = import_ed25519_publickey_from_file(delegation_key_file + '.pub')
# Import the private keys. These private keys are needed to generate the
# signatures included in metadata.
root_private = import_rsa_privatekey_from_file(root_key_file, 'password')
targets_private = import_ed25519_privatekey_from_file(targets_key_file, 'password')
snapshot_private = import_ed25519_privatekey_from_file(snapshot_key_file, 'password')
timestamp_private = import_ed25519_privatekey_from_file(timestamp_key_file, 'password')
delegation_private = import_ed25519_privatekey_from_file(delegation_key_file, 'password')
# Add the verification keys to the top-level roles.
repository.root.add_verification_key(root_public)
repository.targets.add_verification_key(targets_public)
repository.snapshot.add_verification_key(snapshot_public)
repository.timestamp.add_verification_key(timestamp_public)
# Load the signing keys, previously imported, for the top-level roles so that
# valid metadata can be written.
repository.root.load_signing_key(root_private)
repository.targets.load_signing_key(targets_private)
repository.snapshot.load_signing_key(snapshot_private)
repository.timestamp.load_signing_key(timestamp_private)
# Create the target files (downloaded by clients) whose file size and digest
# are specified in the 'targets.json' file.
target1_filepath = 'repository/targets/file1.txt'
securesystemslib.util.ensure_parent_dir(target1_filepath)
target2_filepath = 'repository/targets/file2.txt'
securesystemslib.util.ensure_parent_dir(target2_filepath)
target3_filepath = 'repository/targets/file3.txt'
securesystemslib.util.ensure_parent_dir(target2_filepath)
if not options.dry_run:
with open(target1_filepath, 'wt') as file_object:
file_object.write('This is an example target file.')
# As we will add this file's permissions to the custom_attribute in the
# target's metadata we need to ensure that the file has the same
# permissions when created by this script regardless of umask value on
# the host system generating the data
os.chmod(target1_filepath, 0o644)
with open(target2_filepath, 'wt') as file_object:
file_object.write('This is an another example target file.')
with open(target3_filepath, 'wt') as file_object:
file_object.write('This is role1\'s target file.')
# Add target files to the top-level 'targets.json' role. These target files
# should already exist. 'target1_filepath' contains additional information
# about the target (i.e., file permissions in octal format.)
octal_file_permissions = oct(os.stat(target1_filepath).st_mode)[4:]
file_permissions = {'file_permissions': octal_file_permissions}
repository.targets.add_target(os.path.basename(target1_filepath), file_permissions)
repository.targets.add_target(os.path.basename(target2_filepath))
repository.targets.delegate('role1', [delegation_public],
[os.path.basename(target3_filepath)])
repository.targets('role1').add_target(os.path.basename(target3_filepath))
repository.targets('role1').load_signing_key(delegation_private)
repository.targets('role1').delegate('role2', [delegation_public], [])
repository.targets('role2').load_signing_key(delegation_private)
# Set the top-level expiration times far into the future so that
# they do not expire anytime soon, or else the tests fail. Unit tests may
# modify the expiration datetimes (of the copied files), if they wish.
repository.root.expiration = datetime.datetime(2030, 1, 1, 0, 0)
repository.targets.expiration = datetime.datetime(2030, 1, 1, 0, 0)
repository.snapshot.expiration = datetime.datetime(2030, 1, 1, 0, 0)
repository.timestamp.expiration = datetime.datetime(2030, 1, 1, 0, 0)
repository.targets('role1').expiration = datetime.datetime(2030, 1, 1, 0, 0)
repository.targets('role2').expiration = datetime.datetime(2030, 1, 1, 0, 0)
# Create the actual metadata files, which are saved to 'metadata.staged'.
if not options.dry_run:
repository.writeall()
# Move the staged.metadata to 'metadata' and create the client folder. The
# client folder, which includes the required directory structure and metadata
# files for clients to successfully load an 'tuf.client.updater.py' object.
staged_metadata_directory = 'repository/metadata.staged'
metadata_directory = 'repository/metadata'
if not options.dry_run:
shutil.copytree(staged_metadata_directory, metadata_directory)
# Create the client files (required directory structure and minimal metadata)
# as expected by 'tuf.client.updater'.
if not options.dry_run:
create_tuf_client_directory('repository', os.path.join('client', 'test_repository1'))

109
tests/repository_data/generate_project_data_old.py
View file

@ -1,109 +0,0 @@
#!/usr/bin/env python
# Copyright 2014 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
generate_project_data.py
<Author>
Santiago Torres <torresariass@gmail.com>
<Started>
January 22, 2014.
<Copyright>
See LICENSE-MIT.txt OR LICENSE-APACHE.txt for licensing information.
<Purpose>
Generate a pre-fabricated set of metadata files for
'test_developer_tool_old.py' test cases.
"""
import shutil
import datetime
import optparse
import os
from tuf.developer_tool import *
import securesystemslib
parser = optparse.OptionParser()
parser.add_option("-d","--dry-run", action='store_true', dest="dry_run",
help="Do not write the files, just run", default=False)
(options, args) = parser.parse_args()
project_key_file = 'keystore/root_key'
targets_key_file = 'keystore/targets_key'
delegation_key_file = 'keystore/delegation_key'
# The files we use for signing in the unit tests should exist, if they are not
# populated, run 'generate.py'.
assert os.path.exists(project_key_file)
assert os.path.exists(targets_key_file)
assert os.path.exists(delegation_key_file)
# Import the public keys. These keys are needed so that metadata roles are
# assigned verification keys, which clients use to verify the signatures created
# by the corresponding private keys.
project_public = import_rsa_publickey_from_file(project_key_file + '.pub')
targets_public = import_ed25519_publickey_from_file(targets_key_file + '.pub')
delegation_public = import_ed25519_publickey_from_file(delegation_key_file + '.pub')
# Import the private keys. These private keys are needed to generate the
# signatures included in metadata.
project_private = import_rsa_privatekey_from_file(project_key_file, 'password')
targets_private = import_ed25519_privatekey_from_file(targets_key_file, 'password')
delegation_private = import_ed25519_privatekey_from_file(delegation_key_file, 'password')
os.mkdir("project")
os.mkdir("project/targets")
# Create the target files (downloaded by clients) whose file size and digest
# are specified in the 'targets.json' file.
target1_filepath = 'project/targets/file1.txt'
securesystemslib.util.ensure_parent_dir(target1_filepath)
target2_filepath = 'project/targets/file2.txt'
securesystemslib.util.ensure_parent_dir(target2_filepath)
target3_filepath = 'project/targets/file3.txt'
securesystemslib.util.ensure_parent_dir(target2_filepath)
if not options.dry_run:
with open(target1_filepath, 'wt') as file_object:
file_object.write('This is an example target file.')
with open(target2_filepath, 'wt') as file_object:
file_object.write('This is an another example target file.')
with open(target3_filepath, 'wt') as file_object:
file_object.write('This is role1\'s target file.')
project = create_new_project("test-flat", 'project/test-flat', 'prefix', 'project/targets')
# Add target files to the top-level projects role. These target files should
# already exist.
project.add_target('file1.txt')
project.add_target('file2.txt')
# Add one key to the project.
project.add_verification_key(project_public)
project.load_signing_key(project_private)
# Add the delegated role keys.
project.delegate('role1', [delegation_public], [target3_filepath])
project('role1').load_signing_key(delegation_private)
# Set the project expiration time far into the future so that its metadata does
# not expire anytime soon, or else the tests fail. Unit tests may modify the
# expiration datetimes (of the copied files), if they wish.
project.expiration = datetime.datetime(2030, 1, 1, 0, 0)
project('role1').expiration = datetime.datetime(2030, 1, 1, 0, 0)
# Create the actual metadata files, which are saved to 'metadata.staged'.
if not options.dry_run:
project.write()

59
tests/simple_https_server_old.py
View file

@ -1,59 +0,0 @@
#!/usr/bin/env python
# Copyright 2014 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program>
simple_https_server_old.py
<Author>
Vladimir Diaz.
<Started>
June 17, 2014
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Provide a simple https server that can be used by the unit tests. For
example, 'download.py' can connect to the https server started by this module
to verify that https downloads are permitted.
<Reference>
ssl.SSLContext.wrap_socket:
https://docs.python.org/3/library/ssl.html#ssl.SSLContext.wrap_socket
SimpleHTTPServer:
http://docs.python.org/library/simplehttpserver.html#module-SimpleHTTPServer
"""
import sys
import ssl
import os
import http.server
keyfile = os.path.join('ssl_certs', 'ssl_cert.key')
certfile = os.path.join('ssl_certs', 'ssl_cert.crt')
if len(sys.argv) > 1 and os.path.exists(sys.argv[1]):
certfile = sys.argv[1]
httpd = http.server.HTTPServer(('localhost', 0),
http.server.SimpleHTTPRequestHandler)
context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
context.load_cert_chain(certfile, keyfile)
httpd.socket = context.wrap_socket(httpd.socket, server_side=True)
port_message = 'bind succeeded, server port is: ' \
+ str(httpd.server_address[1])
print(port_message)
if len(sys.argv) > 1 and certfile != sys.argv[1]:
print('simple_https_server_old: cert file was not found: ' + sys.argv[1] +
'; using default: ' + certfile + " certfile")
httpd.serve_forever()

63
tests/slow_retrieval_server_old.py
View file

@ -1,63 +0,0 @@
#!/usr/bin/env python
# Copyright 2012 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
slow_retrieval_server_old.py
<Author>
Konstantin Andrianov.
<Started>
March 13, 2012.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Server that throttles data by sending one byte at a time (specified time
interval 'DELAY'). The server is used in 'test_slow_retrieval_attack_old.py'.
"""
import os
import time
import http.server
# HTTP request handler.
class Handler(http.server.BaseHTTPRequestHandler):
# Overwrite do_GET.
def do_GET(self):
current_dir = os.getcwd()
try:
filepath = os.path.join(current_dir, self.path.lstrip('/'))
data = None
with open(filepath, 'r') as fileobj:
data = fileobj.read()
self.send_response(200)
self.send_header('Content-length', str(len(data)))
self.end_headers()
# Before sending any data, the server does nothing for a long time.
DELAY = 40
time.sleep(DELAY)
self.wfile.write((data.encode('utf-8')))
except IOError as e:
self.send_error(404, 'File Not Found!')
if __name__ == '__main__':
server_address = ('localhost', 0)
httpd = http.server.HTTPServer(server_address, Handler)
port_message = 'bind succeeded, server port is: ' \
+ str(httpd.server_address[1])
print(port_message)
httpd.serve_forever()

17
tests/ssl_certs/proxy_ca.crt
View file

@ -1,17 +0,0 @@
-----BEGIN CERTIFICATE-----
MIICpDCCAYwCCQCFr/EhHmzVajANBgkqhkiG9w0BAQsFADAUMRIwEAYDVQQDDAlw
cm94eTIgQ0EwHhcNMTgwOTIwMTkyOTQ2WhcNMjgwOTE3MTkyOTQ2WjAUMRIwEAYD
VQQDDAlwcm94eTIgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC/
rVOeqSzJb01Vyliw3dnfLJsWfDfs/Lq5HLn+Xqnzl6MqnYirDqHzTErD3vl8lo/o
OJrziO0vYCWGXEylRQlZp+P37bLToSWiVqWZ8pH6CAh+AhA3WtegN5JwTgIUSP7A
aDlxuZrXlJM50QVlXJIPkc74M8ALz0nu5zmyWkGFvmTYS8503T8cXs9Alr4Bo++9
Ilixv6lW4QS7FKTeQXlI49K4TeGGGsfmEO6Uj4WTUkwMZym9wfiqtaWc6I9ZMese
WmU3LuufY+pFCdjsdMWDJpYc+HabTSrbgXSF5Iq9a84Xuum39qhVpYhBwBtLk3ye
cxZmIxde1vnkWAitJFETAgMBAAEwDQYJKoZIhvcNAQELBQADggEBAKV09r/x3WyO
McH0RU4WRVzvQN5F0e7swpDlLUX7YnfvpPEkavqQfmrL1cYyEDgsm/347Gvcs1Aa
iaT77axYroXOvCEJ3DxZdzUErKH6Jr3MmHKcZ/L35u6ZXKnmx/edFjdWr6ENkjuZ
NVvKbTrm4cl6Wy4bXkp6b24rBa9IFJncOouSkIvHENEcH//OD4xeTK8vSJTJ9nmw
TiJ0TjCRujtJWC6yb03ZV32VbeiHa1zLlZhcyKqUtt81dLti5t5+L2hAAVCcnEgI
DBWQdlRs/wilHGWVBo/9srOoMNsmvecTBpLH2JyC5VZ1+faYLPrNlgkWgHIFOTTi
h4ByR95Wbi8=
-----END CERTIFICATE-----

27
tests/ssl_certs/proxy_ca.key
View file

@ -1,27 +0,0 @@
-----BEGIN RSA PRIVATE KEY-----
MIIEpQIBAAKCAQEAv61TnqksyW9NVcpYsN3Z3yybFnw37Py6uRy5/l6p85ejKp2I
qw6h80xKw975fJaP6Dia84jtL2AlhlxMpUUJWafj9+2y06ElolalmfKR+ggIfgIQ
N1rXoDeScE4CFEj+wGg5cbma15STOdEFZVySD5HO+DPAC89J7uc5slpBhb5k2EvO
dN0/HF7PQJa+AaPvvSJYsb+pVuEEuxSk3kF5SOPSuE3hhhrH5hDulI+Fk1JMDGcp
vcH4qrWlnOiPWTHrHlplNy7rn2PqRQnY7HTFgyaWHPh2m00q24F0heSKvWvOF7rp
t/aoVaWIQcAbS5N8nnMWZiMXXtb55FgIrSRREwIDAQABAoIBACxJObbA064+3xlh
RRioSXx86+BIFwvUYLgAYSDacl3rvTFNcJRFLznteKDE1dPpXZqD6Zk3G8YEauce
UD8nMj/awJs5+kVXSEC30E8/cmbYkE284E5J2OQVsunrvCM/skx2SD90aMhCdbm4
B40h1EVwpOdH3alc3XIrTnNc0yK5MWAu41qwkxYxXHmW9Y0L8AjZve9JBrnKsJMB
ETEZFhHgi/IWtfh5PLbJO2dbSe7Nqo4ikyWo3r5b3yvuphFz1il88ZLjJ5nDmtlH
is7sk7pd0tYNsK1Di5G1ku50XvcbOE4F7mOVCxICTwjN+sdyG8o+AVlgbTKBo/JF
uEhthCECgYEA/3YXS9mAEujlstrV4VOksYWtySSrLHC56tLjj8cHVPJ1qkzT4OOC
X9TsWReDG4J8/t0DOHn+5dnhnqGcYjMMAQx095KHU1bQGrcRdmi6cjnNLTvfEbge
IcJTYG5P7NpLfLjB3DOGqFR4o0iz4K9ZLTYJc+BaCB9qJBEw6nuoP+sCgYEAwBTN
WpRDrmch0+LFPQwboLwtEPiFscTj8SInV0KsI/MK8+5Sm+tXS8PQHYJYcECEQxQM
2gfyM8vy33UP4yn4edJGWlaz7a4hyDxn944vv2fBQ3vjJTNz3X3skkhZ2/F+ZW9e
SFxPj+Vbif8VTEU+wK0f5SUmpRec4E7y3fq+kXkCgYEAib8ZbLLI1mlygfBx51/8
rCRSwuTcz8ew2CgCwGInV+ys+bkXfmnuwNHE531AGrNPxvVRaUCO602C1NB7zI+N
53raDyyZf5yN9fnElr592l3EfqGL9Lf8t2NbJeIVgrdqgMP29E9sSpPRwOnQ5FRo
l3JNwoe0xDB8QRpr7+PhoyUCgYEAp+GGmmR7wzLgnhDV00WB4DqYKP0N3RH5KAhx
2hKr4b/LEuh5y00mP1Il06TZJ0M8VmRv1yCa0CqxXB00hZdpVRAz7UFagaJwZFJn
jDb6BJDqmdDt9tXBrxUgb7pMz6+CiaWNAjGsWFheaX5JXyAmeMDX369Y13KL6oEW
RG2jogECgYEA/1vLZcWNK/0yd4ClU+Xbu8xC29q82JUMsaazHtbgSNlOfo9LMQlH
z6xBiMYfHZ/SiHCy9RsO8GD4caXiF0RsTVnhqjSRJf3EARamufelNsu2ApLclkSN
fzSoB7ZHddGaYKYpXkGzcwFcKd/QjAlHm1yIsZu4B52AhCxC/WS2X54=
-----END RSA PRIVATE KEY-----

27
tests/ssl_certs/proxy_cert.key
View file

@ -1,27 +0,0 @@
-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAzZO36nZvb9wLxBNB2cZyHqcX5poChJd1YnFBtxbtQwiISxid
eGdiWImQE80vpUyTQbI7TxM+w1xZeEeu4PXuYrOgdTDRFEnjM2mteG+3WpHQBN4H
xoah0msp3046fMkYqcEvhvHbsc5DAWgLK4JFHQPtG/+CIH0ZY+lBBPQhFIhBLYkt
YxNVqwpsXOGreASSw6mO6cVehCuVFJQO5NnI1sCAvp3SeosMKeIcDZxpZWmZhSwH
n3Rj6RMNM66C8zG4YlpvIniGzgV4UiW8XrTUG8HmzQ2295IcfB4No2DZeJDSR9oq
jOkyqJXll+tSiAMuzBRtTQKvGZ5bpZWW4XELEQIDAQABAoIBAQCAfW2cjD4GimCI
QwkLlq9JXWLg7S3ZtdjWmLdcOmY9WZ3mYhI6aVPcxs5Ysgyvonb/vui2+e5mqNf7
B8LUNKK06lTGKqbjqXLqdYjJF/pgD3cXM7dkbE3EeNqJChogWIijwW11SMHqFmNn
A6LHpPqRshyHPWIV8FroSagr8nKio5BjUEuUiQUUAmSJPGN5qUhdIWXcQu8R1JB8
9qqqtwPR4FELbFVGI2vYHaSWGnf9V0boPOsfFXWbSq/Ksj3Lm3gAqMtlAeOFu84l
fhP9RkgeXfaCXq0VaOM83UDgLqXm4Ni4wAMKRLwNs4LzumqMM/dfUTn+mGncj33q
idp5qnDhAoGBAOXkwuf60F7aBbo98A0vWZli2CbkspsJz2J573pf+lVWI+ZHBZLI
MOM2DgCOEIUfa2TIMkwFr2t9x6uXlACEwFbEtEBpM4J5qUHgGtXZIsnTsv3qUg/C
L89cNrMddOuuRkxQbyK1QMYZZmZQjSKG2jW6m1KING+shtkOzQ/P9ildAoGBAOTs
DLyyPeEZPj1UMqxVNmeYYRfWnt+YyTPulOIbSuFN0DhZPNLsjrhSxvDwe/3sYH/p
nKdjnlFlx8frz9wtkCt0hWvY0pG2Zam4IBCvreFN7rSvpzHwUAK3oXic2TRKKu1m
xUPZqMJwnWAPX+XxGFn0m7UJj+95VTEOJ2d12ClFAoGAdexXMgmM8uqg/3yf8xNz
wWNbfu/W0gJBN8FWXw52aWmrNob9y+IWeaYTnqNAxBhuzR6H9kkAR4IYduNkzrNJ
ufhigZu1CVuAv8LF4SXlW2PVL7wPZff08Efb4xrcC7y0YJbtuv8Af90tkpQFIU3N
Brx2yeoGA7aa4SJfe5nwKh0CgYAo1yP+lh4MBqDf+CGCNUGbgcfwpM17PprGtQ3C
uPPG9kbrhqAfUSy1Ha94VK8KQh2FNHxKMK+R/gKCXEOdGFPcLNGQyAHpFQ1WFg9C
atUumOS5P40oj6L2mSQpjHIDrieyat9Ol4pQBh9Nf/Cv6S9a/RS6W5ZeNttIASpu
fsutsQKBgQCq+BFeDYJH4f+C1233W3PXM0P1ivj+9TJMRUP63RRay6rv2ZTZXyPc
Rx6Lv4OVWh9VMfv1kHRloJ1GKEBo/uD3nid1WqoNxpXv1iwxeGtjXkFHfvCB7Ruu
vTyQhJQQ7WSCJJOfarstusIn0udOG3MLRgG4X1pPQghyS1AT8NUglw==
-----END RSA PRIVATE KEY-----

28
tests/ssl_certs/ssl_cert.crt
View file

@ -1,28 +0,0 @@
-----BEGIN CERTIFICATE-----
MIIE1TCCAz2gAwIBAgIJAKqz8ew7Z44mMA0GCSqGSIb3DQEBCwUAMIGAMQswCQYD
VQQGEwJVUzERMA8GA1UECAwITmV3IFlvcmsxETAPBgNVBAcMCEJyb29rbHluMQww
CgYDVQQKDANOWVUxKTAnBgNVBAsMIENvbXB1dGVyIFNjaWVuY2UgYW5kIEVuZ2lu
ZWVyaW5nMRIwEAYDVQQDDAlsb2NhbGhvc3QwHhcNMTYwMTI3MjEyMTMxWhcNMjYw
MTI0MjEyMTMxWjCBgDELMAkGA1UEBhMCVVMxETAPBgNVBAgMCE5ldyBZb3JrMREw
DwYDVQQHDAhCcm9va2x5bjEMMAoGA1UECgwDTllVMSkwJwYDVQQLDCBDb21wdXRl
ciBTY2llbmNlIGFuZCBFbmdpbmVlcmluZzESMBAGA1UEAwwJbG9jYWxob3N0MIIB
ojANBgkqhkiG9w0BAQEFAAOCAY8AMIIBigKCAYEAxyFVeRsWnb1UlCKBks2azM9W
9K+J/ZkzdSb6eCxOIxv79M/Ug54CfWqkySSaQejsu0U/gJxkFYRvwQAy5lATrspY
2kyiWYiggWXFDWz+i8ETPkL9zn59v13sNIpT/IXQj0S3Mr9ZnsUn1qCyEOOIxJxZ
lyuV/M/XP1DP4tArhEvrex12V6MQIK+8fYzEjHG/W7vIIet+wTStIR8ArvVQi0Kv
PbbGCfrZ+e+gq+UpBLBuAfMzM95TW+YJ5duMchie2n6LDmOeegA4jMEv2ppeOr8Q
JJtZuKpXWVbJvLg81yrDjr1rAwJR/WQrnk8GQWPCyPLneAA4mJbi75LqjLxn0AoJ
b3kzLfGEMJJEWXspxNg06bLQU948hB4L7nKARq6s7KoESjEV+/L4koMPWJoNq6fx
OUVw2+S3ITNrDctecRQ1j3RGVPaj5l6bn03C7KV9uRrfqFY3OUjn7A0kDczvRnmr
e1BZIpe+mfGFB+Uu7JiQoBv6I6fqyrdH9rX1LUKlAgMBAAGjUDBOMB0GA1UdDgQW
BBT8LvRkvodP9bR/bBs/aI+AydRIvTAfBgNVHSMEGDAWgBT8LvRkvodP9bR/bBs/
aI+AydRIvTAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBgQC6kwuSEF0Y
5yLMf1TKfVkeBaZ4tOqR2kzpggzPPog+JcfIQgVmI2QTUDritHWFIM4YUwQ/00WU
uol2BCUpgaLci5gNgyTw8p538Q5cZHXE3kK/CWJA4zKag+oHdmXzGjMalqzvPuVJ
9VdtPrwHhB0Xntf72iWWhE2dIn1QZqVmJ/8hhIU8cQ91pIqTjYjhrYE/GhGH7HMW
bRiRolt37VxbzfXjEBMqVH6fOQq0piTRxwTNPBFp6JO5mRakRmWRvN3dnR8J9qXi
6tQhNNn2uQIpPlKlqVQnh5j5YxFrb50b0FCjDw+eNilXP93yjV4+lWK2QZychcGl
6/7Wu8snZkJCImPbwmcT80XSKesf918zIkauekWiaJE02+ljNtbM7MUAE+XLsKJy
NFGzpyZJ9LihGC/eeVl7K+xqC41jGVOXOOHtbDMbIQfaEZd1nPvy3+V/tublv+am
jPSlj/FW3bLTkjF0OspFjHvJeCeAJdM9kJdYfZoahd6kcejGJc+vjXE=
-----END CERTIFICATE-----

39
tests/ssl_certs/ssl_cert.key
View file

@ -1,39 +0,0 @@
-----BEGIN RSA PRIVATE KEY-----
MIIG4wIBAAKCAYEAxyFVeRsWnb1UlCKBks2azM9W9K+J/ZkzdSb6eCxOIxv79M/U
g54CfWqkySSaQejsu0U/gJxkFYRvwQAy5lATrspY2kyiWYiggWXFDWz+i8ETPkL9
zn59v13sNIpT/IXQj0S3Mr9ZnsUn1qCyEOOIxJxZlyuV/M/XP1DP4tArhEvrex12
V6MQIK+8fYzEjHG/W7vIIet+wTStIR8ArvVQi0KvPbbGCfrZ+e+gq+UpBLBuAfMz
M95TW+YJ5duMchie2n6LDmOeegA4jMEv2ppeOr8QJJtZuKpXWVbJvLg81yrDjr1r
AwJR/WQrnk8GQWPCyPLneAA4mJbi75LqjLxn0AoJb3kzLfGEMJJEWXspxNg06bLQ
U948hB4L7nKARq6s7KoESjEV+/L4koMPWJoNq6fxOUVw2+S3ITNrDctecRQ1j3RG
VPaj5l6bn03C7KV9uRrfqFY3OUjn7A0kDczvRnmre1BZIpe+mfGFB+Uu7JiQoBv6
I6fqyrdH9rX1LUKlAgMBAAECggGAEogMn0ehFC7xdxO7AUF3HYZSLlVDv0EJo+Zr
utFMuEG7ce4Bdfo3exp4mWt5m5akqUzpevuS6Nm5WLm/AuYC3upf2Hj3RuPLJB+n
dfdlvPXL56huXFAzPaLs/3q8FC0T2rFnZyadnYP1kCjGSYITUVDHmaTpwWxKOM85
eX8r/ZTfJkb4o3E+Z/xSy1BVXkibqVrRZi63Th2r2wA6nQ2hYERlcJXY2kbpEDR3
vGeIKLKOmknawwH2uf+vfh+vc1LNE7p9C5w16ex0OcmCo6G1ln7/dcwmXmcS3M0S
Bax5Jzu5ozaJFL9G59o0AUGJoZj9Gj9leeKPZvShsGcA0JmBMQiLIdhgRwj0B83x
HrYXTZ6P5BjJmwrIv4mGdv2bHV20pbWKAATUwo8EVBzylipexhhAtQJ5B6OsPDPS
HTluaEC2niD6lE613uRnzzbjw4SlwkoMLE0aqOhQyWIPS9/8oRjTzQi4otL7Dt69
oMrVhmSfxUqZhh2R3KMHDcMKt5nBAoHBAOXkDovYOhTMD3ei0WbKpbSB1sJp5t2d
/9gVil4nWLa4ahw7/TsZi3Co+c9gD2UJku1L9JbOy6TVZ2LoXOybLvIJfeAjNdYH
vi/ElG7498fgsSyw6bua/1VEd7VtbtpWJIQt1LdJG1+O3ZbJNTY6tbLbYVuy4FIO
e/484F8kdZ9PtRsn+I0I7kfoYJ2IFoM0UWgwQETOBguBCua43ZnHoxrvyHKABAO+
Iuvw4RBZKphGVxMCEjvTCB9S/CpGCRAkkQKBwQDdvu3reA/lVdFDN56VNUn0u3vr
zPSoiOjojlHDyWVAWiLB9I0qaE61UMvVgChM8VkmjhHYQEW6Cj0XMZMkCnsfKDQn
TYF16jt/sTteWSTcx0PTeiCGs3yM5wK4B8q9coOlzSqDd39mjDIFiUz4e+44OIcU
+ISc8pGbwxw0W8qRwIUJPTSVoaUZDnupuR/IE48q8CTPT1Gf00sMLWuv3SYuFHKX
djpcMLWVf4HclIY6y3BqNIZ0JaUAOd+OZT2kdtUCgcBLWPwLics/lcJcC9lmP3Ug
PI4PGna4nFiGkkjPo0XIXZkpt9+/xxeUzU1TUsC49PJbJFH+O7kzRV6lZFNQmWxB
mCrRk7jJdbA4J84esStFL7fiVfnFq3+UiuRRapSyqxk82WimyidWopSuHzR5mbSD
8rNuQqqTOnwZUAqaJHEIzi8lv2wPjaXLm7ZO65O1XShxZZ8q7fu9OYZBKMY46N3k
rkKchKjMMT1w53pcyVzUm/leGYewY/J9kc1kbZ/60oECgcEAj/qdzwt4/sa3BncB
wA4GxCJL9zJwFVI4MG/gRUjqNluQP/GDC2sI2A/rGeiJwlPfN/p9ObWZ0I8/VWT6
DifEA9n96xsXGTIKigHQ85TcK4Iy1whwQCYgk/iXOljM2i+VrT1HAm+/yBz1icS5
ton5hoWlqAcpTCLwSnvoP1Lud67ScspL73Aym89cmjo6mZWhmxasP/NXo3f1PaXs
SxdD6B2cvh2lDSEPdk+BSXEiquBXUI5kUtvyg/AP6Qxxdu01AoHAO05qTh9zokkT
yg0sZf4Z5i01em2ys4ZhQjhhbw+I5lIO76e/ZyUWpEZusBVd9TV5BHgiATOHw4yr
nbjEZKwLEb3SXoHl3/CD/l9vWk4gKAYDJdW+oPZttDlkp6dfPJVDupQwLhrxXYmE
fgs4WFmY3Q5b1wut2pnSs1UEPDqJBvykt59gFgn7yVwyTy8VLihNVtH4mwVPYXha
jz2T6BzRAPlYqx/FpkK2YHHNcyj+HFtnBUMMzacnSl/aXpJgHTKw
-----END RSA PRIVATE KEY-----

30
tests/ssl_certs/ssl_cert_2.crt
View file

@ -1,30 +0,0 @@
-----BEGIN CERTIFICATE-----
MIIFOTCCA6GgAwIBAgIJAO+bbero+zKtMA0GCSqGSIb3DQEBCwUAMIGAMQswCQYD
VQQGEwJVUzERMA8GA1UECAwITmV3IFlvcmsxETAPBgNVBAcMCEJyb29rbHluMQww
CgYDVQQKDANOWVUxKTAnBgNVBAsMIENvbXB1dGVyIFNjaWVuY2UgYW5kIEVuZ2lu
ZWVyaW5nMRIwEAYDVQQDDAlsb2NhbGhvc3QwHhcNMTgwOTI2MTgwMDAzWhcNMzgw
OTIxMTgwMDAzWjCBgDELMAkGA1UEBhMCVVMxETAPBgNVBAgMCE5ldyBZb3JrMREw
DwYDVQQHDAhCcm9va2x5bjEMMAoGA1UECgwDTllVMSkwJwYDVQQLDCBDb21wdXRl
ciBTY2llbmNlIGFuZCBFbmdpbmVlcmluZzESMBAGA1UEAwwJbG9jYWxob3N0MIIB
ojANBgkqhkiG9w0BAQEFAAOCAY8AMIIBigKCAYEAxyFVeRsWnb1UlCKBks2azM9W
9K+J/ZkzdSb6eCxOIxv79M/Ug54CfWqkySSaQejsu0U/gJxkFYRvwQAy5lATrspY
2kyiWYiggWXFDWz+i8ETPkL9zn59v13sNIpT/IXQj0S3Mr9ZnsUn1qCyEOOIxJxZ
lyuV/M/XP1DP4tArhEvrex12V6MQIK+8fYzEjHG/W7vIIet+wTStIR8ArvVQi0Kv
PbbGCfrZ+e+gq+UpBLBuAfMzM95TW+YJ5duMchie2n6LDmOeegA4jMEv2ppeOr8Q
JJtZuKpXWVbJvLg81yrDjr1rAwJR/WQrnk8GQWPCyPLneAA4mJbi75LqjLxn0AoJ
b3kzLfGEMJJEWXspxNg06bLQU948hB4L7nKARq6s7KoESjEV+/L4koMPWJoNq6fx
OUVw2+S3ITNrDctecRQ1j3RGVPaj5l6bn03C7KV9uRrfqFY3OUjn7A0kDczvRnmr
e1BZIpe+mfGFB+Uu7JiQoBv6I6fqyrdH9rX1LUKlAgMBAAGjgbMwgbAwgZ8GA1Ud
IwSBlzCBlKGBhqSBgzCBgDELMAkGA1UEBhMCVVMxETAPBgNVBAgMCE5ldyBZb3Jr
MREwDwYDVQQHDAhCcm9va2x5bjEMMAoGA1UECgwDTllVMSkwJwYDVQQLDCBDb21w
dXRlciBTY2llbmNlIGFuZCBFbmdpbmVlcmluZzESMBAGA1UEAwwJbG9jYWxob3N0
ggkA75tt6uj7Mq0wDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAYEAFWcl
1tAmt/3DJDjk0ppF62jbwcEOu1N9Nono9a70ojAQYYuMC7Ditw6rLbeXS8tP8ae/
drlci3VxlE5PpmAjuP67Uv2CuGu/2iMqa99AWZ4mVN+x4YL6awvYs8ea6I1Xe8tQ
5+RqvNA+QtnjtfOeb6yWQBAGrc2eTX87IzqvV/EewkdKAs4GZUWG1Zjv3effqjTO
qRX94ltW1GWud7fVcqpZLOaK9U+4IaI2nNHuCtWODoyQmMoVApXyig/YQqFe0eyj
76m1T+2SZLRtn0xn1fTHuLZ2bdtTMZ7k5PTAKnBNEn1Rr9MAS+WEASN1ZyoQ3reL
VYrgkMTrrXPO8bdDTvP7z1Jzv5Cq9WMHFvOLfnj/vN9ZPH6w4QT3Zb97SAAOSPK/
gzOzRtIe+hqCYBh/cwMoeeoAzes/nJgorj3IOTu8JXmtZrZGrdLIhu2Q8U+yKasf
+TUrr6xdcJI/fyVM5BVelpGhqHzzOQe1tO4VYQlAVaaVvFidDPHqTI2/S272
-----END CERTIFICATE-----

30
tests/ssl_certs/ssl_cert_expired.crt
View file

@ -1,30 +0,0 @@
-----BEGIN CERTIFICATE-----
MIIFOTCCA6GgAwIBAgIJALtyUsChEIJpMA0GCSqGSIb3DQEBCwUAMIGAMQswCQYD
VQQGEwJVUzERMA8GA1UECAwITmV3IFlvcmsxETAPBgNVBAcMCEJyb29rbHluMQww
CgYDVQQKDANOWVUxKTAnBgNVBAsMIENvbXB1dGVyIFNjaWVuY2UgYW5kIEVuZ2lu
ZWVyaW5nMRIwEAYDVQQDDAlsb2NhbGhvc3QwHhcNMTgwOTI2MTc0NTM2WhcNMTgw
OTI1MTc0NTM2WjCBgDELMAkGA1UEBhMCVVMxETAPBgNVBAgMCE5ldyBZb3JrMREw
DwYDVQQHDAhCcm9va2x5bjEMMAoGA1UECgwDTllVMSkwJwYDVQQLDCBDb21wdXRl
ciBTY2llbmNlIGFuZCBFbmdpbmVlcmluZzESMBAGA1UEAwwJbG9jYWxob3N0MIIB
ojANBgkqhkiG9w0BAQEFAAOCAY8AMIIBigKCAYEAxyFVeRsWnb1UlCKBks2azM9W
9K+J/ZkzdSb6eCxOIxv79M/Ug54CfWqkySSaQejsu0U/gJxkFYRvwQAy5lATrspY
2kyiWYiggWXFDWz+i8ETPkL9zn59v13sNIpT/IXQj0S3Mr9ZnsUn1qCyEOOIxJxZ
lyuV/M/XP1DP4tArhEvrex12V6MQIK+8fYzEjHG/W7vIIet+wTStIR8ArvVQi0Kv
PbbGCfrZ+e+gq+UpBLBuAfMzM95TW+YJ5duMchie2n6LDmOeegA4jMEv2ppeOr8Q
JJtZuKpXWVbJvLg81yrDjr1rAwJR/WQrnk8GQWPCyPLneAA4mJbi75LqjLxn0AoJ
b3kzLfGEMJJEWXspxNg06bLQU948hB4L7nKARq6s7KoESjEV+/L4koMPWJoNq6fx
OUVw2+S3ITNrDctecRQ1j3RGVPaj5l6bn03C7KV9uRrfqFY3OUjn7A0kDczvRnmr
e1BZIpe+mfGFB+Uu7JiQoBv6I6fqyrdH9rX1LUKlAgMBAAGjgbMwgbAwgZ8GA1Ud
IwSBlzCBlKGBhqSBgzCBgDELMAkGA1UEBhMCVVMxETAPBgNVBAgMCE5ldyBZb3Jr
MREwDwYDVQQHDAhCcm9va2x5bjEMMAoGA1UECgwDTllVMSkwJwYDVQQLDCBDb21w
dXRlciBTY2llbmNlIGFuZCBFbmdpbmVlcmluZzESMBAGA1UEAwwJbG9jYWxob3N0
ggkAu3JSwKEQgmkwDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAYEAW4I1
TacdFv3L9ENFkSLciPb7zFMckLUZfk/P+4VjdapWrfuydO4W/ogMxA4DK09thTsK
N/BgcExyKjDldGUfUv57Tqv3v2E5kbygNcNtP53fwMz3y+7QourzkDE5HWciw1Lb
hmbnCBTzt/UioSBdJnAH29GWpSS+Jzu745sRaI48AS/J5ApH2aVEnNQTCE7v1LNH
2bTTPYl3eDXiD8yOhvyiW1F4y2BSFbQRH/3aE6Goe4A75m8sX50+JlOgjyyQnAMf
vbfvZsjGfqdXv9Qpci50qKCFxHJLXXNAUbX3fDgKE+RoZUNZnmn2VDgJYnToz6on
RcVnppV09kmSjHXZBT04XXUA0vG3p+oU0TO4puJlePVf4Oz23/DRCPHSfVWgMeB2
c1PpKit4+Bz7mypnsWVw8kk//l0GJ1cHnkkZElKJtPEB7I587jgTCDcN811TGNBc
rLLd/JwtYAvi1CPFt2ICGDvA4AKLY3rBNg5z1DrSE/iom1NTC00SFZJztYiX
-----END CERTIFICATE-----

31
tests/ssl_certs/ssl_cert_wronghost.crt
View file

@ -1,31 +0,0 @@
-----BEGIN CERTIFICATE-----
MIIFRTCCA62gAwIBAgIJAKY6b706lpuDMA0GCSqGSIb3DQEBCwUAMIGEMQswCQYD
VQQGEwJVUzERMA8GA1UECAwITmV3IFlvcmsxETAPBgNVBAcMCEJyb29rbHluMQww
CgYDVQQKDANOWVUxKTAnBgNVBAsMIENvbXB1dGVyIFNjaWVuY2UgYW5kIEVuZ2lu
ZWVyaW5nMRYwFAYDVQQDDA1ub3RteWhvc3RuYW1lMB4XDTE4MDkxMjE2NTkxN1oX
DTM4MDkwNzE2NTkxN1owgYQxCzAJBgNVBAYTAlVTMREwDwYDVQQIDAhOZXcgWW9y
azERMA8GA1UEBwwIQnJvb2tseW4xDDAKBgNVBAoMA05ZVTEpMCcGA1UECwwgQ29t
cHV0ZXIgU2NpZW5jZSBhbmQgRW5naW5lZXJpbmcxFjAUBgNVBAMMDW5vdG15aG9z
dG5hbWUwggGiMA0GCSqGSIb3DQEBAQUAA4IBjwAwggGKAoIBgQDHIVV5GxadvVSU
IoGSzZrMz1b0r4n9mTN1Jvp4LE4jG/v0z9SDngJ9aqTJJJpB6Oy7RT+AnGQVhG/B
ADLmUBOuyljaTKJZiKCBZcUNbP6LwRM+Qv3Ofn2/Xew0ilP8hdCPRLcyv1mexSfW
oLIQ44jEnFmXK5X8z9c/UM/i0CuES+t7HXZXoxAgr7x9jMSMcb9bu8gh637BNK0h
HwCu9VCLQq89tsYJ+tn576Cr5SkEsG4B8zMz3lNb5gnl24xyGJ7afosOY556ADiM
wS/aml46vxAkm1m4qldZVsm8uDzXKsOOvWsDAlH9ZCueTwZBY8LI8ud4ADiYluLv
kuqMvGfQCglveTMt8YQwkkRZeynE2DTpstBT3jyEHgvucoBGrqzsqgRKMRX78viS
gw9Ymg2rp/E5RXDb5LchM2sNy15xFDWPdEZU9qPmXpufTcLspX25Gt+oVjc5SOfs
DSQNzO9Geat7UFkil76Z8YUH5S7smJCgG/ojp+rKt0f2tfUtQqUCAwEAAaOBtzCB
tDCBowYDVR0jBIGbMIGYoYGKpIGHMIGEMQswCQYDVQQGEwJVUzERMA8GA1UECAwI
TmV3IFlvcmsxETAPBgNVBAcMCEJyb29rbHluMQwwCgYDVQQKDANOWVUxKTAnBgNV
BAsMIENvbXB1dGVyIFNjaWVuY2UgYW5kIEVuZ2luZWVyaW5nMRYwFAYDVQQDDA1u
b3RteWhvc3RuYW1lggkApjpvvTqWm4MwDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0B
AQsFAAOCAYEAvpBMce3kxwo9W0o4RqezkSxnNyax0ezbUNodIkx5kbzX09qQLqhK
SkhQY3CNmtrpsczUg1W2nldxioEouwfTlhi15H98E/8XytpGaHO7Rnbtq8nkOp3E
N1+DMfFR95OynbHSd7bfK9UEmH1CmCnttvCuQkLTxDCpEsQNAxvmU/yDONoDr+cu
jGo80XTnYTqHl5/UtGbCS4SAIdWgrXTIqVvY/eF+mR+3nQEYjBuqW0cNfXLyYLXH
XMc6qtfGX1P+NRWtlrWgGQmc0fry+GczRHMJuKtJMV2xZzPJAJqwwvj3Fjz8HNGu
ZX3kVdbkDjf8is2cWgyZqDecqPHDBW4Ey539s/5eurgOkEvhriS4/9RnVhgdzduj
nRdXkD10ficrFcBQO0KaTWT+iFBc9duuYPuLRyRTye5p3t0liOikH2XrRXs4IBfz
2mT4npXQl1liNixcCf/yUEUOSQAJDG6aRjDjD4SZBUPDLjfqKLid8M0BpLQrks9L
5hAg1WZXorY6
-----END CERTIFICATE-----

287
tests/test_arbitrary_package_attack_old.py
View file

@ -1,287 +0,0 @@
#!/usr/bin/env python
# Copyright 2012 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
test_arbitrary_package_attack_old.py
<Author>
Konstantin Andrianov.
<Started>
February 22, 2012.
March 21, 2014.
Refactored to use the 'unittest' module (test conditions in code, rather
than verifying text output), use pre-generated repository files, and
discontinue use of the old repository tools. -vladimir.v.diaz
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Simulate an arbitrary package attack, where an updater client attempts to
download a malicious file. TUF and non-TUF client scenarios are tested.
There is no difference between 'updates' and 'target' files.
"""
import os
import tempfile
import shutil
import json
import logging
import unittest
import sys
from urllib import request
import tuf
import tuf.formats
import tuf.roledb
import tuf.keydb
import tuf.log
import tuf.client.updater as updater
import tuf.unittest_toolbox as unittest_toolbox
from tests import utils
import securesystemslib
logger = logging.getLogger(__name__)
class TestArbitraryPackageAttack(unittest_toolbox.Modified_TestCase):
@classmethod
def setUpClass(cls):
# Create a temporary directory to store the repository, metadata, and target
# files. 'temporary_directory' must be deleted in TearDownModule() so that
# temporary files are always removed, even when exceptions occur.
cls.temporary_directory = tempfile.mkdtemp(dir=os.getcwd())
# Launch a SimpleHTTPServer (serves files in the current directory).
# Test cases will request metadata and target files that have been
# pre-generated in 'tuf/tests/repository_data', which will be served by the
# SimpleHTTPServer launched here. The test cases of this unit test assume
# the pre-generated metadata files have a specific structure, such
# as a delegated role 'targets/role1', three target files, five key files,
# etc.
cls.server_process_handler = utils.TestServerProcess(log=logger)
@classmethod
def tearDownClass(cls):
# Cleans the resources and flush the logged lines (if any).
cls.server_process_handler.clean()
# Remove the temporary repository directory, which should contain all the
# metadata, targets, and key files generated of all the test cases.
shutil.rmtree(cls.temporary_directory)
def setUp(self):
# We are inheriting from custom class.
unittest_toolbox.Modified_TestCase.setUp(self)
self.repository_name = 'test_repository1'
# Copy the original repository files provided in the test folder so that
# any modifications made to repository files are restricted to the copies.
# The 'repository_data' directory is expected to exist in 'tuf/tests/'.
original_repository_files = os.path.join(os.getcwd(), 'repository_data')
temporary_repository_root = \
self.make_temp_directory(directory=self.temporary_directory)
# The original repository, keystore, and client directories will be copied
# for each test case.
original_repository = os.path.join(original_repository_files, 'repository')
original_client = os.path.join(original_repository_files, 'client')
# Save references to the often-needed client repository directories.
# Test cases need these references to access metadata and target files.
self.repository_directory = \
os.path.join(temporary_repository_root, 'repository')
self.client_directory = os.path.join(temporary_repository_root, 'client')
# Copy the original 'repository', 'client', and 'keystore' directories
# to the temporary repository the test cases can use.
shutil.copytree(original_repository, self.repository_directory)
shutil.copytree(original_client, self.client_directory)
# Set the url prefix required by the 'tuf/client/updater.py' updater.
# 'path/to/tmp/repository' -> 'localhost:8001/tmp/repository'.
repository_basepath = self.repository_directory[len(os.getcwd()):]
url_prefix = 'http://' + utils.TEST_HOST_ADDRESS + ':' \
+ str(self.server_process_handler.port) + repository_basepath
# Setting 'tuf.settings.repository_directory' with the temporary client
# directory copied from the original repository files.
tuf.settings.repositories_directory = self.client_directory
self.repository_mirrors = {'mirror1': {'url_prefix': url_prefix,
'metadata_path': 'metadata',
'targets_path': 'targets'}}
# Create the repository instance. The test cases will use this client
# updater to refresh metadata, fetch target files, etc.
self.repository_updater = updater.Updater(self.repository_name,
self.repository_mirrors)
def tearDown(self):
# updater.Updater() populates the roledb with the name "test_repository1"
tuf.roledb.clear_roledb(clear_all=True)
tuf.keydb.clear_keydb(clear_all=True)
# Logs stdout and stderr from the sever subprocess.
self.server_process_handler.flush_log()
# Remove temporary directory
unittest_toolbox.Modified_TestCase.tearDown(self)
def test_without_tuf(self):
# Verify that a target file replaced with a malicious version is downloaded
# by a non-TUF client (i.e., a non-TUF client that does not verify hashes,
# detect mix-and-mix attacks, etc.) A tuf client, on the other hand, should
# detect that the downloaded target file is invalid.
# Test: Download a valid target file from the repository.
# Ensure the target file to be downloaded has not already been downloaded,
# and generate its file size and digest. The file size and digest is needed
# to check that the malicious file was indeed downloaded.
target_path = os.path.join(self.repository_directory, 'targets', 'file1.txt')
client_target_path = os.path.join(self.client_directory, 'file1.txt')
self.assertFalse(os.path.exists(client_target_path))
length, hashes = securesystemslib.util.get_file_details(target_path)
fileinfo = tuf.formats.make_targets_fileinfo(length, hashes)
url_prefix = self.repository_mirrors['mirror1']['url_prefix']
url_file = os.path.join(url_prefix, 'targets', 'file1.txt')
# On Windows, the URL portion should not contain back slashes.
request.urlretrieve(url_file.replace('\\', '/'), client_target_path)
self.assertTrue(os.path.exists(client_target_path))
length, hashes = securesystemslib.util.get_file_details(client_target_path)
download_fileinfo = tuf.formats.make_targets_fileinfo(length, hashes)
self.assertEqual(fileinfo, download_fileinfo)
# Test: Download a target file that has been modified by an attacker.
with open(target_path, 'wt') as file_object:
file_object.write('add malicious content.')
length, hashes = securesystemslib.util.get_file_details(target_path)
malicious_fileinfo = tuf.formats.make_targets_fileinfo(length, hashes)
# On Windows, the URL portion should not contain back slashes.
request.urlretrieve(url_file.replace('\\', '/'), client_target_path)
length, hashes = securesystemslib.util.get_file_details(client_target_path)
download_fileinfo = tuf.formats.make_targets_fileinfo(length, hashes)
# Verify 'download_fileinfo' is unequal to the original trusted version.
self.assertNotEqual(download_fileinfo, fileinfo)
# Verify 'download_fileinfo' is equal to the malicious version.
self.assertEqual(download_fileinfo, malicious_fileinfo)
def test_with_tuf(self):
# Verify that a target file (on the remote repository) modified by an
# attacker is not downloaded by the TUF client.
# First test that the valid target file is successfully downloaded.
file1_fileinfo = self.repository_updater.get_one_valid_targetinfo('file1.txt')
destination = os.path.join(self.client_directory)
self.repository_updater.download_target(file1_fileinfo, destination)
client_target_path = os.path.join(destination, 'file1.txt')
self.assertTrue(os.path.exists(client_target_path))
# Modify 'file1.txt' and confirm that the TUF client rejects it.
target_path = os.path.join(self.repository_directory, 'targets', 'file1.txt')
with open(target_path, 'wt') as file_object:
file_object.write('malicious content, size 33 bytes.')
try:
self.repository_updater.download_target(file1_fileinfo, destination)
except tuf.exceptions.NoWorkingMirrorError as exception:
url_prefix = self.repository_mirrors['mirror1']['url_prefix']
url_file = os.path.join(url_prefix, 'targets', 'file1.txt')
# Verify that only one exception is raised for 'url_file'.
self.assertTrue(len(exception.mirror_errors), 1)
# Verify that the expected 'tuf.exceptions.DownloadLengthMismatchError' exception
# is raised for 'url_file'.
self.assertTrue(url_file.replace('\\', '/') in exception.mirror_errors)
self.assertTrue(
isinstance(exception.mirror_errors[url_file.replace('\\', '/')],
securesystemslib.exceptions.BadHashError))
else:
self.fail('TUF did not prevent an arbitrary package attack.')
def test_with_tuf_and_metadata_tampering(self):
# Test that a TUF client does not download a malicious target file, and a
# 'targets.json' metadata file that has also been modified by the attacker.
# The attacker does not attach a valid signature to 'targets.json'
# An attacker modifies 'file1.txt'.
target_path = os.path.join(self.repository_directory, 'targets', 'file1.txt')
with open(target_path, 'wt') as file_object:
file_object.write('malicious content, size 33 bytes.')
# An attacker also tries to add the malicious target's length and digest
# to its metadata file.
length, hashes = securesystemslib.util.get_file_details(target_path)
metadata_path = \
os.path.join(self.repository_directory, 'metadata', 'targets.json')
metadata = securesystemslib.util.load_json_file(metadata_path)
metadata['signed']['targets']['file1.txt']['hashes'] = hashes
metadata['signed']['targets']['file1.txt']['length'] = length
tuf.formats.check_signable_object_format(metadata)
with open(metadata_path, 'wb') as file_object:
file_object.write(json.dumps(metadata, indent=1,
separators=(',', ': '), sort_keys=True).encode('utf-8'))
# Verify that the malicious 'targets.json' is not downloaded. Perform
# a refresh of top-level metadata to demonstrate that the malicious
# 'targets.json' is not downloaded.
try:
self.repository_updater.refresh()
file1_fileinfo = self.repository_updater.get_one_valid_targetinfo('file1.txt')
destination = os.path.join(self.client_directory)
self.repository_updater.download_target(file1_fileinfo, destination)
except tuf.exceptions.NoWorkingMirrorError as exception:
url_prefix = self.repository_mirrors['mirror1']['url_prefix']
url_file = os.path.join(url_prefix, 'targets', 'file1.txt')
# Verify that an exception raised for only the malicious 'url_file'.
self.assertTrue(len(exception.mirror_errors), 1)
# Verify that the specific and expected mirror exception is raised.
self.assertTrue(url_file.replace('\\', '/') in exception.mirror_errors)
self.assertTrue(
isinstance(exception.mirror_errors[url_file.replace('\\', '/')],
securesystemslib.exceptions.BadHashError))
else:
self.fail('TUF did not prevent an arbitrary package attack.')
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

428
tests/test_developer_tool_old.py
View file

@ -1,428 +0,0 @@
#!/usr/bin/env python
# Copyright 2014 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
test_developer_tool_old.py.
<Authors>
Santiago Torres Arias <torresariass@gmail.com>
Zane Fisher <zanefisher@gmail.com>
<Started>
January 22, 2014.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Unit test for the 'developer_tool.py' module.
"""
import os
import unittest
import logging
import tempfile
import shutil
import sys
import tuf
import tuf.log
import tuf.roledb
import tuf.keydb
import tuf.developer_tool as developer_tool
import tuf.exceptions
import securesystemslib
import securesystemslib.exceptions
from tuf.developer_tool import METADATA_DIRECTORY_NAME
from tuf.developer_tool import TARGETS_DIRECTORY_NAME
from tests import utils
logger = logging.getLogger(__name__)
developer_tool.disable_console_log_messages()
class TestProject(unittest.TestCase):
tmp_dir = None
@classmethod
def setUpClass(cls):
cls.tmp_dir = tempfile.mkdtemp(dir = os.getcwd())
@classmethod
def tearDownClass(cls):
shutil.rmtree(cls.tmp_dir)
def tearDown(self):
# called after every test case
tuf.roledb.clear_roledb(clear_all=True)
tuf.keydb.clear_keydb(clear_all=True)
def test_create_new_project(self):
# Test cases for the create_new_project function. In this test we will
# check input, correct file creation and format. We also check
# that a proper object is generated. We will use the normal layout for this
# test suite.
# Create a local subfolder for this test.
local_tmp = tempfile.mkdtemp(dir = self.tmp_dir)
# These are the usual values we will be throwing to the function, however
# we will swap these for nulls or malformed values every now and then to
# test input.
project_name = 'test_suite'
metadata_directory = local_tmp
location_in_repository = '/prefix'
targets_directory = None
key = None
# Create a blank project.
project = developer_tool.create_new_project(project_name, metadata_directory,
location_in_repository)
self.assertTrue(isinstance(project, developer_tool.Project))
self.assertTrue(project.layout_type == 'repo-like')
self.assertTrue(project.prefix == location_in_repository)
self.assertTrue(project.project_name == project_name)
self.assertTrue(project.metadata_directory ==
os.path.join(metadata_directory,METADATA_DIRECTORY_NAME))
self.assertTrue(project.targets_directory ==
os.path.join(metadata_directory,TARGETS_DIRECTORY_NAME))
# Create a blank project without a prefix.
project = developer_tool.create_new_project(project_name, metadata_directory)
self.assertTrue(isinstance(project, developer_tool.Project))
self.assertTrue(project.layout_type == 'repo-like')
self.assertTrue(project.prefix == '')
self.assertTrue(project.project_name == project_name)
self.assertTrue(project.metadata_directory ==
os.path.join(metadata_directory,METADATA_DIRECTORY_NAME))
self.assertTrue(project.targets_directory ==
os.path.join(metadata_directory,TARGETS_DIRECTORY_NAME))
# Create a blank project without a valid metadata directory.
self.assertRaises(securesystemslib.exceptions.FormatError, developer_tool.create_new_project,
0, metadata_directory, location_in_repository)
self.assertRaises(securesystemslib.exceptions.FormatError, developer_tool.create_new_project,
project_name, 0, location_in_repository)
self.assertRaises(securesystemslib.exceptions.FormatError, developer_tool.create_new_project,
project_name, metadata_directory, 0)
# Create a new project with a flat layout.
targets_directory = tempfile.mkdtemp(dir = local_tmp)
metadata_directory = tempfile.mkdtemp(dir = local_tmp)
project = developer_tool.create_new_project(project_name, metadata_directory,
location_in_repository, targets_directory)
self.assertTrue(isinstance(project, developer_tool.Project))
self.assertTrue(project.layout_type == 'flat')
self.assertTrue(project.prefix == location_in_repository)
self.assertTrue(project.project_name == project_name)
self.assertTrue(project.metadata_directory == metadata_directory)
self.assertTrue(project.targets_directory == targets_directory)
# Finally, check that if targets_directory is set, it is valid.
self.assertRaises(securesystemslib.exceptions.FormatError, developer_tool.create_new_project,
project_name, metadata_directory, location_in_repository, 0)
# Copy a key to our workspace and create a new project with it.
keystore_path = os.path.join('repository_data','keystore')
# I will use the same key as the one provided in the repository
# tool tests for the root role, but this is not a root role...
root_key_path = os.path.join(keystore_path,'root_key.pub')
project_key = developer_tool.import_rsa_publickey_from_file(root_key_path)
# Test create new project with a key added by default.
project = developer_tool.create_new_project(project_name, metadata_directory,
location_in_repository, targets_directory, project_key)
self.assertTrue(isinstance(project, developer_tool.Project))
self.assertTrue(project.layout_type == 'flat')
self.assertTrue(project.prefix == location_in_repository)
self.assertTrue(project.project_name == project_name)
self.assertTrue(project.metadata_directory == metadata_directory)
self.assertTrue(project.targets_directory == targets_directory)
self.assertTrue(len(project.keys) == 1)
self.assertTrue(project.keys[0] == project_key['keyid'])
# Try to write to an invalid location. The OSError should be re-raised by
# create_new_project().
shutil.rmtree(targets_directory)
tuf.roledb.clear_roledb()
tuf.keydb.clear_keydb()
metadata_directory = '/'
valid_metadata_directory_name = developer_tool.METADATA_DIRECTORY_NAME
developer_tool.METADATA_DIRECTORY_NAME = '/'
try:
developer_tool.create_new_project(project_name, metadata_directory,
location_in_repository, targets_directory, project_key)
except (OSError, tuf.exceptions.RepositoryError):
pass
developer_tool.METADATA_DIRECTORY_NAME = valid_metadata_directory_name
def test_load_project(self):
# This test case will first try to load an existing project and test for
# verify the loaded object. It will next try to load a nonexisting project
# and expect a correct error handler. Finally, it will try to overwrite the
# existing prefix of the loaded project.
# Create a local subfolder for this test.
local_tmp = tempfile.mkdtemp(dir = self.tmp_dir)
# Test non-existent project filepath.
nonexistent_path = os.path.join(local_tmp, 'nonexistent')
self.assertRaises(securesystemslib.exceptions.StorageError,
developer_tool.load_project, nonexistent_path)
# Copy the pregenerated metadata.
project_data_filepath = os.path.join('repository_data', 'project')
target_project_data_filepath = os.path.join(local_tmp, 'project')
shutil.copytree('repository_data/project', target_project_data_filepath)
# Properly load a project.
repo_filepath = os.path.join(local_tmp, 'project', 'test-flat')
new_targets_path = os.path.join(local_tmp, 'project', 'targets')
project = developer_tool.load_project(repo_filepath,
new_targets_location = new_targets_path)
self.assertTrue(project._targets_directory == new_targets_path)
self.assertTrue(project.layout_type == 'flat')
# Load a project overwriting the prefix.
project = developer_tool.load_project(repo_filepath, prefix='new')
self.assertTrue(project.prefix == 'new')
# Load a project with a file missing.
file_to_corrupt = os.path.join(repo_filepath, 'test-flat.json')
with open(file_to_corrupt, 'wt') as fp:
fp.write('this is not a json file')
self.assertRaises(securesystemslib.exceptions.Error, developer_tool.load_project, repo_filepath)
def test_add_verification_keys(self):
# Create a new project instance.
project = developer_tool.Project('test_verification_keys', 'somepath',
'someotherpath', 'prefix')
# Add invalid verification key.
self.assertRaises(securesystemslib.exceptions.FormatError, project.add_verification_key, 'invalid')
# Add verification key.
# - load it first
keystore_path = os.path.join('repository_data', 'keystore')
first_verification_key_path = os.path.join(keystore_path,'root_key.pub')
first_verification_key = \
developer_tool.import_rsa_publickey_from_file(first_verification_key_path)
project.add_verification_key(first_verification_key)
# Add another verification key (should expect exception.)
second_verification_key_path = os.path.join(keystore_path, 'snapshot_key.pub')
second_verification_key = \
developer_tool.import_ed25519_publickey_from_file(second_verification_key_path)
self.assertRaises(securesystemslib.exceptions.Error,
project.add_verification_key,(second_verification_key))
# Add a verification key for the delegation.
project.delegate('somedelegation', [], [])
project('somedelegation').add_verification_key(first_verification_key)
project('somedelegation').add_verification_key(second_verification_key)
# Add another delegation of the delegation.
project('somedelegation').delegate('somesubdelegation', [], [])
project('somesubdelegation').add_verification_key(first_verification_key)
project('somesubdelegation').add_verification_key(second_verification_key)
def test_write(self):
# Create tmp directory.
local_tmp = tempfile.mkdtemp(dir=self.tmp_dir)
# Create new project inside tmp directory.
project = developer_tool.create_new_project('new_project', local_tmp,
'prefix');
# Create some target files inside the tmp directory.
target_filepath = os.path.join(local_tmp, 'targets', 'test_target')
with open(target_filepath, 'wt') as fp:
fp.write('testing file')
# Add the targets.
project.add_target(os.path.basename(target_filepath))
# Add verification keys.
keystore_path = os.path.join('repository_data', 'keystore')
project_key_path = os.path.join(keystore_path, 'root_key.pub')
project_key = \
developer_tool.import_rsa_publickey_from_file(project_key_path)
# Call status (for the sake of doing it and to improve test coverage by
# executing its statements.)
project.status()
project.add_verification_key(project_key)
# Add another verification key (should expect exception.)
delegation_key_path = os.path.join(keystore_path, 'snapshot_key.pub')
delegation_key = \
developer_tool.import_ed25519_publickey_from_file(delegation_key_path)
# Add a subdelegation.
subdelegation_key_path = os.path.join(keystore_path, 'timestamp_key.pub')
subdelegation_key = \
developer_tool.import_ed25519_publickey_from_file(subdelegation_key_path)
# Add a delegation.
project.delegate('delegation', [delegation_key], [])
project('delegation').delegate('subdelegation', [subdelegation_key], [])
# call write (except)
self.assertRaises(securesystemslib.exceptions.Error, project.write, ())
# Call status (for the sake of doing it and executing its statements.)
project.status()
# Load private keys.
project_private_key_path = os.path.join(keystore_path, 'root_key')
project_private_key = \
developer_tool.import_rsa_privatekey_from_file(project_private_key_path,
'password')
delegation_private_key_path = os.path.join(keystore_path, 'snapshot_key')
delegation_private_key = \
developer_tool.import_ed25519_privatekey_from_file(delegation_private_key_path,
'password')
subdelegation_private_key_path = \
os.path.join(keystore_path, 'timestamp_key')
subdelegation_private_key = \
developer_tool.import_ed25519_privatekey_from_file(subdelegation_private_key_path,
'password')
# Test partial write.
# backup everything (again)
# + backup targets.
targets_backup = project.target_files
# + backup delegations.
delegations_backup = \
tuf.roledb.get_delegated_rolenames(project.project_name)
# + backup layout type.
layout_type_backup = project.layout_type
# + backup keyids.
keys_backup = project.keys
delegation_keys_backup = project('delegation').keys
# + backup the prefix.
prefix_backup = project.prefix
# + backup the name.
name_backup = project.project_name
# Write and reload.
self.assertRaises(securesystemslib.exceptions.Error, project.write)
project.write(write_partial=True)
project = developer_tool.load_project(local_tmp)
# Check against backup.
self.assertEqual(list(project.target_files.keys()), list(targets_backup.keys()))
new_delegations = tuf.roledb.get_delegated_rolenames(project.project_name)
self.assertEqual(new_delegations, delegations_backup)
self.assertEqual(project.layout_type, layout_type_backup)
self.assertEqual(project.keys, keys_backup)
self.assertEqual(project('delegation').keys, delegation_keys_backup)
self.assertEqual(project.prefix, prefix_backup)
self.assertEqual(project.project_name, name_backup)
roleinfo = tuf.roledb.get_roleinfo(project.project_name)
self.assertEqual(roleinfo['partial_loaded'], True)
# Load_signing_keys.
project('delegation').load_signing_key(delegation_private_key)
project.status()
project.load_signing_key(project_private_key)
# Backup everything.
# + backup targets.
targets_backup = project.target_files
# + backup delegations.
delegations_backup = \
tuf.roledb.get_delegated_rolenames(project.project_name)
# + backup layout type.
layout_type_backup = project.layout_type
# + backup keyids
keys_backup = project.keys
delegation_keys_backup = project('delegation').keys
# + backup the prefix.
prefix_backup = project.prefix
# + backup the name.
name_backup = project.project_name
# Call status (for the sake of doing it.)
project.status()
# Call write.
project.write()
# Call load.
project = developer_tool.load_project(local_tmp)
# Check against backup.
self.assertEqual(list(project.target_files.keys()), list(targets_backup.keys()))
new_delegations = tuf.roledb.get_delegated_rolenames(project.project_name)
self.assertEqual(new_delegations, delegations_backup)
self.assertEqual(project.layout_type, layout_type_backup)
self.assertEqual(project.keys, keys_backup)
self.assertEqual(project('delegation').keys, delegation_keys_backup)
self.assertEqual(project.prefix, prefix_backup)
self.assertEqual(project.project_name, name_backup)
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

392
tests/test_download_old.py
View file

@ -1,392 +0,0 @@
#!/usr/bin/env python
# Copyright 2014 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program>
test_download_old.py
<Author>
Konstantin Andrianov.
<Started>
March 26, 2012.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Unit test for 'download.py'.
NOTE: Make sure test_download_old.py is ran in 'tuf/tests/' directory.
Otherwise, module that launches simple server would not be found.
TODO: Adopt the environment variable management from test_proxy_use.py here.
"""
import hashlib
import logging
import os
import sys
import unittest
import urllib3
import warnings
import tuf
import tuf.download as download
import tuf.requests_fetcher
import tuf.log
import tuf.unittest_toolbox as unittest_toolbox
import tuf.exceptions
from tests import utils
import requests.exceptions
import securesystemslib
logger = logging.getLogger(__name__)
class TestDownload(unittest_toolbox.Modified_TestCase):
def setUp(self):
"""
Create a temporary file and launch a simple server in the
current working directory.
"""
unittest_toolbox.Modified_TestCase.setUp(self)
# Making a temporary file.
current_dir = os.getcwd()
target_filepath = self.make_temp_data_file(directory=current_dir)
self.target_fileobj = open(target_filepath, 'r')
self.target_data = self.target_fileobj.read()
self.target_data_length = len(self.target_data)
# Launch a SimpleHTTPServer (serves files in the current dir).
self.server_process_handler = utils.TestServerProcess(log=logger)
rel_target_filepath = os.path.basename(target_filepath)
self.url = 'http://' + utils.TEST_HOST_ADDRESS + ':' \
+ str(self.server_process_handler.port) + '/' + rel_target_filepath
# Computing hash of target file data.
m = hashlib.md5()
m.update(self.target_data.encode('utf-8'))
digest = m.hexdigest()
self.target_hash = {'md5':digest}
# Initialize the default fetcher for the download
self.fetcher = tuf.requests_fetcher.RequestsFetcher()
# Stop server process and perform clean up.
def tearDown(self):
# Cleans the resources and flush the logged lines (if any).
self.server_process_handler.clean()
self.target_fileobj.close()
# Remove temp directory
unittest_toolbox.Modified_TestCase.tearDown(self)
# Test: Normal case.
def test_download_url_to_tempfileobj(self):
download_file = download.safe_download
with download_file(self.url, self.target_data_length, self.fetcher) as temp_fileobj:
temp_fileobj.seek(0)
temp_file_data = temp_fileobj.read().decode('utf-8')
self.assertEqual(self.target_data, temp_file_data)
self.assertEqual(self.target_data_length, len(temp_file_data))
# Test: Download url in more than one chunk.
def test_download_url_in_chunks(self):
# Set smaller chunk size to ensure that the file will be downloaded
# in more than one chunk
default_chunk_size = tuf.settings.CHUNK_SIZE
tuf.settings.CHUNK_SIZE = 4
# We don't have access to chunks from download_file()
# so we just confirm that the expectation of more than one chunk is
# correct and verify that no errors are raised during download
chunks_count = self.target_data_length/tuf.settings.CHUNK_SIZE
self.assertGreater(chunks_count, 1)
download_file = download.safe_download
with download_file(self.url, self.target_data_length, self.fetcher) as temp_fileobj:
temp_fileobj.seek(0)
temp_file_data = temp_fileobj.read().decode('utf-8')
self.assertEqual(self.target_data, temp_file_data)
self.assertEqual(self.target_data_length, len(temp_file_data))
# Restore default settings
tuf.settings.CHUNK_SIZE = default_chunk_size
# Test: Incorrect lengths.
def test_download_url_to_tempfileobj_and_lengths(self):
# We do *not* catch
# 'securesystemslib.exceptions.DownloadLengthMismatchError' in the
# following two calls because the file at 'self.url' contains enough bytes
# to satisfy the smaller number of required bytes requested.
# safe_download() and unsafe_download() will only log a warning when the
# the server-reported length of the file does not match the
# required_length. 'updater.py' *does* verify the hashes of downloaded
# content.
download.safe_download(self.url, self.target_data_length - 4, self.fetcher).close()
download.unsafe_download(self.url, self.target_data_length - 4, self.fetcher).close()
# We catch 'tuf.exceptions.DownloadLengthMismatchError' for safe_download()
# because it will not download more bytes than requested (in this case, a
# length greater than the size of the target file).
self.assertRaises(tuf.exceptions.DownloadLengthMismatchError,
download.safe_download, self.url, self.target_data_length + 1, self.fetcher)
# Calling unsafe_download() with a mismatched length should not raise an
# exception.
download.unsafe_download(self.url, self.target_data_length + 1, self.fetcher).close()
def test_download_url_to_tempfileobj_and_performance(self):
"""
# Measuring performance of 'auto_flush = False' vs. 'auto_flush = True'
# in download._download_file() during write. No change was observed.
star_cpu = time.clock()
star_real = time.time()
temp_fileobj = download_file(self.url,
self.target_data_length)
end_cpu = time.clock()
end_real = time.time()
self.assertEqual(self.target_data, temp_fileobj.read())
self.assertEqual(self.target_data_length, len(temp_fileobj.read()))
temp_fileobj.close()
print "Performance cpu time: "+str(end_cpu - star_cpu)
print "Performance real time: "+str(end_real - star_real)
# TODO: [Not urgent] Show the difference by setting write(auto_flush=False)
"""
# Test: Incorrect/Unreachable URLs.
def test_download_url_to_tempfileobj_and_urls(self):
download_file = download.safe_download
unsafe_download_file = download.unsafe_download
with self.assertRaises(securesystemslib.exceptions.FormatError):
download_file(None, self.target_data_length, self.fetcher)
url = 'http://' + utils.TEST_HOST_ADDRESS + ':' \
+ str(self.server_process_handler.port) + '/' + self.random_string()
with self.assertRaises(tuf.exceptions.FetcherHTTPError) as cm:
download_file(url, self.target_data_length, self.fetcher)
self.assertEqual(cm.exception.status_code, 404)
url1 = 'http://' + utils.TEST_HOST_ADDRESS + ':' \
+ str(self.server_process_handler.port + 1) + '/' + self.random_string()
with self.assertRaises(requests.exceptions.ConnectionError):
download_file(url1, self.target_data_length, self.fetcher)
# Specify an unsupported URI scheme.
url_with_unsupported_uri = self.url.replace('http', 'file')
self.assertRaises(requests.exceptions.InvalidSchema, download_file, url_with_unsupported_uri,
self.target_data_length, self.fetcher)
self.assertRaises(requests.exceptions.InvalidSchema, unsafe_download_file,
url_with_unsupported_uri, self.target_data_length, self.fetcher)
'''
# This test uses sites on the internet, requiring a net connection to succeed.
# Since this is the only such test in TUF, I'm not going to enable it... but
# it's here in case it's useful for diagnosis.
def test_https_validation(self):
"""
Use some known URLs on the net to ensure that TUF download checks SSL
certificates appropriately.
"""
# We should never get as far as the target file download itself, so the
# length we pass to safe_download and unsafe_download shouldn't matter.
irrelevant_length = 10
for bad_url in [
'https://expired.badssl.com/', # expired certificate
'https://wrong.host.badssl.com/', ]: # hostname verification fail
with self.assertRaises(requests.exceptions.SSLError):
download.safe_download(bad_url, irrelevant_length)
with self.assertRaises(requests.exceptions.SSLError):
download.unsafe_download(bad_url, irrelevant_length)
'''
def test_https_connection(self):
"""
Try various HTTPS downloads using trusted and untrusted certificates with
and without the correct hostname listed in the SSL certificate.
"""
# Make a temporary file to be served to the client.
current_directory = os.getcwd()
target_filepath = self.make_temp_data_file(directory=current_directory)
with open(target_filepath, 'r') as target_file_object:
target_data_length = len(target_file_object.read())
# These cert files provide various test cases:
# good: A valid cert from an older generation of test_download.py tests.
# good2: A valid cert made simultaneous to the bad certs below, with the
# same settings otherwise, tested here in case the difference
# between the way the new bad certs and the old good cert were
# generated turns out to matter at some point.
# bad: An otherwise-valid cert with the wrong hostname. The good certs
# list "localhost", but this lists "notmyhostname".
# expired: An otherwise-valid cert but which is expired (no valid dates
# exist, fwiw: startdate > enddate).
good_cert_fname = os.path.join('ssl_certs', 'ssl_cert.crt')
good2_cert_fname = os.path.join('ssl_certs', 'ssl_cert_2.crt')
bad_cert_fname = os.path.join('ssl_certs', 'ssl_cert_wronghost.crt')
expired_cert_fname = os.path.join('ssl_certs', 'ssl_cert_expired.crt')
# Launch four HTTPS servers (serve files in the current dir).
# 1: we expect to operate correctly
# 2: also good; uses a slightly different cert (controls for the cert
# generation method used for the next two, in case it comes to matter)
# 3: run with an HTTPS certificate with an unexpected hostname
# 4: run with an HTTPS certificate that is expired
# Be sure to offset from the port used in setUp to avoid collision.
good_https_server_handler = utils.TestServerProcess(log=logger,
server='simple_https_server_old.py',
extra_cmd_args=[good_cert_fname])
good2_https_server_handler = utils.TestServerProcess(log=logger,
server='simple_https_server_old.py',
extra_cmd_args=[good2_cert_fname])
bad_https_server_handler = utils.TestServerProcess(log=logger,
server='simple_https_server_old.py',
extra_cmd_args=[bad_cert_fname])
expd_https_server_handler = utils.TestServerProcess(log=logger,
server='simple_https_server_old.py',
extra_cmd_args=[expired_cert_fname])
suffix = '/' + os.path.basename(target_filepath)
good_https_url = 'https://localhost:' \
+ str(good_https_server_handler.port) + suffix
good2_https_url = 'https://localhost:' \
+ str(good2_https_server_handler.port) + suffix
bad_https_url = 'https://localhost:' \
+ str(bad_https_server_handler.port) + suffix
expired_https_url = 'https://localhost:' \
+ str(expd_https_server_handler.port) + suffix
# Download the target file using an HTTPS connection.
# Use try-finally solely to ensure that the server processes are killed.
try:
# Trust the certfile that happens to use a different hostname than we
# will expect.
os.environ['REQUESTS_CA_BUNDLE'] = bad_cert_fname
# Clear sessions to ensure that the certificate we just specified is used.
# TODO: Confirm necessity of this session clearing and lay out mechanics.
self.fetcher._sessions = {}
# Try connecting to the server process with the bad cert while trusting
# the bad cert. Expect failure because even though we trust it, the
# hostname we're connecting to does not match the hostname in the cert.
logger.info('Trying HTTPS download of target file: ' + bad_https_url)
with warnings.catch_warnings():
# We're ok with a slightly fishy localhost cert
warnings.filterwarnings('ignore',
category=urllib3.exceptions.SubjectAltNameWarning)
with self.assertRaises(requests.exceptions.SSLError):
download.safe_download(bad_https_url, target_data_length, self.fetcher)
with self.assertRaises(requests.exceptions.SSLError):
download.unsafe_download(bad_https_url, target_data_length, self.fetcher)
# Try connecting to the server processes with the good certs while not
# trusting the good certs (trusting the bad cert instead). Expect failure
# because even though the server's cert file is otherwise OK, we don't
# trust it.
logger.info('Trying HTTPS download of target file: ' + good_https_url)
with self.assertRaises(requests.exceptions.SSLError):
download.safe_download(good_https_url, target_data_length, self.fetcher)
with self.assertRaises(requests.exceptions.SSLError):
download.unsafe_download(good_https_url, target_data_length, self.fetcher)
logger.info('Trying HTTPS download of target file: ' + good2_https_url)
with self.assertRaises(requests.exceptions.SSLError):
download.safe_download(good2_https_url, target_data_length, self.fetcher)
with self.assertRaises(requests.exceptions.SSLError):
download.unsafe_download(good2_https_url, target_data_length, self.fetcher)
# Configure environment to now trust the certfile that is expired.
os.environ['REQUESTS_CA_BUNDLE'] = expired_cert_fname
# Clear sessions to ensure that the certificate we just specified is used.
# TODO: Confirm necessity of this session clearing and lay out mechanics.
self.fetcher._sessions = {}
# Try connecting to the server process with the expired cert while
# trusting the expired cert. Expect failure because even though we trust
# it, it is expired.
logger.info('Trying HTTPS download of target file: ' + expired_https_url)
with self.assertRaises(requests.exceptions.SSLError):
download.safe_download(expired_https_url, target_data_length, self.fetcher)
with self.assertRaises(requests.exceptions.SSLError):
download.unsafe_download(expired_https_url, target_data_length, self.fetcher)
# Try connecting to the server processes with the good certs while
# trusting the appropriate good certs. Expect success.
# TODO: expand testing to switch expected certificates back and forth a
# bit more while clearing / not clearing sessions.
os.environ['REQUESTS_CA_BUNDLE'] = good_cert_fname
# Clear sessions to ensure that the certificate we just specified is used.
# TODO: Confirm necessity of this session clearing and lay out mechanics.
self.fetcher._sessions = {}
logger.info('Trying HTTPS download of target file: ' + good_https_url)
download.safe_download(good_https_url, target_data_length, self.fetcher).close()
download.unsafe_download(good_https_url, target_data_length,self.fetcher).close()
os.environ['REQUESTS_CA_BUNDLE'] = good2_cert_fname
# Clear sessions to ensure that the certificate we just specified is used.
# TODO: Confirm necessity of this session clearing and lay out mechanics.
self.fetcher._sessions = {}
logger.info('Trying HTTPS download of target file: ' + good2_https_url)
download.safe_download(good2_https_url, target_data_length, self.fetcher).close()
download.unsafe_download(good2_https_url, target_data_length, self.fetcher).close()
finally:
for proc_handler in [
good_https_server_handler,
good2_https_server_handler,
bad_https_server_handler,
expd_https_server_handler]:
# Cleans the resources and flush the logged lines (if any).
proc_handler.clean()
# Run unit test.
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

272
tests/test_endless_data_attack_old.py
View file

@ -1,272 +0,0 @@
#!/usr/bin/env python
# Copyright 2012 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
test_endless_data_attack_old.py
<Author>
Konstantin Andrianov.
<Started>
March 13, 2012.
April 3, 2014.
Refactored to use the 'unittest' module (test conditions in code, rather
than verifying text output), use pre-generated repository files, and
discontinue use of the old repository tools. Minor edits to the test cases.
-vladimir.v.diaz
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Simulate an endless data attack, where an updater client tries to download a
target file modified by an attacker to contain a large amount of data (a TUF
client should only download up to the file's expected length). TUF and
non-TUF client scenarios are tested.
There is no difference between 'updates' and 'target' files.
"""
import os
import tempfile
import shutil
import json
import logging
import unittest
import sys
from urllib import request
import tuf
import tuf.formats
import tuf.log
import tuf.client.updater as updater
import tuf.unittest_toolbox as unittest_toolbox
import tuf.roledb
from tests import utils
import securesystemslib
logger = logging.getLogger(__name__)
class TestEndlessDataAttack(unittest_toolbox.Modified_TestCase):
@classmethod
def setUpClass(cls):
# Create a temporary directory to store the repository, metadata, and target
# files. 'temporary_directory' must be deleted in TearDownModule() so that
# temporary files are always removed, even when exceptions occur.
cls.temporary_directory = tempfile.mkdtemp(dir=os.getcwd())
# Launch a SimpleHTTPServer (serves files in the current directory).
# Test cases will request metadata and target files that have been
# pre-generated in 'tuf/tests/repository_data', which will be served by the
# SimpleHTTPServer launched here. The test cases of this unit test assume
# the pre-generated metadata files have a specific structure, such
# as a delegated role 'targets/role1', three target files, five key files,
# etc.
cls.server_process_handler = utils.TestServerProcess(log=logger)
@classmethod
def tearDownClass(cls):
# Cleans the resources and flush the logged lines (if any).
cls.server_process_handler.clean()
# Remove the temporary repository directory, which should contain all the
# metadata, targets, and key files generated of all the test cases.
shutil.rmtree(cls.temporary_directory)
def setUp(self):
# We are inheriting from custom class.
unittest_toolbox.Modified_TestCase.setUp(self)
self.repository_name = 'test_repository1'
# Copy the original repository files provided in the test folder so that
# any modifications made to repository files are restricted to the copies.
# The 'repository_data' directory is expected to exist in 'tuf/tests/'.
original_repository_files = os.path.join(os.getcwd(), 'repository_data')
temporary_repository_root = \
self.make_temp_directory(directory=self.temporary_directory)
# The original repository, keystore, and client directories will be copied
# for each test case.
original_repository = os.path.join(original_repository_files, 'repository')
original_client = os.path.join(original_repository_files, 'client')
# Save references to the often-needed client repository directories.
# Test cases need these references to access metadata and target files.
self.repository_directory = \
os.path.join(temporary_repository_root, 'repository')
self.client_directory = os.path.join(temporary_repository_root, 'client')
# Copy the original 'repository', 'client', and 'keystore' directories
# to the temporary repository the test cases can use.
shutil.copytree(original_repository, self.repository_directory)
shutil.copytree(original_client, self.client_directory)
# Set the url prefix required by the 'tuf/client/updater.py' updater.
# 'path/to/tmp/repository' -> 'localhost:8001/tmp/repository'.
repository_basepath = self.repository_directory[len(os.getcwd()):]
url_prefix = 'http://' + utils.TEST_HOST_ADDRESS + ':' \
+ str(self.server_process_handler.port) + repository_basepath
# Setting 'tuf.settings.repository_directory' with the temporary client
# directory copied from the original repository files.
tuf.settings.repositories_directory = self.client_directory
self.repository_mirrors = {'mirror1': {'url_prefix': url_prefix,
'metadata_path': 'metadata',
'targets_path': 'targets'}}
# Create the repository instance. The test cases will use this client
# updater to refresh metadata, fetch target files, etc.
self.repository_updater = updater.Updater(self.repository_name,
self.repository_mirrors)
def tearDown(self):
tuf.roledb.clear_roledb(clear_all=True)
tuf.keydb.clear_keydb(clear_all=True)
# Logs stdout and stderr from the sever subprocess.
self.server_process_handler.flush_log()
# Remove temporary directory
unittest_toolbox.Modified_TestCase.tearDown(self)
def test_without_tuf(self):
# Verify that a target file replaced with a larger malicious version (to
# simulate an endless data attack) is downloaded by a non-TUF client (i.e.,
# a non-TUF client that does not verify hashes, detect mix-and-mix attacks,
# etc.) A tuf client, on the other hand, should only download target files
# up to their expected lengths, as explicitly specified in metadata, or
# 'tuf.settings.py' (when retrieving 'timestamp.json' and 'root.json unsafely'.)
# Test: Download a valid target file from the repository.
# Ensure the target file to be downloaded has not already been downloaded,
# and generate its file size and digest. The file size and digest is needed
# to verify that the malicious file was indeed downloaded.
target_path = os.path.join(self.repository_directory, 'targets', 'file1.txt')
client_target_path = os.path.join(self.client_directory, 'file1.txt')
self.assertFalse(os.path.exists(client_target_path))
length, hashes = securesystemslib.util.get_file_details(target_path)
fileinfo = tuf.formats.make_targets_fileinfo(length, hashes)
url_prefix = self.repository_mirrors['mirror1']['url_prefix']
url_file = os.path.join(url_prefix, 'targets', 'file1.txt')
# On Windows, the URL portion should not contain backslashes.
request.urlretrieve(url_file.replace('\\', '/'), client_target_path)
self.assertTrue(os.path.exists(client_target_path))
length, hashes = securesystemslib.util.get_file_details(client_target_path)
download_fileinfo = tuf.formats.make_targets_fileinfo(length, hashes)
self.assertEqual(fileinfo, download_fileinfo)
# Test: Download a target file that has been modified by an attacker with
# extra data.
with open(target_path, 'a') as file_object:
file_object.write('append large amount of data' * 100000)
large_length, hashes = securesystemslib.util.get_file_details(target_path)
malicious_fileinfo = tuf.formats.make_targets_fileinfo(large_length, hashes)
# Is the modified file actually larger?
self.assertTrue(large_length > length)
# On Windows, the URL portion should not contain backslashes.
request.urlretrieve(url_file.replace('\\', '/'), client_target_path)
length, hashes = securesystemslib.util.get_file_details(client_target_path)
download_fileinfo = tuf.formats.make_targets_fileinfo(length, hashes)
# Verify 'download_fileinfo' is unequal to the original trusted version.
self.assertNotEqual(download_fileinfo, fileinfo)
# Verify 'download_fileinfo' is equal to the malicious version.
self.assertEqual(download_fileinfo, malicious_fileinfo)
def test_with_tuf(self):
# Verify that a target file (on the remote repository) modified by an
# attacker, to contain a large amount of extra data, is not downloaded by
# the TUF client. First test that the valid target file is successfully
# downloaded.
file1_fileinfo = self.repository_updater.get_one_valid_targetinfo('file1.txt')
destination = os.path.join(self.client_directory)
self.repository_updater.download_target(file1_fileinfo, destination)
client_target_path = os.path.join(destination, 'file1.txt')
self.assertTrue(os.path.exists(client_target_path))
# Verify the client's downloaded file matches the repository's.
target_path = os.path.join(self.repository_directory, 'targets', 'file1.txt')
length, hashes = securesystemslib.util.get_file_details(client_target_path)
fileinfo = tuf.formats.make_targets_fileinfo(length, hashes)
length, hashes = securesystemslib.util.get_file_details(client_target_path)
download_fileinfo = tuf.formats.make_targets_fileinfo(length, hashes)
self.assertEqual(fileinfo, download_fileinfo)
# Modify 'file1.txt' and confirm that the TUF client only downloads up to
# the expected file length.
with open(target_path, 'a') as file_object:
file_object.write('append large amount of data' * 10000)
# Is the modified file actually larger?
large_length, hashes = securesystemslib.util.get_file_details(target_path)
self.assertTrue(large_length > length)
os.remove(client_target_path)
self.repository_updater.download_target(file1_fileinfo, destination)
# A large amount of data has been appended to the original content. The
# extra data appended should be discarded by the client, so the downloaded
# file size and hash should not have changed.
length, hashes = securesystemslib.util.get_file_details(client_target_path)
download_fileinfo = tuf.formats.make_targets_fileinfo(length, hashes)
self.assertEqual(fileinfo, download_fileinfo)
# Test that the TUF client does not download large metadata files, as well.
timestamp_path = os.path.join(self.repository_directory, 'metadata',
'timestamp.json')
original_length, hashes = securesystemslib.util.get_file_details(timestamp_path)
with open(timestamp_path, 'r+') as file_object:
timestamp_content = securesystemslib.util.load_json_file(timestamp_path)
large_data = 'LargeTimestamp' * 10000
timestamp_content['signed']['_type'] = large_data
json.dump(timestamp_content, file_object, indent=1, sort_keys=True)
modified_length, hashes = securesystemslib.util.get_file_details(timestamp_path)
self.assertTrue(modified_length > original_length)
# Does the TUF client download the upper limit of an unsafely fetched
# 'timestamp.json'? 'timestamp.json' must not be greater than
# 'tuf.settings.DEFAULT_TIMESTAMP_REQUIRED_LENGTH'.
try:
self.repository_updater.refresh()
except tuf.exceptions.NoWorkingMirrorError as exception:
for mirror_url, mirror_error in exception.mirror_errors.items():
self.assertTrue(isinstance(mirror_error, securesystemslib.exceptions.Error))
else:
self.fail('TUF did not prevent an endless data attack.')
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

214
tests/test_extraneous_dependencies_attack_old.py
View file

@ -1,214 +0,0 @@
#!/usr/bin/env python
# Copyright 2013 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
test_extraneous_dependencies_attack_old.py
<Author>
Zane Fisher.
<Started>
August 19, 2013.
April 6, 2014.
Refactored to use the 'unittest' module (test conditions in code, rather
than verifying text output), use pre-generated repository files, and
discontinue use of the old repository tools. Modify the previous scenario
simulated for the mix-and-match attack. The metadata that specified the
dependencies of a project modified (previously a text file.)
-vladimir.v.diaz
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Simulate an extraneous dependencies attack. The client attempts to download
a file, which lists all the target dependencies, with one legitimate
dependency, and one extraneous dependency. A client should not download a
target dependency even if it is found on the repository. Valid targets are
listed and verified by TUF metadata, such as 'targets.txt'.
There is no difference between 'updates' and 'target' files.
"""
import os
import tempfile
import shutil
import json
import logging
import unittest
import sys
import tuf.formats
import tuf.log
import tuf.client.updater as updater
import tuf.roledb
import tuf.keydb
import tuf.unittest_toolbox as unittest_toolbox
from tests import utils
import securesystemslib
logger = logging.getLogger(__name__)
class TestExtraneousDependenciesAttack(unittest_toolbox.Modified_TestCase):
@classmethod
def setUpClass(cls):
# Create a temporary directory to store the repository, metadata, and target
# files. 'temporary_directory' must be deleted in TearDownModule() so that
# temporary files are always removed, even when exceptions occur.
cls.temporary_directory = tempfile.mkdtemp(dir=os.getcwd())
# Launch a SimpleHTTPServer (serves files in the current directory).
# Test cases will request metadata and target files that have been
# pre-generated in 'tuf/tests/repository_data', which will be served by the
# SimpleHTTPServer launched here. The test cases of this unit test assume
# the pre-generated metadata files have a specific structure, such
# as a delegated role 'targets/role1', three target files, five key files,
# etc.
cls.server_process_handler = utils.TestServerProcess(log=logger)
@classmethod
def tearDownClass(cls):
# Cleans the resources and flush the logged lines (if any).
cls.server_process_handler.clean()
# Remove the temporary repository directory, which should contain all the
# metadata, targets, and key files generated of all the test cases.
shutil.rmtree(cls.temporary_directory)
def setUp(self):
# We are inheriting from custom class.
unittest_toolbox.Modified_TestCase.setUp(self)
self.repository_name = 'test_repository1'
# Copy the original repository files provided in the test folder so that
# any modifications made to repository files are restricted to the copies.
# The 'repository_data' directory is expected to exist in 'tuf/tests/'.
original_repository_files = os.path.join(os.getcwd(), 'repository_data')
temporary_repository_root = \
self.make_temp_directory(directory=self.temporary_directory)
# The original repository, keystore, and client directories will be copied
# for each test case.
original_repository = os.path.join(original_repository_files, 'repository')
original_client = os.path.join(original_repository_files, 'client')
original_keystore = os.path.join(original_repository_files, 'keystore')
# Save references to the often-needed client repository directories.
# Test cases need these references to access metadata and target files.
self.repository_directory = \
os.path.join(temporary_repository_root, 'repository')
self.client_directory = os.path.join(temporary_repository_root, 'client')
self.keystore_directory = os.path.join(temporary_repository_root, 'keystore')
# Copy the original 'repository', 'client', and 'keystore' directories
# to the temporary repository the test cases can use.
shutil.copytree(original_repository, self.repository_directory)
shutil.copytree(original_client, self.client_directory)
shutil.copytree(original_keystore, self.keystore_directory)
# Set the url prefix required by the 'tuf/client/updater.py' updater.
# 'path/to/tmp/repository' -> 'localhost:8001/tmp/repository'.
repository_basepath = self.repository_directory[len(os.getcwd()):]
url_prefix = 'http://' + utils.TEST_HOST_ADDRESS + ':' \
+ str(self.server_process_handler.port) + repository_basepath
# Setting 'tuf.settings.repository_directory' with the temporary client
# directory copied from the original repository files.
tuf.settings.repositories_directory = self.client_directory
self.repository_mirrors = {'mirror1': {'url_prefix': url_prefix,
'metadata_path': 'metadata',
'targets_path': 'targets'}}
# Create the repository instance. The test cases will use this client
# updater to refresh metadata, fetch target files, etc.
self.repository_updater = updater.Updater(self.repository_name,
self.repository_mirrors)
def tearDown(self):
tuf.roledb.clear_roledb(clear_all=True)
tuf.keydb.clear_keydb(clear_all=True)
# Logs stdout and stderr from the sever subprocess.
self.server_process_handler.flush_log()
# Remove temporary directory
unittest_toolbox.Modified_TestCase.tearDown(self)
def test_with_tuf(self):
# An attacker tries to trick a client into installing an extraneous target
# file (a valid file on the repository, in this case) by listing it in the
# project's metadata file. For the purposes of test_with_tuf(),
# 'role1.json' is treated as the metadata file that indicates all
# the files needed to install/update the 'role1' project. The attacker
# simply adds the extraneous target file to 'role1.json', which the TUF
# client should reject as improperly signed.
role1_filepath = os.path.join(self.repository_directory, 'metadata',
'role1.json')
file1_filepath = os.path.join(self.repository_directory, 'targets',
'file1.txt')
length, hashes = securesystemslib.util.get_file_details(file1_filepath)
role1_metadata = securesystemslib.util.load_json_file(role1_filepath)
role1_metadata['signed']['targets']['/file2.txt'] = {}
role1_metadata['signed']['targets']['/file2.txt']['hashes'] = hashes
role1_metadata['signed']['targets']['/file2.txt']['length'] = length
tuf.formats.check_signable_object_format(role1_metadata)
with open(role1_filepath, 'wt') as file_object:
json.dump(role1_metadata, file_object, indent=1, sort_keys=True)
# Un-install the metadata of the top-level roles so that the client can
# download and detect the invalid 'role1.json'.
os.remove(os.path.join(self.client_directory, self.repository_name,
'metadata', 'current', 'snapshot.json'))
os.remove(os.path.join(self.client_directory, self.repository_name,
'metadata', 'current', 'targets.json'))
os.remove(os.path.join(self.client_directory, self.repository_name,
'metadata', 'current', 'timestamp.json'))
os.remove(os.path.join(self.client_directory, self.repository_name,
'metadata', 'current', 'role1.json'))
# Verify that the TUF client rejects the invalid metadata and refuses to
# continue the update process.
self.repository_updater.refresh()
try:
with utils.ignore_deprecation_warnings('tuf.client.updater'):
self.repository_updater.targets_of_role('role1')
# Verify that the specific 'tuf.exceptions.ForbiddenTargetError' exception is raised
# by each mirror.
except tuf.exceptions.NoWorkingMirrorError as exception:
for mirror_url, mirror_error in exception.mirror_errors.items():
url_prefix = self.repository_mirrors['mirror1']['url_prefix']
url_file = os.path.join(url_prefix, 'metadata', 'role1.json')
# Verify that 'role1.json' is the culprit.
self.assertEqual(url_file.replace('\\', '/'), mirror_url)
self.assertTrue(isinstance(mirror_error, securesystemslib.exceptions.BadSignatureError))
else:
self.fail('TUF did not prevent an extraneous dependencies attack.')
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

134
tests/test_fetcher_old.py
View file

@ -1,134 +0,0 @@
#!/usr/bin/env python
# Copyright 2021, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""Unit test for RequestsFetcher.
"""
import io
import logging
import math
import os
import sys
import tempfile
import unittest
import tuf
import tuf.exceptions
import tuf.requests_fetcher
from tests import utils
from tuf import unittest_toolbox
logger = logging.getLogger(__name__)
class TestFetcher(unittest_toolbox.Modified_TestCase):
"""Unit tests for RequestFetcher."""
def setUp(self):
"""
Create a temporary file and launch a simple server in the
current working directory.
"""
unittest_toolbox.Modified_TestCase.setUp(self)
# Making a temporary file.
current_dir = os.getcwd()
target_filepath = self.make_temp_data_file(directory=current_dir)
with open(target_filepath, "r", encoding="utf8") as target_fileobj:
self.file_contents = target_fileobj.read()
self.file_length = len(self.file_contents)
# Launch a SimpleHTTPServer (serves files in the current dir).
self.server_process_handler = utils.TestServerProcess(log=logger)
rel_target_filepath = os.path.basename(target_filepath)
self.url = (
"http://"
+ utils.TEST_HOST_ADDRESS
+ ":"
+ str(self.server_process_handler.port)
+ "/"
+ rel_target_filepath
)
# Create a temporary file where the target file chunks are written
# during fetching
# pylint: disable-next=consider-using-with
self.temp_file = tempfile.TemporaryFile()
self.fetcher = tuf.requests_fetcher.RequestsFetcher()
# Stop server process and perform clean up.
def tearDown(self):
# Cleans the resources and flush the logged lines (if any).
self.server_process_handler.clean()
self.temp_file.close()
# Remove temporary directory
unittest_toolbox.Modified_TestCase.tearDown(self)
# Test: Normal case.
def test_fetch(self):
for chunk in self.fetcher.fetch(self.url, self.file_length):
self.temp_file.write(chunk)
self.temp_file.seek(0)
temp_file_data = self.temp_file.read().decode("utf-8")
self.assertEqual(self.file_contents, temp_file_data)
# Test if fetcher downloads file up to a required length
def test_fetch_restricted_length(self):
for chunk in self.fetcher.fetch(self.url, self.file_length - 4):
self.temp_file.write(chunk)
self.temp_file.seek(0, io.SEEK_END)
self.assertEqual(self.temp_file.tell(), self.file_length - 4)
# Test that fetcher does not download more than actual file length
def test_fetch_upper_length(self):
for chunk in self.fetcher.fetch(self.url, self.file_length + 4):
self.temp_file.write(chunk)
self.temp_file.seek(0, io.SEEK_END)
self.assertEqual(self.temp_file.tell(), self.file_length)
# Test incorrect URL parsing
def test_url_parsing(self):
with self.assertRaises(tuf.exceptions.URLParsingError):
self.fetcher.fetch(self.random_string(), self.file_length)
# Test: Normal case with url data downloaded in more than one chunk
def test_fetch_in_chunks(self):
# Set smaller chunk size to ensure that the file will be downloaded
# in more than one chunk
default_chunk_size = tuf.settings.CHUNK_SIZE
tuf.settings.CHUNK_SIZE = 4
# expected_chunks_count: 3
expected_chunks_count = math.ceil(
self.file_length / tuf.settings.CHUNK_SIZE
)
self.assertEqual(expected_chunks_count, 3)
chunks_count = 0
for chunk in self.fetcher.fetch(self.url, self.file_length):
self.temp_file.write(chunk)
chunks_count += 1
self.temp_file.seek(0)
temp_file_data = self.temp_file.read().decode("utf-8")
self.assertEqual(self.file_contents, temp_file_data)
# Check that we calculate chunks as expected
self.assertEqual(chunks_count, expected_chunks_count)
# Restore default settings
tuf.settings.CHUNK_SIZE = default_chunk_size
# Run unit test.
if __name__ == "__main__":
utils.configure_test_logging(sys.argv)
unittest.main()

971
tests/test_formats_old.py
View file

@ -1,971 +0,0 @@
#!/usr/bin/env python
# Copyright 2012 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
test_formats_old.py
<Author>
Vladimir Diaz <vladimir.v.diaz@gmail.com>
<Started>
October 2012.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Unit test for 'formats.py'
"""
import unittest
import datetime
import sys
import os
import tuf
import tuf.formats
from tests import utils
import securesystemslib
import securesystemslib.util
class TestFormats(unittest.TestCase):
def setUp(self):
pass
def tearDown(self):
pass
def test_schemas(self):
# Test conditions for valid schemas.
valid_schemas = {
'ISO8601_DATETIME_SCHEMA': (securesystemslib.formats.ISO8601_DATETIME_SCHEMA,
'1985-10-21T13:20:00Z'),
'UNIX_TIMESTAMP_SCHEMA': (securesystemslib.formats.UNIX_TIMESTAMP_SCHEMA, 499137720),
'HASH_SCHEMA': (securesystemslib.formats.HASH_SCHEMA, 'A4582BCF323BCEF'),
'HASHDICT_SCHEMA': (securesystemslib.formats.HASHDICT_SCHEMA,
{'sha256': 'A4582BCF323BCEF'}),
'HEX_SCHEMA': (securesystemslib.formats.HEX_SCHEMA, 'A4582BCF323BCEF'),
'KEYID_SCHEMA': (securesystemslib.formats.KEYID_SCHEMA, '123456789abcdef'),
'KEYIDS_SCHEMA': (securesystemslib.formats.KEYIDS_SCHEMA,
['123456789abcdef', '123456789abcdef']),
'SCHEME_SCHEMA': (securesystemslib.formats.SCHEME_SCHEMA, 'rsassa-pss-sha256'),
'RELPATH_SCHEMA': (tuf.formats.RELPATH_SCHEMA, 'metadata/root/'),
'RELPATHS_SCHEMA': (tuf.formats.RELPATHS_SCHEMA,
['targets/role1/', 'targets/role2/']),
'PATH_SCHEMA': (securesystemslib.formats.PATH_SCHEMA, '/home/someuser/'),
'PATHS_SCHEMA': (securesystemslib.formats.PATHS_SCHEMA,
['/home/McFly/', '/home/Tannen/']),
'URL_SCHEMA': (securesystemslib.formats.URL_SCHEMA,
'https://www.updateframework.com/'),
'VERSION_SCHEMA': (tuf.formats.VERSION_SCHEMA,
{'major': 1, 'minor': 0, 'fix': 8}),
'LENGTH_SCHEMA': (tuf.formats.LENGTH_SCHEMA, 8),
'NAME_SCHEMA': (securesystemslib.formats.NAME_SCHEMA, 'Marty McFly'),
'BOOLEAN_SCHEMA': (securesystemslib.formats.BOOLEAN_SCHEMA, True),
'THRESHOLD_SCHEMA': (tuf.formats.THRESHOLD_SCHEMA, 1),
'ROLENAME_SCHEMA': (tuf.formats.ROLENAME_SCHEMA, 'Root'),
'RSAKEYBITS_SCHEMA': (securesystemslib.formats.RSAKEYBITS_SCHEMA, 4096),
'PASSWORD_SCHEMA': (securesystemslib.formats.PASSWORD_SCHEMA, 'secret'),
'PASSWORDS_SCHEMA': (securesystemslib.formats.PASSWORDS_SCHEMA, ['pass1', 'pass2']),
'KEYVAL_SCHEMA': (securesystemslib.formats.KEYVAL_SCHEMA,
{'public': 'pubkey', 'private': 'privkey'}),
'KEY_SCHEMA': (securesystemslib.formats.KEY_SCHEMA,
{'keytype': 'rsa',
'scheme': 'rsassa-pss-sha256',
'keyval': {'public': 'pubkey',
'private': 'privkey'}}),
'RSAKEY_SCHEMA': (securesystemslib.formats.RSAKEY_SCHEMA,
{'keytype': 'rsa',
'scheme': 'rsassa-pss-sha256',
'keyid': '123456789abcdef',
'keyval': {'public': 'pubkey',
'private': 'privkey'}}),
'TARGETS_FILEINFO_SCHEMA': (tuf.formats.TARGETS_FILEINFO_SCHEMA,
{'length': 1024,
'hashes': {'sha256': 'A4582BCF323BCEF'},
'custom': {'type': 'paintjob'}}),
'METADATA_FILEINFO_SCHEMA': (tuf.formats.METADATA_FILEINFO_SCHEMA,
{'length': 1024,
'hashes': {'sha256': 'A4582BCF323BCEF'},
'version': 1}),
'FILEDICT_SCHEMA': (tuf.formats.FILEDICT_SCHEMA,
{'metadata/root.json': {'length': 1024,
'hashes': {'sha256': 'ABCD123'},
'custom': {'type': 'metadata'}}}),
'TARGETINFO_SCHEMA': (tuf.formats.TARGETINFO_SCHEMA,
{'filepath': 'targets/target1.gif',
'fileinfo': {'length': 1024,
'hashes': {'sha256': 'ABCD123'},
'custom': {'type': 'target'}}}),
'TARGETINFOS_SCHEMA': (tuf.formats.TARGETINFOS_SCHEMA,
[{'filepath': 'targets/target1.gif',
'fileinfo': {'length': 1024,
'hashes': {'sha256': 'ABCD123'},
'custom': {'type': 'target'}}}]),
'SIGNATURE_SCHEMA': (securesystemslib.formats.SIGNATURE_SCHEMA,
{'keyid': '123abc',
'sig': 'A4582BCF323BCEF'}),
'SIGNATURESTATUS_SCHEMA': (tuf.formats.SIGNATURESTATUS_SCHEMA,
{'threshold': 1,
'good_sigs': ['123abc'],
'bad_sigs': ['123abc'],
'unknown_sigs': ['123abc'],
'untrusted_sigs': ['123abc'],
'unknown_signing_schemes': ['123abc']}),
'SIGNABLE_SCHEMA': (tuf.formats.SIGNABLE_SCHEMA,
{'signed': 'signer',
'signatures': [{'keyid': '123abc',
'sig': 'A4582BCF323BCEF'}]}),
'KEYDICT_SCHEMA': (securesystemslib.formats.KEYDICT_SCHEMA,
{'123abc': {'keytype': 'rsa',
'scheme': 'rsassa-pss-sha256',
'keyval': {'public': 'pubkey',
'private': 'privkey'}}}),
'KEYDB_SCHEMA': (tuf.formats.KEYDB_SCHEMA,
{'123abc': {'keytype': 'rsa',
'scheme': 'rsassa-pss-sha256',
'keyid': '123456789abcdef',
'keyval': {'public': 'pubkey',
'private': 'privkey'}}}),
'SCPCONFIG_SCHEMA': (tuf.formats.SCPCONFIG_SCHEMA,
{'general': {'transfer_module': 'scp',
'metadata_path': '/path/meta.json',
'targets_directory': '/targets'},
'scp': {'host': 'http://localhost:8001',
'user': 'McFly',
'identity_file': '/home/.ssh/file',
'remote_directory': '/home/McFly'}}),
'RECEIVECONFIG_SCHEMA': (tuf.formats.RECEIVECONFIG_SCHEMA,
{'general': {'transfer_module': 'scp',
'pushroots': ['/pushes'],
'repository_directory': '/repo',
'metadata_directory': '/repo/meta',
'targets_directory': '/repo/targets',
'backup_directory': '/repo/backup'}}),
'ROLE_SCHEMA': (tuf.formats.ROLE_SCHEMA,
{'keyids': ['123abc'],
'threshold': 1,
'paths': ['path1/', 'path2']}),
'ROLEDICT_SCHEMA': (tuf.formats.ROLEDICT_SCHEMA,
{'root': {'keyids': ['123abc'],
'threshold': 1,
'paths': ['path1/', 'path2']}}),
'ROOT_SCHEMA': (tuf.formats.ROOT_SCHEMA,
{'_type': 'root',
'spec_version': '1.0.0',
'version': 8,
'consistent_snapshot': False,
'expires': '1985-10-21T13:20:00Z',
'keys': {'123abc': {'keytype': 'rsa',
'scheme': 'rsassa-pss-sha256',
'keyval': {'public': 'pubkey',
'private': 'privkey'}}},
'roles': {'root': {'keyids': ['123abc'],
'threshold': 1,
'paths': ['path1/', 'path2']}}}),
'TARGETS_SCHEMA': (tuf.formats.TARGETS_SCHEMA,
{'_type': 'targets',
'spec_version': '1.0.0',
'version': 8,
'expires': '1985-10-21T13:20:00Z',
'targets': {'metadata/targets.json': {'length': 1024,
'hashes': {'sha256': 'ABCD123'},
'custom': {'type': 'metadata'}}},
'delegations': {'keys': {'123abc': {'keytype':'rsa',
'scheme': 'rsassa-pss-sha256',
'keyval': {'public': 'pubkey',
'private': 'privkey'}}},
'roles': [{'name': 'root', 'keyids': ['123abc'],
'threshold': 1,
'paths': ['path1/', 'path2']}]}}),
'SNAPSHOT_SCHEMA': (tuf.formats.SNAPSHOT_SCHEMA,
{'_type': 'snapshot',
'spec_version': '1.0.0',
'version': 8,
'expires': '1985-10-21T13:20:00Z',
'meta': {'snapshot.json': {'version': 1024}}}),
'TIMESTAMP_SCHEMA': (tuf.formats.TIMESTAMP_SCHEMA,
{'_type': 'timestamp',
'spec_version': '1.0.0',
'version': 8,
'expires': '1985-10-21T13:20:00Z',
'meta': {'metadattimestamp.json': {'length': 1024,
'hashes': {'sha256': 'AB1245'},
'version': 1}}}),
'MIRROR_SCHEMA': (tuf.formats.MIRROR_SCHEMA,
{'url_prefix': 'http://localhost:8001',
'metadata_path': 'metadata/',
'targets_path': 'targets/',
'confined_target_dirs': ['path1/', 'path2/'],
'custom': {'type': 'mirror'}}),
'MIRROR_SCHEMA_NO_CONFINED_TARGETS': (tuf.formats.MIRROR_SCHEMA,
{'url_prefix': 'http://localhost:8001',
'metadata_path': 'metadata/',
'targets_path': 'targets/',
'custom': {'type': 'mirror'}}),
'MIRRORDICT_SCHEMA': (tuf.formats.MIRRORDICT_SCHEMA,
{'mirror1': {'url_prefix': 'http://localhost:8001',
'metadata_path': 'metadata/',
'targets_path': 'targets/',
'confined_target_dirs': ['path1/', 'path2/'],
'custom': {'type': 'mirror'}}}),
'MIRRORLIST_SCHEMA': (tuf.formats.MIRRORLIST_SCHEMA,
{'_type': 'mirrors',
'version': 8,
'spec_version': '1.0.0',
'expires': '1985-10-21T13:20:00Z',
'mirrors': [{'url_prefix': 'http://localhost:8001',
'metadata_path': 'metadata/',
'targets_path': 'targets/',
'confined_target_dirs': ['path1/', 'path2/'],
'custom': {'type': 'mirror'}}]})}
# Iterate 'valid_schemas', ensuring each 'valid_schema' correctly matches
# its respective 'schema_type'.
for schema_name, (schema_type, valid_schema) in valid_schemas.items():
if not schema_type.matches(valid_schema):
print('bad schema: ' + repr(valid_schema))
self.assertEqual(True, schema_type.matches(valid_schema))
# Test conditions for invalid schemas.
# Set the 'valid_schema' of 'valid_schemas' to an invalid
# value and test that it does not match 'schema_type'.
for schema_name, (schema_type, valid_schema) in valid_schemas.items():
invalid_schema = 0xBAD
if isinstance(schema_type, securesystemslib.schema.Integer):
invalid_schema = 'BAD'
self.assertEqual(False, schema_type.matches(invalid_schema))
def test_specfication_version_schema(self):
"""Test valid and invalid SPECIFICATION_VERSION_SCHEMAs, using examples
from 'regex101.com/r/Ly7O1x/3/', referenced by
'semver.org/spec/v2.0.0.html'. """
valid_schemas = [
"0.0.4",
"1.2.3",
"10.20.30",
"1.1.2-prerelease+meta",
"1.1.2+meta",
"1.1.2+meta-valid",
"1.0.0-alpha",
"1.0.0-beta",
"1.0.0-alpha.beta",
"1.0.0-alpha.beta.1",
"1.0.0-alpha.1",
"1.0.0-alpha0.valid",
"1.0.0-alpha.0valid",
"1.0.0-alpha-a.b-c-somethinglong+build.1-aef.1-its-okay",
"1.0.0-rc.1+build.1",
"2.0.0-rc.1+build.123",
"1.2.3-beta",
"10.2.3-DEV-SNAPSHOT",
"1.2.3-SNAPSHOT-123",
"1.0.0",
"2.0.0",
"1.1.7",
"2.0.0+build.1848",
"2.0.1-alpha.1227",
"1.0.0-alpha+beta",
"1.2.3----RC-SNAPSHOT.12.9.1--.12+788",
"1.2.3----R-S.12.9.1--.12+meta",
"1.2.3----RC-SNAPSHOT.12.9.1--.12",
"1.0.0+0.build.1-rc.10000aaa-kk-0.1",
"99999999999999999999999.999999999999999999.99999999999999999",
"1.0.0-0A.is.legal"]
for valid_schema in valid_schemas:
self.assertTrue(
tuf.formats.SPECIFICATION_VERSION_SCHEMA.matches(valid_schema),
"'{}' should match 'SPECIFICATION_VERSION_SCHEMA'.".format(
valid_schema))
invalid_schemas = [
"1",
"1.2",
"1.2.3-0123",
"1.2.3-0123.0123",
"1.1.2+.123",
"+invalid",
"-invalid",
"-invalid+invalid",
"-invalid.01",
"alpha",
"alpha.beta",
"alpha.beta.1",
"alpha.1",
"alpha+beta",
"alpha_beta",
"alpha.",
"alpha..",
"beta",
"1.0.0-alpha_beta",
"-alpha.",
"1.0.0-alpha..",
"1.0.0-alpha..1",
"1.0.0-alpha...1",
"1.0.0-alpha....1",
"1.0.0-alpha.....1",
"1.0.0-alpha......1",
"1.0.0-alpha.......1",
"01.1.1",
"1.01.1",
"1.1.01",
"1.2",
"1.2.3.DEV",
"1.2-SNAPSHOT",
"1.2.31.2.3----RC-SNAPSHOT.12.09.1--..12+788",
"1.2-RC-SNAPSHOT",
"-1.0.3-gamma+b7718",
"+justmeta",
"9.8.7+meta+meta",
"9.8.7-whatever+meta+meta",
"99999999999999999999999.999999999999999999.99999999999999999----RC-SNAPSHOT.12.09.1--------------------------------..12"]
for invalid_schema in invalid_schemas:
self.assertFalse(
tuf.formats.SPECIFICATION_VERSION_SCHEMA.matches(invalid_schema),
"'{}' should not match 'SPECIFICATION_VERSION_SCHEMA'.".format(
invalid_schema))
def test_build_dict_conforming_to_schema(self):
# Test construction of a few metadata formats using
# build_dict_conforming_to_schema().
# Try the wrong type of schema object.
STRING_SCHEMA = securesystemslib.schema.AnyString()
with self.assertRaises(ValueError):
tuf.formats.build_dict_conforming_to_schema(
STRING_SCHEMA, string='some string')
# Try building Timestamp metadata.
spec_version = tuf.SPECIFICATION_VERSION
version = 8
length = 88
hashes = {'sha256': '3c7fe3eeded4a34'}
expires = '1985-10-21T13:20:00Z'
filedict = {'snapshot.json': {'length': length, 'hashes': hashes, 'version': 1}}
# Try with and without _type and spec_version, both of which are
# automatically populated if they are not included.
self.assertTrue(tuf.formats.TIMESTAMP_SCHEMA.matches( # both
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.TIMESTAMP_SCHEMA,
_type='timestamp',
spec_version=spec_version,
version=version,
expires=expires,
meta=filedict)))
self.assertTrue(tuf.formats.TIMESTAMP_SCHEMA.matches( # neither
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.TIMESTAMP_SCHEMA,
version=version,
expires=expires,
meta=filedict)))
self.assertTrue(tuf.formats.TIMESTAMP_SCHEMA.matches( # one
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.TIMESTAMP_SCHEMA,
spec_version=spec_version,
version=version,
expires=expires,
meta=filedict)))
self.assertTrue(tuf.formats.TIMESTAMP_SCHEMA.matches( # the other
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.TIMESTAMP_SCHEMA,
_type='timestamp',
version=version,
expires=expires,
meta=filedict)))
# Try test arguments for invalid Timestamp creation.
bad_spec_version = 123
bad_version = 'eight'
bad_expires = '2000'
bad_filedict = 123
with self.assertRaises(securesystemslib.exceptions.FormatError):
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.TIMESTAMP_SCHEMA,
_type='timestamp',
spec_version=bad_spec_version,
version=version,
expires=expires,
meta=filedict)
with self.assertRaises(securesystemslib.exceptions.FormatError):
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.TIMESTAMP_SCHEMA,
_type='timestamp',
spec_version=spec_version,
version=bad_version,
expires=expires,
meta=filedict)
with self.assertRaises(securesystemslib.exceptions.FormatError):
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.TIMESTAMP_SCHEMA,
_type='timestamp',
spec_version=spec_version,
version=version,
expires=bad_expires,
meta=filedict)
with self.assertRaises(securesystemslib.exceptions.FormatError):
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.TIMESTAMP_SCHEMA,
_type='timestamp',
spec_version=spec_version,
version=version,
expires=expires,
meta=bad_filedict)
with self.assertRaises(ValueError):
tuf.formats.build_dict_conforming_to_schema(123)
# Try building Root metadata.
consistent_snapshot = False
keydict = {'123abc': {'keytype': 'rsa',
'scheme': 'rsassa-pss-sha256',
'keyval': {'public': 'pubkey',
'private': 'privkey'}}}
roledict = {'root': {'keyids': ['123abc'],
'threshold': 1,
'paths': ['path1/', 'path2']}}
self.assertTrue(tuf.formats.ROOT_SCHEMA.matches(
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROOT_SCHEMA,
_type='root',
spec_version=spec_version,
version=version,
expires=expires,
keys=keydict,
roles=roledict,
consistent_snapshot=consistent_snapshot)))
# Additional test arguments for invalid Root creation.
bad_keydict = 123
bad_roledict = 123
# TODO: Later on, write a test looper that takes pairs of key-value args
# to substitute in on each run to shorten this.... There's a lot of
# test code that looks like this, and it'd be easier to use a looper.
with self.assertRaises(securesystemslib.exceptions.FormatError):
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROOT_SCHEMA,
_type='root',
spec_version=bad_spec_version,
version=version,
expires=expires,
keys=keydict,
roles=roledict,
consistent_snapshot=consistent_snapshot)
with self.assertRaises(securesystemslib.exceptions.FormatError):
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROOT_SCHEMA,
_type='root',
spec_version=spec_version,
version=bad_version,
expires=expires,
keys=keydict,
roles=roledict,
consistent_snapshot=consistent_snapshot)
with self.assertRaises(securesystemslib.exceptions.FormatError):
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROOT_SCHEMA,
_type='root',
spec_version=spec_version,
version=version,
expires=bad_expires,
keys=keydict,
roles=roledict,
consistent_snapshot=consistent_snapshot)
with self.assertRaises(securesystemslib.exceptions.FormatError):
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROOT_SCHEMA,
_type='root',
spec_version=spec_version,
version=version,
expires=expires,
keys=bad_keydict,
roles=roledict,
consistent_snapshot=consistent_snapshot)
with self.assertRaises(securesystemslib.exceptions.FormatError):
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROOT_SCHEMA,
_type='root',
spec_version=spec_version,
version=version,
expires=expires,
keys=keydict,
roles=bad_roledict,
consistent_snapshot=consistent_snapshot)
with self.assertRaises(TypeError):
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROOT_SCHEMA, 'bad')
with self.assertRaises(ValueError):
tuf.formats.build_dict_conforming_to_schema(
'bad',
_type='root',
spec_version=spec_version,
version=version,
expires=expires,
keys=keydict,
roles=roledict,
consistent_snapshot=consistent_snapshot)
# Try building Snapshot metadata.
versiondict = {'targets.json' : {'version': version}}
self.assertTrue(tuf.formats.SNAPSHOT_SCHEMA.matches(
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.SNAPSHOT_SCHEMA,
_type='snapshot',
spec_version=spec_version,
version=version,
expires=expires,
meta=versiondict)))
# Additional test arguments for invalid Snapshot creation.
bad_versiondict = 123
with self.assertRaises(securesystemslib.exceptions.FormatError):
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.SNAPSHOT_SCHEMA,
_type='snapshot',
spec_version=bad_spec_version,
version=version,
expires=expires,
meta=versiondict)
with self.assertRaises(securesystemslib.exceptions.FormatError):
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.SNAPSHOT_SCHEMA,
_type='snapshot',
spec_version=spec_version,
version=bad_version,
expires=expires,
meta=versiondict)
with self.assertRaises(securesystemslib.exceptions.FormatError):
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.SNAPSHOT_SCHEMA,
_type='snapshot',
spec_version=spec_version,
version=version,
expires=bad_expires,
meta=versiondict)
with self.assertRaises(securesystemslib.exceptions.FormatError):
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.SNAPSHOT_SCHEMA,
_type='snapshot',
spec_version=spec_version,
version=version,
expires=expires,
meta=bad_versiondict)
# Try building Targets metadata.
filedict = {'metadata/targets.json': {'length': 1024,
'hashes': {'sha256': 'ABCD123'},
'custom': {'type': 'metadata'}}}
delegations = {'keys': {'123abc': {'keytype':'rsa',
'scheme': 'rsassa-pss-sha256',
'keyval': {'public': 'pubkey',
'private': 'privkey'}}},
'roles': [{'name': 'root', 'keyids': ['123abc'],
'threshold': 1, 'paths': ['path1/', 'path2']}]}
self.assertTrue(tuf.formats.TARGETS_SCHEMA.matches(
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.TARGETS_SCHEMA,
_type='targets',
spec_version=spec_version,
version=version,
expires=expires,
targets=filedict,
delegations=delegations)))
# Try with no delegations included (should work, since they're optional).
self.assertTrue(tuf.formats.TARGETS_SCHEMA.matches(
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.TARGETS_SCHEMA,
_type='targets',
spec_version=spec_version,
version=version,
expires=expires,
targets=filedict)))
# Additional test arguments for invalid Targets creation.
bad_filedict = 123
bad_delegations = 123
with self.assertRaises(securesystemslib.exceptions.FormatError):
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.TARGETS_SCHEMA,
_type='targets',
spec_version=spec_version,
version=bad_version,
expires=expires,
targets=filedict,
delegations=delegations)
with self.assertRaises(securesystemslib.exceptions.FormatError):
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.TARGETS_SCHEMA,
_type='targets',
spec_version=spec_version,
version=version,
expires=bad_expires,
targets=filedict,
delegations=delegations)
with self.assertRaises(securesystemslib.exceptions.FormatError):
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.TARGETS_SCHEMA,
_type='targets',
spec_version=spec_version,
version=version,
expires=expires,
targets=bad_filedict,
delegations=delegations)
with self.assertRaises(securesystemslib.exceptions.FormatError):
tuf.formats.build_dict_conforming_to_schema(
tuf.formats.TARGETS_SCHEMA,
_type='targets',
spec_version=spec_version,
version=version,
expires=expires,
targets=filedict,
delegations=bad_delegations)
def test_expiry_string_to_datetime(self):
dt = tuf.formats.expiry_string_to_datetime('1985-10-21T13:20:00Z')
self.assertEqual(dt, datetime.datetime(1985, 10, 21, 13, 20, 0))
dt = tuf.formats.expiry_string_to_datetime('2038-01-19T03:14:08Z')
self.assertEqual(dt, datetime.datetime(2038, 1, 19, 3, 14, 8))
# First 3 fail via securesystemslib schema, last one because of strptime()
invalid_inputs = [
'2038-1-19T03:14:08Z', # leading zeros not optional
'2038-01-19T031408Z', # strict time parsing
'2038-01-19T03:14:08Z-06:00', # timezone not allowed
'2038-13-19T03:14:08Z', # too many months
]
for invalid_input in invalid_inputs:
with self.assertRaises(securesystemslib.exceptions.FormatError):
tuf.formats.expiry_string_to_datetime(invalid_input)
def test_unix_timestamp_to_datetime(self):
# Test conditions for valid arguments.
UNIX_TIMESTAMP_SCHEMA = securesystemslib.formats.UNIX_TIMESTAMP_SCHEMA
self.assertTrue(datetime.datetime, tuf.formats.unix_timestamp_to_datetime(499137720))
datetime_object = datetime.datetime(1985, 10, 26, 1, 22)
self.assertEqual(datetime_object, tuf.formats.unix_timestamp_to_datetime(499137720))
# Test conditions for invalid arguments.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.formats.unix_timestamp_to_datetime, 'bad')
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.formats.unix_timestamp_to_datetime, 1000000000000000000000)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.formats.unix_timestamp_to_datetime, -1)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.formats.unix_timestamp_to_datetime, ['5'])
def test_datetime_to_unix_timestamp(self):
# Test conditions for valid arguments.
datetime_object = datetime.datetime(2015, 10, 21, 19, 28)
self.assertEqual(1445455680, tuf.formats.datetime_to_unix_timestamp(datetime_object))
# Test conditions for invalid arguments.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.formats.datetime_to_unix_timestamp, 'bad')
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.formats.datetime_to_unix_timestamp, 1000000000000000000000)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.formats.datetime_to_unix_timestamp, ['1'])
def test_format_base64(self):
# Test conditions for valid arguments.
data = 'updateframework'.encode('utf-8')
self.assertEqual('dXBkYXRlZnJhbWV3b3Jr', tuf.formats.format_base64(data))
self.assertTrue(isinstance(tuf.formats.format_base64(data), str))
# Test conditions for invalid arguments.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.formats.format_base64, 123)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.formats.format_base64, True)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.formats.format_base64, ['123'])
def test_parse_base64(self):
# Test conditions for valid arguments.
base64 = 'dXBkYXRlZnJhbWV3b3Jr'
self.assertEqual(b'updateframework', tuf.formats.parse_base64(base64))
self.assertTrue(isinstance(tuf.formats.parse_base64(base64), bytes))
# Test conditions for invalid arguments.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.formats.parse_base64, 123)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.formats.parse_base64, True)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.formats.parse_base64, ['123'])
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.formats.parse_base64, '/')
def test_make_signable(self):
# Test conditions for expected make_signable() behavior.
SIGNABLE_SCHEMA = tuf.formats.SIGNABLE_SCHEMA
root_file = os.path.join('repository_data', 'repository', 'metadata',
'root.json')
root = securesystemslib.util.load_json_file(root_file)
self.assertTrue(SIGNABLE_SCHEMA.matches(tuf.formats.make_signable(root)))
signable = tuf.formats.make_signable(root)
self.assertEqual('root', tuf.formats.check_signable_object_format(signable))
self.assertEqual(signable, tuf.formats.make_signable(signable))
# Test conditions for miscellaneous arguments.
self.assertTrue(SIGNABLE_SCHEMA.matches(tuf.formats.make_signable('123')))
self.assertTrue(SIGNABLE_SCHEMA.matches(tuf.formats.make_signable(123)))
def test_make_targets_fileinfo(self):
# Test conditions for valid arguments.
length = 1024
hashes = {'sha256': 'A4582BCF323BCEF', 'sha512': 'A4582BCF323BFEF'}
custom = {'type': 'paintjob'}
TARGETS_FILEINFO_SCHEMA = tuf.formats.TARGETS_FILEINFO_SCHEMA
make_targets_fileinfo = tuf.formats.make_targets_fileinfo
self.assertTrue(TARGETS_FILEINFO_SCHEMA.matches(make_targets_fileinfo(length, hashes, custom)))
self.assertTrue(TARGETS_FILEINFO_SCHEMA.matches(make_targets_fileinfo(length, hashes)))
# Test conditions for invalid arguments.
bad_length = 'bad'
bad_hashes = 'bad'
bad_custom = 'bad'
self.assertRaises(securesystemslib.exceptions.FormatError, make_targets_fileinfo,
bad_length, hashes, custom)
self.assertRaises(securesystemslib.exceptions.FormatError, make_targets_fileinfo,
length, bad_hashes, custom)
self.assertRaises(securesystemslib.exceptions.FormatError, make_targets_fileinfo,
length, hashes, bad_custom)
self.assertRaises(securesystemslib.exceptions.FormatError, make_targets_fileinfo,
bad_length, hashes)
self.assertRaises(securesystemslib.exceptions.FormatError, make_targets_fileinfo,
length, bad_hashes)
def test_make_metadata_fileinfo(self):
# Test conditions for valid arguments.
length = 1024
hashes = {'sha256': 'A4582BCF323BCEF', 'sha512': 'A4582BCF323BFEF'}
version = 8
METADATA_FILEINFO_SCHEMA = tuf.formats.METADATA_FILEINFO_SCHEMA
make_metadata_fileinfo = tuf.formats.make_metadata_fileinfo
self.assertTrue(METADATA_FILEINFO_SCHEMA.matches(make_metadata_fileinfo(
version, length, hashes)))
self.assertTrue(METADATA_FILEINFO_SCHEMA.matches(make_metadata_fileinfo(version)))
# Test conditions for invalid arguments.
bad_version = 'bad'
bad_length = 'bad'
bad_hashes = 'bad'
self.assertRaises(securesystemslib.exceptions.FormatError, make_metadata_fileinfo,
bad_version, length, hashes)
self.assertRaises(securesystemslib.exceptions.FormatError, make_metadata_fileinfo,
version, bad_length, hashes)
self.assertRaises(securesystemslib.exceptions.FormatError, make_metadata_fileinfo,
version, length, bad_hashes)
self.assertRaises(securesystemslib.exceptions.FormatError, make_metadata_fileinfo,
bad_version)
def test_make_versioninfo(self):
# Test conditions for valid arguments.
version_number = 8
versioninfo = {'version': version_number}
VERSIONINFO_SCHEMA = tuf.formats.VERSIONINFO_SCHEMA
make_versioninfo = tuf.formats.make_versioninfo
self.assertTrue(VERSIONINFO_SCHEMA.matches(make_versioninfo(version_number)))
# Test conditions for invalid arguments.
bad_version_number = '8'
self.assertRaises(securesystemslib.exceptions.FormatError, make_versioninfo, bad_version_number)
def test_expected_meta_rolename(self):
# Test conditions for valid arguments.
expected_rolename = tuf.formats.expected_meta_rolename
self.assertEqual('root', expected_rolename('Root'))
self.assertEqual('targets', expected_rolename('Targets'))
self.assertEqual('snapshot', expected_rolename('Snapshot'))
self.assertEqual('timestamp', expected_rolename('Timestamp'))
self.assertEqual('mirrors', expected_rolename('Mirrors'))
self.assertEqual('targets role', expected_rolename('Targets Role'))
self.assertEqual('root', expected_rolename('Root'))
# Test conditions for invalid arguments.
self.assertRaises(securesystemslib.exceptions.FormatError, expected_rolename, 123)
self.assertRaises(securesystemslib.exceptions.FormatError, expected_rolename, tuf.formats.ROOT_SCHEMA)
self.assertRaises(securesystemslib.exceptions.FormatError, expected_rolename, True)
def test_check_signable_object_format(self):
# Test condition for a valid argument.
root_file = os.path.join('repository_data', 'repository', 'metadata',
'root.json')
root = securesystemslib.util.load_json_file(root_file)
root = tuf.formats.make_signable(root)
self.assertEqual('root', tuf.formats.check_signable_object_format(root))
# Test conditions for invalid arguments.
check_signable = tuf.formats.check_signable_object_format
self.assertRaises(securesystemslib.exceptions.FormatError, check_signable, 'root')
self.assertRaises(securesystemslib.exceptions.FormatError, check_signable, 123)
self.assertRaises(securesystemslib.exceptions.FormatError, check_signable, tuf.formats.ROOT_SCHEMA)
self.assertRaises(securesystemslib.exceptions.FormatError, check_signable, True)
saved_type = root['signed']['_type']
del root['signed']['_type']
self.assertRaises(securesystemslib.exceptions.FormatError, check_signable, root)
root['signed']['_type'] = saved_type
root['signed']['_type'] = 'Root'
self.assertRaises(securesystemslib.exceptions.FormatError, check_signable, root)
root['signed']['_type'] = 'root'
del root['signed']['expires']
self.assertRaises(securesystemslib.exceptions.FormatError, check_signable, root)
def test_encode_canonical(self):
# Test conditions for valid arguments.
encode = securesystemslib.formats.encode_canonical
result = []
output = result.append
bad_output = 123
self.assertEqual('""', encode(""))
self.assertEqual('[1,2,3]', encode([1, 2, 3]))
self.assertEqual('[1,2,3]', encode([1,2,3]))
self.assertEqual('[]', encode([]))
self.assertEqual('{"A":[99]}', encode({"A": [99]}))
self.assertEqual('{"x":3,"y":2}', encode({"x": 3, "y": 2}))
self.assertEqual('{"x":3,"y":null}', encode({"x": 3, "y": None}))
# Condition where 'encode()' sends the result to the callable
# 'output'.
self.assertEqual(None, encode([1, 2, 3], output))
self.assertEqual('[1,2,3]', ''.join(result))
# Test conditions for invalid arguments.
self.assertRaises(securesystemslib.exceptions.FormatError, encode, tuf.formats.ROOT_SCHEMA)
self.assertRaises(securesystemslib.exceptions.FormatError, encode, 8.0)
self.assertRaises(securesystemslib.exceptions.FormatError, encode, {"x": 8.0})
self.assertRaises(securesystemslib.exceptions.FormatError, encode, 8.0, output)
self.assertRaises(securesystemslib.exceptions.FormatError, encode, {"x": securesystemslib.exceptions.FormatError})
# Run unit test.
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

461
tests/test_indefinite_freeze_attack_old.py
View file

@ -1,461 +0,0 @@
#!/usr/bin/env python
# Copyright 2012 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
test_indefinite_freeze_attack_old.py
<Author>
Konstantin Andrianov.
<Started>
March 10, 2012.
April 1, 2014.
Refactored to use the 'unittest' module (test conditions in code, rather
than verifying text output), use pre-generated repository files, and
discontinue use of the old repository tools. -vladimir.v.diaz
March 9, 2016.
Additional test added relating to issue:
https://github.com/theupdateframework/python-tuf/issues/322
If a metadata file is not updated (no indication of a new version
available), the expiration of the pre-existing, locally trusted metadata
must still be detected. This additional test complains if such does not
occur, and accompanies code in tuf.client.updater:refresh() to detect it.
-sebastien.awwad
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Simulate an indefinite freeze attack. In an indefinite freeze attack,
attacker is able to respond to client's requests with the same, outdated
metadata without the client being aware.
"""
import os
import time
import tempfile
import shutil
import json
import logging
import unittest
import sys
from urllib import request
import unittest.mock as mock
import tuf.formats
import tuf.log
import tuf.client.updater as updater
import tuf.repository_tool as repo_tool
import tuf.unittest_toolbox as unittest_toolbox
import tuf.roledb
import tuf.keydb
import tuf.exceptions
from tests import utils
import securesystemslib
# The repository tool is imported and logs console messages by default. Disable
# console log messages generated by this unit test.
repo_tool.disable_console_log_messages()
logger = logging.getLogger(__name__)
class TestIndefiniteFreezeAttack(unittest_toolbox.Modified_TestCase):
@classmethod
def setUpClass(cls):
# Create a temporary directory to store the repository, metadata, and target
# files. 'temporary_directory' must be deleted in TearDownModule() so that
# temporary files are always removed, even when exceptions occur.
cls.temporary_directory = tempfile.mkdtemp(dir=os.getcwd())
# Launch a SimpleHTTPServer (serves files in the current directory).
# Test cases will request metadata and target files that have been
# pre-generated in 'tuf/tests/repository_data', which will be served by the
# SimpleHTTPServer launched here. The test cases of this unit test assume
# the pre-generated metadata files have a specific structure, such
# as a delegated role 'targets/role1', three target files, five key files,
# etc.
cls.server_process_handler = utils.TestServerProcess(log=logger)
@classmethod
def tearDownClass(cls):
# Cleans the resources and flush the logged lines (if any).
cls.server_process_handler.clean()
# Remove the temporary repository directory, which should contain all the
# metadata, targets, and key files generated of all the test cases.
shutil.rmtree(cls.temporary_directory)
def setUp(self):
# We are inheriting from custom class.
unittest_toolbox.Modified_TestCase.setUp(self)
self.repository_name = 'test_repository1'
# Copy the original repository files provided in the test folder so that
# any modifications made to repository files are restricted to the copies.
# The 'repository_data' directory is expected to exist in 'tuf/tests/'.
original_repository_files = os.path.join(os.getcwd(), 'repository_data')
temporary_repository_root = \
self.make_temp_directory(directory=self.temporary_directory)
# The original repository, keystore, and client directories will be copied
# for each test case.
original_repository = os.path.join(original_repository_files, 'repository')
original_client = os.path.join(original_repository_files, 'client')
original_keystore = os.path.join(original_repository_files, 'keystore')
# Save references to the often-needed client repository directories.
# Test cases need these references to access metadata and target files.
self.repository_directory = \
os.path.join(temporary_repository_root, 'repository')
self.client_directory = os.path.join(temporary_repository_root, 'client')
self.keystore_directory = os.path.join(temporary_repository_root, 'keystore')
# Copy the original 'repository', 'client', and 'keystore' directories
# to the temporary repository the test cases can use.
shutil.copytree(original_repository, self.repository_directory)
shutil.copytree(original_client, self.client_directory)
shutil.copytree(original_keystore, self.keystore_directory)
# Set the url prefix required by the 'tuf/client/updater.py' updater.
# 'path/to/tmp/repository' -> 'localhost:8001/tmp/repository'.
repository_basepath = self.repository_directory[len(os.getcwd()):]
url_prefix = 'http://' + utils.TEST_HOST_ADDRESS + ':' \
+ str(self.server_process_handler.port) + repository_basepath
# Setting 'tuf.settings.repository_directory' with the temporary client
# directory copied from the original repository files.
tuf.settings.repositories_directory = self.client_directory
self.repository_mirrors = {'mirror1': {'url_prefix': url_prefix,
'metadata_path': 'metadata',
'targets_path': 'targets'}}
# Create the repository instance. The test cases will use this client
# updater to refresh metadata, fetch target files, etc.
self.repository_updater = updater.Updater(self.repository_name,
self.repository_mirrors)
def tearDown(self):
tuf.roledb.clear_roledb(clear_all=True)
tuf.keydb.clear_keydb(clear_all=True)
# Logs stdout and stderr from the sever subprocess.
self.server_process_handler.flush_log()
# Remove temporary directory
unittest_toolbox.Modified_TestCase.tearDown(self)
def test_without_tuf(self):
# Without TUF, Test 1 and Test 2 are functionally equivalent, so we skip
# Test 1 and only perform Test 2.
#
# Test 1: If we find that the timestamp acquired from a mirror indicates
# that there is no new snapshot file, and our current snapshot
# file is expired, is it recognized as such?
# Test 2: If an expired timestamp is downloaded, is it recognized as such?
# Test 2 Begin:
#
# 'timestamp.json' specifies the latest version of the repository files. A
# client should only accept the same version of this file up to a certain
# point, or else it cannot detect that new files are available for
# download. Modify the repository's timestamp.json' so that it expires
# soon, copy it over to the client, and attempt to re-fetch the same
# expired version.
#
# A non-TUF client (without a way to detect when metadata has expired) is
# expected to download the same version, and thus the same outdated files.
# Verify that the downloaded 'timestamp.json' contains the same file size
# and hash as the one available locally.
timestamp_path = os.path.join(self.repository_directory, 'metadata',
'timestamp.json')
timestamp_metadata = securesystemslib.util.load_json_file(timestamp_path)
expiry_time = time.time() - 10
expires = tuf.formats.unix_timestamp_to_datetime(int(expiry_time))
expires = expires.isoformat() + 'Z'
timestamp_metadata['signed']['expires'] = expires
tuf.formats.check_signable_object_format(timestamp_metadata)
with open(timestamp_path, 'wb') as file_object:
# Explicitly specify the JSON separators for Python 2 + 3 consistency.
timestamp_content = \
json.dumps(timestamp_metadata, indent=1, separators=(',', ': '),
sort_keys=True).encode('utf-8')
file_object.write(timestamp_content)
client_timestamp_path = os.path.join(self.client_directory, 'timestamp.json')
shutil.copy(timestamp_path, client_timestamp_path)
length, hashes = securesystemslib.util.get_file_details(timestamp_path)
fileinfo = tuf.formats.make_targets_fileinfo(length, hashes)
url_prefix = self.repository_mirrors['mirror1']['url_prefix']
url_file = os.path.join(url_prefix, 'metadata', 'timestamp.json')
request.urlretrieve(url_file.replace('\\', '/'), client_timestamp_path)
length, hashes = securesystemslib.util.get_file_details(client_timestamp_path)
download_fileinfo = tuf.formats.make_targets_fileinfo(length, hashes)
# Verify 'download_fileinfo' is equal to the current local file.
self.assertEqual(download_fileinfo, fileinfo)
def test_with_tuf(self):
# Three tests are conducted here.
#
# Test 1: If we find that the timestamp acquired from a mirror indicates
# that there is no new snapshot file, and our current snapshot
# file is expired, is it recognized as such?
# Test 2: If an expired timestamp is downloaded, is it recognized as such?
# Test 3: If an expired Snapshot is downloaded, is it (1) rejected? (2) the
# local Snapshot file deleted? (3) and is the client able to recover when
# given a new, valid Snapshot?
# Test 1 Begin:
#
# Addresses this issue: https://github.com/theupdateframework/python-tuf/issues/322
#
# If time has passed and our snapshot or targets role is expired, and
# the mirror whose timestamp we fetched doesn't indicate the existence of a
# new snapshot version, we still need to check that it's expired and notify
# the software update system / application / user. This test creates that
# scenario. The correct behavior is to raise an exception.
#
# Background: Expiration checks (updater._ensure_not_expired) were
# previously conducted when the metadata file was downloaded. If no new
# metadata file was downloaded, no expiry check would occur. In particular,
# while root was checked for expiration at the beginning of each
# updater.refresh() cycle, and timestamp was always checked because it was
# always fetched, snapshot and targets were never checked if the user did
# not receive evidence that they had changed. This bug allowed a class of
# freeze attacks.
# That bug was fixed and this test tests that fix going forward.
# Modify the timestamp file on the remote repository. 'timestamp.json'
# must be properly updated and signed with 'repository_tool.py', otherwise
# the client will reject it as invalid metadata.
# Load the repository
repository = repo_tool.load_repository(self.repository_directory)
# Load the snapshot and timestamp keys
key_file = os.path.join(self.keystore_directory, 'timestamp_key')
timestamp_private = repo_tool.import_ed25519_privatekey_from_file(key_file,
'password')
repository.timestamp.load_signing_key(timestamp_private)
key_file = os.path.join(self.keystore_directory, 'snapshot_key')
snapshot_private = repo_tool.import_ed25519_privatekey_from_file(key_file,
'password')
repository.snapshot.load_signing_key(snapshot_private)
# sign snapshot with expiry in near future (earlier than e.g. timestamp)
expiry = int(time.time() + 60*60)
repository.snapshot.expiration = tuf.formats.unix_timestamp_to_datetime(
expiry)
repository.mark_dirty(['snapshot', 'timestamp'])
repository.writeall()
# And move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# Refresh metadata on the client. For this refresh, all data is not expired.
logger.info('Test: Refreshing #1 - Initial metadata refresh occurring.')
self.repository_updater.refresh()
logger.info('Test: Refreshing #2 - refresh after local snapshot expiry.')
# mock current time to one second after snapshot expiry
mock_time = mock.Mock()
mock_time.return_value = expiry + 1
with mock.patch('time.time', mock_time):
try:
self.repository_updater.refresh() # We expect this to fail!
except tuf.exceptions.ExpiredMetadataError:
logger.info('Test: Refresh #2 - failed as expected. Expired local'
' snapshot case generated a tuf.exceptions.ExpiredMetadataError'
' exception as expected. Test pass.')
else:
self.fail('TUF failed to detect expired stale snapshot metadata. Freeze'
' attack successful.')
# Test 2 Begin:
#
# 'timestamp.json' specifies the latest version of the repository files.
# A client should only accept the same version of this file up to a certain
# point, or else it cannot detect that new files are available for download.
# Modify the repository's 'timestamp.json' so that it is about to expire,
# copy it over the to client, wait a moment until it expires, and attempt to
# re-fetch the same expired version.
# The same scenario as in test_without_tuf() is followed here, except with
# a TUF client. The TUF client performs a refresh of top-level metadata,
# which includes 'timestamp.json', and should detect a freeze attack if
# the repository serves an outdated 'timestamp.json'.
# Modify the timestamp file on the remote repository. 'timestamp.json'
# must be properly updated and signed with 'repository_tool.py', otherwise
# the client will reject it as invalid metadata. The resulting
# 'timestamp.json' should be valid metadata, but expired (as intended).
repository = repo_tool.load_repository(self.repository_directory)
key_file = os.path.join(self.keystore_directory, 'timestamp_key')
timestamp_private = repo_tool.import_ed25519_privatekey_from_file(key_file,
'password')
repository.timestamp.load_signing_key(timestamp_private)
# Set timestamp metadata to expire soon.
# We cannot set the timestamp expiration with
# 'repository.timestamp.expiration = ...' with already-expired timestamp
# metadata because of consistency checks that occur during that assignment.
expiry_time = time.time() + 60*60
datetime_object = tuf.formats.unix_timestamp_to_datetime(int(expiry_time))
repository.timestamp.expiration = datetime_object
repository.writeall()
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# mock current time to one second after timestamp expiry
mock_time = mock.Mock()
mock_time.return_value = expiry_time + 1
with mock.patch('time.time', mock_time):
try:
self.repository_updater.refresh() # We expect NoWorkingMirrorError.
except tuf.exceptions.NoWorkingMirrorError as e:
# Make sure the contained error is ExpiredMetadataError
for mirror_url, mirror_error in e.mirror_errors.items():
self.assertTrue(isinstance(mirror_error, tuf.exceptions.ExpiredMetadataError))
else:
self.fail('TUF failed to detect expired, stale timestamp metadata.'
' Freeze attack successful.')
# Test 3 Begin:
#
# Serve the client expired Snapshot. The client should reject the given,
# expired Snapshot and the locally trusted one, which should now be out of
# date.
# After the attack, attempt to re-issue a valid Snapshot to verify that
# the client is still able to update. A bug previously caused snapshot
# expiration or replay to result in an indefinite freeze; see
# github.com/theupdateframework/python-tuf/issues/736
repository = repo_tool.load_repository(self.repository_directory)
ts_key_file = os.path.join(self.keystore_directory, 'timestamp_key')
snapshot_key_file = os.path.join(self.keystore_directory, 'snapshot_key')
timestamp_private = repo_tool.import_ed25519_privatekey_from_file(
ts_key_file, 'password')
snapshot_private = repo_tool.import_ed25519_privatekey_from_file(
snapshot_key_file, 'password')
repository.timestamp.load_signing_key(timestamp_private)
repository.snapshot.load_signing_key(snapshot_private)
# Set ts to expire in 1 month.
ts_expiry_time = time.time() + 2630000
# Set snapshot to expire in 1 hour.
snapshot_expiry_time = time.time() + 60*60
ts_datetime_object = tuf.formats.unix_timestamp_to_datetime(
int(ts_expiry_time))
snapshot_datetime_object = tuf.formats.unix_timestamp_to_datetime(
int(snapshot_expiry_time))
repository.timestamp.expiration = ts_datetime_object
repository.snapshot.expiration = snapshot_datetime_object
repository.writeall()
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# mock current time to one second after snapshot expiry
mock_time = mock.Mock()
mock_time.return_value = snapshot_expiry_time + 1
with mock.patch('time.time', mock_time):
try:
# We expect the following refresh() to raise a NoWorkingMirrorError.
self.repository_updater.refresh()
except tuf.exceptions.NoWorkingMirrorError as e:
# Make sure the contained error is ExpiredMetadataError
for mirror_url, mirror_error in e.mirror_errors.items():
self.assertTrue(isinstance(mirror_error, tuf.exceptions.ExpiredMetadataError))
self.assertTrue(mirror_url.endswith('snapshot.json'))
else:
self.fail('TUF failed to detect expired, stale Snapshot metadata.'
' Freeze attack successful.')
# The client should have rejected the malicious Snapshot metadata, and
# distrusted the local snapshot file that is no longer valid.
self.assertTrue('snapshot' not in self.repository_updater.metadata['current'])
self.assertEqual(sorted(['root', 'targets', 'timestamp']),
sorted(self.repository_updater.metadata['current']))
# Verify that the client is able to recover from the malicious Snapshot.
# Re-sign a valid Snapshot file that the client should accept.
repository = repo_tool.load_repository(self.repository_directory)
repository.timestamp.load_signing_key(timestamp_private)
repository.snapshot.load_signing_key(snapshot_private)
# Set snapshot to expire in 1 month.
snapshot_expiry_time = time.time() + 2630000
snapshot_datetime_object = tuf.formats.unix_timestamp_to_datetime(
int(snapshot_expiry_time))
repository.snapshot.expiration = snapshot_datetime_object
repository.writeall()
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# Verify that the client accepts the valid metadata file.
self.repository_updater.refresh()
self.assertTrue('snapshot' in self.repository_updater.metadata['current'])
self.assertEqual(sorted(['root', 'targets', 'timestamp', 'snapshot']),
sorted(self.repository_updater.metadata['current']))
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

495
tests/test_key_revocation_integration_old.py
View file

@ -1,495 +0,0 @@
#!/usr/bin/env python
# Copyright 2016 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
test_key_revocation_integration_old.py
<Author>
Vladimir Diaz.
<Started>
April 28, 2016.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Integration test that verifies top-level roles are updated after all of their
keys have been revoked. There are unit tests in 'test_repository_tool_old.py'
that verify key and role revocation of specific roles, but these should be
expanded to verify key revocations over the span of multiple snapshots of the
repository.
The 'unittest_toolbox.py' module was created to provide additional testing
tools, such as automatically deleting temporary files created in test cases.
For more information on the additional testing tools, see
'tests/unittest_toolbox.py'.
"""
import os
import shutil
import tempfile
import logging
import unittest
import sys
import tuf
import tuf.log
import tuf.roledb
import tuf.keydb
import tuf.repository_tool as repo_tool
import tuf.unittest_toolbox as unittest_toolbox
import tuf.client.updater as updater
from tests import utils
import securesystemslib
logger = logging.getLogger(__name__)
repo_tool.disable_console_log_messages()
class TestKeyRevocation(unittest_toolbox.Modified_TestCase):
@classmethod
def setUpClass(cls):
# Create a temporary directory to store the repository, metadata, and target
# files. 'temporary_directory' must be deleted in TearDownModule() so that
# temporary files are always removed, even when exceptions occur.
cls.temporary_directory = tempfile.mkdtemp(dir=os.getcwd())
# Launch a SimpleHTTPServer (serves files in the current directory). Test
# cases will request metadata and target files that have been pre-generated
# in 'tuf/tests/repository_data', which will be served by the
# SimpleHTTPServer launched here. The test cases of
# 'test_key_revocation.py' assume the pre-generated metadata files have a
# specific structure, such as a delegated role, three target files, five
# key files, etc.
cls.server_process_handler = utils.TestServerProcess(log=logger)
@classmethod
def tearDownClass(cls):
# Cleans the resources and flush the logged lines (if any).
cls.server_process_handler.clean()
# Remove the temporary repository directory, which should contain all the
# metadata, targets, and key files generated for the test cases.
shutil.rmtree(cls.temporary_directory)
def setUp(self):
# We are inheriting from custom class.
unittest_toolbox.Modified_TestCase.setUp(self)
self.repository_name = 'test_repository1'
# Copy the original repository files provided in the test folder so that
# any modifications made to repository files are restricted to the copies.
# The 'repository_data' directory is expected to exist in 'tuf.tests/'.
original_repository_files = os.path.join(os.getcwd(), 'repository_data')
temporary_repository_root = \
self.make_temp_directory(directory=self.temporary_directory)
# The original repository, keystore, and client directories will be copied
# for each test case.
original_repository = os.path.join(original_repository_files, 'repository')
original_keystore = os.path.join(original_repository_files, 'keystore')
original_client = os.path.join(original_repository_files, 'client')
# Save references to the often-needed client repository directories.
# Test cases need these references to access metadata and target files.
self.repository_directory = \
os.path.join(temporary_repository_root, 'repository')
self.keystore_directory = \
os.path.join(temporary_repository_root, 'keystore')
self.client_directory = os.path.join(temporary_repository_root, 'client')
self.client_metadata = os.path.join(self.client_directory,
self.repository_name, 'metadata')
self.client_metadata_current = os.path.join(self.client_metadata, 'current')
self.client_metadata_previous = os.path.join(self.client_metadata, 'previous')
# Copy the original 'repository', 'client', and 'keystore' directories
# to the temporary repository the test cases can use.
shutil.copytree(original_repository, self.repository_directory)
shutil.copytree(original_client, self.client_directory)
shutil.copytree(original_keystore, self.keystore_directory)
# 'path/to/tmp/repository' -> 'localhost:8001/tmp/repository'.
repository_basepath = self.repository_directory[len(os.getcwd()):]
url_prefix = 'http://' + utils.TEST_HOST_ADDRESS + ':' \
+ str(self.server_process_handler.port) + repository_basepath
# Setting 'tuf.settings.repository_directory' with the temporary client
# directory copied from the original repository files.
tuf.settings.repositories_directory = self.client_directory
self.repository_mirrors = {'mirror1': {'url_prefix': url_prefix,
'metadata_path': 'metadata',
'targets_path': 'targets'}}
# Creating repository instance. The test cases will use this client
# updater to refresh metadata, fetch target files, etc.
self.repository_updater = updater.Updater(self.repository_name,
self.repository_mirrors)
# Metadata role keys are needed by the test cases to make changes to the
# repository (e.g., adding a new target file to 'targets.json' and then
# requesting a refresh()).
self.role_keys = _load_role_keys(self.keystore_directory)
def tearDown(self):
tuf.roledb.clear_roledb(clear_all=True)
tuf.keydb.clear_keydb(clear_all=True)
# Logs stdout and stderr from the sever subprocess.
self.server_process_handler.flush_log()
# Remove temporary directory
unittest_toolbox.Modified_TestCase.tearDown(self)
# UNIT TESTS.
def test_timestamp_key_revocation(self):
# First verify that the Timestamp role is properly signed. Calling
# refresh() should not raise an exception.
self.repository_updater.refresh()
# There should only be one key for Timestamp. Store the keyid to later
# verify that it has been revoked.
timestamp_roleinfo = tuf.roledb.get_roleinfo('timestamp', self.repository_name)
timestamp_keyid = timestamp_roleinfo['keyids']
self.assertEqual(len(timestamp_keyid), 1)
# Remove 'timestamp_keyid' and add a new key. Verify that the client
# detects the removal and addition of keys to the Timestamp role.
repository = repo_tool.load_repository(self.repository_directory)
repository.timestamp.remove_verification_key(self.role_keys['timestamp']['public'])
repository.timestamp.add_verification_key(self.role_keys['snapshot']['public'])
# Root, Snapshot, and Timestamp must be rewritten. Root must be written
# because the timestamp key has changed; Snapshot, because Root has
# changed, and ...
repository.root.load_signing_key(self.role_keys['root']['private'])
repository.snapshot.load_signing_key(self.role_keys['snapshot']['private'])
repository.timestamp.load_signing_key(self.role_keys['snapshot']['private'])
repository.writeall()
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# The client performs a refresh of top-level metadata to get the latest
# changes.
self.repository_updater.refresh()
# Verify that the client is able to recognize that a new set of keys have
# been added to the Timestamp role.
# First, has 'timestamp_keyid' been removed?
timestamp_roleinfo = tuf.roledb.get_roleinfo('timestamp', self.repository_name)
self.assertTrue(timestamp_keyid not in timestamp_roleinfo['keyids'])
# Second, is Timestamp's new key correct? The new key should be Snapshot's.
self.assertEqual(len(timestamp_roleinfo['keyids']), 1)
snapshot_roleinfo = tuf.roledb.get_roleinfo('snapshot', self.repository_name)
self.assertEqual(timestamp_roleinfo['keyids'], snapshot_roleinfo['keyids'])
def test_snapshot_key_revocation(self):
# First verify that the Snapshot role is properly signed. Calling
# refresh() should not raise an exception.
self.repository_updater.refresh()
# There should only be one key for Snapshot. Store the keyid to later
# verify that it has been revoked.
snapshot_roleinfo = tuf.roledb.get_roleinfo('snapshot', self.repository_name)
snapshot_keyid = snapshot_roleinfo['keyids']
self.assertEqual(len(snapshot_keyid), 1)
# Remove 'snapshot_keyid' and add a new key. Verify that the client
# detects the removal and addition of keys to the Snapshot role.
repository = repo_tool.load_repository(self.repository_directory)
repository.snapshot.remove_verification_key(self.role_keys['snapshot']['public'])
repository.snapshot.add_verification_key(self.role_keys['timestamp']['public'])
# Root, Snapshot, and Timestamp must be rewritten. Root must be written
# because the timestamp key has changed; Snapshot, because Root has
# changed, and Timesamp, because it must sign its metadata with a new key.
repository.root.load_signing_key(self.role_keys['root']['private'])
# Note: we added Timestamp's key to the Snapshot role.
repository.snapshot.load_signing_key(self.role_keys['timestamp']['private'])
repository.timestamp.load_signing_key(self.role_keys['timestamp']['private'])
repository.writeall()
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# The client performs a refresh of top-level metadata to get the latest
# changes.
self.repository_updater.refresh()
# Verify that the client is able to recognize that a new set of keys have
# been added to the Snapshot role.
# First, has 'snapshot_keyid' been removed?
snapshot_roleinfo = tuf.roledb.get_roleinfo('snapshot', self.repository_name)
self.assertTrue(snapshot_keyid not in snapshot_roleinfo['keyids'])
# Second, is Snapshot's new key correct? The new key should be
# Timestamp's.
self.assertEqual(len(snapshot_roleinfo['keyids']), 1)
timestamp_roleinfo = tuf.roledb.get_roleinfo('timestamp', self.repository_name)
self.assertEqual(snapshot_roleinfo['keyids'], timestamp_roleinfo['keyids'])
def test_targets_key_revocation(self):
# First verify that the Targets role is properly signed. Calling
# refresh() should not raise an exception.
self.repository_updater.refresh()
# There should only be one key for Targets. Store the keyid to later
# verify that it has been revoked.
targets_roleinfo = tuf.roledb.get_roleinfo('targets', self.repository_name)
targets_keyid = targets_roleinfo['keyids']
self.assertEqual(len(targets_keyid), 1)
# Remove 'targets_keyid' and add a new key. Verify that the client
# detects the removal and addition of keys to the Targets role.
repository = repo_tool.load_repository(self.repository_directory)
repository.targets.remove_verification_key(self.role_keys['targets']['public'])
repository.targets.add_verification_key(self.role_keys['timestamp']['public'])
# Root, Snapshot, and Timestamp must be rewritten. Root must be written
# because the timestamp key has changed; Snapshot, because Root has
# changed, and Timestamp because it must sign its metadata with a new key.
repository.root.load_signing_key(self.role_keys['root']['private'])
# Note: we added Timestamp's key to the Targets role.
repository.targets.load_signing_key(self.role_keys['timestamp']['private'])
repository.snapshot.load_signing_key(self.role_keys['snapshot']['private'])
repository.timestamp.load_signing_key(self.role_keys['timestamp']['private'])
repository.writeall()
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# The client performs a refresh of top-level metadata to get the latest
# changes.
self.repository_updater.refresh()
# Verify that the client is able to recognize that a new set of keys have
# been added to the Targets role.
# First, has 'targets_keyid' been removed?
targets_roleinfo = tuf.roledb.get_roleinfo('targets', self.repository_name)
self.assertTrue(targets_keyid not in targets_roleinfo['keyids'])
# Second, is Targets's new key correct? The new key should be
# Timestamp's.
self.assertEqual(len(targets_roleinfo['keyids']), 1)
timestamp_roleinfo = tuf.roledb.get_roleinfo('timestamp', self.repository_name)
self.assertEqual(targets_roleinfo['keyids'], timestamp_roleinfo['keyids'])
def test_root_key_revocation(self):
# First verify that the Root role is properly signed. Calling
# refresh() should not raise an exception.
self.repository_updater.refresh()
# There should only be one key for Root. Store the keyid to later verify
# that it has been revoked.
root_roleinfo = tuf.roledb.get_roleinfo('root', self.repository_name)
root_keyid = root_roleinfo['keyids']
self.assertEqual(len(root_keyid), 1)
# Remove 'root_keyid' and add a new key. Verify that the client detects
# the removal and addition of keys to the Root file.
repository = repo_tool.load_repository(self.repository_directory)
repository.root.add_verification_key(self.role_keys['snapshot']['public'])
repository.root.add_verification_key(self.role_keys['targets']['public'])
repository.root.add_verification_key(self.role_keys['timestamp']['public'])
# Root, Snapshot, and Timestamp must be rewritten. Root must be written
# because the timestamp key has changed; Snapshot, because Root has
# changed, and Timestamp because it must sign its metadata with a new key.
repository.root.load_signing_key(self.role_keys['snapshot']['private'])
repository.root.load_signing_key(self.role_keys['targets']['private'])
repository.root.load_signing_key(self.role_keys['timestamp']['private'])
# Note: We added the Snapshot, Targets, and Timestampkeys to the Root role.
# The Root's expected private key has not been loaded yet, so that we can
# verify that refresh() correctly raises a
# securesystemslib.exceptions.BadSignatureError exception.
repository.snapshot.load_signing_key(self.role_keys['snapshot']['private'])
repository.timestamp.load_signing_key(self.role_keys['timestamp']['private'])
# Root's version number = 2 after the following writeall().
repository.writeall()
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# Note well: The client should reject the new Root file because the
# repository has revoked the only Root key that the client trusts.
try:
self.repository_updater.refresh()
except tuf.exceptions.NoWorkingMirrorError as exception:
for mirror_exception in exception.mirror_errors.values():
self.assertTrue(isinstance(mirror_exception,
securesystemslib.exceptions.BadSignatureError))
repository.root.add_verification_key(self.role_keys['root']['public'])
repository.root.load_signing_key(self.role_keys['root']['private'])
# root, snapshot, and timestamp should be dirty
repository.dirty_roles()
repository.write('root', increment_version_number=False)
repository.write('snapshot')
repository.write('timestamp')
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# Root's version number = 2...
# The client successfully performs a refresh of top-level metadata to get
# the latest changes.
self.repository_updater.refresh()
self.assertEqual(self.repository_updater.metadata['current']['root']['version'], 2)
# Revoke the snapshot and targets keys (added to root) so that multiple
# snapshots are created. Discontinue signing with the old root key now
# that the client has successfully updated (note: the old Root key
# was revoked, but the repository continued signing with it to allow
# the client to update).
repository.root.remove_verification_key(self.role_keys['root']['public'])
repository.root.unload_signing_key(self.role_keys['root']['private'])
repository.root.remove_verification_key(self.role_keys['snapshot']['public'])
repository.root.unload_signing_key(self.role_keys['snapshot']['private'])
repository.writeall()
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# Root's version number = 3...
self.repository_updater.refresh()
repository.root.remove_verification_key(self.role_keys['targets']['public'])
repository.root.unload_signing_key(self.role_keys['targets']['private'])
# The following should fail because root rotation requires the new Root
# to be signed with the previous self.role_keys['targets'] key.
self.assertRaises(tuf.exceptions.UnsignedMetadataError,
repository.writeall)
repository.root.load_signing_key(self.role_keys['targets']['private'])
repository.writeall()
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# Root's version number = 4...
self.repository_updater.refresh()
self.assertEqual(self.repository_updater.metadata['current']['root']['version'], 4)
# Verify that the client is able to recognize that a new set of keys have
# been added to the Root role.
# First, has 'root_keyid' been removed?
root_roleinfo = tuf.roledb.get_roleinfo('root', self.repository_name)
self.assertTrue(root_keyid not in root_roleinfo['keyids'])
# Second, is Root's new key correct? The new key should be
# Timestamp's.
self.assertEqual(len(root_roleinfo['keyids']), 1)
timestamp_roleinfo = tuf.roledb.get_roleinfo('timestamp', self.repository_name)
self.assertEqual(root_roleinfo['keyids'], timestamp_roleinfo['keyids'])
def _load_role_keys(keystore_directory):
# Populating 'self.role_keys' by importing the required public and private
# keys of 'tuf/tests/repository_data/'. The role keys are needed when
# modifying the remote repository used by the test cases in this unit test.
# The pre-generated key files in 'repository_data/keystore' are all encrypted with
# a 'password' passphrase.
EXPECTED_KEYFILE_PASSWORD = 'password'
# Store and return the cryptography keys of the top-level roles, including 1
# delegated role.
role_keys = {}
root_key_file = os.path.join(keystore_directory, 'root_key')
targets_key_file = os.path.join(keystore_directory, 'targets_key')
snapshot_key_file = os.path.join(keystore_directory, 'snapshot_key')
timestamp_key_file = os.path.join(keystore_directory, 'timestamp_key')
delegation_key_file = os.path.join(keystore_directory, 'delegation_key')
role_keys = {'root': {}, 'targets': {}, 'snapshot': {}, 'timestamp': {},
'role1': {}}
# Import the top-level and delegated role public keys.
role_keys['root']['public'] = \
repo_tool.import_rsa_publickey_from_file(root_key_file+'.pub')
role_keys['targets']['public'] = \
repo_tool.import_ed25519_publickey_from_file(targets_key_file + '.pub')
role_keys['snapshot']['public'] = \
repo_tool.import_ed25519_publickey_from_file(snapshot_key_file + '.pub')
role_keys['timestamp']['public'] = \
repo_tool.import_ed25519_publickey_from_file(timestamp_key_file + '.pub')
role_keys['role1']['public'] = \
repo_tool.import_ed25519_publickey_from_file(delegation_key_file + '.pub')
# Import the private keys of the top-level and delegated roles.
role_keys['root']['private'] = \
repo_tool.import_rsa_privatekey_from_file(root_key_file,
EXPECTED_KEYFILE_PASSWORD)
role_keys['targets']['private'] = \
repo_tool.import_ed25519_privatekey_from_file(targets_key_file,
EXPECTED_KEYFILE_PASSWORD)
role_keys['snapshot']['private'] = \
repo_tool.import_ed25519_privatekey_from_file(snapshot_key_file,
EXPECTED_KEYFILE_PASSWORD)
role_keys['timestamp']['private'] = \
repo_tool.import_ed25519_privatekey_from_file(timestamp_key_file,
EXPECTED_KEYFILE_PASSWORD)
role_keys['role1']['private'] = \
repo_tool.import_ed25519_privatekey_from_file(delegation_key_file,
EXPECTED_KEYFILE_PASSWORD)
return role_keys
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

407
tests/test_keydb_old.py
View file

@ -1,407 +0,0 @@
#!/usr/bin/env python
# Copyright 2012 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
test_keydb_old.py
<Author>
Vladimir Diaz <vladimir.v.diaz@gmail.com>
<Started>
October 2012.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Unit test for 'keydb.py'.
"""
import unittest
import logging
import sys
import tuf
import tuf.formats
import securesystemslib.keys
import securesystemslib.settings
import tuf.keydb
import tuf.log
from tests import utils
logger = logging.getLogger(__name__)
# Generate the three keys to use in our test cases.
KEYS = []
for junk in range(3):
rsa_key = securesystemslib.keys.generate_rsa_key(2048)
rsa_key['keyid_hash_algorithms'] = securesystemslib.settings.HASH_ALGORITHMS
KEYS.append(rsa_key)
class TestKeydb(unittest.TestCase):
def setUp(self):
tuf.keydb.clear_keydb(clear_all=True)
def tearDown(self):
tuf.keydb.clear_keydb(clear_all=True)
def test_create_keydb(self):
# Test condition for normal behaviour.
repository_name = 'example_repository'
# The keydb dictionary should contain only the 'default' repository entry.
self.assertTrue('default' in tuf.keydb._keydb_dict)
self.assertEqual(1, len(tuf.keydb._keydb_dict))
tuf.keydb.create_keydb(repository_name)
self.assertEqual(2, len(tuf.keydb._keydb_dict))
# Verify that a keydb cannot be created for a name that already exists.
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.keydb.create_keydb, repository_name)
# Ensure that the key database for 'example_repository' is deleted so that
# the key database is returned to its original, default state.
tuf.keydb.remove_keydb(repository_name)
def test_remove_keydb(self):
# Test condition for expected behaviour.
rsakey = KEYS[0]
keyid = KEYS[0]['keyid']
repository_name = 'example_repository'
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.keydb.remove_keydb, 'default')
tuf.keydb.create_keydb(repository_name)
tuf.keydb.remove_keydb(repository_name)
# tuf.keydb.remove_keydb() logs a warning if a keydb for a non-existent
# repository is specified.
tuf.keydb.remove_keydb(repository_name)
# Test condition for improperly formatted argument, and unexpected argument.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.remove_keydb, 123)
self.assertRaises(TypeError, tuf.keydb.remove_keydb, rsakey, 123)
def test_clear_keydb(self):
# Test condition ensuring 'clear_keydb()' clears the keydb database.
# Test the length of the keydb before and after adding a key.
self.assertEqual(0, len(tuf.keydb._keydb_dict['default']))
rsakey = KEYS[0]
keyid = KEYS[0]['keyid']
tuf.keydb._keydb_dict['default'][keyid] = rsakey
self.assertEqual(1, len(tuf.keydb._keydb_dict['default']))
tuf.keydb.clear_keydb()
self.assertEqual(0, len(tuf.keydb._keydb_dict['default']))
# Test condition for unexpected argument.
self.assertRaises(TypeError, tuf.keydb.clear_keydb, 'default', False, 'unexpected_argument')
# Test condition for improperly formatted arguments.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.clear_keydb, 0)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.clear_keydb, 'default', 0)
# Test condition for non-existent repository name.
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.keydb.clear_keydb, 'non-existent')
# Test condition for keys added to a non-default key database. Unlike the
# test conditions above, this test makes use of the public functions
# add_key(), create_keydb(), and get_key() to more easily verify
# clear_keydb()'s behaviour.
rsakey = KEYS[0]
keyid = KEYS[0]['keyid']
repository_name = 'example_repository'
tuf.keydb.create_keydb(repository_name)
self.assertRaises(tuf.exceptions.UnknownKeyError, tuf.keydb.get_key, keyid, repository_name)
tuf.keydb.add_key(rsakey, keyid, repository_name)
self.assertEqual(rsakey, tuf.keydb.get_key(keyid, repository_name))
tuf.keydb.clear_keydb(repository_name)
self.assertRaises(tuf.exceptions.UnknownKeyError, tuf.keydb.get_key, keyid, repository_name)
# Remove 'repository_name' from the key database to revert it back to its
# original, default state (i.e., only the 'default' repository exists).
tuf.keydb.remove_keydb(repository_name)
def test_get_key(self):
# Test conditions using valid 'keyid' arguments.
rsakey = KEYS[0]
keyid = KEYS[0]['keyid']
tuf.keydb._keydb_dict['default'][keyid] = rsakey
rsakey2 = KEYS[1]
keyid2 = KEYS[1]['keyid']
tuf.keydb._keydb_dict['default'][keyid2] = rsakey2
self.assertEqual(rsakey, tuf.keydb.get_key(keyid))
self.assertEqual(rsakey2, tuf.keydb.get_key(keyid2))
self.assertNotEqual(rsakey2, tuf.keydb.get_key(keyid))
self.assertNotEqual(rsakey, tuf.keydb.get_key(keyid2))
# Test conditions using invalid arguments.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.get_key, None)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.get_key, 123)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.get_key, ['123'])
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.get_key, {'keyid': '123'})
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.get_key, '')
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.get_key, keyid, 123)
# Test condition using a 'keyid' that has not been added yet.
keyid3 = KEYS[2]['keyid']
self.assertRaises(tuf.exceptions.UnknownKeyError, tuf.keydb.get_key, keyid3)
# Test condition for a key added to a non-default repository.
repository_name = 'example_repository'
rsakey3 = KEYS[2]
tuf.keydb.create_keydb(repository_name)
tuf.keydb.add_key(rsakey3, keyid3, repository_name)
# Test condition for a key added to a non-existent repository.
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.keydb.get_key,
keyid, 'non-existent')
# Verify that 'rsakey3' is added to the expected repository name.
# If not supplied, the 'default' repository name is searched.
self.assertRaises(tuf.exceptions.UnknownKeyError, tuf.keydb.get_key, keyid3)
self.assertEqual(rsakey3, tuf.keydb.get_key(keyid3, repository_name))
# Remove the 'example_repository' so that other test functions have access
# to a default state of the keydb.
tuf.keydb.remove_keydb(repository_name)
def test_add_key(self):
# Test conditions using valid 'keyid' arguments.
rsakey = KEYS[0]
keyid = KEYS[0]['keyid']
rsakey2 = KEYS[1]
keyid2 = KEYS[1]['keyid']
rsakey3 = KEYS[2]
keyid3 = KEYS[2]['keyid']
self.assertEqual(None, tuf.keydb.add_key(rsakey, keyid))
self.assertEqual(None, tuf.keydb.add_key(rsakey2, keyid2))
self.assertEqual(None, tuf.keydb.add_key(rsakey3))
self.assertEqual(rsakey, tuf.keydb.get_key(keyid))
self.assertEqual(rsakey2, tuf.keydb.get_key(keyid2))
self.assertEqual(rsakey3, tuf.keydb.get_key(keyid3))
# Test conditions using arguments with invalid formats.
tuf.keydb.clear_keydb()
rsakey3['keytype'] = 'bad_keytype'
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.add_key, None, keyid)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.add_key, '', keyid)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.add_key, ['123'], keyid)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.add_key, {'a': 'b'}, keyid)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.add_key, rsakey, {'keyid': ''})
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.add_key, rsakey, 123)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.add_key, rsakey, False)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.add_key, rsakey, ['keyid'])
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.add_key, rsakey3, keyid3)
rsakey3['keytype'] = 'rsa'
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.add_key, rsakey3, keyid3, 123)
# Test conditions where keyid does not match the rsakey.
self.assertRaises(securesystemslib.exceptions.Error, tuf.keydb.add_key, rsakey, keyid2)
self.assertRaises(securesystemslib.exceptions.Error, tuf.keydb.add_key, rsakey2, keyid)
# Test conditions using keyids that have already been added.
tuf.keydb.add_key(rsakey, keyid)
tuf.keydb.add_key(rsakey2, keyid2)
self.assertRaises(tuf.exceptions.KeyAlreadyExistsError, tuf.keydb.add_key, rsakey)
self.assertRaises(tuf.exceptions.KeyAlreadyExistsError, tuf.keydb.add_key, rsakey2)
# Test condition for key added to the keydb of a non-default repository.
repository_name = 'example_repository'
tuf.keydb.create_keydb(repository_name)
self.assertRaises(tuf.exceptions.UnknownKeyError, tuf.keydb.get_key, keyid3, repository_name)
tuf.keydb.add_key(rsakey3, keyid3, repository_name)
self.assertRaises(tuf.exceptions.UnknownKeyError, tuf.keydb.get_key, keyid3)
self.assertEqual(rsakey3, tuf.keydb.get_key(keyid3, repository_name))
# Test condition for key added to the keydb of a non-existent repository.
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.keydb.add_key,
rsakey3, keyid3, 'non-existent')
# Reset the keydb to its original, default state. Other test functions
# expect only the 'default' repository to exist.
tuf.keydb.remove_keydb(repository_name)
def test_remove_key(self):
# Test conditions using valid keyids.
rsakey = KEYS[0]
keyid = KEYS[0]['keyid']
rsakey2 = KEYS[1]
keyid2 = KEYS[1]['keyid']
rsakey3 = KEYS[2]
keyid3 = KEYS[2]['keyid']
tuf.keydb.add_key(rsakey, keyid)
tuf.keydb.add_key(rsakey2, keyid2)
tuf.keydb.add_key(rsakey3, keyid3)
self.assertEqual(None, tuf.keydb.remove_key(keyid))
self.assertEqual(None, tuf.keydb.remove_key(keyid2))
# Ensure the keys were actually removed.
self.assertRaises(tuf.exceptions.UnknownKeyError, tuf.keydb.get_key, keyid)
self.assertRaises(tuf.exceptions.UnknownKeyError, tuf.keydb.get_key, keyid2)
# Test for 'keyid' not in keydb.
self.assertRaises(tuf.exceptions.UnknownKeyError, tuf.keydb.remove_key, keyid)
# Test condition for unknown key argument.
self.assertRaises(tuf.exceptions.UnknownKeyError, tuf.keydb.remove_key, '1')
# Test condition for removal of keys from a non-default repository.
repository_name = 'example_repository'
tuf.keydb.create_keydb(repository_name)
tuf.keydb.add_key(rsakey, keyid, repository_name)
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.keydb.remove_key, keyid, 'non-existent')
tuf.keydb.remove_key(keyid, repository_name)
self.assertRaises(tuf.exceptions.UnknownKeyError, tuf.keydb.remove_key, keyid, repository_name)
# Reset the keydb so that subsequent tests have access to the original,
# default keydb.
tuf.keydb.remove_keydb(repository_name)
# Test conditions for arguments with invalid formats.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.remove_key, None)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.remove_key, '')
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.remove_key, 123)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.remove_key, ['123'])
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.remove_key, keyid, 123)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.keydb.remove_key, {'bad': '123'})
self.assertRaises(securesystemslib.exceptions.Error, tuf.keydb.remove_key, rsakey3)
def test_create_keydb_from_root_metadata(self):
# Test condition using a valid 'root_metadata' argument.
rsakey = KEYS[0]
keyid = KEYS[0]['keyid']
rsakey2 = KEYS[1]
keyid2 = KEYS[1]['keyid']
keydict = {keyid: rsakey, keyid2: rsakey2}
roledict = {'Root': {'keyids': [keyid], 'threshold': 1},
'Targets': {'keyids': [keyid2, keyid], 'threshold': 1}}
version = 8
consistent_snapshot = False
expires = '1985-10-21T01:21:00Z'
root_metadata = tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROOT_SCHEMA,
_type='root',
spec_version='1.0.0',
version=version,
expires=expires,
keys=keydict,
roles=roledict,
consistent_snapshot=consistent_snapshot)
self.assertEqual(None, tuf.keydb.create_keydb_from_root_metadata(root_metadata))
tuf.keydb.create_keydb_from_root_metadata(root_metadata)
# Ensure 'keyid' and 'keyid2' were added to the keydb database.
self.assertEqual(rsakey, tuf.keydb.get_key(keyid))
self.assertEqual(rsakey2, tuf.keydb.get_key(keyid2))
# Verify that the keydb is populated for a non-default repository.
repository_name = 'example_repository'
tuf.keydb.create_keydb_from_root_metadata(root_metadata, repository_name)
# Test conditions for arguments with invalid formats.
self.assertRaises(securesystemslib.exceptions.FormatError,
tuf.keydb.create_keydb_from_root_metadata, None)
self.assertRaises(securesystemslib.exceptions.FormatError,
tuf.keydb.create_keydb_from_root_metadata, '')
self.assertRaises(securesystemslib.exceptions.FormatError,
tuf.keydb.create_keydb_from_root_metadata, 123)
self.assertRaises(securesystemslib.exceptions.FormatError,
tuf.keydb.create_keydb_from_root_metadata, ['123'])
self.assertRaises(securesystemslib.exceptions.FormatError,
tuf.keydb.create_keydb_from_root_metadata, {'bad': '123'})
self.assertRaises(securesystemslib.exceptions.FormatError,
tuf.keydb.create_keydb_from_root_metadata, root_metadata, 123)
# Verify that a keydb cannot be created for a non-existent repository name.
tuf.keydb.create_keydb_from_root_metadata(root_metadata, 'non-existent')
# Remove the 'non-existent' and 'example_repository' key database so that
# subsequent test functions have access to a default keydb.
tuf.keydb.remove_keydb(repository_name)
tuf.keydb.remove_keydb('non-existent')
# Test conditions for correctly formatted 'root_metadata' arguments but
# containing incorrect keyids or key types. In these conditions, the keys
# should not be added to the keydb database and a warning should be logged.
tuf.keydb.clear_keydb()
# 'keyid' does not match 'rsakey2'.
# In this case, the key will be added to the keydb
keydict[keyid] = rsakey2
# Key with invalid keytype.
rsakey3 = KEYS[2]
keyid3 = KEYS[2]['keyid']
rsakey3['keytype'] = 'bad_keytype'
keydict[keyid3] = rsakey3
version = 8
expires = '1985-10-21T01:21:00Z'
root_metadata = tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROOT_SCHEMA,
_type='root',
spec_version='1.0.0',
version=version,
expires=expires,
keys=keydict,
roles=roledict,
consistent_snapshot=consistent_snapshot)
self.assertEqual(None, tuf.keydb.create_keydb_from_root_metadata(root_metadata))
# Ensure only 'keyid2' and 'keyid' were added to the keydb database.
# 'keyid3' should not be stored.
self.maxDiff = None
self.assertEqual(rsakey2, tuf.keydb.get_key(keyid2))
test_key = rsakey2
test_key['keyid'] = keyid
self.assertEqual(test_key, tuf.keydb.get_key(keyid))
self.assertRaises(tuf.exceptions.UnknownKeyError, tuf.keydb.get_key, keyid3)
# reset values
rsakey3['keytype'] = 'rsa'
rsakey2['keyid'] = keyid2
# Run unit test.
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

210
tests/test_log_old.py
View file

@ -1,210 +0,0 @@
#!/usr/bin/env python
# Copyright 2014 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
test_log_old.py
<Authors>
Vladimir Diaz <vladimir.v.diaz@gmail.com>
<Started>
May 1, 2014.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Unit test for 'log.py'.
"""
import logging
import unittest
import os
import shutil
import sys
import importlib
import tuf
import tuf.log
import tuf.settings
import securesystemslib
import securesystemslib.util
from tests import utils
# We explicitly create a logger which is a child of the tuf hierarchy,
# instead of using the standard getLogger(__name__) pattern, because the
# tests are not part of the tuf hierarchy and we are testing functionality
# of the tuf package explicitly enabled on the tuf hierarchy
logger = logging.getLogger('tuf.test_log')
log_levels = [logging.CRITICAL, logging.ERROR, logging.WARNING,
logging.INFO, logging.DEBUG]
class TestLog(unittest.TestCase):
def setUp(self):
# store the current log level so it can be restored after the test
self._initial_level = logging.getLogger('tuf').level
def tearDown(self):
tuf.log.remove_console_handler()
tuf.log.disable_file_logging()
logging.getLogger('tuf').level = self._initial_level
def test_set_log_level(self):
# Test normal case.
global log_levels
global logger
tuf.log.set_log_level()
self.assertTrue(logger.isEnabledFor(logging.DEBUG))
for level in log_levels:
tuf.log.set_log_level(level)
self.assertTrue(logger.isEnabledFor(level))
# Test for improperly formatted argument.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.log.set_log_level, '123')
# Test for invalid argument.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.log.set_log_level, 51)
def test_set_filehandler_log_level(self):
# Normal case. Default log level.
# A file handler is not set by default. Add one now before attempting to
# set the log level.
self.assertRaises(tuf.exceptions.Error, tuf.log.set_filehandler_log_level)
tuf.log.enable_file_logging()
tuf.log.set_filehandler_log_level()
# Expected log levels.
for level in log_levels:
tuf.log.set_log_level(level)
# Test that the log level of the file handler cannot be set because
# file logging is disabled (via tuf.settings.ENABLE_FILE_LOGGING).
tuf.settings.ENABLE_FILE_LOGGING = False
importlib.reload(tuf.log)
# Test for improperly formatted argument.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.log.set_filehandler_log_level, '123')
# Test for invalid argument.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.log.set_filehandler_log_level, 51)
def test_set_console_log_level(self):
# Test setting a console log level without first adding one.
self.assertRaises(securesystemslib.exceptions.Error, tuf.log.set_console_log_level)
# Normal case. Default log level. Setting the console log level first
# requires adding a console logger.
tuf.log.add_console_handler()
tuf.log.set_console_log_level()
# Expected log levels.
for level in log_levels:
tuf.log.set_console_log_level(level)
# Test for improperly formatted argument.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.log.set_console_log_level, '123')
# Test for invalid argument.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.log.set_console_log_level, 51)
def test_add_console_handler(self):
# Normal case. Default log level.
tuf.log.add_console_handler()
# Adding a console handler when one has already been added.
tuf.log.add_console_handler()
# Expected log levels.
for level in log_levels:
tuf.log.set_console_log_level(level)
# Test for improperly formatted argument.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.log.add_console_handler, '123')
# Test for invalid argument.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.log.add_console_handler, 51)
# Test that an exception is printed to the console. Note: A stack trace
# is not included in the exception output because 'log.py' applies a filter
# to minimize the amount of output to the console.
try:
raise TypeError('Test exception output in the console.')
except TypeError as e:
logger.exception(e)
def test_remove_console_handler(self):
# Normal case.
tuf.log.remove_console_handler()
# Removing a console handler that has not been added. Logs a warning.
tuf.log.remove_console_handler()
def test_enable_file_logging(self):
# Normal case.
if os.path.exists(tuf.settings.LOG_FILENAME):
shutil.move(
tuf.settings.LOG_FILENAME, tuf.settings.LOG_FILENAME + '.backup')
tuf.log.enable_file_logging()
self.assertTrue(os.path.exists(tuf.settings.LOG_FILENAME))
if os.path.exists(tuf.settings.LOG_FILENAME + '.backup'):
shutil.move(
tuf.settings.LOG_FILENAME + '.backup', tuf.settings.LOG_FILENAME)
# The file logger must first be unset before attempting to re-add it.
self.assertRaises(tuf.exceptions.Error, tuf.log.enable_file_logging)
tuf.log.disable_file_logging()
tuf.log.enable_file_logging('my_log_file.log')
logger.debug('testing file logging')
self.assertTrue(os.path.exists('my_log_file.log'))
# Test for an improperly formatted argument.
tuf.log.disable_file_logging()
self.assertRaises(securesystemslib.exceptions.FormatError,
tuf.log.enable_file_logging, 1)
def test_disable_file_logging(self):
# Normal case.
tuf.log.enable_file_logging('my.log')
logger.debug('debug message')
junk, hashes = securesystemslib.util.get_file_details('my.log')
tuf.log.disable_file_logging()
logger.debug('new debug message')
junk, hashes2 = securesystemslib.util.get_file_details('my.log')
self.assertEqual(hashes, hashes2)
# An exception should not be raised if an attempt is made to disable
# the file logger if it has already been disabled.
tuf.log.disable_file_logging()
# Run unit test.
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

138
tests/test_mirrors_old.py
View file

@ -1,138 +0,0 @@
#!/usr/bin/env python
# Copyright 2012 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program>
test_mirrors_old.py
<Author>
Konstantin Andrianov.
<Started>
March 26, 2012.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Unit test for 'mirrors.py'.
"""
import unittest
import sys
import tuf.mirrors as mirrors
import tuf.unittest_toolbox as unittest_toolbox
from tests import utils
import securesystemslib
import securesystemslib.util
class TestMirrors(unittest_toolbox.Modified_TestCase):
def setUp(self):
unittest_toolbox.Modified_TestCase.setUp(self)
self.mirrors = \
{'mirror1': {'url_prefix' : 'http://mirror1.com',
'metadata_path' : 'metadata',
'targets_path' : 'targets'},
'mirror2': {'url_prefix' : 'http://mirror2.com',
'metadata_path' : 'metadata',
'targets_path' : 'targets',
'confined_target_dirs' : ['targets/release/',
'targets/release/']},
'mirror3': {'url_prefix' : 'http://mirror3.com',
'targets_path' : 'targets',
'confined_target_dirs' : ['targets/release/v2/']},
# confined_target_dirs = [] means that none of the targets on
# that mirror is available.
'mirror4': {'url_prefix' : 'http://mirror4.com',
'metadata_path' : 'metadata',
'confined_target_dirs' : []},
# Make sure we are testing when confined_target_dirs is [''] which means
# that all targets are available on that mirror.
'mirror5': {'url_prefix' : 'http://mirror5.com',
'targets_path' : 'targets',
'confined_target_dirs' : ['']}
}
def test_get_list_of_mirrors(self):
# Test: Normal case.
# 1 match: a mirror without target directory confinement
mirror_list = mirrors.get_list_of_mirrors('target', 'a.txt', self.mirrors)
self.assertEqual(len(mirror_list), 2)
self.assertTrue(self.mirrors['mirror1']['url_prefix']+'/targets/a.txt' in \
mirror_list)
self.assertTrue(self.mirrors['mirror5']['url_prefix']+'/targets/a.txt' in \
mirror_list)
mirror_list = mirrors.get_list_of_mirrors('target', 'a/b', self.mirrors)
self.assertEqual(len(mirror_list), 2)
self.assertTrue(self.mirrors['mirror1']['url_prefix']+'/targets/a/b' in \
mirror_list)
self.assertTrue(self.mirrors['mirror5']['url_prefix']+'/targets/a/b' in \
mirror_list)
# 2 matches: One with non-confined targets and one with matching confinement
mirror_list = mirrors.get_list_of_mirrors('target', 'release/v2/c', self.mirrors)
self.assertEqual(len(mirror_list), 3)
self.assertTrue(self.mirrors['mirror1']['url_prefix']+'/targets/release/v2/c' in \
mirror_list)
self.assertTrue(self.mirrors['mirror3']['url_prefix']+'/targets/release/v2/c' in \
mirror_list)
self.assertTrue(self.mirrors['mirror5']['url_prefix']+'/targets/release/v2/c' in \
mirror_list)
# 3 matches: Metadata found on 3 mirrors
mirror_list = mirrors.get_list_of_mirrors('meta', 'release.txt', self.mirrors)
self.assertEqual(len(mirror_list), 3)
self.assertTrue(self.mirrors['mirror1']['url_prefix']+'/metadata/release.txt' in \
mirror_list)
self.assertTrue(self.mirrors['mirror2']['url_prefix']+'/metadata/release.txt' in \
mirror_list)
self.assertTrue(self.mirrors['mirror4']['url_prefix']+'/metadata/release.txt' in \
mirror_list)
# No matches
del self.mirrors['mirror1']
del self.mirrors['mirror5']
mirror_list = mirrors.get_list_of_mirrors('target', 'a/b', self.mirrors)
self.assertFalse(mirror_list)
# Test: Invalid 'file_type'.
self.assertRaises(securesystemslib.exceptions.Error, mirrors.get_list_of_mirrors,
self.random_string(), 'a', self.mirrors)
self.assertRaises(securesystemslib.exceptions.Error, mirrors.get_list_of_mirrors,
12345, 'a', self.mirrors)
# Test: Improperly formatted 'file_path'.
self.assertRaises(securesystemslib.exceptions.FormatError, mirrors.get_list_of_mirrors,
'meta', 12345, self.mirrors)
# Test: Improperly formatted 'mirrors_dict' object.
self.assertRaises(securesystemslib.exceptions.FormatError, mirrors.get_list_of_mirrors,
'meta', 'a', 12345)
self.assertRaises(securesystemslib.exceptions.FormatError, mirrors.get_list_of_mirrors,
'meta', 'a', ['a'])
self.assertRaises(securesystemslib.exceptions.FormatError, mirrors.get_list_of_mirrors,
'meta', 'a', {'a':'b'})
# Run the unittests
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

236
tests/test_mix_and_match_attack_old.py
View file

@ -1,236 +0,0 @@
#!/usr/bin/env python
# Copyright 2012 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
test_mix_and_match_attack_old.py
<Author>
Konstantin Andrianov.
<Started>
March 27, 2012.
April 6, 2014.
Refactored to use the 'unittest' module (test conditions in code, rather
than verifying text output), use pre-generated repository files, and
discontinue use of the old repository tools. Modify the previous scenario
simulated for the mix-and-match attack. -vladimir.v.diaz
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Simulate a mix-and-match attack. In a mix-and-match attack, an attacker is
able to trick clients into using a combination of metadata that never existed
together on the repository at the same time.
Note: There is no difference between 'updates' and 'target' files.
"""
import os
import tempfile
import shutil
import logging
import unittest
import sys
import tuf.exceptions
import tuf.log
import tuf.client.updater as updater
import tuf.repository_tool as repo_tool
import tuf.unittest_toolbox as unittest_toolbox
import tuf.roledb
import tuf.keydb
from tests import utils
# The repository tool is imported and logs console messages by default.
# Disable console log messages generated by this unit test.
repo_tool.disable_console_log_messages()
logger = logging.getLogger(__name__)
class TestMixAndMatchAttack(unittest_toolbox.Modified_TestCase):
@classmethod
def setUpClass(cls):
# Create a temporary directory to store the repository, metadata, and
# target files. 'temporary_directory' must be deleted in TearDownModule()
# so that temporary files are always removed, even when exceptions occur.
cls.temporary_directory = tempfile.mkdtemp(dir=os.getcwd())
# Launch a SimpleHTTPServer (serves files in the current directory).
# Test cases will request metadata and target files that have been
# pre-generated in 'tuf/tests/repository_data', which will be served by the
# SimpleHTTPServer launched here. The test cases of this unit test assume
# the pre-generated metadata files have a specific structure, such
# as a delegated role 'targets/role1', three target files, five key files,
# etc.
cls.server_process_handler = utils.TestServerProcess(log=logger)
@classmethod
def tearDownClass(cls):
# Cleans the resources and flush the logged lines (if any).
cls.server_process_handler.clean()
# Remove the temporary repository directory, which should contain all the
# metadata, targets, and key files generated of all the test cases.
shutil.rmtree(cls.temporary_directory)
def setUp(self):
# We are inheriting from custom class.
unittest_toolbox.Modified_TestCase.setUp(self)
self.repository_name = 'test_repository1'
# Copy the original repository files provided in the test folder so that
# any modifications made to repository files are restricted to the copies.
# The 'repository_data' directory is expected to exist in 'tuf/tests/'.
original_repository_files = os.path.join(os.getcwd(), 'repository_data')
temporary_repository_root = \
self.make_temp_directory(directory=self.temporary_directory)
# The original repository, keystore, and client directories will be copied
# for each test case.
original_repository = os.path.join(original_repository_files, 'repository')
original_client = os.path.join(original_repository_files, 'client')
original_keystore = os.path.join(original_repository_files, 'keystore')
# Save references to the often-needed client repository directories.
# Test cases need these references to access metadata and target files.
self.repository_directory = \
os.path.join(temporary_repository_root, 'repository')
self.client_directory = os.path.join(temporary_repository_root, 'client')
self.keystore_directory = os.path.join(temporary_repository_root, 'keystore')
# Copy the original 'repository', 'client', and 'keystore' directories
# to the temporary repository the test cases can use.
shutil.copytree(original_repository, self.repository_directory)
shutil.copytree(original_client, self.client_directory)
shutil.copytree(original_keystore, self.keystore_directory)
# Set the url prefix required by the 'tuf/client/updater.py' updater.
# 'path/to/tmp/repository' -> 'localhost:8001/tmp/repository'.
repository_basepath = self.repository_directory[len(os.getcwd()):]
url_prefix = 'http://' + utils.TEST_HOST_ADDRESS + ':' \
+ str(self.server_process_handler.port) + repository_basepath
# Setting 'tuf.settings.repository_directory' with the temporary client
# directory copied from the original repository files.
tuf.settings.repositories_directory = self.client_directory
self.repository_mirrors = {'mirror1': {'url_prefix': url_prefix,
'metadata_path': 'metadata',
'targets_path': 'targets'}}
# Create the repository instance. The test cases will use this client
# updater to refresh metadata, fetch target files, etc.
self.repository_updater = updater.Updater(self.repository_name,
self.repository_mirrors)
def tearDown(self):
tuf.roledb.clear_roledb(clear_all=True)
tuf.keydb.clear_keydb(clear_all=True)
# Logs stdout and stderr from the sever subprocess.
self.server_process_handler.flush_log()
# Remove temporary directory
unittest_toolbox.Modified_TestCase.tearDown(self)
def test_with_tuf(self):
# Scenario:
# An attacker tries to trick the client into installing files indicated by
# a previous release of its corresponding metatadata. The outdated metadata
# is properly named and was previously valid, but is no longer current
# according to the latest 'snapshot.json' role. Generate a new snapshot of
# the repository after modifying a target file of 'role1.json'.
# Backup 'role1.json' (the delegated role to be updated, and then inserted
# again for the mix-and-match attack.)
role1_path = os.path.join(self.repository_directory, 'metadata', 'role1.json')
backup_role1 = os.path.join(self.repository_directory, 'role1.json.backup')
shutil.copy(role1_path, backup_role1)
# Backup 'file3.txt', specified by 'role1.json'.
file3_path = os.path.join(self.repository_directory, 'targets', 'file3.txt')
shutil.copy(file3_path, file3_path + '.backup')
# Re-generate the required metadata on the remote repository. The affected
# metadata must be properly updated and signed with 'repository_tool.py',
# otherwise the client will reject them as invalid metadata. The resulting
# metadata should be valid metadata.
repository = repo_tool.load_repository(self.repository_directory)
# Load the signing keys so that newly generated metadata is properly signed.
timestamp_keyfile = os.path.join(self.keystore_directory, 'timestamp_key')
role1_keyfile = os.path.join(self.keystore_directory, 'delegation_key')
snapshot_keyfile = os.path.join(self.keystore_directory, 'snapshot_key')
timestamp_private = \
repo_tool.import_ed25519_privatekey_from_file(timestamp_keyfile, 'password')
role1_private = \
repo_tool.import_ed25519_privatekey_from_file(role1_keyfile, 'password')
snapshot_private = \
repo_tool.import_ed25519_privatekey_from_file(snapshot_keyfile, 'password')
repository.targets('role1').load_signing_key(role1_private)
repository.snapshot.load_signing_key(snapshot_private)
repository.timestamp.load_signing_key(timestamp_private)
# Modify a 'role1.json' target file, and add it to its metadata so that a
# new version is generated.
with open(file3_path, 'wt') as file_object:
file_object.write('This is role2\'s target file.')
repository.targets('role1').add_target(os.path.basename(file3_path))
repository.writeall()
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# Insert the previously valid 'role1.json'. The TUF client should reject it.
shutil.move(backup_role1, role1_path)
# Verify that the TUF client detects unexpected metadata (previously valid,
# but not up-to-date with the latest snapshot of the repository) and
# refuses to continue the update process. Refresh top-level metadata so
# that the client is aware of the latest snapshot of the repository.
self.repository_updater.refresh()
try:
with utils.ignore_deprecation_warnings('tuf.client.updater'):
self.repository_updater.targets_of_role('role1')
# Verify that the specific
# 'tuf.exceptions.BadVersionNumberError' exception is raised by
# each mirror.
except tuf.exceptions.NoWorkingMirrorError as exception:
for mirror_url, mirror_error in exception.mirror_errors.items():
url_prefix = self.repository_mirrors['mirror1']['url_prefix']
url_file = os.path.join(url_prefix, 'metadata', 'role1.json')
# Verify that 'role1.json' is the culprit.
self.assertEqual(url_file.replace('\\', '/'), mirror_url)
self.assertTrue(isinstance(
mirror_error, tuf.exceptions.BadVersionNumberError))
else:
self.fail('TUF did not prevent a mix-and-match attack.')
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

268
tests/test_multiple_repositories_integration_old.py
View file

@ -1,268 +0,0 @@
#!/usr/bin/env python
# Copyright 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
test_multiple_repositories_integration_old.py
<Author>
Vladimir Diaz <vladimir.v.diaz@gmail.com>
<Started>
February 2, 2017
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Verify that clients and the repository tools are able to keep track of
multiple repositories and separate sets of metadata for each.
"""
import os
import tempfile
import logging
import shutil
import unittest
import json
import sys
import tuf
import tuf.log
import tuf.roledb
import tuf.client.updater as updater
import tuf.settings
import tuf.unittest_toolbox as unittest_toolbox
import tuf.repository_tool as repo_tool
from tests import utils
import securesystemslib
logger = logging.getLogger(__name__)
repo_tool.disable_console_log_messages()
class TestMultipleRepositoriesIntegration(unittest_toolbox.Modified_TestCase):
def setUp(self):
# Modified_Testcase can handle temp dir removal
unittest_toolbox.Modified_TestCase.setUp(self)
self.temporary_directory = self.make_temp_directory(directory=os.getcwd())
# Copy the original repository files provided in the test folder so that
# any modifications made to repository files are restricted to the copies.
# The 'repository_data' directory is expected to exist in 'tuf/tests/'.
original_repository_files = os.path.join(os.getcwd(), 'repository_data')
self.temporary_repository_root = tempfile.mkdtemp(dir=self.temporary_directory)
# The original repository, keystore, and client directories will be copied
# for each test case.
original_repository = os.path.join(original_repository_files, 'repository')
original_client = os.path.join(original_repository_files, 'client', 'test_repository1')
original_keystore = os.path.join(original_repository_files, 'keystore')
original_map_file = os.path.join(original_repository_files, 'map.json')
# Save references to the often-needed client repository directories.
# Test cases need these references to access metadata and target files.
self.repository_directory = os.path.join(self.temporary_repository_root,
'repository_server1')
self.repository_directory2 = os.path.join(self.temporary_repository_root,
'repository_server2')
# Setting 'tuf.settings.repositories_directory' with the temporary client
# directory copied from the original repository files.
tuf.settings.repositories_directory = self.temporary_repository_root
self.repository_name = 'test_repository1'
self.repository_name2 = 'test_repository2'
self.client_directory = os.path.join(self.temporary_repository_root,
self.repository_name)
self.client_directory2 = os.path.join(self.temporary_repository_root,
self.repository_name2)
self.keystore_directory = os.path.join(self.temporary_repository_root, 'keystore')
self.map_file = os.path.join(self.client_directory, 'map.json')
self.map_file2 = os.path.join(self.client_directory2, 'map.json')
# Copy the original 'repository', 'client', and 'keystore' directories
# to the temporary repository the test cases can use.
shutil.copytree(original_repository, self.repository_directory)
shutil.copytree(original_repository, self.repository_directory2)
shutil.copytree(original_client, self.client_directory)
shutil.copytree(original_client, self.client_directory2)
shutil.copyfile(original_map_file, self.map_file)
shutil.copyfile(original_map_file, self.map_file2)
shutil.copytree(original_keystore, self.keystore_directory)
# Launch a SimpleHTTPServer (serves files in the current directory).
# Test cases will request metadata and target files that have been
# pre-generated in 'tuf/tests/repository_data', which will be served by the
# SimpleHTTPServer launched here. The test cases of this unit test assume
# the pre-generated metadata files have a specific structure, such
# as a delegated role 'targets/role1', three target files, five key files,
# etc.
# Needed because in some tests simple_server.py cannot be found.
# The reason is that the current working directory
# has been changed when executing a subprocess.
SIMPLE_SERVER_PATH = os.path.join(os.getcwd(), 'simple_server.py')
# Creates a subprocess running a server.
self.server_process_handler = utils.TestServerProcess(log=logger,
server=SIMPLE_SERVER_PATH, popen_cwd=self.repository_directory)
logger.debug('Server process started.')
# Creates a subprocess running a server.
self.server_process_handler2 = utils.TestServerProcess(log=logger,
server=SIMPLE_SERVER_PATH, popen_cwd=self.repository_directory2)
logger.debug('Server process 2 started.')
url_prefix = \
'http://' + utils.TEST_HOST_ADDRESS + ':' + \
str(self.server_process_handler.port)
url_prefix2 = \
'http://' + utils.TEST_HOST_ADDRESS + ':' + \
str(self.server_process_handler2.port)
self.repository_mirrors = {'mirror1': {'url_prefix': url_prefix,
'metadata_path': 'metadata',
'targets_path': 'targets'}}
self.repository_mirrors2 = {'mirror1': {'url_prefix': url_prefix2,
'metadata_path': 'metadata',
'targets_path': 'targets'}}
# Create the repository instances. The test cases will use these client
# updaters to refresh metadata, fetch target files, etc.
self.repository_updater = updater.Updater(self.repository_name,
self.repository_mirrors)
self.repository_updater2 = updater.Updater(self.repository_name2,
self.repository_mirrors2)
def tearDown(self):
# Cleans the resources and flush the logged lines (if any).
self.server_process_handler.clean()
self.server_process_handler2.clean()
# updater.Updater() populates the roledb with the name "test_repository1"
tuf.roledb.clear_roledb(clear_all=True)
tuf.keydb.clear_keydb(clear_all=True)
# Remove top-level temporary directory
unittest_toolbox.Modified_TestCase.tearDown(self)
def test_update(self):
self.assertEqual('test_repository1', str(self.repository_updater))
self.assertEqual('test_repository2', str(self.repository_updater2))
self.assertEqual(sorted(['role1', 'root', 'snapshot', 'targets', 'timestamp']),
sorted(tuf.roledb.get_rolenames('test_repository1')))
self.assertEqual(sorted(['role1', 'root', 'snapshot', 'targets', 'timestamp']),
sorted(tuf.roledb.get_rolenames('test_repository2')))
# Note: refresh() resets the known metadata and updates the latest
# top-level metadata.
self.repository_updater.refresh()
self.assertEqual(sorted(['root', 'snapshot', 'targets', 'timestamp']),
sorted(tuf.roledb.get_rolenames('test_repository1')))
# test_repository2 wasn't refreshed and should still know about delegated
# roles.
self.assertEqual(sorted(['root', 'role1', 'snapshot', 'targets', 'timestamp']),
sorted(tuf.roledb.get_rolenames('test_repository2')))
# 'role1.json' should be downloaded, because it provides info for the
# requested 'file3.txt'.
valid_targetinfo = self.repository_updater.get_one_valid_targetinfo('file3.txt')
self.assertEqual(sorted(['role2', 'role1', 'root', 'snapshot', 'targets', 'timestamp']),
sorted(tuf.roledb.get_rolenames('test_repository1')))
def test_repository_tool(self):
self.assertEqual(self.repository_name, str(self.repository_updater))
self.assertEqual(self.repository_name2, str(self.repository_updater2))
repository = repo_tool.load_repository(self.repository_directory,
self.repository_name)
repository2 = repo_tool.load_repository(self.repository_directory2,
self.repository_name2)
repository.timestamp.version = 88
self.assertEqual(['timestamp'], tuf.roledb.get_dirty_roles(
self.repository_name))
self.assertEqual([], tuf.roledb.get_dirty_roles(self.repository_name2))
repository2.timestamp.version = 100
self.assertEqual(['timestamp'], tuf.roledb.get_dirty_roles(
self.repository_name2))
key_file = os.path.join(self.keystore_directory, 'timestamp_key')
timestamp_private = repo_tool.import_ed25519_privatekey_from_file(key_file, "password")
repository.timestamp.load_signing_key(timestamp_private)
repository2.timestamp.load_signing_key(timestamp_private)
repository.write('timestamp', increment_version_number=False)
repository2.write('timestamp', increment_version_number=False)
# And move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.rmtree(os.path.join(self.repository_directory2, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory2, 'metadata.staged'),
os.path.join(self.repository_directory2, 'metadata'))
# Verify that the client retrieves the expected updates.
logger.info('Downloading timestamp from server 1.')
self.repository_updater.refresh()
self.assertEqual(
88, self.repository_updater.metadata['current']['timestamp']['version'])
logger.info('Downloading timestamp from server 2.')
self.repository_updater2.refresh()
self.assertEqual(
100, self.repository_updater2.metadata['current']['timestamp']['version'])
# Test the behavior of the multi-repository updater.
map_file = securesystemslib.util.load_json_file(self.map_file)
map_file['repositories'][self.repository_name] = ['http://localhost:' \
+ str(self.server_process_handler.port)]
map_file['repositories'][self.repository_name2] = ['http://localhost:' \
+ str(self.server_process_handler2.port)]
with open(self.map_file, 'w') as file_object:
file_object.write(json.dumps(map_file))
# Try to load a non-existent map file.
self.assertRaises(tuf.exceptions.Error, updater.MultiRepoUpdater, 'bad_path')
multi_repo_updater = updater.MultiRepoUpdater(self.map_file)
valid_targetinfo = multi_repo_updater.get_valid_targetinfo('file3.txt')
for my_updater, my_targetinfo in valid_targetinfo.items():
my_updater.download_target(my_targetinfo, self.temporary_directory)
self.assertTrue(os.path.exists(os.path.join(self.temporary_directory, 'file3.txt')))
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

321
tests/test_replay_attack_old.py
View file

@ -1,321 +0,0 @@
#!/usr/bin/env python
# Copyright 2012 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
test_replay_attack_old.py
<Author>
Konstantin Andrianov.
<Started>
February 22, 2012.
April 5, 2014.
Refactored to use the 'unittest' module (test conditions in code, rather
than verifying text output), use pre-generated repository files, and
discontinue use of the old repository tools. Expanded comments.
-vladimir.v.diaz
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Simulate a replay, or rollback, attack. In a replay attack, a client is
tricked into installing software that is older than that which the client
previously knew to be available.
Note: There is no difference between 'updates' and 'target' files.
"""
import os
import tempfile
import datetime
import shutil
import logging
import unittest
import sys
from urllib import request
import tuf.formats
import tuf.log
import tuf.client.updater as updater
import tuf.repository_tool as repo_tool
import tuf.unittest_toolbox as unittest_toolbox
from tests import utils
import securesystemslib
# The repository tool is imported and logs console messages by default.
# Disable console log messages generated by this unit test.
repo_tool.disable_console_log_messages()
logger = logging.getLogger(__name__)
class TestReplayAttack(unittest_toolbox.Modified_TestCase):
@classmethod
def setUpClass(cls):
# Create a temporary directory to store the repository, metadata, and target
# files. 'temporary_directory' must be deleted in TearDownModule() so that
# temporary files are always removed, even when exceptions occur.
cls.temporary_directory = tempfile.mkdtemp(dir=os.getcwd())
# Launch a SimpleHTTPServer (serves files in the current directory).
# Test cases will request metadata and target files that have been
# pre-generated in 'tuf/tests/repository_data', which will be served by the
# SimpleHTTPServer launched here. The test cases of this unit test assume
# the pre-generated metadata files have a specific structure, such
# as a delegated role 'targets/role1', three target files, five key files,
# etc.
cls.server_process_handler = utils.TestServerProcess(log=logger)
@classmethod
def tearDownClass(cls):
# Cleans the resources and flush the logged lines (if any).
cls.server_process_handler.clean()
# Remove the temporary repository directory, which should contain all the
# metadata, targets, and key files generated of all the test cases.
shutil.rmtree(cls.temporary_directory)
def setUp(self):
# We are inheriting from custom class.
unittest_toolbox.Modified_TestCase.setUp(self)
self.repository_name = 'test_repository1'
# Copy the original repository files provided in the test folder so that
# any modifications made to repository files are restricted to the copies.
# The 'repository_data' directory is expected to exist in 'tuf/tests/'.
original_repository_files = os.path.join(os.getcwd(), 'repository_data')
temporary_repository_root = \
self.make_temp_directory(directory=self.temporary_directory)
# The original repository, keystore, and client directories will be copied
# for each test case.
original_repository = os.path.join(original_repository_files, 'repository')
original_client = os.path.join(original_repository_files, 'client')
original_keystore = os.path.join(original_repository_files, 'keystore')
# Save references to the often-needed client repository directories.
# Test cases need these references to access metadata and target files.
self.repository_directory = \
os.path.join(temporary_repository_root, 'repository')
self.client_directory = os.path.join(temporary_repository_root, 'client')
self.keystore_directory = os.path.join(temporary_repository_root, 'keystore')
# Copy the original 'repository', 'client', and 'keystore' directories
# to the temporary repository the test cases can use.
shutil.copytree(original_repository, self.repository_directory)
shutil.copytree(original_client, self.client_directory)
shutil.copytree(original_keystore, self.keystore_directory)
# Set the url prefix required by the 'tuf/client/updater.py' updater.
# 'path/to/tmp/repository' -> 'localhost:8001/tmp/repository'.
repository_basepath = self.repository_directory[len(os.getcwd()):]
url_prefix = 'http://' + utils.TEST_HOST_ADDRESS + ':' \
+ str(self.server_process_handler.port) + repository_basepath
# Setting 'tuf.settings.repository_directory' with the temporary client
# directory copied from the original repository files.
tuf.settings.repositories_directory = self.client_directory
self.repository_mirrors = {'mirror1': {'url_prefix': url_prefix,
'metadata_path': 'metadata',
'targets_path': 'targets'}}
# Create the repository instance. The test cases will use this client
# updater to refresh metadata, fetch target files, etc.
self.repository_updater = updater.Updater(self.repository_name,
self.repository_mirrors)
def tearDown(self):
tuf.roledb.clear_roledb(clear_all=True)
tuf.keydb.clear_keydb(clear_all=True)
# Logs stdout and stderr from the sever subprocess.
self.server_process_handler.flush_log()
# Remove temporary directory
unittest_toolbox.Modified_TestCase.tearDown(self)
def test_without_tuf(self):
# Scenario:
# 'timestamp.json' specifies the latest version of the repository files.
# A client should only accept the same version number (specified in the
# file) of the metadata, or greater. A version number less than the one
# currently trusted should be rejected. A non-TUF client may use a
# different mechanism for determining versions of metadata, but version
# numbers in this integrations because that is what TUF uses.
#
# Modify the repository's timestamp.json' so that a new version is generated
# and accepted by the client, and backup the previous version. The previous
# is then returned the next time the client requests an update. A non-TUF
# client (without a way to detect older versions of metadata, and thus
# updates) is expected to download older metadata and outdated files.
# Verify that the older version of timestamp.json' is downloaded by the
# non-TUF client.
# Backup the current version of 'timestamp'. It will be used as the
# outdated version returned to the client. The repository tool removes
# obsolete metadadata, so do *not* save the backup version in the
# repository's metadata directory.
timestamp_path = os.path.join(self.repository_directory, 'metadata',
'timestamp.json')
backup_timestamp = os.path.join(self.repository_directory,
'timestamp.json.backup')
shutil.copy(timestamp_path, backup_timestamp)
# The fileinfo of the previous version is saved to verify that it is indeed
# accepted by the non-TUF client.
length, hashes = securesystemslib.util.get_file_details(backup_timestamp)
previous_fileinfo = tuf.formats.make_targets_fileinfo(length, hashes)
# Modify the timestamp file on the remote repository.
repository = repo_tool.load_repository(self.repository_directory)
key_file = os.path.join(self.keystore_directory, 'timestamp_key')
timestamp_private = repo_tool.import_ed25519_privatekey_from_file(key_file,
'password')
repository.timestamp.load_signing_key(timestamp_private)
# Set an arbitrary expiration so that the repository tool generates a new
# version.
repository.timestamp.expiration = datetime.datetime(2030, 1, 1, 12, 12)
repository.writeall()
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# Save the fileinfo of the new version generated to verify that it is
# saved by the client.
length, hashes = securesystemslib.util.get_file_details(timestamp_path)
new_fileinfo = tuf.formats.make_targets_fileinfo(length, hashes)
url_prefix = self.repository_mirrors['mirror1']['url_prefix']
url_file = os.path.join(url_prefix, 'metadata', 'timestamp.json')
client_timestamp_path = os.path.join(self.client_directory,
self.repository_name, 'metadata', 'current', 'timestamp.json')
# On Windows, the URL portion should not contain back slashes.
request.urlretrieve(url_file.replace('\\', '/'), client_timestamp_path)
length, hashes = securesystemslib.util.get_file_details(client_timestamp_path)
download_fileinfo = tuf.formats.make_targets_fileinfo(length, hashes)
# Verify 'download_fileinfo' is equal to the new version.
self.assertEqual(download_fileinfo, new_fileinfo)
# Restore the previous version of 'timestamp.json' on the remote repository
# and verify that the non-TUF client downloads it (expected, but not ideal).
shutil.move(backup_timestamp, timestamp_path)
# On Windows, the URL portion should not contain back slashes.
request.urlretrieve(url_file.replace('\\', '/'), client_timestamp_path)
length, hashes = securesystemslib.util.get_file_details(client_timestamp_path)
download_fileinfo = tuf.formats.make_targets_fileinfo(length, hashes)
# Verify 'download_fileinfo' is equal to the previous version.
self.assertEqual(download_fileinfo, previous_fileinfo)
self.assertNotEqual(download_fileinfo, new_fileinfo)
def test_with_tuf(self):
# The same scenario outlined in test_without_tuf() is followed here, except
# with a TUF client (scenario description provided in the opening comment
# block of that test case.) The TUF client performs a refresh of top-level
# metadata, which also includes 'timestamp.json'.
# Backup the current version of 'timestamp'. It will be used as the
# outdated version returned to the client. The repository tool removes
# obsolete metadadata, so do *not* save the backup version in the
# repository's metadata directory.
timestamp_path = os.path.join(self.repository_directory, 'metadata',
'timestamp.json')
backup_timestamp = os.path.join(self.repository_directory,
'timestamp.json.backup')
shutil.copy(timestamp_path, backup_timestamp)
# The fileinfo of the previous version is saved to verify that it is indeed
# accepted by the non-TUF client.
length, hashes = securesystemslib.util.get_file_details(backup_timestamp)
previous_fileinfo = tuf.formats.make_targets_fileinfo(length, hashes)
# Modify the timestamp file on the remote repository.
repository = repo_tool.load_repository(self.repository_directory)
key_file = os.path.join(self.keystore_directory, 'timestamp_key')
timestamp_private = repo_tool.import_ed25519_privatekey_from_file(key_file,
'password')
repository.timestamp.load_signing_key(timestamp_private)
# Set an arbitrary expiration so that the repository tool generates a new
# version.
repository.timestamp.expiration = datetime.datetime(2030, 1, 1, 12, 12)
repository.writeall()
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# Save the fileinfo of the new version generated to verify that it is
# saved by the client.
length, hashes = securesystemslib.util.get_file_details(timestamp_path)
new_fileinfo = tuf.formats.make_targets_fileinfo(length, hashes)
# Refresh top-level metadata, including 'timestamp.json'. Installation of
# new version of 'timestamp.json' is expected.
self.repository_updater.refresh()
client_timestamp_path = os.path.join(self.client_directory,
self.repository_name, 'metadata', 'current', 'timestamp.json')
length, hashes = securesystemslib.util.get_file_details(client_timestamp_path)
download_fileinfo = tuf.formats.make_targets_fileinfo(length, hashes)
# Verify 'download_fileinfo' is equal to the new version.
self.assertEqual(download_fileinfo, new_fileinfo)
# Restore the previous version of 'timestamp.json' on the remote repository
# and verify that the non-TUF client downloads it (expected, but not ideal).
shutil.move(backup_timestamp, timestamp_path)
logger.info('Moving the timestamp.json backup to the current version.')
# Verify that the TUF client detects replayed metadata and refuses to
# continue the update process.
try:
self.repository_updater.refresh()
# Verify that the specific 'tuf.exceptions.ReplayedMetadataError' is raised by each
# mirror.
except tuf.exceptions.NoWorkingMirrorError as exception:
for mirror_url, mirror_error in exception.mirror_errors.items():
url_prefix = self.repository_mirrors['mirror1']['url_prefix']
url_file = os.path.join(url_prefix, 'metadata', 'timestamp.json')
# Verify that 'timestamp.json' is the culprit.
self.assertEqual(url_file.replace('\\', '/'), mirror_url)
self.assertTrue(isinstance(mirror_error, tuf.exceptions.ReplayedMetadataError))
else:
self.fail('TUF did not prevent a replay attack.')
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

1102
tests/test_repository_lib_old.py
View file

File diff suppressed because it is too large Load diff

2199
tests/test_repository_tool_old.py
View file

File diff suppressed because it is too large Load diff

787
tests/test_roledb_old.py
View file

@ -1,787 +0,0 @@
#!/usr/bin/env python
# Copyright 2012 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
test_roledb_old.py
<Author>
Vladimir Diaz <vladimir.v.diaz@gmail.com>
<Started>
October 2012.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Unit test for 'roledb.py'.
"""
import unittest
import logging
import sys
import tuf
import tuf.formats
import tuf.roledb
import tuf.exceptions
import tuf.log
from tests import utils
import securesystemslib
import securesystemslib.keys
logger = logging.getLogger(__name__)
# Generate the three keys to use in our test cases.
KEYS = []
for junk in range(3):
KEYS.append(securesystemslib.keys.generate_rsa_key(2048))
class TestRoledb(unittest.TestCase):
def setUp(self):
tuf.roledb.clear_roledb(clear_all=True)
def tearDown(self):
tuf.roledb.clear_roledb(clear_all=True)
def test_create_roledb(self):
# Verify that a roledb is created for a named repository.
self.assertTrue('default' in tuf.roledb._roledb_dict)
self.assertEqual(1, len(tuf.roledb._roledb_dict))
repository_name = 'example_repository'
tuf.roledb.create_roledb(repository_name)
self.assertEqual(2, len(tuf.roledb._roledb_dict))
self.assertTrue(repository_name in tuf.roledb._roledb_dict)
# Test for invalid and improperly formatted arguments.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.create_roledb, 123)
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.create_roledb, 'default')
# Reset the roledb so that subsequent test functions have access to the
# original, default roledb.
tuf.roledb.remove_roledb(repository_name)
def test_remove_roledb(self):
# Verify that the named repository is removed from the roledb.
repository_name = 'example_repository'
rolename = 'targets'
roleinfo = {'keyids': ['123'], 'threshold': 1}
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.remove_roledb, 'default')
tuf.roledb.create_roledb(repository_name)
tuf.roledb.remove_roledb(repository_name)
# remove_roledb() should not raise an exception if a non-existent
# 'repository_name' is specified.
tuf.roledb.remove_roledb(repository_name)
# Ensure the roledb is reset to its original, default state. Subsequent
# test functions expect only the 'default' repository to exist in the roledb.
tuf.roledb.remove_roledb(repository_name)
def test_clear_roledb(self):
# Test for an empty roledb, a length of 1 after adding a key, and finally
# an empty roledb after calling 'clear_roledb()'.
self.assertEqual(0, len(tuf.roledb._roledb_dict['default']))
tuf.roledb._roledb_dict['default']['Root'] = {'keyids': ['123'], 'threshold': 1}
self.assertEqual(1, len(tuf.roledb._roledb_dict['default']))
tuf.roledb.clear_roledb()
self.assertEqual(0, len(tuf.roledb._roledb_dict['default']))
# Verify that the roledb can be cleared for a non-default repository.
rolename = 'targets'
roleinfo = {'keyids': ['123'], 'threshold': 1}
repository_name = 'example_repository'
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.clear_roledb, repository_name)
tuf.roledb.create_roledb(repository_name)
tuf.roledb.add_role(rolename, roleinfo, repository_name)
self.assertEqual(roleinfo['keyids'], tuf.roledb.get_role_keyids(rolename, repository_name))
tuf.roledb.clear_roledb(repository_name)
self.assertFalse(tuf.roledb.role_exists(rolename, repository_name))
# Reset the roledb so that subsequent tests have access to the original,
# default roledb.
tuf.roledb.remove_roledb(repository_name)
# Test condition for invalid and unexpected arguments.
self.assertRaises(TypeError, tuf.roledb.clear_roledb, 'default', False, 'unexpected_argument')
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.clear_roledb, 123)
def test_add_role(self):
# Test conditions where the arguments are valid.
self.assertEqual(0, len(tuf.roledb._roledb_dict['default']))
rolename = 'targets'
roleinfo = {'keyids': ['123'], 'threshold': 1}
rolename2 = 'role1'
self.assertEqual(None, tuf.roledb.add_role(rolename, roleinfo))
self.assertEqual(1, len(tuf.roledb._roledb_dict['default']))
tuf.roledb.clear_roledb()
self.assertEqual(None, tuf.roledb.add_role(rolename, roleinfo))
self.assertEqual(1, len(tuf.roledb._roledb_dict['default']))
# Verify that a role can be added to a non-default repository.
repository_name = 'example_repository'
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.clear_roledb,
repository_name)
tuf.roledb.create_roledb(repository_name)
tuf.roledb.add_role(rolename, roleinfo, repository_name)
self.assertEqual(roleinfo['keyids'], tuf.roledb.get_role_keyids(rolename,
repository_name))
# Reset the roledb so that subsequent tests have access to a default
# roledb.
tuf.roledb.remove_roledb(repository_name)
# Test conditions where the arguments are improperly formatted.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.add_role, None, roleinfo)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.add_role, 123, roleinfo)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.add_role, [''], roleinfo)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.add_role, rolename, None)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.add_role, rolename, 123)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.add_role, rolename, [''])
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.add_role, rolename, roleinfo, 123)
# Test condition where the rolename already exists in the role database.
self.assertRaises(tuf.exceptions.RoleAlreadyExistsError, tuf.roledb.add_role,
rolename, roleinfo)
# Test where the repository name does not exist in the role database.
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.add_role,
'new_role', roleinfo, 'non-existent')
# Test conditions for invalid rolenames.
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.add_role, ' badrole ',
roleinfo)
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.add_role, '/badrole/',
roleinfo)
def test_role_exists(self):
# Test conditions where the arguments are valid.
rolename = 'targets'
roleinfo = {'keyids': ['123'], 'threshold': 1}
rolename2 = 'role1'
self.assertEqual(False, tuf.roledb.role_exists(rolename))
tuf.roledb.add_role(rolename, roleinfo)
tuf.roledb.add_role(rolename2, roleinfo)
self.assertEqual(True, tuf.roledb.role_exists(rolename))
self.assertEqual(True, tuf.roledb.role_exists(rolename2))
# Verify that a role can be queried for a non-default repository.
repository_name = 'example_repository'
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.clear_roledb, repository_name)
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.role_exists, rolename, repository_name)
tuf.roledb.create_roledb(repository_name)
self.assertEqual(False, tuf.roledb.role_exists(rolename, repository_name))
tuf.roledb.add_role(rolename, roleinfo, repository_name)
self.assertTrue(tuf.roledb.role_exists(rolename, repository_name))
# Reset the roledb so that subsequent tests have access to the original,
# default roledb.
tuf.roledb.remove_roledb(repository_name)
# Test conditions where the arguments are improperly formatted.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.role_exists, None)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.role_exists, 123)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.role_exists, ['rolename'])
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.role_exists, rolename, 123)
# Test conditions for invalid rolenames.
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.role_exists, '')
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.role_exists, ' badrole ')
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.role_exists, '/badrole/')
def test_remove_role(self):
# Test conditions where the arguments are valid.
rolename = 'targets'
rolename2 = 'release'
rolename3 = 'django'
roleinfo = {'keyids': ['123'], 'threshold': 1}
roleinfo2 = {'keyids': ['123'], 'threshold': 1, 'delegations':
{'roles': [{'name': 'django', 'keyids': ['456'], 'threshold': 1}],
'keys': {'456': {'keytype': 'rsa', 'keyval': {'public': '456'}},
}}}
tuf.roledb.add_role(rolename, roleinfo)
tuf.roledb.add_role(rolename2, roleinfo2)
tuf.roledb.add_role(rolename3, roleinfo)
self.assertEqual(None, tuf.roledb.remove_role(rolename))
self.assertEqual(True, rolename not in tuf.roledb._roledb_dict)
# Verify that a role can be removed from a non-default repository.
repository_name = 'example_repository'
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.remove_role, rolename, repository_name)
tuf.roledb.create_roledb(repository_name)
tuf.roledb.add_role(rolename, roleinfo, repository_name)
self.assertEqual(roleinfo['keyids'], tuf.roledb.get_role_keyids(rolename, repository_name))
self.assertEqual(None, tuf.roledb.remove_role(rolename, repository_name))
# Verify that a role cannot be removed from a non-existent repository name.
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.remove_role, rolename, 'non-existent')
# Reset the roledb so that subsequent test have access to the original,
# default roledb.
tuf.roledb.remove_roledb(repository_name)
# Test conditions where removing a role does not cause the removal of its
# delegated roles. The 'django' role should now only exist (after the
# removal of 'targets' in the previous test condition, and the removal
# of 'release' in the remove_role() call next.
self.assertEqual(None, tuf.roledb.remove_role(rolename2))
self.assertEqual(1, len(tuf.roledb._roledb_dict['default']))
# Test conditions where the arguments are improperly formatted,
# contain invalid names, or haven't been added to the role database.
self._test_rolename(tuf.roledb.remove_role)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.remove_role, rolename, 123)
def test_get_rolenames(self):
# Test conditions where the arguments are valid.
rolename = 'targets'
rolename2 = 'role1'
roleinfo = {'keyids': ['123'], 'threshold': 1}
self.assertEqual([], tuf.roledb.get_rolenames())
tuf.roledb.add_role(rolename, roleinfo)
tuf.roledb.add_role(rolename2, roleinfo)
self.assertEqual(set(['targets', 'role1']),
set(tuf.roledb.get_rolenames()))
# Verify that rolenames can be retrieved for a role in a non-default
# repository.
repository_name = 'example_repository'
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.get_rolenames, repository_name)
tuf.roledb.create_roledb(repository_name)
tuf.roledb.add_role(rolename, roleinfo, repository_name)
tuf.roledb.add_role(rolename2, roleinfo, repository_name)
self.assertEqual(set(['targets', 'role1']),
set(tuf.roledb.get_rolenames()))
# Reset the roledb so that subsequent tests have access to the original,
# default repository.
tuf.roledb.remove_roledb(repository_name)
# Test for invalid or improperly formatted arguments.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.get_rolenames, 123)
def test_get_role_info(self):
# Test conditions where the arguments are valid.
rolename = 'targets'
rolename2 = 'role1'
roleinfo = {'keyids': ['123'], 'threshold': 1}
roleinfo2 = {'keyids': ['456', '789'], 'threshold': 2}
self.assertRaises(tuf.exceptions.UnknownRoleError, tuf.roledb.get_roleinfo, rolename)
tuf.roledb.add_role(rolename, roleinfo)
tuf.roledb.add_role(rolename2, roleinfo2)
self.assertEqual(roleinfo, tuf.roledb.get_roleinfo(rolename))
self.assertEqual(roleinfo2, tuf.roledb.get_roleinfo(rolename2))
# Verify that a roleinfo can be retrieved for a role in a non-default
# repository.
repository_name = 'example_repository'
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.get_roleinfo,
rolename, repository_name)
tuf.roledb.create_roledb(repository_name)
tuf.roledb.add_role(rolename, roleinfo, repository_name)
self.assertEqual(roleinfo, tuf.roledb.get_roleinfo(rolename, repository_name))
# Verify that a roleinfo cannot be retrieved for a non-existent repository
# name.
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.get_roleinfo, rolename,
'non-existent')
# Reset the roledb so that subsequent tests have access to the original,
# default roledb
tuf.roledb.remove_roledb(repository_name)
# Test conditions where the arguments are improperly formatted, contain
# invalid names, or haven't been added to the role database.
self._test_rolename(tuf.roledb.get_roleinfo)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.get_roleinfo, rolename, 123)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.get_roleinfo, 123)
def test_get_role_keyids(self):
# Test conditions where the arguments are valid.
rolename = 'targets'
rolename2 = 'role1'
roleinfo = {'keyids': ['123'], 'threshold': 1}
roleinfo2 = {'keyids': ['456', '789'], 'threshold': 2}
self.assertRaises(tuf.exceptions.UnknownRoleError, tuf.roledb.get_role_keyids, rolename)
tuf.roledb.add_role(rolename, roleinfo)
tuf.roledb.add_role(rolename2, roleinfo2)
self.assertEqual(['123'], tuf.roledb.get_role_keyids(rolename))
self.assertEqual(set(['456', '789']),
set(tuf.roledb.get_role_keyids(rolename2)))
# Verify that the role keyids can be retrieved for a role in a non-default
# repository.
repository_name = 'example_repository'
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.get_role_keyids,
rolename, repository_name)
tuf.roledb.create_roledb(repository_name)
tuf.roledb.add_role(rolename, roleinfo, repository_name)
self.assertEqual(['123'], tuf.roledb.get_role_keyids(rolename, repository_name))
# Verify that rolekeyids cannot be retrieved from a non-existent repository
# name.
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.get_role_keyids, rolename,
'non-existent')
# Reset the roledb so that subsequent tests have access to the original,
# default roledb
tuf.roledb.remove_roledb(repository_name)
# Test conditions where the arguments are improperly formatted, contain
# invalid names, or haven't been added to the role database.
self._test_rolename(tuf.roledb.get_role_keyids)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.get_role_keyids, rolename, 123)
def test_get_role_threshold(self):
# Test conditions where the arguments are valid.
rolename = 'targets'
rolename2 = 'role1'
roleinfo = {'keyids': ['123'], 'threshold': 1}
roleinfo2 = {'keyids': ['456', '789'], 'threshold': 2}
self.assertRaises(tuf.exceptions.UnknownRoleError, tuf.roledb.get_role_threshold, rolename)
tuf.roledb.add_role(rolename, roleinfo)
tuf.roledb.add_role(rolename2, roleinfo2)
self.assertEqual(1, tuf.roledb.get_role_threshold(rolename))
self.assertEqual(2, tuf.roledb.get_role_threshold(rolename2))
# Verify that the threshold can be retrieved for a role in a non-default
# repository.
repository_name = 'example_repository'
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.get_role_threshold,
rolename, repository_name)
tuf.roledb.create_roledb(repository_name)
tuf.roledb.add_role(rolename, roleinfo, repository_name)
self.assertEqual(roleinfo['threshold'], tuf.roledb.get_role_threshold(rolename, repository_name))
# Verify that a role's threshold cannot be retrieved from a non-existent
# repository name.
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.get_role_threshold,
rolename, 'non-existent')
# Reset the roledb so that subsequent tests have access to the original,
# default roledb.
tuf.roledb.remove_roledb(repository_name)
# Test conditions where the arguments are improperly formatted,
# contain invalid names, or haven't been added to the role database.
self._test_rolename(tuf.roledb.get_role_threshold)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.get_role_threshold, rolename, 123)
def test_get_role_paths(self):
# Test conditions where the arguments are valid.
rolename = 'targets'
rolename2 = 'role1'
roleinfo = {'keyids': ['123'], 'threshold': 1}
paths = ['a/b', 'c/d']
roleinfo2 = {'keyids': ['456', '789'], 'threshold': 2, 'paths': paths}
self.assertRaises(tuf.exceptions.UnknownRoleError, tuf.roledb.get_role_paths, rolename)
tuf.roledb.add_role(rolename, roleinfo)
tuf.roledb.add_role(rolename2, roleinfo2)
self.assertEqual({}, tuf.roledb.get_role_paths(rolename))
self.assertEqual(paths, tuf.roledb.get_role_paths(rolename2))
# Verify that role paths can be queried for roles in non-default
# repositories.
repository_name = 'example_repository'
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.get_role_paths,
rolename, repository_name)
tuf.roledb.create_roledb(repository_name)
tuf.roledb.add_role(rolename2, roleinfo2, repository_name)
self.assertEqual(roleinfo2['paths'], tuf.roledb.get_role_paths(rolename2,
repository_name))
# Reset the roledb so that subsequent roles have access to the original,
# default roledb.
tuf.roledb.remove_roledb(repository_name)
# Test conditions where the arguments are improperly formatted,
# contain invalid names, or haven't been added to the role database.
self._test_rolename(tuf.roledb.get_role_paths)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.get_role_paths, rolename, 123)
def test_get_delegated_rolenames(self):
# Test conditions where the arguments are valid.
rolename = 'unclaimed'
rolename2 = 'django'
rolename3 = 'release'
rolename4 = 'tuf'
# unclaimed's roleinfo.
roleinfo = {'keyids': ['123'], 'threshold': 1, 'delegations':
{'roles': [{'name': 'django', 'keyids': ['456'], 'threshold': 1},
{'name': 'tuf', 'keyids': ['888'], 'threshold': 1}],
'keys': {'456': {'keytype': 'rsa', 'keyval': {'public': '456'}},
}}}
# django's roleinfo.
roleinfo2 = {'keyids': ['456'], 'threshold': 1, 'delegations':
{'roles': [{'name': 'release', 'keyids': ['789'], 'threshold': 1}],
'keys': {'789': {'keytype': 'rsa', 'keyval': {'public': '789'}},
}}}
# release's roleinfo.
roleinfo3 = {'keyids': ['789'], 'threshold': 1, 'delegations':
{'roles': [],
'keys': {}}}
# tuf's roleinfo.
roleinfo4 = {'keyids': ['888'], 'threshold': 1, 'delegations':
{'roles': [],
'keys': {}}}
self.assertRaises(tuf.exceptions.UnknownRoleError, tuf.roledb.get_delegated_rolenames,
rolename)
tuf.roledb.add_role(rolename, roleinfo)
tuf.roledb.add_role(rolename2, roleinfo2)
tuf.roledb.add_role(rolename3, roleinfo3)
tuf.roledb.add_role(rolename4, roleinfo4)
self.assertEqual(set(['django', 'tuf']),
set(tuf.roledb.get_delegated_rolenames(rolename)))
self.assertEqual(set(['release']),
set(tuf.roledb.get_delegated_rolenames(rolename2)))
self.assertEqual(set([]),
set(tuf.roledb.get_delegated_rolenames(rolename3)))
self.assertEqual(set([]),
set(tuf.roledb.get_delegated_rolenames(rolename4)))
# Verify that the delegated rolenames of a role in a non-default
# repository can be accessed.
repository_name = 'example_repository'
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.get_delegated_rolenames,
rolename, repository_name)
tuf.roledb.create_roledb(repository_name)
tuf.roledb.add_role(rolename, roleinfo, repository_name)
self.assertEqual(set(['django', 'tuf']),
set(tuf.roledb.get_delegated_rolenames(rolename, repository_name)))
# Reset the roledb so that subsequent tests have access to the original,
# default roledb.
tuf.roledb.remove_roledb(repository_name)
# Test conditions where the arguments are improperly formatted,
# contain invalid names, or haven't been added to the role database.
self._test_rolename(tuf.roledb.get_delegated_rolenames)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.get_delegated_rolenames, rolename, 123)
def test_create_roledb_from_root_metadata(self):
# Test condition using a valid 'root_metadata' argument.
rsakey = KEYS[0]
keyid = KEYS[0]['keyid']
rsakey2 = KEYS[1]
keyid2 = KEYS[1]['keyid']
rsakey3 = KEYS[2]
keyid3 = KEYS[2]['keyid']
keydict = {keyid: rsakey, keyid2: rsakey2}
roledict = {'root': {'keyids': [keyid], 'threshold': 1},
'targets': {'keyids': [keyid2], 'threshold': 1}}
version = 8
consistent_snapshot = False
expires = '1985-10-21T01:21:00Z'
root_metadata = tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROOT_SCHEMA,
_type='root',
spec_version='1.0.0',
version=version,
expires=expires,
keys=keydict,
roles=roledict,
consistent_snapshot=consistent_snapshot)
self.assertEqual(None,
tuf.roledb.create_roledb_from_root_metadata(root_metadata))
# Ensure 'Root' and 'Targets' were added to the role database.
self.assertEqual([keyid], tuf.roledb.get_role_keyids('root'))
self.assertEqual([keyid2], tuf.roledb.get_role_keyids('targets'))
# Test that a roledb is created for a non-default repository.
repository_name = 'example_repository'
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.clear_roledb,
repository_name)
tuf.roledb.create_roledb_from_root_metadata(root_metadata, repository_name)
self.assertEqual([keyid], tuf.roledb.get_role_keyids('root', repository_name))
self.assertEqual([keyid2], tuf.roledb.get_role_keyids('targets', repository_name))
# Remove the example repository added to the roledb so that subsequent
# tests have access to an original, default roledb.
tuf.roledb.remove_roledb(repository_name)
# Test conditions for arguments with invalid formats.
self.assertRaises(securesystemslib.exceptions.FormatError,
tuf.roledb.create_roledb_from_root_metadata, None)
self.assertRaises(securesystemslib.exceptions.FormatError,
tuf.roledb.create_roledb_from_root_metadata, '')
self.assertRaises(securesystemslib.exceptions.FormatError,
tuf.roledb.create_roledb_from_root_metadata, 123)
self.assertRaises(securesystemslib.exceptions.FormatError,
tuf.roledb.create_roledb_from_root_metadata, ['123'])
self.assertRaises(securesystemslib.exceptions.FormatError,
tuf.roledb.create_roledb_from_root_metadata, {'bad': '123'})
self.assertRaises(securesystemslib.exceptions.FormatError,
tuf.roledb.create_roledb_from_root_metadata, root_metadata, 123)
# Verify that the expected roles of a Root file are properly loaded.
tuf.roledb.clear_roledb()
roledict = {'root': {'keyids': [keyid], 'threshold': 1},
'release': {'keyids': [keyid3], 'threshold': 1}}
version = 8
# Add a third key for 'release'.
keydict[keyid3] = rsakey3
# Generate 'root_metadata' to verify that 'release' and 'root' are added
# to the role database.
root_metadata = tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROOT_SCHEMA,
_type='root',
spec_version='1.0.0',
version=version,
expires=expires,
keys=keydict,
roles=roledict,
consistent_snapshot=consistent_snapshot)
self.assertEqual(None,
tuf.roledb.create_roledb_from_root_metadata(root_metadata))
# Ensure only 'root' and 'release' were added to the role database.
self.assertEqual(2, len(tuf.roledb._roledb_dict['default']))
self.assertEqual(True, tuf.roledb.role_exists('root'))
self.assertEqual(True, tuf.roledb.role_exists('release'))
def test_update_roleinfo(self):
rolename = 'targets'
roleinfo = {'keyids': ['123'], 'threshold': 1}
tuf.roledb.add_role(rolename, roleinfo)
# Test normal case.
tuf.roledb.update_roleinfo(rolename, roleinfo)
# Verify that a roleinfo can be updated for a role in a non-default
# repository.
repository_name = 'example_repository'
mark_role_as_dirty = True
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.clear_roledb, repository_name)
tuf.roledb.create_roledb(repository_name)
tuf.roledb.add_role(rolename, roleinfo, repository_name)
tuf.roledb.update_roleinfo(rolename, roleinfo, mark_role_as_dirty, repository_name)
self.assertEqual(roleinfo['keyids'], tuf.roledb.get_role_keyids(rolename, repository_name))
# Reset the roledb so that subsequent tests can access the default roledb.
tuf.roledb.remove_roledb(repository_name)
# Test for an unknown role.
self.assertRaises(tuf.exceptions.UnknownRoleError, tuf.roledb.update_roleinfo,
'unknown_rolename', roleinfo)
# Verify that a roleinfo cannot be updated to a non-existent repository
# name.
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.update_roleinfo,
'new_rolename', roleinfo, False, 'non-existent')
# Test improperly formatted arguments.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.update_roleinfo, 1, roleinfo)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.update_roleinfo, rolename, 1)
repository_name = 'example_repository'
mark_role_as_dirty = True
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.update_roleinfo, rolename,
roleinfo, 1, repository_name)
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.update_roleinfo,
rolename, mark_role_as_dirty, 123)
def test_get_dirty_roles(self):
# Verify that the dirty roles of a role are returned.
rolename = 'targets'
roleinfo1 = {'keyids': ['123'], 'threshold': 1}
tuf.roledb.add_role(rolename, roleinfo1)
roleinfo2 = {'keyids': ['123'], 'threshold': 2}
mark_role_as_dirty = True
tuf.roledb.update_roleinfo(rolename, roleinfo2, mark_role_as_dirty)
# Note: The 'default' repository is searched if the repository name is
# not given to get_dirty_roles().
self.assertEqual([rolename], tuf.roledb.get_dirty_roles())
# Verify that a list of dirty roles is returned for a non-default
# repository.
repository_name = 'example_repository'
tuf.roledb.create_roledb(repository_name)
tuf.roledb.add_role(rolename, roleinfo1, repository_name)
tuf.roledb.update_roleinfo(rolename, roleinfo2, mark_role_as_dirty, repository_name)
self.assertEqual([rolename], tuf.roledb.get_dirty_roles(repository_name))
# Verify that dirty roles are not returned for a non-existent repository.
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.get_dirty_roles, 'non-existent')
# Reset the roledb so that subsequent tests have access to a default
# roledb.
tuf.roledb.remove_roledb(repository_name)
# Test for improperly formatted argument.
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.roledb.get_dirty_roles, 123)
def test_mark_dirty(self):
# Add a dirty role to roledb.
rolename = 'targets'
roleinfo1 = {'keyids': ['123'], 'threshold': 1}
tuf.roledb.add_role(rolename, roleinfo1)
rolename2 = 'dirty_role'
roleinfo2 = {'keyids': ['123'], 'threshold': 2}
mark_role_as_dirty = True
tuf.roledb.update_roleinfo(rolename, roleinfo1, mark_role_as_dirty)
# Note: The 'default' repository is searched if the repository name is
# not given to get_dirty_roles().
self.assertEqual([rolename], tuf.roledb.get_dirty_roles())
tuf.roledb.mark_dirty(['dirty_role'])
self.assertEqual([rolename2, rolename], tuf.roledb.get_dirty_roles())
# Verify that a role cannot be marked as dirty for a non-existent
# repository.
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.mark_dirty,
['dirty_role'], 'non-existent')
def test_unmark_dirty(self):
# Add a dirty role to roledb.
rolename = 'targets'
roleinfo1 = {'keyids': ['123'], 'threshold': 1}
tuf.roledb.add_role(rolename, roleinfo1)
rolename2 = 'dirty_role'
roleinfo2 = {'keyids': ['123'], 'threshold': 2}
tuf.roledb.add_role(rolename2, roleinfo2)
mark_role_as_dirty = True
tuf.roledb.update_roleinfo(rolename, roleinfo1, mark_role_as_dirty)
# Note: The 'default' repository is searched if the repository name is
# not given to get_dirty_roles().
self.assertEqual([rolename], tuf.roledb.get_dirty_roles())
tuf.roledb.update_roleinfo(rolename2, roleinfo2, mark_role_as_dirty)
tuf.roledb.unmark_dirty(['dirty_role'])
self.assertEqual([rolename], tuf.roledb.get_dirty_roles())
tuf.roledb.unmark_dirty(['targets'])
self.assertEqual([], tuf.roledb.get_dirty_roles())
# What happens for a role that isn't dirty? unmark_dirty() should just
# log a message.
tuf.roledb.unmark_dirty(['unknown_role'])
# Verify that a role cannot be unmarked as dirty for a non-existent
# repository.
self.assertRaises(securesystemslib.exceptions.InvalidNameError, tuf.roledb.unmark_dirty,
['dirty_role'], 'non-existent')
def _test_rolename(self, test_function):
# Private function that tests the 'rolename' argument of 'test_function'
# for format, invalid name, and unknown role exceptions.
# Test conditions where the arguments are improperly formatted.
self.assertRaises(securesystemslib.exceptions.FormatError, test_function, None)
self.assertRaises(securesystemslib.exceptions.FormatError, test_function, 123)
self.assertRaises(securesystemslib.exceptions.FormatError, test_function, ['rolename'])
self.assertRaises(securesystemslib.exceptions.FormatError, test_function, {'a': 'b'})
self.assertRaises(securesystemslib.exceptions.FormatError, test_function, ('a', 'b'))
self.assertRaises(securesystemslib.exceptions.FormatError, test_function, True)
# Test condition where the 'rolename' has not been added to the role database.
self.assertRaises(tuf.exceptions.UnknownRoleError, test_function, 'badrole')
# Test conditions for invalid rolenames.
self.assertRaises(securesystemslib.exceptions.InvalidNameError, test_function, '')
self.assertRaises(securesystemslib.exceptions.InvalidNameError, test_function, ' badrole ')
self.assertRaises(securesystemslib.exceptions.InvalidNameError, test_function, '/badrole/')
def setUpModule():
# setUpModule() is called before any test cases run.
# Ensure the roledb has not been modified by a previous test, which may
# affect assumptions (i.e., empty roledb) made by the tests cases in this
# unit test.
tuf.roledb.clear_roledb()
def tearDownModule():
# tearDownModule() is called after all the tests have run.
# Ensure we clean up roledb. Courtesy is contagious, and it begins with
# test_roledb_old.py.
tuf.roledb.clear_roledb()
# Run the unit tests.
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

230
tests/test_root_versioning_integration_old.py
View file

@ -1,230 +0,0 @@
#!/usr/bin/env python
# Copyright 2016 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
test_root_versioning_integration_old.py
<Author>
Evan Cordell.
<Started>
July 21, 2016.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Test root versioning for efficient root key rotation.
"""
import os
import logging
import tempfile
import shutil
import unittest
import sys
import tuf
import tuf.log
import tuf.formats
import tuf.exceptions
import tuf.roledb
import tuf.keydb
import tuf.repository_tool as repo_tool
from tests import utils
import securesystemslib
import securesystemslib.storage
logger = logging.getLogger(__name__)
repo_tool.disable_console_log_messages()
class TestRepository(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.temporary_directory = tempfile.mkdtemp(dir=os.getcwd())
@classmethod
def tearDownClass(cls):
shutil.rmtree(cls.temporary_directory)
def tearDown(self):
tuf.roledb.clear_roledb()
tuf.keydb.clear_keydb()
def test_init(self):
# Test normal case.
storage_backend = securesystemslib.storage.FilesystemBackend()
repository = repo_tool.Repository('repository_directory/',
'metadata_directory/',
'targets_directory/',
storage_backend)
self.assertTrue(isinstance(repository.root, repo_tool.Root))
self.assertTrue(isinstance(repository.snapshot, repo_tool.Snapshot))
self.assertTrue(isinstance(repository.timestamp, repo_tool.Timestamp))
self.assertTrue(isinstance(repository.targets, repo_tool.Targets))
# Test improperly formatted arguments.
self.assertRaises(securesystemslib.exceptions.FormatError, repo_tool.Repository, 3,
'metadata_directory/', 'targets_directory', storage_backend)
self.assertRaises(securesystemslib.exceptions.FormatError, repo_tool.Repository,
'repository_directory', 3, 'targets_directory', storage_backend)
self.assertRaises(securesystemslib.exceptions.FormatError, repo_tool.Repository,
'repository_directory', 'metadata_directory', storage_backend, 3)
def test_root_role_versioning(self):
# Test root role versioning
#
# 1. Import public and private keys.
# 2. Add verification keys.
# 3. Load signing keys.
# 4. Add target files.
# 5. Perform delegation.
# 6. writeall()
#
# Copy the target files from 'tuf/tests/repository_data' so that writeall()
# has target fileinfo to include in metadata.
temporary_directory = tempfile.mkdtemp(dir=self.temporary_directory)
targets_directory = os.path.join(temporary_directory, 'repository',
repo_tool.TARGETS_DIRECTORY_NAME)
original_targets_directory = os.path.join('repository_data',
'repository', 'targets')
shutil.copytree(original_targets_directory, targets_directory)
# In this case, create_new_repository() creates the 'repository/'
# sub-directory in 'temporary_directory' if it does not exist.
repository_directory = os.path.join(temporary_directory, 'repository')
metadata_directory = os.path.join(repository_directory,
repo_tool.METADATA_STAGED_DIRECTORY_NAME)
repository = repo_tool.create_new_repository(repository_directory)
# (1) Load the public and private keys of the top-level roles, and one
# delegated role.
keystore_directory = os.path.join('repository_data', 'keystore')
# Load the public keys.
root_pubkey_path = os.path.join(keystore_directory, 'root_key.pub')
targets_pubkey_path = os.path.join(keystore_directory, 'targets_key.pub')
snapshot_pubkey_path = os.path.join(keystore_directory, 'snapshot_key.pub')
timestamp_pubkey_path = os.path.join(keystore_directory, 'timestamp_key.pub')
role1_pubkey_path = os.path.join(keystore_directory, 'delegation_key.pub')
root_pubkey = repo_tool.import_rsa_publickey_from_file(root_pubkey_path)
targets_pubkey = repo_tool.import_ed25519_publickey_from_file(targets_pubkey_path)
snapshot_pubkey = \
repo_tool.import_ed25519_publickey_from_file(snapshot_pubkey_path)
timestamp_pubkey = \
repo_tool.import_ed25519_publickey_from_file(timestamp_pubkey_path)
role1_pubkey = repo_tool.import_ed25519_publickey_from_file(role1_pubkey_path)
# Load the private keys.
root_privkey_path = os.path.join(keystore_directory, 'root_key')
targets_privkey_path = os.path.join(keystore_directory, 'targets_key')
snapshot_privkey_path = os.path.join(keystore_directory, 'snapshot_key')
timestamp_privkey_path = os.path.join(keystore_directory, 'timestamp_key')
role1_privkey_path = os.path.join(keystore_directory, 'delegation_key')
root_privkey = \
repo_tool.import_rsa_privatekey_from_file(root_privkey_path, 'password')
targets_privkey = \
repo_tool.import_ed25519_privatekey_from_file(targets_privkey_path, 'password')
snapshot_privkey = \
repo_tool.import_ed25519_privatekey_from_file(snapshot_privkey_path,
'password')
timestamp_privkey = \
repo_tool.import_ed25519_privatekey_from_file(timestamp_privkey_path,
'password')
role1_privkey = \
repo_tool.import_ed25519_privatekey_from_file(role1_privkey_path,
'password')
# (2) Add top-level verification keys.
repository.root.add_verification_key(root_pubkey)
repository.targets.add_verification_key(targets_pubkey)
repository.snapshot.add_verification_key(snapshot_pubkey)
repository.timestamp.add_verification_key(timestamp_pubkey)
# (3) Load top-level signing keys.
repository.root.load_signing_key(root_privkey)
repository.targets.load_signing_key(targets_privkey)
repository.snapshot.load_signing_key(snapshot_privkey)
repository.timestamp.load_signing_key(timestamp_privkey)
# (4) Add target files.
target1 = 'file1.txt'
target2 = 'file2.txt'
target3 = 'file3.txt'
repository.targets.add_target(target1)
repository.targets.add_target(target2)
# (5) Perform delegation.
repository.targets.delegate('role1', [role1_pubkey], [target3])
repository.targets('role1').load_signing_key(role1_privkey)
# (6) Write repository.
repository.writeall()
self.assertTrue(os.path.exists(os.path.join(metadata_directory, 'root.json')))
self.assertTrue(os.path.exists(os.path.join(metadata_directory, '1.root.json')))
# Verify that the expected metadata is written.
root_filepath = os.path.join(metadata_directory, 'root.json')
root_1_filepath = os.path.join(metadata_directory, '1.root.json')
root_2_filepath = os.path.join(metadata_directory, '2.root.json')
old_root_signable = securesystemslib.util.load_json_file(root_filepath)
root_1_signable = securesystemslib.util.load_json_file(root_1_filepath)
# Make a change to the root keys
repository.root.add_verification_key(targets_pubkey)
repository.root.load_signing_key(targets_privkey)
repository.root.threshold = 2
repository.writeall()
new_root_signable = securesystemslib.util.load_json_file(root_filepath)
root_2_signable = securesystemslib.util.load_json_file(root_2_filepath)
for role_signable in [old_root_signable, new_root_signable, root_1_signable, root_2_signable]:
# Raise 'securesystemslib.exceptions.FormatError' if 'role_signable' is an
# invalid signable.
tuf.formats.check_signable_object_format(role_signable)
# Verify contents of versioned roots
self.assertEqual(old_root_signable, root_1_signable)
self.assertEqual(new_root_signable, root_2_signable)
self.assertEqual(root_1_signable['signed']['version'], 1)
self.assertEqual(root_2_signable['signed']['version'], 2)
repository.root.remove_verification_key(root_pubkey)
repository.root.unload_signing_key(root_privkey)
repository.root.threshold = 2
# Errors, not enough signing keys to satisfy old threshold
self.assertRaises(tuf.exceptions.UnsignedMetadataError, repository.writeall)
# No error, write() ignore's root's threshold and allows it to be written
# to disk partially signed.
repository.write('root')
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

546
tests/test_sig_old.py
View file

@ -1,546 +0,0 @@
#!/usr/bin/env python
# Copyright 2012 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
test_sig_old.py
<Author>
Geremy Condra
Vladimir Diaz <vladimir.v.diaz@gmail.com>
<Started>
February 28, 2012. Based on a previous version of this module.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Test cases for sig.py.
"""
import unittest
import logging
import copy
import sys
import tuf
import tuf.log
import tuf.formats
import tuf.keydb
import tuf.roledb
import tuf.sig
import tuf.exceptions
from tests import utils
import securesystemslib
import securesystemslib.keys
logger = logging.getLogger(__name__)
# Setup the keys to use in our test cases.
KEYS = []
for _ in range(3):
KEYS.append(securesystemslib.keys.generate_rsa_key(2048))
class TestSig(unittest.TestCase):
def setUp(self):
pass
def tearDown(self):
tuf.roledb.clear_roledb()
tuf.keydb.clear_keydb()
def test_get_signature_status_no_role(self):
signable = {'signed': 'test', 'signatures': []}
# A valid, but empty signature status.
sig_status = tuf.sig.get_signature_status(signable)
self.assertTrue(tuf.formats.SIGNATURESTATUS_SCHEMA.matches(sig_status))
self.assertEqual(0, sig_status['threshold'])
self.assertEqual([], sig_status['good_sigs'])
self.assertEqual([], sig_status['bad_sigs'])
self.assertEqual([], sig_status['unknown_sigs'])
self.assertEqual([], sig_status['untrusted_sigs'])
self.assertEqual([], sig_status['unknown_signing_schemes'])
# A valid signable, but non-existent role argument.
self.assertRaises(tuf.exceptions.UnknownRoleError,
tuf.sig.get_signature_status, signable, 'unknown_role')
# Should verify we are not adding a duplicate signature
# when doing the following action. Here we know 'signable'
# has only one signature so it's okay.
signed = securesystemslib.formats.encode_canonical(signable['signed']).encode('utf-8')
signable['signatures'].append(securesystemslib.keys.create_signature(
KEYS[0], signed))
tuf.keydb.add_key(KEYS[0])
# Improperly formatted role.
self.assertRaises(securesystemslib.exceptions.FormatError,
tuf.sig.get_signature_status, signable, 1)
# Not allowed to call verify() without having specified a role.
args = (signable, None)
self.assertRaises(securesystemslib.exceptions.Error, tuf.sig.verify, *args)
# Done. Let's remove the added key(s) from the key database.
tuf.keydb.remove_key(KEYS[0]['keyid'])
def test_get_signature_status_bad_sig(self):
signable = {'signed' : 'test', 'signatures' : []}
signed = securesystemslib.formats.encode_canonical(signable['signed']).encode('utf-8')
signable['signatures'].append(securesystemslib.keys.create_signature(
KEYS[0], signed))
signable['signed'] += 'signature no longer matches signed data'
tuf.keydb.add_key(KEYS[0])
threshold = 1
roleinfo = tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROLE_SCHEMA, keyids=[KEYS[0]['keyid']], threshold=threshold)
tuf.roledb.add_role('Root', roleinfo)
sig_status = tuf.sig.get_signature_status(signable, 'Root')
self.assertEqual(1, sig_status['threshold'])
self.assertEqual([], sig_status['good_sigs'])
self.assertEqual([KEYS[0]['keyid']], sig_status['bad_sigs'])
self.assertEqual([], sig_status['unknown_sigs'])
self.assertEqual([], sig_status['untrusted_sigs'])
self.assertEqual([], sig_status['unknown_signing_schemes'])
self.assertFalse(tuf.sig.verify(signable, 'Root'))
# Done. Let's remove the added key(s) from the key database.
tuf.keydb.remove_key(KEYS[0]['keyid'])
# Remove the role.
tuf.roledb.remove_role('Root')
def test_get_signature_status_unknown_signing_scheme(self):
signable = {'signed' : 'test', 'signatures' : []}
signed = securesystemslib.formats.encode_canonical(signable['signed']).encode('utf-8')
signable['signatures'].append(securesystemslib.keys.create_signature(
KEYS[0], signed))
valid_scheme = KEYS[0]['scheme']
KEYS[0]['scheme'] = 'unknown_signing_scheme'
tuf.keydb.add_key(KEYS[0])
threshold = 1
roleinfo = tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROLE_SCHEMA, keyids=[KEYS[0]['keyid']], threshold=threshold)
tuf.roledb.add_role('root', roleinfo)
sig_status = tuf.sig.get_signature_status(signable, 'root')
self.assertEqual(1, sig_status['threshold'])
self.assertEqual([], sig_status['good_sigs'])
self.assertEqual([], sig_status['bad_sigs'])
self.assertEqual([], sig_status['unknown_sigs'])
self.assertEqual([], sig_status['untrusted_sigs'])
self.assertEqual([KEYS[0]['keyid']],
sig_status['unknown_signing_schemes'])
self.assertFalse(tuf.sig.verify(signable, 'root'))
# Done. Let's remove the added key(s) from the key database.
KEYS[0]['scheme'] = valid_scheme
tuf.keydb.remove_key(KEYS[0]['keyid'])
# Remove the role.
tuf.roledb.remove_role('root')
def test_get_signature_status_single_key(self):
signable = {'signed' : 'test', 'signatures' : []}
signed = securesystemslib.formats.encode_canonical(signable['signed']).encode('utf-8')
signable['signatures'].append(securesystemslib.keys.create_signature(
KEYS[0], signed))
threshold = 1
roleinfo = tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROLE_SCHEMA, keyids=[KEYS[0]['keyid']], threshold=threshold)
tuf.roledb.add_role('Root', roleinfo)
tuf.keydb.add_key(KEYS[0])
sig_status = tuf.sig.get_signature_status(signable, 'Root')
self.assertEqual(1, sig_status['threshold'])
self.assertEqual([KEYS[0]['keyid']], sig_status['good_sigs'])
self.assertEqual([], sig_status['bad_sigs'])
self.assertEqual([], sig_status['unknown_sigs'])
self.assertEqual([], sig_status['untrusted_sigs'])
self.assertEqual([], sig_status['unknown_signing_schemes'])
self.assertTrue(tuf.sig.verify(signable, 'Root'))
# Test for an unknown signature when 'role' is left unspecified.
sig_status = tuf.sig.get_signature_status(signable)
self.assertEqual(0, sig_status['threshold'])
self.assertEqual([], sig_status['good_sigs'])
self.assertEqual([], sig_status['bad_sigs'])
self.assertEqual([KEYS[0]['keyid']], sig_status['unknown_sigs'])
self.assertEqual([], sig_status['untrusted_sigs'])
self.assertEqual([], sig_status['unknown_signing_schemes'])
# Done. Let's remove the added key(s) from the key database.
tuf.keydb.remove_key(KEYS[0]['keyid'])
# Remove the role.
tuf.roledb.remove_role('Root')
def test_get_signature_status_below_threshold(self):
signable = {'signed' : 'test', 'signatures' : []}
signed = securesystemslib.formats.encode_canonical(signable['signed']).encode('utf-8')
signable['signatures'].append(securesystemslib.keys.create_signature(
KEYS[0], signed))
tuf.keydb.add_key(KEYS[0])
threshold = 2
roleinfo = tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROLE_SCHEMA,
keyids=[KEYS[0]['keyid'], KEYS[2]['keyid']],
threshold=threshold)
tuf.roledb.add_role('Root', roleinfo)
sig_status = tuf.sig.get_signature_status(signable, 'Root')
self.assertEqual(2, sig_status['threshold'])
self.assertEqual([KEYS[0]['keyid']], sig_status['good_sigs'])
self.assertEqual([], sig_status['bad_sigs'])
self.assertEqual([], sig_status['unknown_sigs'])
self.assertEqual([], sig_status['untrusted_sigs'])
self.assertEqual([], sig_status['unknown_signing_schemes'])
self.assertFalse(tuf.sig.verify(signable, 'Root'))
# Done. Let's remove the added key(s) from the key database.
tuf.keydb.remove_key(KEYS[0]['keyid'])
# Remove the role.
tuf.roledb.remove_role('Root')
def test_get_signature_status_below_threshold_unrecognized_sigs(self):
signable = {'signed' : 'test', 'signatures' : []}
signed = securesystemslib.formats.encode_canonical(signable['signed']).encode('utf-8')
# Two keys sign it, but only one of them will be trusted.
signable['signatures'].append(securesystemslib.keys.create_signature(
KEYS[0], signed))
signable['signatures'].append(securesystemslib.keys.create_signature(
KEYS[2], signed))
tuf.keydb.add_key(KEYS[0])
tuf.keydb.add_key(KEYS[1])
threshold = 2
roleinfo = tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROLE_SCHEMA,
keyids=[KEYS[0]['keyid'], KEYS[1]['keyid']],
threshold=threshold)
tuf.roledb.add_role('Root', roleinfo)
sig_status = tuf.sig.get_signature_status(signable, 'Root')
self.assertEqual(2, sig_status['threshold'])
self.assertEqual([KEYS[0]['keyid']], sig_status['good_sigs'])
self.assertEqual([], sig_status['bad_sigs'])
self.assertEqual([KEYS[2]['keyid']], sig_status['unknown_sigs'])
self.assertEqual([], sig_status['untrusted_sigs'])
self.assertEqual([], sig_status['unknown_signing_schemes'])
self.assertFalse(tuf.sig.verify(signable, 'Root'))
# Done. Let's remove the added key(s) from the key database.
tuf.keydb.remove_key(KEYS[0]['keyid'])
tuf.keydb.remove_key(KEYS[1]['keyid'])
# Remove the role.
tuf.roledb.remove_role('Root')
def test_get_signature_status_below_threshold_unauthorized_sigs(self):
signable = {'signed' : 'test', 'signatures' : []}
signed = securesystemslib.formats.encode_canonical(signable['signed']).encode('utf-8')
# Two keys sign it, but one of them is only trusted for a different
# role.
signable['signatures'].append(securesystemslib.keys.create_signature(
KEYS[0], signed))
signable['signatures'].append(securesystemslib.keys.create_signature(
KEYS[1], signed))
tuf.keydb.add_key(KEYS[0])
tuf.keydb.add_key(KEYS[1])
threshold = 2
roleinfo = tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROLE_SCHEMA,
keyids=[KEYS[0]['keyid'], KEYS[2]['keyid']],
threshold=threshold)
tuf.roledb.add_role('Root', roleinfo)
roleinfo = tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROLE_SCHEMA,
keyids=[KEYS[1]['keyid'], KEYS[2]['keyid']],
threshold=threshold)
tuf.roledb.add_role('Release', roleinfo)
sig_status = tuf.sig.get_signature_status(signable, 'Root')
self.assertEqual(2, sig_status['threshold'])
self.assertEqual([KEYS[0]['keyid']], sig_status['good_sigs'])
self.assertEqual([], sig_status['bad_sigs'])
self.assertEqual([], sig_status['unknown_sigs'])
self.assertEqual([KEYS[1]['keyid']], sig_status['untrusted_sigs'])
self.assertEqual([], sig_status['unknown_signing_schemes'])
self.assertFalse(tuf.sig.verify(signable, 'Root'))
self.assertRaises(tuf.exceptions.UnknownRoleError,
tuf.sig.get_signature_status, signable, 'unknown_role')
# Done. Let's remove the added key(s) from the key database.
tuf.keydb.remove_key(KEYS[0]['keyid'])
tuf.keydb.remove_key(KEYS[1]['keyid'])
# Remove the roles.
tuf.roledb.remove_role('Root')
tuf.roledb.remove_role('Release')
def test_check_signatures_no_role(self):
signable = {'signed' : 'test', 'signatures' : []}
signed = securesystemslib.formats.encode_canonical(signable['signed']).encode('utf-8')
signable['signatures'].append(securesystemslib.keys.create_signature(
KEYS[0], signed))
tuf.keydb.add_key(KEYS[0])
# No specific role we're considering. It's invalid to use the
# function tuf.sig.verify() without a role specified because
# tuf.sig.verify() is checking trust, as well.
args = (signable, None)
self.assertRaises(securesystemslib.exceptions.Error, tuf.sig.verify, *args)
# Done. Let's remove the added key(s) from the key database.
tuf.keydb.remove_key(KEYS[0]['keyid'])
def test_verify_single_key(self):
signable = {'signed' : 'test', 'signatures' : []}
signed = securesystemslib.formats.encode_canonical(signable['signed']).encode('utf-8')
signable['signatures'].append(securesystemslib.keys.create_signature(
KEYS[0], signed))
tuf.keydb.add_key(KEYS[0])
threshold = 1
roleinfo = tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROLE_SCHEMA, keyids=[KEYS[0]['keyid']], threshold=threshold)
tuf.roledb.add_role('Root', roleinfo)
# This will call verify() and return True if 'signable' is valid,
# False otherwise.
self.assertTrue(tuf.sig.verify(signable, 'Root'))
# Done. Let's remove the added key(s) from the key database.
tuf.keydb.remove_key(KEYS[0]['keyid'])
# Remove the roles.
tuf.roledb.remove_role('Root')
def test_verify_must_not_count_duplicate_keyids_towards_threshold(self):
# Create and sign dummy metadata twice with same key
# Note that we use the non-deterministic rsassa-pss signing scheme, so
# creating the signature twice shows that we don't only detect duplicate
# signatures but also different signatures from the same key.
signable = {"signed" : "test", "signatures" : []}
signed = securesystemslib.formats.encode_canonical(
signable["signed"]).encode("utf-8")
signable["signatures"].append(
securesystemslib.keys.create_signature(KEYS[0], signed))
signable["signatures"].append(
securesystemslib.keys.create_signature(KEYS[0], signed))
# 'get_signature_status' uses keys from keydb for verification
tuf.keydb.add_key(KEYS[0])
# Assert that 'get_signature_status' returns two good signatures ...
status = tuf.sig.get_signature_status(
signable, "root", keyids=[KEYS[0]["keyid"]], threshold=2)
self.assertTrue(len(status["good_sigs"]) == 2)
# ... but only one counts towards the threshold
self.assertFalse(
tuf.sig.verify(signable, "root", keyids=[KEYS[0]["keyid"]], threshold=2))
# Clean-up keydb
tuf.keydb.remove_key(KEYS[0]["keyid"])
def test_verify_count_different_keyids_for_same_key_towards_threshold(self):
# Create and sign dummy metadata twice with same key but different keyids
signable = {"signed" : "test", "signatures" : []}
key_sha256 = copy.deepcopy(KEYS[0])
key_sha256["keyid"] = "deadbeef256"
key_sha512 = copy.deepcopy(KEYS[0])
key_sha512["keyid"] = "deadbeef512"
signed = securesystemslib.formats.encode_canonical(
signable["signed"]).encode("utf-8")
signable["signatures"].append(
securesystemslib.keys.create_signature(key_sha256, signed))
signable["signatures"].append(
securesystemslib.keys.create_signature(key_sha512, signed))
# 'get_signature_status' uses keys from keydb for verification
tuf.keydb.add_key(key_sha256)
tuf.keydb.add_key(key_sha512)
# Assert that the key only counts toward the threshold once
keyids = [key_sha256["keyid"], key_sha512["keyid"]]
self.assertFalse(
tuf.sig.verify(signable, "root", keyids=keyids, threshold=2))
# Clean-up keydb
tuf.keydb.remove_key(key_sha256["keyid"])
tuf.keydb.remove_key(key_sha512["keyid"])
def test_verify_unrecognized_sig(self):
signable = {'signed' : 'test', 'signatures' : []}
signed = securesystemslib.formats.encode_canonical(signable['signed']).encode('utf-8')
# Two keys sign it, but only one of them will be trusted.
signable['signatures'].append(securesystemslib.keys.create_signature(
KEYS[0], signed))
signable['signatures'].append(securesystemslib.keys.create_signature(
KEYS[2], signed))
tuf.keydb.add_key(KEYS[0])
tuf.keydb.add_key(KEYS[1])
threshold = 2
roleinfo = tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROLE_SCHEMA,
keyids=[KEYS[0]['keyid'], KEYS[1]['keyid']],
threshold=threshold)
tuf.roledb.add_role('Root', roleinfo)
self.assertFalse(tuf.sig.verify(signable, 'Root'))
# Done. Let's remove the added key(s) from the key database.
tuf.keydb.remove_key(KEYS[0]['keyid'])
tuf.keydb.remove_key(KEYS[1]['keyid'])
# Remove the roles.
tuf.roledb.remove_role('Root')
def test_generate_rsa_signature(self):
signable = {'signed' : 'test', 'signatures' : []}
signed = securesystemslib.formats.encode_canonical(signable['signed']).encode('utf-8')
signable['signatures'].append(securesystemslib.keys.create_signature(
KEYS[0], signed))
self.assertEqual(1, len(signable['signatures']))
signature = signable['signatures'][0]
self.assertEqual(KEYS[0]['keyid'], signature['keyid'])
returned_signature = tuf.sig.generate_rsa_signature(signable['signed'], KEYS[0])
self.assertTrue(securesystemslib.formats.SIGNATURE_SCHEMA.matches(returned_signature))
signable['signatures'].append(securesystemslib.keys.create_signature(
KEYS[1], signed))
self.assertEqual(2, len(signable['signatures']))
signature = signable['signatures'][1]
self.assertEqual(KEYS[1]['keyid'], signature['keyid'])
def test_may_need_new_keys(self):
# One untrusted key in 'signable'.
signable = {'signed' : 'test', 'signatures' : []}
signed = securesystemslib.formats.encode_canonical(signable['signed']).encode('utf-8')
signable['signatures'].append(securesystemslib.keys.create_signature(
KEYS[0], signed))
tuf.keydb.add_key(KEYS[1])
threshold = 1
roleinfo = tuf.formats.build_dict_conforming_to_schema(
tuf.formats.ROLE_SCHEMA, keyids=[KEYS[1]['keyid']], threshold=threshold)
tuf.roledb.add_role('Root', roleinfo)
sig_status = tuf.sig.get_signature_status(signable, 'Root')
self.assertTrue(tuf.sig.may_need_new_keys(sig_status))
# Done. Let's remove the added key(s) from the key database.
tuf.keydb.remove_key(KEYS[1]['keyid'])
# Remove the roles.
tuf.roledb.remove_role('Root')
def test_signable_has_invalid_format(self):
# get_signature_status() and verify() validate 'signable' before continuing.
# 'signable' must be of the form: {'signed': , 'signatures': [{}]}.
# Object types are checked as well.
signable = {'not_signed' : 'test', 'signatures' : []}
args = (signable['not_signed'], KEYS[0])
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.sig.get_signature_status, *args)
# 'signatures' value must be a list. Let's try a dict.
signable = {'signed' : 'test', 'signatures' : {}}
args = (signable['signed'], KEYS[0])
self.assertRaises(securesystemslib.exceptions.FormatError, tuf.sig.get_signature_status, *args)
# Run unit test.
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

216
tests/test_slow_retrieval_attack_old.py
View file

@ -1,216 +0,0 @@
#!/usr/bin/env python
# Copyright 2012 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
test_slow_retrieval_attack_old.py
<Author>
Konstantin Andrianov.
<Started>
March 13, 2012.
April 5, 2014.
Refactored to use the 'unittest' module (test conditions in code, rather
than verifying text output), use pre-generated repository files, and
discontinue use of the old repository tools. Expanded comments and modified
previous setup. -vladimir.v.diaz
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Simulate a slow retrieval attack, where an attacker is able to prevent clients
from receiving updates by responding to client requests so slowly that updates
never complete. Test cases included for two types of slow retrievals: data
that slowly trickles in, and data that is only returned after a long time
delay. TUF prevents slow retrieval attacks by ensuring the download rate
does not fall below a required rate (tuf.settings.MIN_AVERAGE_DOWNLOAD_SPEED).
Note: There is no difference between 'updates' and 'target' files.
# TODO: Consider additional tests for slow metadata download. Tests here only
use slow target download.
"""
import os
import tempfile
import shutil
import logging
import unittest
import sys
import tuf.log
import tuf.client.updater as updater
import tuf.unittest_toolbox as unittest_toolbox
import tuf.repository_tool as repo_tool
import tuf.roledb
import tuf.keydb
from tests import utils
logger = logging.getLogger(__name__)
repo_tool.disable_console_log_messages()
class TestSlowRetrieval(unittest_toolbox.Modified_TestCase):
def setUp(self):
# Modified_Testcase can handle temp dir removal
unittest_toolbox.Modified_TestCase.setUp(self)
self.temporary_directory = self.make_temp_directory(directory=os.getcwd())
self.repository_name = 'test_repository1'
# Copy the original repository files provided in the test folder so that
# any modifications made to repository files are restricted to the copies.
# The 'repository_data' directory is expected to exist in 'tuf/tests/'.
original_repository_files = os.path.join(os.getcwd(), 'repository_data')
temporary_repository_root = tempfile.mkdtemp(dir=self.temporary_directory)
# The original repository, keystore, and client directories will be copied
# for each test case.
original_repository = os.path.join(original_repository_files, 'repository')
original_client = os.path.join(original_repository_files, 'client')
original_keystore = os.path.join(original_repository_files, 'keystore')
# Save references to the often-needed client repository directories.
# Test cases need these references to access metadata and target files.
self.repository_directory = \
os.path.join(temporary_repository_root, 'repository')
self.client_directory = os.path.join(temporary_repository_root, 'client')
self.keystore_directory = os.path.join(temporary_repository_root, 'keystore')
# Copy the original 'repository', 'client', and 'keystore' directories
# to the temporary repository the test cases can use.
shutil.copytree(original_repository, self.repository_directory)
shutil.copytree(original_client, self.client_directory)
shutil.copytree(original_keystore, self.keystore_directory)
# Produce a longer target file than exists in the other test repository
# data, to provide for a long-duration slow attack. Then we'll write new
# top-level metadata that includes a hash over that file, and provide that
# metadata to the client as well.
# The slow retrieval server, in mode 2 (1 byte per second), will only
# sleep for a total of (target file size) seconds. Add a target file
# that contains sufficient number of bytes to trigger a slow retrieval
# error. A transfer should not be permitted to take 1 second per byte
# transferred. Because this test is currently expected to fail, I'm
# limiting the size to 10 bytes (10 seconds) to avoid expected testing
# delays.... Consider increasing again after fix, to, e.g. 400.
total_bytes = 10
repository = repo_tool.load_repository(self.repository_directory)
file1_filepath = os.path.join(self.repository_directory, 'targets',
'file1.txt')
with open(file1_filepath, 'wb') as file_object:
data = 'a' * int(round(total_bytes))
file_object.write(data.encode('utf-8'))
key_file = os.path.join(self.keystore_directory, 'timestamp_key')
timestamp_private = repo_tool.import_ed25519_privatekey_from_file(key_file,
'password')
key_file = os.path.join(self.keystore_directory, 'snapshot_key')
snapshot_private = repo_tool.import_ed25519_privatekey_from_file(key_file,
'password')
key_file = os.path.join(self.keystore_directory, 'targets_key')
targets_private = repo_tool.import_ed25519_privatekey_from_file(key_file,
'password')
repository.targets.load_signing_key(targets_private)
repository.snapshot.load_signing_key(snapshot_private)
repository.timestamp.load_signing_key(timestamp_private)
repository.writeall()
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# Since we've changed the repository metadata in this setup (by lengthening
# a target file and then writing new metadata), we also have to update the
# client metadata to get to the expected initial state, where the client
# knows the right target info (and so expects the right, longer target
# length.
# We'll skip using updater.refresh since we don't have a server running,
# and we'll update the metadata locally, manually.
shutil.rmtree(os.path.join(
self.client_directory, self.repository_name, 'metadata', 'current'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata'),
os.path.join(self.client_directory, self.repository_name, 'metadata',
'current'))
# Set the url prefix required by the 'tuf/client/updater.py' updater.
# 'path/to/tmp/repository' -> 'localhost:8001/tmp/repository'.
repository_basepath = self.repository_directory[len(os.getcwd()):]
self.server_process_handler = utils.TestServerProcess(log=logger,
server='slow_retrieval_server_old.py')
logger.info('Slow Retrieval Server process started.')
url_prefix = 'http://' + utils.TEST_HOST_ADDRESS + ':' \
+ str(self.server_process_handler.port) + repository_basepath
# Setting 'tuf.settings.repository_directory' with the temporary client
# directory copied from the original repository files.
tuf.settings.repositories_directory = self.client_directory
self.repository_mirrors = {'mirror1': {'url_prefix': url_prefix,
'metadata_path': 'metadata',
'targets_path': 'targets'}}
# Create the repository instance. The test cases will use this client
# updater to refresh metadata, fetch target files, etc.
self.repository_updater = updater.Updater(self.repository_name,
self.repository_mirrors)
def tearDown(self):
tuf.roledb.clear_roledb(clear_all=True)
tuf.keydb.clear_keydb(clear_all=True)
# Cleans the resources and flush the logged lines (if any).
self.server_process_handler.clean()
# Remove temporary directory
unittest_toolbox.Modified_TestCase.tearDown(self)
def test_delay_before_send(self):
# Simulate a slow retrieval attack.
# When download begins,the server blocks the download for a long
# time by doing nothing before it sends the first byte of data.
# Verify that the TUF client detects replayed metadata and refuses to
# continue the update process.
try:
file1_target = self.repository_updater.get_one_valid_targetinfo('file1.txt')
self.repository_updater.download_target(file1_target, self.client_directory)
# Verify that the specific 'tuf.exceptions.SlowRetrievalError' exception is raised by
# each mirror.
except tuf.exceptions.NoWorkingMirrorError as exception:
for mirror_url, mirror_error in exception.mirror_errors.items():
url_prefix = self.repository_mirrors['mirror1']['url_prefix']
url_file = os.path.join(url_prefix, 'targets', 'file1.txt')
# Verify that 'file1.txt' is the culprit.
self.assertEqual(url_file.replace('\\', '/'), mirror_url)
self.assertTrue(isinstance(mirror_error, tuf.exceptions.SlowRetrievalError))
else:
self.fail('TUF did not prevent a slow retrieval attack.')
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

407
tests/test_tutorial_old.py
View file

@ -1,407 +0,0 @@
#!/usr/bin/env python
"""
<Program Name>
test_tutorial_old.py
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Regression test for the TUF tutorial as laid out in TUTORIAL.md.
This essentially runs the tutorial and checks some results.
There are a few deviations from the TUTORIAL.md instructions:
- steps that involve user input (like passphrases) are modified slightly
to not require user input
- use of path separators '/' is replaced by join() calls. (We assume that
when following the tutorial, users will correctly deal with path
separators for their system if they happen to be using non-Linux systems.)
- shell instructions are mimicked using Python commands
"""
import unittest
import datetime # part of TUTORIAL.md
import os # part of TUTORIAL.md, but also needed separately
import shutil
import tempfile
import sys
import unittest.mock as mock
from tuf.repository_tool import * # part of TUTORIAL.md
from tests import utils
import securesystemslib.exceptions
from securesystemslib.formats import encode_canonical # part of TUTORIAL.md
from securesystemslib.keys import create_signature # part of TUTORIAL.md
class TestTutorial(unittest.TestCase):
def setUp(self):
self.working_dir = os.getcwd()
self.test_dir = os.path.realpath(tempfile.mkdtemp())
os.chdir(self.test_dir)
def tearDown(self):
os.chdir(self.working_dir)
shutil.rmtree(self.test_dir)
def test_tutorial(self):
"""
Run the TUTORIAL.md tutorial.
Note that anywhere the tutorial provides a command that prompts for the
user to enter a passphrase/password, this test is changed to simply provide
that as an argument. It's not worth trying to arrange automated testing of
the interactive password entry process here. Anywhere user entry has been
skipped from the tutorial instructions, "# Skipping user entry of password"
is written, with the original line below it, starting with ##.
"""
# ----- Tutorial Section: Keys
generate_and_write_rsa_keypair(password='password', filepath='root_key', bits=2048)
# Skipping user entry of password
## generate_and_write_rsa_keypair_with_prompt('root_key2')
generate_and_write_rsa_keypair(password='password', filepath='root_key2')
# Tutorial tells users to expect these files to exist:
# ['root_key', 'root_key.pub', 'root_key2', 'root_key2.pub']
for fname in ['root_key', 'root_key.pub', 'root_key2', 'root_key2.pub']:
self.assertTrue(os.path.exists(fname))
# Generate key pair at /path/to/KEYID
fname = generate_and_write_rsa_keypair(password="password")
self.assertTrue(os.path.exists(fname))
# ----- Tutorial Section: Import RSA Keys
public_root_key = import_rsa_publickey_from_file('root_key.pub')
# Skipping user entry of password
## private_root_key = import_rsa_privatekey_from_file('root_key')
private_root_key = import_rsa_privatekey_from_file('root_key', 'password')
# Skipping user entry of password
## import_rsa_privatekey_from_file('root_key')
with self.assertRaises(securesystemslib.exceptions.CryptoError):
import_rsa_privatekey_from_file('root_key', 'not_the_real_pw')
# ----- Tutorial Section: Create and Import Ed25519 Keys
# Skipping user entry of password
## generate_and_write_ed25519_keypair_with_prompt('ed25519_key')
generate_and_write_ed25519_keypair(password='password', filepath='ed25519_key')
public_ed25519_key = import_ed25519_publickey_from_file('ed25519_key.pub')
# Skipping user entry of password
## private_ed25519_key = import_ed25519_privatekey_from_file('ed25519_key')
private_ed25519_key = import_ed25519_privatekey_from_file(
'ed25519_key', 'password')
# ----- Tutorial Section: Create Top-level Metadata
repository = create_new_repository('repository')
repository.root.add_verification_key(public_root_key)
self.assertTrue(repository.root.keys)
public_root_key2 = import_rsa_publickey_from_file('root_key2.pub')
repository.root.add_verification_key(public_root_key2)
repository.root.threshold = 2
private_root_key2 = import_rsa_privatekey_from_file(
'root_key2', password='password')
repository.root.load_signing_key(private_root_key)
repository.root.load_signing_key(private_root_key2)
# NOTE: The tutorial does not call dirty_roles anymore due to #964 and
# #958. We still call it here to see if roles are dirty as expected.
with mock.patch("tuf.repository_tool.logger") as mock_logger:
repository.dirty_roles()
# Concat strings to avoid Python2/3 unicode prefix problems ('' vs. u'')
mock_logger.info.assert_called_with("Dirty roles: " + str(['root']))
# Patch logger to assert that it accurately logs the repo's status. Since
# the logger is called multiple times, we have to assert for the accurate
# sequence of calls or rather its call arguments.
with mock.patch("tuf.repository_lib.logger") as mock_logger:
repository.status()
# Concat strings to avoid Python2/3 unicode prefix problems ('' vs. u'')
self.assertListEqual([
repr('targets') + " role contains 0 / 1 public keys.",
repr('snapshot') + " role contains 0 / 1 public keys.",
repr('timestamp') + " role contains 0 / 1 public keys.",
repr('root') + " role contains 2 / 2 signatures.",
repr('targets') + " role contains 0 / 1 signatures."
], [args[0] for args, _ in mock_logger.info.call_args_list])
generate_and_write_rsa_keypair(password='password', filepath='targets_key')
generate_and_write_rsa_keypair(password='password', filepath='snapshot_key')
generate_and_write_rsa_keypair(password='password', filepath='timestamp_key')
repository.targets.add_verification_key(import_rsa_publickey_from_file(
'targets_key.pub'))
repository.snapshot.add_verification_key(import_rsa_publickey_from_file(
'snapshot_key.pub'))
repository.timestamp.add_verification_key(import_rsa_publickey_from_file(
'timestamp_key.pub'))
# Skipping user entry of password
## private_targets_key = import_rsa_privatekey_from_file('targets_key')
private_targets_key = import_rsa_privatekey_from_file(
'targets_key', 'password')
# Skipping user entry of password
## private_snapshot_key = import_rsa_privatekey_from_file('snapshot_key')
private_snapshot_key = import_rsa_privatekey_from_file(
'snapshot_key', 'password')
# Skipping user entry of password
## private_timestamp_key = import_rsa_privatekey_from_file('timestamp_key')
private_timestamp_key = import_rsa_privatekey_from_file(
'timestamp_key', 'password')
repository.targets.load_signing_key(private_targets_key)
repository.snapshot.load_signing_key(private_snapshot_key)
repository.timestamp.load_signing_key(private_timestamp_key)
repository.timestamp.expiration = datetime.datetime(2080, 10, 28, 12, 8)
# NOTE: The tutorial does not call dirty_roles anymore due to #964 and
# #958. We still call it here to see if roles are dirty as expected.
with mock.patch("tuf.repository_tool.logger") as mock_logger:
repository.dirty_roles()
# Concat strings to avoid Python2/3 unicode prefix problems ('' vs. u'')
mock_logger.info.assert_called_with("Dirty roles: " +
str(['root', 'snapshot', 'targets', 'timestamp']))
repository.writeall()
# ----- Tutorial Section: Targets
# These next commands in the tutorial are shown as bash commands, so I'll
# just simulate this with some Python commands.
## $ cd repository/targets/
## $ echo 'file1' > file1.txt
## $ echo 'file2' > file2.txt
## $ echo 'file3' > file3.txt
## $ mkdir myproject; echo 'file4' > myproject/file4.txt
## $ cd ../../
with open(os.path.join('repository', 'targets', 'file1.txt'), 'w') as fobj:
fobj.write('file1')
with open(os.path.join('repository', 'targets', 'file2.txt'), 'w') as fobj:
fobj.write('file2')
with open(os.path.join('repository', 'targets', 'file3.txt'), 'w') as fobj:
fobj.write('file3')
os.mkdir(os.path.join('repository', 'targets', 'myproject'))
with open(os.path.join('repository', 'targets', 'myproject', 'file4.txt'),
'w') as fobj:
fobj.write('file4')
repository = load_repository('repository')
# TODO: replace the hard-coded list of targets with a helper
# method that returns a list of normalized relative target paths
list_of_targets = ['file1.txt', 'file2.txt', 'file3.txt']
repository.targets.add_targets(list_of_targets)
self.assertTrue('file1.txt' in repository.targets.target_files)
self.assertTrue('file2.txt' in repository.targets.target_files)
self.assertTrue('file3.txt' in repository.targets.target_files)
target4_filepath = 'myproject/file4.txt'
target4_abspath = os.path.abspath(os.path.join(
'repository', 'targets', target4_filepath))
octal_file_permissions = oct(os.stat(target4_abspath).st_mode)[4:]
custom_file_permissions = {'file_permissions': octal_file_permissions}
repository.targets.add_target(target4_filepath, custom_file_permissions)
# Note that target filepaths specified in the repo use '/' even on Windows.
# (This is important to make metadata platform-independent.)
self.assertTrue(
os.path.join(target4_filepath) in repository.targets.target_files)
# Skipping user entry of password
## private_targets_key = import_rsa_privatekey_from_file('targets_key')
private_targets_key = import_rsa_privatekey_from_file(
'targets_key', 'password')
repository.targets.load_signing_key(private_targets_key)
# Skipping user entry of password
## private_snapshot_key = import_rsa_privatekey_from_file('snapshot_key')
private_snapshot_key = import_rsa_privatekey_from_file(
'snapshot_key', 'password')
repository.snapshot.load_signing_key(private_snapshot_key)
# Skipping user entry of password
## private_timestamp_key = import_rsa_privatekey_from_file('timestamp_key')
private_timestamp_key = import_rsa_privatekey_from_file(
'timestamp_key', 'password')
repository.timestamp.load_signing_key(private_timestamp_key)
# NOTE: The tutorial does not call dirty_roles anymore due to #964 and
# #958. We still call it here to see if roles are dirty as expected.
with mock.patch("tuf.repository_tool.logger") as mock_logger:
repository.dirty_roles()
# Concat strings to avoid Python2/3 unicode prefix problems ('' vs. u'')
mock_logger.info.assert_called_with(
"Dirty roles: " + str(['snapshot', 'targets', 'timestamp']))
repository.writeall()
repository.targets.remove_target('myproject/file4.txt')
self.assertTrue(os.path.exists(os.path.join(
'repository','targets', 'myproject', 'file4.txt')))
# NOTE: The tutorial does not call dirty_roles anymore due to #964 and
# #958. We still call it here to see if roles are dirty as expected.
with mock.patch("tuf.repository_tool.logger") as mock_logger:
repository.dirty_roles()
# Concat strings to avoid Python2/3 unicode prefix problems ('' vs. u'')
mock_logger.info.assert_called_with(
"Dirty roles: " + str(['targets']))
repository.mark_dirty(['snapshot', 'timestamp'])
repository.writeall()
# ----- Tutorial Section: Excursion: Dump Metadata and Append Signature
signable_content = dump_signable_metadata(
os.path.join('repository', 'metadata.staged', 'timestamp.json'))
# Skipping user entry of password
## private_ed25519_key = import_ed25519_privatekey_from_file('ed25519_key')
private_ed25519_key = import_ed25519_privatekey_from_file('ed25519_key', 'password')
signature = create_signature(
private_ed25519_key, encode_canonical(signable_content).encode())
append_signature(
signature,
os.path.join('repository', 'metadata.staged', 'timestamp.json'))
# ----- Tutorial Section: Delegations
generate_and_write_rsa_keypair(
password='password', filepath='unclaimed_key', bits=2048)
public_unclaimed_key = import_rsa_publickey_from_file('unclaimed_key.pub')
repository.targets.delegate(
'unclaimed', [public_unclaimed_key], ['myproject/*.txt'])
repository.targets("unclaimed").add_target("myproject/file4.txt")
# Skipping user entry of password
## private_unclaimed_key = import_rsa_privatekey_from_file('unclaimed_key')
private_unclaimed_key = import_rsa_privatekey_from_file(
'unclaimed_key', 'password')
repository.targets("unclaimed").load_signing_key(private_unclaimed_key)
# NOTE: The tutorial does not call dirty_roles anymore due to #964 and
# #958. We still call it here to see if roles are dirty as expected.
with mock.patch("tuf.repository_tool.logger") as mock_logger:
repository.dirty_roles()
# Concat strings to avoid Python2/3 unicode prefix problems ('' vs. u'')
mock_logger.info.assert_called_with(
"Dirty roles: " + str(['targets', 'unclaimed']))
repository.mark_dirty(["snapshot", "timestamp"])
repository.writeall()
# Simulate the following shell command:
## $ cp -r "repository/metadata.staged/" "repository/metadata/"
shutil.copytree(
os.path.join('repository', 'metadata.staged'),
os.path.join('repository', 'metadata'))
# ----- Tutorial Section: Delegate to Hashed Bins
repository.targets('unclaimed').remove_target("myproject/file4.txt")
targets = ['myproject/file4.txt']
# Patch logger to assert that it accurately logs the output of hashed bin
# delegation. The logger is called multiple times, first with info level
# then with warning level. So we have to assert for the accurate sequence
# of calls or rather its call arguments.
with mock.patch("tuf.repository_tool.logger") as mock_logger:
repository.targets('unclaimed').delegate_hashed_bins(
targets, [public_unclaimed_key], 32)
self.assertListEqual([
"Creating hashed bin delegations.\n"
"1 total targets.\n"
"32 hashed bins.\n"
"256 total hash prefixes.\n"
"Each bin ranges over 8 hash prefixes."
] + ["Adding a verification key that has already been used."] * 32,
[
args[0] for args, _ in
mock_logger.info.call_args_list + mock_logger.warning.call_args_list
])
for delegation in repository.targets('unclaimed').delegations:
delegation.load_signing_key(private_unclaimed_key)
# NOTE: The tutorial does not call dirty_roles anymore due to #964 and
# #958. We still call it here to see if roles are dirty as expected.
with mock.patch("tuf.repository_tool.logger") as mock_logger:
repository.dirty_roles()
# Concat strings to avoid Python2/3 unicode prefix problems ('' vs. u'')
mock_logger.info.assert_called_with(
"Dirty roles: " + str(['00-07', '08-0f', '10-17', '18-1f', '20-27',
'28-2f', '30-37', '38-3f', '40-47', '48-4f', '50-57', '58-5f',
'60-67', '68-6f', '70-77', '78-7f', '80-87', '88-8f', '90-97',
'98-9f', 'a0-a7', 'a8-af', 'b0-b7', 'b8-bf', 'c0-c7', 'c8-cf',
'd0-d7', 'd8-df', 'e0-e7', 'e8-ef', 'f0-f7', 'f8-ff', 'unclaimed']))
repository.mark_dirty(["snapshot", "timestamp"])
repository.writeall()
# ----- Tutorial Section: How to Perform an Update
# A separate tutorial is linked to for client use. That is not tested here.
create_tuf_client_directory("repository/", "client/tufrepo/")
# ----- Tutorial Section: Test TUF Locally
# TODO: Run subprocess to simulate the following bash instructions:
# $ cd "repository/"; python3 -m http.server 8001
# We next retrieve targets from the TUF repository and save them to client/. The client.py script is available to download metadata and files from a specified repository. In a different command-line prompt . . .
# $ cd "client/"
# $ ls
# metadata/
# $ client.py --repo http://localhost:8001 file1.txt
# $ ls . targets/
# .:
# metadata targets
# targets/:
# file1.txt
# Run unit test.
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

54
tests/test_unittest_toolbox_old.py
View file

@ -1,54 +0,0 @@
#!/usr/bin/env python
# Copyright 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
test_unittest_toolbox_old.py
<Author>
Vladimir Diaz
<Started>
July 14, 2017.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Test cases for unittest_toolbox.py.
"""
import unittest
import logging
import shutil
import sys
import tuf.unittest_toolbox as unittest_toolbox
from tests import utils
logger = logging.getLogger(__name__)
class TestUnittestToolbox(unittest_toolbox.Modified_TestCase):
def setUp(self):
unittest_toolbox.Modified_TestCase.setUp(self)
def tearDown(self):
unittest_toolbox.Modified_TestCase.tearDown(self)
def test_tear_down_already_deleted_dir(self):
temp_directory = self.make_temp_directory()
# Delete the temp directory to make sure unittest_toolbox doesn't
# complain about the missing temp_directory.
shutil.rmtree(temp_directory)
# Run the unit tests.
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

2138
tests/test_updater_old.py
View file

File diff suppressed because it is too large Load diff

685
tests/test_updater_root_rotation_integration_old.py
View file

@ -1,685 +0,0 @@
#!/usr/bin/env python
# Copyright 2016 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
test_updater_root_rotation_integration_old.py
<Author>
Evan Cordell.
<Started>
August 8, 2016.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
'test_updater_root_rotation.py' provides a collection of methods that test
root key rotation in the example client.
<Methodology>
Test cases here should follow a specific order (i.e., independent methods are
tested before dependent methods). More accurately, least dependent methods
are tested before most dependent methods. There is no reason to rewrite or
construct other methods that replicate already-tested methods solely for
testing purposes. This is possible because the 'unittest.TestCase' class
guarantees the order of unit tests. The 'test_something_A' method would
be tested before 'test_something_B'. To ensure the expected order of tests,
a number is placed after 'test' and before methods name like so:
'test_1_check_directory'. The number is a measure of dependence, where 1 is
less dependent than 2.
"""
import os
import shutil
import tempfile
import logging
import unittest
import filecmp
import sys
import tuf
import tuf.log
import tuf.keydb
import tuf.roledb
import tuf.exceptions
import tuf.repository_tool as repo_tool
import tuf.unittest_toolbox as unittest_toolbox
import tuf.client.updater as updater
import tuf.settings
from tests import utils
import securesystemslib
logger = logging.getLogger(__name__)
repo_tool.disable_console_log_messages()
class TestUpdater(unittest_toolbox.Modified_TestCase):
@classmethod
def setUpClass(cls):
# Create a temporary directory to store the repository, metadata, and target
# files. 'temporary_directory' must be deleted in TearDownModule() so that
# temporary files are always removed, even when exceptions occur.
cls.temporary_directory = tempfile.mkdtemp(dir=os.getcwd())
# Launch a SimpleHTTPServer (serves files in the current directory). Test
# cases will request metadata and target files that have been pre-generated
# in 'tuf/tests/repository_data', which will be served by the
# SimpleHTTPServer launched here. The test cases of
# 'test_updater_root_rotation_integration_old.py' assume the
# pre-generated metadata files have a specific structure, such
# as a delegated role 'targets/role1', three target files, five key files,
# etc.
cls.server_process_handler = utils.TestServerProcess(log=logger)
@classmethod
def tearDownClass(cls):
# Cleans the resources and flush the logged lines (if any).
cls.server_process_handler.clean()
# Remove the temporary repository directory, which should contain all the
# metadata, targets, and key files generated for the test cases.
shutil.rmtree(cls.temporary_directory)
def setUp(self):
# We are inheriting from custom class.
unittest_toolbox.Modified_TestCase.setUp(self)
self.repository_name = 'test_repository1'
# Copy the original repository files provided in the test folder so that
# any modifications made to repository files are restricted to the copies.
# The 'repository_data' directory is expected to exist in 'tuf.tests/'.
original_repository_files = os.path.join(os.getcwd(), 'repository_data')
temporary_repository_root = \
self.make_temp_directory(directory=self.temporary_directory)
# The original repository, keystore, and client directories will be copied
# for each test case.
original_repository = os.path.join(original_repository_files, 'repository')
original_keystore = os.path.join(original_repository_files, 'keystore')
original_client = os.path.join(original_repository_files, 'client')
# Save references to the often-needed client repository directories.
# Test cases need these references to access metadata and target files.
self.repository_directory = \
os.path.join(temporary_repository_root, 'repository')
self.keystore_directory = \
os.path.join(temporary_repository_root, 'keystore')
self.client_directory = os.path.join(temporary_repository_root, 'client')
self.client_metadata = os.path.join(self.client_directory,
self.repository_name, 'metadata')
self.client_metadata_current = os.path.join(self.client_metadata, 'current')
self.client_metadata_previous = os.path.join(self.client_metadata, 'previous')
# Copy the original 'repository', 'client', and 'keystore' directories
# to the temporary repository the test cases can use.
shutil.copytree(original_repository, self.repository_directory)
shutil.copytree(original_client, self.client_directory)
shutil.copytree(original_keystore, self.keystore_directory)
# 'path/to/tmp/repository' -> 'localhost:8001/tmp/repository'.
repository_basepath = self.repository_directory[len(os.getcwd()):]
url_prefix = 'http://' + utils.TEST_HOST_ADDRESS + ':' \
+ str(self.server_process_handler.port) + repository_basepath
# Setting 'tuf.settings.repository_directory' with the temporary client
# directory copied from the original repository files.
tuf.settings.repositories_directory = self.client_directory
self.repository_mirrors = {'mirror1': {'url_prefix': url_prefix,
'metadata_path': 'metadata',
'targets_path': 'targets'}}
# Creating a repository instance. The test cases will use this client
# updater to refresh metadata, fetch target files, etc.
self.repository_updater = updater.Updater(self.repository_name,
self.repository_mirrors)
# Metadata role keys are needed by the test cases to make changes to the
# repository (e.g., adding a new target file to 'targets.json' and then
# requesting a refresh()).
self.role_keys = _load_role_keys(self.keystore_directory)
def tearDown(self):
tuf.roledb.clear_roledb(clear_all=True)
tuf.keydb.clear_keydb(clear_all=True)
# Logs stdout and stderr from the sever subprocess.
self.server_process_handler.flush_log()
# Remove temporary directory
unittest_toolbox.Modified_TestCase.tearDown(self)
# UNIT TESTS.
def test_root_rotation(self):
repository = repo_tool.load_repository(self.repository_directory)
repository.root.threshold = 2
repository.snapshot.load_signing_key(self.role_keys['snapshot']['private'])
repository.timestamp.load_signing_key(self.role_keys['timestamp']['private'])
# Errors, not enough signing keys to satisfy root's threshold.
self.assertRaises(tuf.exceptions.UnsignedMetadataError, repository.writeall)
repository.root.add_verification_key(self.role_keys['role1']['public'])
repository.root.load_signing_key(self.role_keys['root']['private'])
repository.root.load_signing_key(self.role_keys['role1']['private'])
repository.writeall()
repository.root.add_verification_key(self.role_keys['snapshot']['public'])
repository.root.load_signing_key(self.role_keys['snapshot']['private'])
repository.root.threshold = 3
repository.writeall()
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
self.repository_updater.refresh()
def test_verify_root_with_current_keyids_and_threshold(self):
"""
Each root file is signed by the current root threshold of keys as well
as the previous root threshold of keys. Test that a root file which is
not 'self-signed' with the current root threshold of keys causes the
update to fail
"""
# Load repository with root.json == 1.root.json (available on client)
# Signing key: "root", Threshold: 1
repository = repo_tool.load_repository(self.repository_directory)
# Rotate keys and update root: 1.root.json --> 2.root.json
# Signing key: "root" (previous) and "root2" (current)
# Threshold (for both): 1
repository.root.load_signing_key(self.role_keys['root']['private'])
repository.root.add_verification_key(self.role_keys['root2']['public'])
repository.root.load_signing_key(self.role_keys['root2']['private'])
# Remove the previous "root" key from the list of current
# verification keys
repository.root.remove_verification_key(self.role_keys['root']['public'])
repository.writeall()
# Move staged metadata to "live" metadata
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# Intercept 2.root.json and tamper with "root2" (current) key signature
root2_path_live = os.path.join(
self.repository_directory, 'metadata', '2.root.json')
root2 = securesystemslib.util.load_json_file(root2_path_live)
for idx, sig in enumerate(root2['signatures']):
if sig['keyid'] == self.role_keys['root2']['public']['keyid']:
sig_len = len(root2['signatures'][idx]['sig'])
root2['signatures'][idx]['sig'] = "deadbeef".ljust(sig_len, '0')
roo2_fobj = tempfile.TemporaryFile()
roo2_fobj.write(tuf.repository_lib._get_written_metadata(root2))
securesystemslib.util.persist_temp_file(roo2_fobj, root2_path_live)
# Update 1.root.json -> 2.root.json
# Signature verification with current keys should fail because we replaced
with self.assertRaises(tuf.exceptions.NoWorkingMirrorError) as cm:
self.repository_updater.refresh()
for mirror_url, mirror_error in cm.exception.mirror_errors.items():
self.assertTrue(mirror_url.endswith('/2.root.json'))
self.assertTrue(isinstance(mirror_error,
securesystemslib.exceptions.BadSignatureError))
# Assert that the current 'root.json' on the client side is the verified one
self.assertTrue(filecmp.cmp(
os.path.join(self.repository_directory, 'metadata', '1.root.json'),
os.path.join(self.client_metadata_current, 'root.json')))
def test_verify_root_with_duplicate_current_keyids(self):
"""
Each root file is signed by the current root threshold of keys as well
as the previous root threshold of keys. In each case, a keyid must only
count once towards the threshold. Test that the new root signatures
specific signature verification implemented in _verify_root_self_signed()
only counts one signature per keyid towards the threshold.
"""
# Load repository with root.json == 1.root.json (available on client)
# Signing key: "root", Threshold: 1
repository = repo_tool.load_repository(self.repository_directory)
# Add an additional signing key and bump the threshold to 2
repository.root.load_signing_key(self.role_keys['root']['private'])
repository.root.add_verification_key(self.role_keys['root2']['public'])
repository.root.load_signing_key(self.role_keys['root2']['private'])
repository.root.threshold = 2
repository.writeall()
# Move staged metadata to "live" metadata
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# Modify 2.root.json and list two signatures with the same keyid
root2_path_live = os.path.join(
self.repository_directory, 'metadata', '2.root.json')
root2 = securesystemslib.util.load_json_file(root2_path_live)
signatures = []
signatures.append(root2['signatures'][0])
signatures.append(root2['signatures'][0])
root2['signatures'] = signatures
root2_fobj = tempfile.TemporaryFile()
root2_fobj.write(tuf.repository_lib._get_written_metadata(root2))
securesystemslib.util.persist_temp_file(root2_fobj, root2_path_live)
# Update 1.root.json -> 2.root.json
# Signature verification with new keys should fail because the threshold
# can only be met by two signatures with the same keyid
with self.assertRaises(tuf.exceptions.NoWorkingMirrorError) as cm:
self.repository_updater.refresh()
for mirror_url, mirror_error in cm.exception.mirror_errors.items():
self.assertTrue(mirror_url.endswith('/2.root.json'))
self.assertTrue(isinstance(mirror_error,
securesystemslib.exceptions.BadSignatureError))
# Assert that the current 'root.json' on the client side is the verified one
self.assertTrue(filecmp.cmp(
os.path.join(self.repository_directory, 'metadata', '1.root.json'),
os.path.join(self.client_metadata_current, 'root.json')))
def test_root_rotation_full(self):
"""Test that a client whose root is outdated by multiple versions and who
has none of the latest nor next-to-latest root keys can still update and
does so by incrementally verifying all roots until the most recent one. """
# Load initial repository with 1.root.json == root.json, signed by "root"
# key. This is the root.json that is already on the client.
repository = repo_tool.load_repository(self.repository_directory)
# 1st rotation: 1.root.json --> 2.root.json
# 2.root.json will be signed by previous "root" key and by new "root2" key
repository.root.load_signing_key(self.role_keys['root']['private'])
repository.root.add_verification_key(self.role_keys['root2']['public'])
repository.root.load_signing_key(self.role_keys['root2']['private'])
repository.writeall()
# 2nd rotation: 2.root.json --> 3.root.json
# 3.root.json will be signed by previous "root2" key and by new "root3" key
repository.root.unload_signing_key(self.role_keys['root']['private'])
repository.root.remove_verification_key(self.role_keys['root']['public'])
repository.root.add_verification_key(self.role_keys['root3']['public'])
repository.root.load_signing_key(self.role_keys['root3']['private'])
repository.writeall()
# Move staged metadata to "live" metadata
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# Update on client 1.root.json --> 2.root.json --> 3.root.json
self.repository_updater.refresh()
# Assert that client updated to the latest root from the repository
self.assertTrue(filecmp.cmp(
os.path.join(self.repository_directory, 'metadata', '3.root.json'),
os.path.join(self.client_metadata_current, 'root.json')))
def test_root_rotation_max(self):
"""Test that client does not rotate beyond a configured upper bound, i.e.
`current_version + MAX_NUMBER_ROOT_ROTATIONS`. """
# NOTE: The nature of below root changes is irrelevant. Here we only want
# the client to update but not beyond a configured upper bound.
# 1.root.json --> 2.root.json (add root2 and root3 keys)
repository = repo_tool.load_repository(self.repository_directory)
repository.root.load_signing_key(self.role_keys['root']['private'])
repository.root.add_verification_key(self.role_keys['root2']['public'])
repository.root.load_signing_key(self.role_keys['root2']['private'])
repository.root.add_verification_key(self.role_keys['root3']['public'])
repository.root.load_signing_key(self.role_keys['root3']['private'])
repository.writeall()
# 2.root.json --> 3.root.json (change threshold)
repository.root.threshold = 2
repository.writeall()
# 3.root.json --> 4.root.json (change threshold again)
repository.root.threshold = 3
repository.writeall()
# Move staged metadata to "live" metadata
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# Assert that repo indeed has "4.root.json" and that it's the latest root
self.assertTrue(filecmp.cmp(
os.path.join(self.repository_directory, 'metadata', '4.root.json'),
os.path.join(self.repository_directory, 'metadata', 'root.json')))
# Lower max root rotation cap so that client stops updating early
max_rotation_backup = tuf.settings.MAX_NUMBER_ROOT_ROTATIONS
tuf.settings.MAX_NUMBER_ROOT_ROTATIONS = 2
# Update on client 1.root.json --> 2.root.json --> 3.root.json,
# but stop before updating to 4.root.json
self.repository_updater.refresh()
# Assert that the client indeed only updated until 3.root.json
self.assertTrue(filecmp.cmp(
os.path.join(self.repository_directory, 'metadata', '3.root.json'),
os.path.join(self.client_metadata_current, 'root.json')))
# reset
tuf.settings.MAX_NUMBER_ROOT_ROTATIONS = max_rotation_backup
def test_root_rotation_missing_keys(self):
repository = repo_tool.load_repository(self.repository_directory)
# A partially written root.json (threshold = 2, and signed with only 1 key)
# causes an invalid root chain later.
repository.root.threshold = 2
repository.root.load_signing_key(self.role_keys['root']['private'])
repository.snapshot.load_signing_key(self.role_keys['snapshot']['private'])
repository.timestamp.load_signing_key(self.role_keys['timestamp']['private'])
repository.write('root')
repository.write('snapshot')
repository.write('timestamp')
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# Create a new, valid root.json.
# Still not valid, because it is not written with a threshold of 2
# previous keys
repository.root.add_verification_key(self.role_keys['role1']['public'])
repository.root.load_signing_key(self.role_keys['role1']['private'])
repository.writeall()
repository.root.add_verification_key(self.role_keys['snapshot']['public'])
repository.root.load_signing_key(self.role_keys['snapshot']['private'])
repository.root.threshold = 3
repository.writeall()
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
with self.assertRaises(tuf.exceptions.NoWorkingMirrorError) as cm:
self.repository_updater.refresh()
for mirror_url, mirror_error in cm.exception.mirror_errors.items():
self.assertTrue(mirror_url.endswith('/2.root.json'))
self.assertTrue(isinstance(mirror_error,
securesystemslib.exceptions.BadSignatureError))
# Assert that the current 'root.json' on the client side is the verified one
self.assertTrue(filecmp.cmp(
os.path.join(self.repository_directory, 'metadata', '1.root.json'),
os.path.join(self.client_metadata_current, 'root.json')))
def test_root_rotation_unmet_last_version_threshold(self):
"""Test that client detects a root.json version that is not signed
by a previous threshold of signatures """
repository = repo_tool.load_repository(self.repository_directory)
# Add verification keys
repository.root.add_verification_key(self.role_keys['root']['public'])
repository.root.add_verification_key(self.role_keys['role1']['public'])
repository.targets.add_verification_key(self.role_keys['targets']['public'])
repository.snapshot.add_verification_key(self.role_keys['snapshot']['public'])
repository.timestamp.add_verification_key(self.role_keys['timestamp']['public'])
repository.snapshot.load_signing_key(self.role_keys['snapshot']['private'])
repository.timestamp.load_signing_key(self.role_keys['timestamp']['private'])
# Add signing keys
repository.root.load_signing_key(self.role_keys['root']['private'])
repository.root.load_signing_key(self.role_keys['role1']['private'])
# Set root threshold
repository.root.threshold = 2
repository.writeall()
# Unload Root's previous signing keys to ensure that these keys are not
# used by mistake.
repository.root.unload_signing_key(self.role_keys['role1']['private'])
repository.root.unload_signing_key(self.role_keys['root']['private'])
# Add new verification key
repository.root.add_verification_key(self.role_keys['snapshot']['public'])
# Remove one of the original signing keys
repository.root.remove_verification_key(self.role_keys['role1']['public'])
# Set the threshold for the new Root file, but note that the previous
# threshold of 2 must still be met.
repository.root.threshold = 1
repository.root.load_signing_key(self.role_keys['role1']['private'])
repository.root.load_signing_key(self.role_keys['snapshot']['private'])
repository.snapshot.load_signing_key(self.role_keys['snapshot']['private'])
repository.timestamp.load_signing_key(self.role_keys['timestamp']['private'])
# We use write() rather than writeall() because the latter should fail due
# to the missing self.role_keys['root'] signature.
repository.write('root', increment_version_number=True)
repository.write('snapshot', increment_version_number=True)
repository.write('timestamp', increment_version_number=True)
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# The following refresh should fail because root must be signed by the
# previous self.role_keys['root'] key, which wasn't loaded.
with self.assertRaises(tuf.exceptions.NoWorkingMirrorError) as cm:
self.repository_updater.refresh()
for mirror_url, mirror_error in cm.exception.mirror_errors.items():
self.assertTrue(mirror_url.endswith('/3.root.json'))
self.assertTrue(isinstance(mirror_error,
securesystemslib.exceptions.BadSignatureError))
# Assert that the current 'root.json' on the client side is the verified one
self.assertTrue(filecmp.cmp(
os.path.join(self.repository_directory, 'metadata', '2.root.json'),
os.path.join(self.client_metadata_current, 'root.json')))
def test_root_rotation_unmet_new_threshold(self):
"""Test that client detects a root.json version that is not signed
by a current threshold of signatures """
repository = repo_tool.load_repository(self.repository_directory)
# Create a new, valid root.json.
repository.root.threshold = 2
repository.root.load_signing_key(self.role_keys['root']['private'])
repository.root.add_verification_key(self.role_keys['root2']['public'])
repository.root.load_signing_key(self.role_keys['root2']['private'])
repository.writeall()
# Increase the threshold and add a new verification key without
# actually loading the signing key
repository.root.threshold = 3
repository.root.add_verification_key(self.role_keys['root3']['public'])
# writeall fails as expected since the third signature is missing
self.assertRaises(tuf.exceptions.UnsignedMetadataError, repository.writeall)
# write an invalid '3.root.json' as partially signed
repository.write('root')
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# The following refresh should fail because root must be signed by the
# current self.role_keys['root3'] key, which wasn't loaded.
with self.assertRaises(tuf.exceptions.NoWorkingMirrorError) as cm:
self.repository_updater.refresh()
for mirror_url, mirror_error in cm.exception.mirror_errors.items():
self.assertTrue(mirror_url.endswith('/3.root.json'))
self.assertTrue(isinstance(mirror_error,
securesystemslib.exceptions.BadSignatureError))
# Assert that the current 'root.json' on the client side is the verified one
self.assertTrue(filecmp.cmp(
os.path.join(self.repository_directory, 'metadata', '2.root.json'),
os.path.join(self.client_metadata_current, 'root.json')))
def test_root_rotation_discard_untrusted_version(self):
"""Test that client discards root.json version that failed the
signature verification """
repository = repo_tool.load_repository(self.repository_directory)
# Rotate the root key without signing with the previous version key 'root'
repository.root.remove_verification_key(self.role_keys['root']['public'])
repository.root.add_verification_key(self.role_keys['root2']['public'])
repository.root.load_signing_key(self.role_keys['root2']['private'])
# 2.root.json
repository.writeall()
# Move the staged metadata to the "live" metadata.
shutil.rmtree(os.path.join(self.repository_directory, 'metadata'))
shutil.copytree(os.path.join(self.repository_directory, 'metadata.staged'),
os.path.join(self.repository_directory, 'metadata'))
# Refresh on the client side should fail because 2.root.json is not signed
# with a threshold of prevous keys
with self.assertRaises(tuf.exceptions.NoWorkingMirrorError) as cm:
self.repository_updater.refresh()
for mirror_url, mirror_error in cm.exception.mirror_errors.items():
self.assertTrue(mirror_url.endswith('/2.root.json'))
self.assertTrue(isinstance(mirror_error,
securesystemslib.exceptions.BadSignatureError))
# Assert that the current 'root.json' on the client side is the trusted one
# and 2.root.json is discarded
self.assertTrue(filecmp.cmp(
os.path.join(self.repository_directory, 'metadata', '1.root.json'),
os.path.join(self.client_metadata_current, 'root.json')))
def _load_role_keys(keystore_directory):
# Populating 'self.role_keys' by importing the required public and private
# keys of 'tuf/tests/repository_data/'. The role keys are needed when
# modifying the remote repository used by the test cases in this unit test.
# The pre-generated key files in 'repository_data/keystore' are all encrypted
# with a 'password' passphrase.
EXPECTED_KEYFILE_PASSWORD = 'password'
# Store and return the cryptography keys of the top-level roles, including 1
# delegated role.
role_keys = {}
root_key_file = os.path.join(keystore_directory, 'root_key')
root2_key_file = os.path.join(keystore_directory, 'root_key2')
root3_key_file = os.path.join(keystore_directory, 'root_key3')
targets_key_file = os.path.join(keystore_directory, 'targets_key')
snapshot_key_file = os.path.join(keystore_directory, 'snapshot_key')
timestamp_key_file = os.path.join(keystore_directory, 'timestamp_key')
delegation_key_file = os.path.join(keystore_directory, 'delegation_key')
role_keys = {'root': {}, 'root2': {}, 'root3': {}, 'targets': {}, 'snapshot':
{}, 'timestamp': {}, 'role1': {}}
# Import the top-level and delegated role public keys.
role_keys['root']['public'] = \
repo_tool.import_rsa_publickey_from_file(root_key_file+'.pub')
role_keys['root2']['public'] = \
repo_tool.import_ed25519_publickey_from_file(root2_key_file+'.pub')
role_keys['root3']['public'] = \
repo_tool.import_ecdsa_publickey_from_file(root3_key_file+'.pub')
role_keys['targets']['public'] = \
repo_tool.import_ed25519_publickey_from_file(targets_key_file+'.pub')
role_keys['snapshot']['public'] = \
repo_tool.import_ed25519_publickey_from_file(snapshot_key_file+'.pub')
role_keys['timestamp']['public'] = \
repo_tool.import_ed25519_publickey_from_file(timestamp_key_file+'.pub')
role_keys['role1']['public'] = \
repo_tool.import_ed25519_publickey_from_file(delegation_key_file+'.pub')
# Import the private keys of the top-level and delegated roles.
role_keys['root']['private'] = \
repo_tool.import_rsa_privatekey_from_file(root_key_file,
EXPECTED_KEYFILE_PASSWORD)
role_keys['root2']['private'] = \
repo_tool.import_ed25519_privatekey_from_file(root2_key_file,
EXPECTED_KEYFILE_PASSWORD)
role_keys['root3']['private'] = \
repo_tool.import_ecdsa_privatekey_from_file(root3_key_file,
EXPECTED_KEYFILE_PASSWORD)
role_keys['targets']['private'] = \
repo_tool.import_ed25519_privatekey_from_file(targets_key_file,
EXPECTED_KEYFILE_PASSWORD)
role_keys['snapshot']['private'] = \
repo_tool.import_ed25519_privatekey_from_file(snapshot_key_file,
EXPECTED_KEYFILE_PASSWORD)
role_keys['timestamp']['private'] = \
repo_tool.import_ed25519_privatekey_from_file(timestamp_key_file,
EXPECTED_KEYFILE_PASSWORD)
role_keys['role1']['private'] = \
repo_tool.import_ed25519_privatekey_from_file(delegation_key_file,
EXPECTED_KEYFILE_PASSWORD)
return role_keys
if __name__ == '__main__':
utils.configure_test_logging(sys.argv)
unittest.main()

45
tests/test_utils.py
View file

@ -21,7 +21,6 @@
"""
import logging
import os
import socket
import sys
import unittest
@ -57,50 +56,6 @@ def test_simple_server_startup(self) -> None:
self.assertTrue(can_connect(server_process_handler.port))
server_process_handler.clean()
def test_simple_https_server_startup(self) -> None:
# Test normal case
good_cert_path = os.path.join("ssl_certs", "ssl_cert.crt")
server_process_handler = utils.TestServerProcess(
log=logger,
server="simple_https_server_old.py",
extra_cmd_args=[good_cert_path],
)
# Make sure we can connect to the server
self.assertTrue(can_connect(server_process_handler.port))
server_process_handler.clean()
# Test when no cert file is provided
server_process_handler = utils.TestServerProcess(
log=logger, server="simple_https_server_old.py"
)
# Make sure we can connect to the server
self.assertTrue(can_connect(server_process_handler.port))
server_process_handler.clean()
# Test with a non existing cert file.
non_existing_cert_path = os.path.join("ssl_certs", "non_existing.crt")
server_process_handler = utils.TestServerProcess(
log=logger,
server="simple_https_server_old.py",
extra_cmd_args=[non_existing_cert_path],
)
# Make sure we can connect to the server
self.assertTrue(can_connect(server_process_handler.port))
server_process_handler.clean()
def test_slow_retrieval_server_startup(self) -> None:
# Test normal case
server_process_handler = utils.TestServerProcess(
log=logger, server="slow_retrieval_server_old.py"
)
# Make sure we can connect to the server
self.assertTrue(can_connect(server_process_handler.port))
server_process_handler.clean()
def test_cleanup(self) -> None:
# Test normal case
server_process_handler = utils.TestServerProcess(

3
tests/utils.py
View file

@ -35,8 +35,6 @@
from contextlib import contextmanager
from typing import IO, Any, Callable, Dict, Iterator, List, Optional
import tuf.log
logger = logging.getLogger(__name__)
# May may be used to reliably read other files in tests dir regardless of cwd
@ -154,7 +152,6 @@ def configure_test_logging(argv: List[str]) -> None:
loglevel = logging.DEBUG
logging.basicConfig(level=loglevel)
tuf.log.set_log_level(loglevel)
def cleanup_dir(path: str) -> None:

7
tox.ini
View file

@ -38,17 +38,12 @@ commands =
[testenv:lint]
changedir = {toxinidir}
lint_dirs = tuf/api tuf/ngclient examples tests
lint_dirs = tuf examples tests
commands =
# Use different configs for new (tuf/api/*) and legacy code
black --check --diff {[testenv:lint]lint_dirs}
isort --check --diff {[testenv:lint]lint_dirs}
pylint -j 0 --rcfile=pyproject.toml {[testenv:lint]lint_dirs}
# NOTE: Contrary to what the pylint docs suggest, ignoring full paths does
# work, unfortunately each subdirectory has to be ignored explicitly.
pylint -j 0 tuf --ignore=tuf/api,tuf/api/serialization,tuf/ngclient,tuf/ngclient/_internal
mypy {[testenv:lint]lint_dirs}
bandit -r tuf

323
tuf/ATTACKS.md
View file

@ -1,323 +0,0 @@
# Demonstrate protection against malicious updates
## Table of Contents ##
- [Blocking Malicious Updates](#blocking-malicious-updates)
- [Arbitrary Package Attack](#arbitrary-package-attack)
- [Rollback Attack](#rollback-attack)
- [Indefinite Freeze Attack](#indefinite-freeze-attack)
- [Endless Data Attack](#endless-data-attack)
- [Compromised Key Attack](#compromised-key-attack)
- [Slow Retrieval Attack](#slow-retrieval-attack)
- [Conclusion](#conclusion)
## Blocking Malicious Updates ##
TUF protects against a number of attacks, some of which include rollback,
arbitrary package, and mix and match attacks. We begin this document on
blocking malicious updates by demonstrating how the client rejects a target
file downloaded from the software repository that doesn't match what is listed
in TUF metadata.
The following demonstration requires and operates on the repository created in
the [repository management
tutorial](https://github.com/theupdateframework/python-tuf/blob/develop/tuf/README.md).
### Arbitrary Package Attack ###
In an arbitrary package attack, an attacker installs anything they want on the
client system. That is, an attacker can provide arbitrary files in response to
download requests and the files will not be detected as illegitimate. We
simulate an arbitrary package attack by creating a "malicious" target file
that our client attempts to fetch.
```Bash
$ mv 'repository/targets/file2.txt' 'repository/targets/file2.txt.backup'
$ echo 'bad_target' > 'repository/targets/file2.txt'
```
We next reset our local timestamp (so that a new update is prompted), and
the target files previously downloaded by the client.
```Bash
$ rm -rf "client/targets/" "client/metadata/current/timestamp.json"
```
The client now performs an update and should detect the invalid target file...
Note: The following command should be executed in the "client/" directory.
```Bash
$ python3 basic_client.py --repo http://localhost:8001
Error: No working mirror was found:
localhost:8001: BadHashError()
```
The log file (tuf.log) saved to the current working directory contains more
information on the update procedure and the cause of the BadHashError.
```Bash
...
BadHashError: Observed
hash ('f569179171c86aa9ed5e8b1d6c94dfd516123189568d239ed57d818946aaabe7') !=
expected hash (u'67ee5478eaadb034ba59944eb977797b49ca6aa8d3574587f36ebcbeeb65f70e')
[2016-10-20 19:45:16,079 UTC] [tuf.client.updater] [ERROR] [_get_file:1415@updater.py]
Failed to update /file2.txt from all mirrors: {u'http://localhost:8001/targets/file2.txt': BadHashError()}
```
Note: The "malicious" target file should be removed and the original file2.txt
restored, otherwise the following examples will fail with BadHashError
exceptions:
```Bash
$ mv 'repository/targets/file2.txt.backup' 'repository/targets/file2.txt'
```
### Indefinite Freeze Attack ###
In an indefinite freeze attack, an attacker continues to present a software
update system with the same files the client has already seen. The result is
that the client does not know that new files are available. Although the
client would be unable to prevent an attacker or compromised repository from
feeding it stale metadata, it can at least detect when an attacker is doing so
indefinitely. The signed metadata used by TUF contains an "expires" field that
indicates when metadata should no longer be trusted.
In the following simulation, the client first tries to perform an update.
```Bash
$ python3 basic_client.py --repo http://localhost:8001
```
According to the logger (`tuf.log` file in the current working directory),
everything appears to be up-to-date. The remote server should also show that
the client retrieved only the timestamp.json file. Let's suppose now that an
attacker continues to feed our client the same stale metadata. If we were to
move the time to a future date that would cause metadata to expire, the TUF
framework should raise an exception or error to indicate that the metadata
should no longer be trusted.
```Bash
$ sudo date -s '2080-12-25 12:34:56'
Wed Dec 25 12:34:56 EST 2080
$ python3 basic_client.py --repo http://localhost:8001
Error: No working mirror was found:
u'localhost:8001': ExpiredMetadataError(u"Metadata u'root' expired on Tue Jan 1 00:00:00 2030 (UTC).",)
```
Note: Reset the date to continue with the rest of the attacks.
### Rollback Attack ###
In a rollback attack, an attacker presents a software update system with older
files than those the client has already seen, causing the client to use files
older than those the client knows about. We begin this example by saving the
current version of the Timestamp file available on the repository. This saved
file will later be served to the client to see if it is rejected. The client
should not accept versions of metadata that is older than previously trusted.
Navigate to the directory containing the server's files and save the current
timestamp.json to a temporary location:
```Bash
$ cp repository/metadata/timestamp.json /tmp
```
We should next generate a new Timestamp file on the repository side.
```Bash
$ python3
>>> from tuf.repository_tool import *
>>> repository = load_repository('repository')
>>> repository.timestamp.version
1
>>> repository.timestamp.version = 2
>>> repository.dirty_roles()
Dirty roles: [u'timestamp']
>>> private_timestamp_key = import_rsa_privatekey_from_file("keystore/timestamp_key")
Enter a password for the encrypted RSA file (/path/to/keystore/timestamp_key):
>>> repository.timestamp.load_signing_key(private_timestamp_key)
>>> repository.write('timestamp')
$ cp repository/metadata.staged/* repository/metadata
```
Now start the HTTP server from the directory containing the 'repository'
subdirectory.
```Bash
$ python3 -m SimpleHTTPServer 8001
```
And perform an update so that the client retrieves the updated timestamp.json.
```Bash
$ python3 basic_client.py --repo http://localhost:8001
```
Finally, move the previous timestamp.json file to the current live repository
and have the client try to download the outdated version. The client should
reject it!
```Bash
$ cp /tmp/timestamp.json repository/metadata/
$ cd repository; python3 -m SimpleHTTPServer 8001
```
On the client side, perform an update...
```Bash
$ python3 basic_client.py --repo http://localhost:8001
Error: No working mirror was found:
u'localhost:8001': ReplayedMetadataError()
```
The tuf.log file contains more information about the ReplayedMetadataError
exception and update process. Please reset timestamp.json to the latest
version, which can be found in the 'repository/metadata.staged' subdirectory.
```Bash
$ cp repository/metadata.staged/timestamp.json repository/metadata
```
### Endless Data Attack ###
In an endless data attack, an attacker responds to a file download request with
an endless stream of data, causing harm to clients (e.g., a disk partition
filling up or memory exhaustion). In this simulated attack, we append extra
data to one of the target files available on the software repository. The
client should only download the exact number of bytes it expects for a
requested target file (according to what is listed in trusted TUF metadata).
```Bash
$ cp repository/targets/file1.txt /tmp
$ python3 -c "print 'a' * 1000" >> repository/targets/file1.txt
```
Now delete the local metadata and target files on the client side so
that remote metadata and target files are downloaded again.
```Bash
$ rm -rf client/targets/
$ rm client/metadata/current/snapshot.json* client/metadata/current/timestamp.json*
```
Lastly, perform an update to verify that the file1.txt is downloaded up to the
expected size, and no more. The target file available on the software
repository does contain more data than expected, though.
```Bash
$ python3 basic_client.py --repo http://localhost:8001
```
At this point, part of the "file1.txt" file should have been fetched. That is,
up to 31 bytes of it should have been downloaded, and the rest of the maliciously
appended data ignored. If we inspect the logger, we'd discover the following:
```Bash
[2016-10-06 21:37:39,092 UTC] [tuf.download] [INFO] [_download_file:235@download.py]
Downloading: u'http://localhost:8001/targets/file1.txt'
[2016-10-06 21:37:39,145 UTC] [tuf.download] [INFO] [_check_downloaded_length:610@download.py]
Downloaded 31 bytes out of the expected 31 bytes.
[2016-10-06 21:37:39,145 UTC] [tuf.client.updater] [INFO] [_get_file:1372@updater.py]
Not decompressing http://localhost:8001/targets/file1.txt
[2016-10-06 21:37:39,145 UTC] [tuf.client.updater] [INFO] [_check_hashes:778@updater.py]
The file's sha256 hash is correct: 65b8c67f51c993d898250f40aa57a317d854900b3a04895464313e48785440da
```
Indeed, the sha256 sum of the first 31 bytes of the "file1.txt" available
on the repository should match to what is trusted. The client did not
downloaded the appended data.
Note: Restore file1.txt
```Bash
$ cp /tmp/file1.txt repository/targets/
```
### Compromised Key Attack ###
An attacker who compromise less than a given threshold of keys is limited in
scope. This includes relying on a single online key (such as only being
protected by SSL) or a single offline key (such as most software update systems
use to sign files). In this example, we attempt to sign a role file with
less-than-a-threshold number of keys. A single key (suppose this is a
compromised key) is used to demonstrate that roles must be signed with the
total number of keys required for the role. In order to compromise a role, an
attacker would have to compromise a threshold of keys. This approach of
requiring a threshold number of signatures provides compromise resilience.
Let's attempt to sign a new snapshot file with a less-than-threshold number of
keys. The client should reject the partially signed snapshot file served by
the repository (or imagine that it is a compromised software repository).
```Bash
$ python3
>>> from tuf.repository_tool import *
>>> repository = load_repository('repository')
>>> version = repository.root.version
>>> repository.root.version = version + 1
>>> private_root_key = import_rsa_privatekey_from_file("keystore/root_key", password="password")
>>> repository.root.load_signing_key(private_root_key)
>>> private_root_key2 = import_rsa_privatekey_from_file("keystore/root_key2", password="password")
>>> repository.root.load_signing_key(private_root_key2)
>>> repository.snapshot.version = 8
>>> repository.snapshot.threshold = 2
>>> private_snapshot_key = import_rsa_privatekey_from_file("keystore/snapshot_key", password="password")
>>> repository.snapshot.load_signing_key(private_snapshot_key)
>>> repository.timestamp.version = 8
>>> private_timestamp_key = import_rsa_privatekey_from_file("keystore/timestamp_key", password="password")
>>> repository.timestamp.load_signing_key(private_timestamp_key)
>>> repository.write('root')
>>> repository.write('snapshot')
>>> repository.write('timestamp')
$ cp repository/metadata.staged/* repository/metadata
```
The client now attempts to refresh the top-level metadata and the
partially written snapshot.json, which should be rejected.
```Bash
$ python3 basic_client.py --repo http://localhost:8001
Error: No working mirror was found:
u'localhost:8001': BadSignatureError()
```
### Slow Retrieval Attack ###
In a slow retrieval attack, an attacker responds to clients with a very slow
stream of data that essentially results in the client never continuing the
update process. In this example, we simulate a slow retrieval attack by
spawning a server that serves data at a slow rate to our update client data.
TUF should not be vulnerable to this attack, and the framework should raise an
exception or error when it detects that a malicious server is serving it data
at a slow enough rate.
We first spawn the server that slowly streams data to the client. The
'slow_retrieval_server_old.py' module (can be found in the tests/ directory of the
source code) should be copied over to the server's 'repository/' directory from
which to launch it.
```Bash
# Before launching the slow retrieval server, copy 'slow_retrieval_server_old.py'
# to the 'repository/' directory and run it from that directory as follows:
$ python3 slow_retrieval_server_old.py 8002 mode_2
```
The client may now make a request to the slow retrieval server on port 8002.
However, before doing so, we'll reduce (for the purposes of this demo) the
minimum average download rate allowed and download chunk size. Open the
'settings.py' module and set MIN_AVERAGE_DOWNLOAD_SPEED = 5 and CHUNK_SIZE = 1.
This should make it so that the client detects the slow retrieval server's
delayed streaming.
```Bash
$ python3 basic_client.py --verbose 1 --repo http://localhost:8002
Error: No working mirror was found:
u'localhost:8002': SlowRetrievalError()
```
The framework should detect the slow retrieval attack and raise a
SlowRetrievalError exception to the client application.
## Conclusion ##
These are just some of the attacks that TUF provides protection against. For
more attacks and updater weaknesses, please see the
[Security](https://theupdateframework.io/security/)
page.

342
tuf/README-developer-tools.md
View file

@ -1,342 +0,0 @@
# The Update Framework Developer Tool: How to Update your Project Securely on a TUF Repository
## Table of Contents
- [Overview](#overview)
- [Creating a Simple Project](#creating_a_simple_project)
- [Generating a Key](#generating_a_key)
- [The Project Class](#the_project_class)
- [Signing and Writing the Metadata](#signing_and_writing_the_metadata)
- [Loading an Existing Project](#loading_an_existing_project)
- [Delegations](#delegations)
- [Managing Keys](#managing_keys)
- [Managing Targets](#managing_targets)
<a name="overview">
## Overview
The Update Framework (TUF) is a Python-based security system for software
updates. In order to prevent your users from downloading vulnerable or malicious
code disguised as updates to your software, TUF requires that each update you
release include certain metadata verifying your authorship of the files.
The TUF developer tools are a Python Library that enables you to create and
maintain the required metadata for files hosted on a TUF Repository. (We call
these files “targets,” to distinguish them from the metadata associated with
them. Both of these together comprise a complete “project”.) You will use these
tools to generate the keys and metadata you need to claim and secure your files
on the repository, and to update the metadata and sign it with those keys
whenever you upload a new version of those files.
This document will teach you how to use these tools in two parts. The first
part walks through the creation of a minimal-complexity TUF project, which is
all you need to get started, and can be expanded later. The second part details
the full functionality of the tools, which offer a finer degree of control in
securing your project.
<a name="creating_a_simple_project">
## Creating a Simple Project
This section walks through the creation of a small example project with just
one target. Once created, this project will be fully functional, and can be
modified as needed.
<a name="generating_a_key">
### Generating a Key
First, we will need to generate a key to sign the metadata. Keys are generated
in pairs: one public and the other private. The private key is
password-protected and is used to sign metadata. The public key can be shared
freely, and is used to verify signatures made by the private key. You will need
to share your public key with the repository hosting your project so they can
verify your metadata is signed by the right person.
The generate\_and\_write\_rsa\_keypair function will create two key files named
"path/to/key.pub", which is the public key and "path/to/key", which
is the private key.
```
>>> from tuf.developer_tool import *
>>> generate_and_write_rsa_keypair_with_prompt(filepath="path/to/key")
enter password to encrypt private key file 'path/to/key'
(leave empty if key should not be encrypted):
Confirm:
>>>
```
We can also use the bits parameter to set a different key length (the default
is 3072). We can also `generate_and_write_rsa_keypair` with a `password`
parameter if a prompt is not desired.
In this example we will be using rsa keys, but ed25519 keys are also supported.
Now we have a key for our project, we can proceed to create our project.
<a name="the_project_class">
### The Project Class
The TUF developer tool is built around the Project class, which is used to
organize groups of targets associated with a single set of metadata. A single
Project instance is used to keep track of all the target files and metadata
files in one project. The Project also keeps track of the keys and signatures,
so that it can update all the metadata with the correct changes and signatures
on a single command.
Before creating a project, you must know where it will be located in the TUF
Repository. In the following example, we will create a project to be hosted as
"repo/unclaimed/example_project" within the repository, and store a local copy
of the metadata at "path/to/metadata". The project will comprise a single
target file, "local/path/to/example\_project/target\_1" locally, and we will
secure it with the key generated above.
First, we must import the generated keys. We can do that by issuing the
following command:
```
>>> public_key = import_rsa_publickey_from_file("path/to/keys.pub")
```
After importing the key, we can generate a new project with the following
command:
```
>>> project = create_new_project(project_name="example_project",
... metadata_directory="local/path/to/metadata/",
... targets_directory="local/path/to/example_project",
... location_in_repository="repo/unclaimed", key=public_key)
```
Let's list the arguments and make sense out of this rather long function call:
- create a project named example_project: the name of the metadata file will match this name
- the metadata will be located in "local/path/to/metadata", this means all of the generated files
for this project will be located here
- the targets are located in local/path/to/example project. If your targets are located in some other
place, you can point the targets directory there. Files must reside under the path local/path/to/example_project or else it won't be possible to add them.
- location\_in\_repository points to repo/unclaimed, this will be prepended to the paths in the generated metadata so the signatures all match.
Now the project is in memory and we can do different operations on it such as
adding and removing targets, delegating files, changing signatures and keys,
etc. For the moment we are interested in adding our one and only target inside
the project.
To add a target, we issue the following method:
```
>>> project.add_target("local/path/to/example_project/target_1")
```
Note that the file "target\_1" should be located in
"local/path/to/example\_project", or this method will throw an
error.
At this point, the metadata is not valid. We have assigned a key to the
project, but we have not *signed* it with that key. Signing is the process of
generating a signature with our private key so it can be verified with the
public key by the server (upon uploading) and by the clients (when updating).
<a name="signing_and_writing_the_metadata">
### Signing and Writing the Metadata ###
In order to sign the metadata, we need to import the private key corresponding
to the public key we added to the project. One the key is loaded to the project,
it will automatically be used to sign the metadata whenever it is written.
```
>>> private_key = import_rsa_privatekey_from_file("path/to/key")
Enter password for the RSA key:
>>> project.load_signing_key(private_key)
>>> project.write()
```
When all changes to the project have been written, the metadata is ready to be
uploaded to the repository, and it is safe to exit the Python interpreter, or
to delete the Project instance.
The project can be loaded later to update changes to the project. The metadata
contains checksums that have to match the actual files or else it won't be
accepted by the upstream repository.
At this point, if you have followed all the steps in this document so far
(substituting appropriate names and filepaths) you will have created a basic
TUF project, which can be expanded as needed. The simplest way to get your
project secured is to add all your files using add\_target() (or see [Managing
Keys](#managing_keys) on how to add whole directories). If your project has
several contributors, you may want to consider adding
[delegations](#delegations) to your project.
<a name="loading_an_existing_project">
## Loading an Existing Project
To make changes to existing metadata, we will need the Project again. We can
restore it with the load_project() function.
```
>>> from tuf.developer_tool import *
>>> project = load_project("local/path/to/metadata")
```
Each time the project is loaded anew, the necessary private keys must also be
loaded in order to sign metadata.
```
>>> private_key = import_rsa_privatekey_from_file("path/to/key")
Enter a password for the RSA key:
>>> project.load_signing_key(private_key)
>>> project.write()
```
If your project does not use any delegations, the five commands above are all
you need to update your project's metadata.
<a name="delegations">
## Delegations
The project we created above is secured entirely by one key. If you want to
allow someone else to update part of your project independently, you will need
to delegate a new role for them. For example, we can do the following:
```
>>> other_key = import_rsa_publickey_from_file(“another_public_key.pub”)
>>> targets = ['local/path/to/newtarget']
>>> project.delegate(“newrole”, [other_key], targets)
```
The new role is now an attribute of the Project instance, and contains the same
methods as Project. For example, we can add targets in the same way as before:
```
>>> project(“newrole”).add_target(“delegated_1”)
```
Recall that we input the other person’s key as part of a list. That list can
contain any number of public keys. We can also add keys to the role after
creating it using the [add\_verification\_key()](#adding_a_key_to_a_delegation)
method.
### Delegated Paths
By default, a delegated role is permitted to add and modify targets anywhere in
the Project's targets directory. We can delegate trust of paths to a role to
limit this permission.
```
>>> project.add_paths(["delegated/filepath"], "newrole")
```
This will prevent the delegated role from signing targets whose local filepaths
do not begin with "delegated/filepath". We can delegate several filepaths to a
role by adding them to the list in the first parameter, or by invoking the
method again. A role with multiple delegated paths can add targets to any of
them.
Note that this method is invoked from the parent role (in this case, the Project)
and takes the delegated role name as an argument.
### Nested Delegations
It is possible for a delegated role to have delegations of its own. We can do
this by calling delegate() on a delegated role:
```
>>> project("newrole").delegate(“nestedrole”, [key], targets)
```
Nested delegations function no differently than first-order delegations. to
demonstrate, adding a target to nested delegation looks like this:
```
>>> project("newrole")("nestedrole").add_target("foo")
```
### Revoking Delegations
Delegations can be revoked, removing the delegated role from the project.
```
>>> project.revoke("newrole")
```
<a name="managing_keys">
## Managing Keys
This section describes the key-related functions and parameters not covered in
the [Creating a Simple Project](#creating_a_simple_project) section.
### Additional Parameters for Key Generation
When generating keys, it is possible to specify the length of the key in bits
and its password as parameters:
```
>>> generate_and_write_rsa_keypair(password="pw", filepath="path/to/key", bits=2048)
```
The bits parameter defaults to 3072, and values below 2048 will raise an error.
The password parameter is only intended to be used in scripts.
<a name="adding_a_key_to_a_delegation">
### Adding a Key to a Delegation
New verifications keys can be added to an existing delegation using
add\_verification\_key():
```
>>> project("rolename").add_verification_key(pubkey)
```
A delegation can have several verification keys at once. By default, a
delegated role with multiple keys can be written using any one of their
corresponding signing keys. To modify this behavior, you can change the
delegated role's [threshold](#delegation_thrsholds).
### Removing a Key from a Delegation
Verification keys can also be removed, like this:
```
>>> project("rolename").remove_verification_key(pubkey)
```
Remember that a project can only have one key, so this method will return an
error if there is already a key assigned to it. In order to replace a key we
must first delete the existing one and then add the new one. It is possible to
omit the key parameter in the create\_new\_project() function, and add the key
later.
### Changing the Project Key
Each Project instance can only have one verification key. This key can be
replaced by removing it and adding a new key, in that order.
```
>>> project.remove_verification_key(oldkey)
>>> project.add_verification_key(new)
```
<a name="delegation_thresholds">
### Delegation Thresholds
Every delegated role has a threshold, which determines how many of its signing
keys need to be loaded to write the role. The threshold defaults to 1, and
should not exceed the number of verification keys assigned to the role. The
threshold can be accessed as a property of a delegated role.
```
>>> project("rolename").threshold = 2
```
The above line will set the "rolename" role's threshold to 2.
<a name="managing_targets">
## Managing Targets
There are supporting functions of the targets library to make the project
maintenance easier. These functions are described in this section.
### Adding Targets by Directory
This function is especially useful when creating a new project to add all the
files contained in the targets directory. The following code block illustrates
the usage of this function:
```
>>> list_of_targets = \
... project.get_filepaths_in_directory(“path/within/targets/folder”,
... recursive_walk=False, follow_links=False)
>>> project.add_targets(list_of_targets)
```
### Deleting Targets from a Project
It is possible that we want to delete existing targets inside our project. To
stop the developer tool from tracking this file we can issue the following
command:
```
>>> project.remove_target(“target_1”)
```
Now the target file won't be part of the metadata.

5
tuf/README.md
View file

@ -1,5 +0,0 @@
[Quickstart](../docs/QUICKSTART.md)
[CLI](../docs/CLI.md)
[Tutorial](../docs/TUTORIAL.md)

8
tuf/__init__.py
View file

@ -1,3 +1,9 @@
# Copyright New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""TUF
"""
# This value is used in the requests user agent.
# setup.cfg has it hard-coded separately.
# Currently, when the version is changed, it must be set in both locations.
@ -10,4 +16,4 @@
# All downloaded metadata must be equal to our supported major version of 1.
# For example, "1.4.3" and "1.0.0" are supported. "2.0.0" is not supported.
# See https://github.com/theupdateframework/specification
SPECIFICATION_VERSION = '1.0.0'
SPECIFICATION_VERSION = "1.0.0"

151
tuf/client/README.md
View file

@ -1,151 +0,0 @@
# updater.py
**updater.py** is intended as the only TUF module that software update
systems need to utilize for a low-level integration. It provides a single
class representing an updater that includes methods to download, install, and
verify metadata or target files in a secure manner. Importing
**tuf.client.updater** and instantiating its main class is all that is
required by the client prior to a TUF update request. The importation and
instantiation steps allow TUF to load all of the required metadata files
and set the repository mirror information.
The **tuf.repository_tool** module can be used to create a TUF repository. See
[tuf/README](../README.md) for more information on creating TUF repositories.
## Overview of the Update Process
1. The software update system instructs TUF to check for updates.
2. TUF downloads and verifies timestamp.json.
3. If timestamp.json indicates that snapshot.json has changed, TUF downloads and
verifies snapshot.json.
4. TUF determines which metadata files listed in snapshot.json differ from those
described in the last snapshot.json that TUF has seen. If root.json has changed,
the update process starts over using the new root.json.
5. TUF provides the software update system with a list of available files
according to targets.json.
6. The software update system instructs TUF to download a specific target
file.
7. TUF downloads and verifies the file and then makes the file available to
the software update system.
If at any point in the above procedure there is a problem (i.e., if unexpired,
signed, valid metadata cannot be retrieved from the repository), the Root file
is downloaded and the process is retried once more (and only once to avoid an
infinite loop). Optionally, the software update system using the framework
can decide how to proceed rather than automatically downloading a new Root file.
## Example Client
### Refresh TUF Metadata
```Python
# The client first imports the 'updater.py' module, the only module the
# client is required to import. The client will utilize a single class
# from this module.
import tuf.client.updater
import tuf.settings
# The only other module the client interacts with is 'settings'. The
# client accesses this module solely to set the repository directory.
# This directory will hold the files downloaded from a remote repository.
tuf.settings.repositories_directory = 'path/to/local_repository'
# Next, the client creates a dictionary object containing the repository
# mirrors. The client may download content from any one of these mirrors.
# In the example below, a single mirror named 'mirror1' is defined. The
# mirror is located at 'http://localhost:8001', and all of the metadata
# and targets files can be found in the 'metadata' and 'targets' directory,
# respectively. If the client wishes to only download target files from
# specific directories on the mirror, the 'confined_target_dirs' field
# should be set. In this example, the client hasn't set confined_target_dirs,
# which is interpreted as no confinement. In other words, the client can download
# targets from any directory or subdirectories. If the client had chosen
# 'targets1/', they would have been confined to the '/targets/targets1/'
# directory on the 'http://localhost:8001' mirror.
repository_mirrors = {'mirror1': {'url_prefix': 'http://localhost:8001',
'metadata_path': 'metadata',
'targets_path': 'targets'}}
# The updater may now be instantiated. The Updater class of 'updater.py'
# is called with two arguments. The first argument assigns a name to this
# particular updater and the second argument the repository mirrors defined
# above.
updater = tuf.client.updater.Updater('updater', repository_mirrors)
# The client calls the refresh() method to ensure it has the latest
# copies of the top-level metadata files (i.e., Root, Targets, Snapshot,
# Timestamp).
updater.refresh()
```
### Download Specific Target File
```Python
# Example demonstrating an update that downloads a specific target.
# Refresh the metadata of the top-level roles (i.e., Root, Targets, Snapshot, Timestamp).
updater.refresh()
# get_one_valid_targetinfo() updates role metadata when required. In other
# words, if the client doesn't possess the metadata that lists 'LICENSE.txt',
# get_one_valid_targetinfo() will try to fetch / update it.
target = updater.get_one_valid_targetinfo('LICENSE.txt')
updated_target = updater.updated_targets([target], destination_directory)
for target in updated_target:
updater.download_target(target, destination_directory)
# Client code here may also reference target information (including 'custom')
# by directly accessing the dictionary entries of the target. The 'custom'
# entry is additional file information explicitly set by the remote repository.
target_path = target['filepath']
target_length = target['fileinfo']['length']
target_hashes = target['fileinfo']['hashes']
target_custom_data = target['fileinfo']['custom']
# Remove any files from the destination directory that are no longer being
# tracked. For example, a target file from a previous snapshot that has since
# been removed on the remote repository.
updater.remove_obsolete_targets(destination_directory)
```
### A Simple Integration Example with client.py
``` Bash
# Assume a simple TUF repository has been setup with 'repo.py'.
$ client.py --repo http://localhost:8001
# Metadata and target files are silently updated. An exception is only raised if an error,
# or attack, is detected. Inspect 'tuf.log' for the outcome of the update process.
$ cat tuf.log
[2013-12-16 16:17:05,267 UTC] [tuf.download] [INFO][_download_file:726@download.py]
Downloading: http://localhost:8001/metadata/timestamp.json
[2013-12-16 16:17:05,269 UTC] [tuf.download] [WARNING][_check_content_length:589@download.py]
reported_length (545) < required_length (2048)
[2013-12-16 16:17:05,269 UTC] [tuf.download] [WARNING][_check_downloaded_length:656@download.py]
Downloaded 545 bytes, but expected 2048 bytes. There is a difference of 1503 bytes!
[2013-12-16 16:17:05,611 UTC] [tuf.download] [INFO][_download_file:726@download.py]
Downloading: http://localhost:8001/metadata/snapshot.json
[2013-12-16 16:17:05,612 UTC] [tuf.client.updater] [INFO][_check_hashes:636@updater.py]
The file's sha256 hash is correct: 782675fadd650eeb2926d33c401b5896caacf4fd6766498baf2bce2f3b739db4
[2013-12-16 16:17:05,951 UTC] [tuf.download] [INFO][_download_file:726@download.py]
Downloading: http://localhost:8001/metadata/targets.json
[2013-12-16 16:17:05,952 UTC] [tuf.client.updater] [INFO][_check_hashes:636@updater.py]
The file's sha256 hash is correct: a5019c28a1595c43a14cad2b6252c4d1db472dd6412a9204181ad6d61b1dd69a
[2013-12-16 16:17:06,299 UTC] [tuf.download] [INFO][_download_file:726@download.py]
Downloading: http://localhost:8001/targets/file1.txt
[2013-12-16 16:17:06,303 UTC] [tuf.client.updater] [INFO][_check_hashes:636@updater.py]
The file's sha256 hash is correct: ecdc5536f73bdae8816f0ea40726ef5e9b810d914493075903bb90623d97b1d8

0
tuf/client/__init__.py
View file

38
tuf/client/fetcher.py
View file

@ -1,38 +0,0 @@
# Copyright 2021, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""Provides an interface for network IO abstraction.
"""
# Imports
import abc
# Classes
class FetcherInterface():
"""Defines an interface for abstract network download.
By providing a concrete implementation of the abstract interface,
users of the framework can plug-in their preferred/customized
network stack.
"""
__metaclass__ = abc.ABCMeta
@abc.abstractmethod
def fetch(self, url, required_length):
"""Fetches the contents of HTTP/HTTPS url from a remote server.
Ensures the length of the downloaded data is up to 'required_length'.
Arguments:
url: A URL string that represents a file location.
required_length: An integer value representing the file length in bytes.
Raises:
tuf.exceptions.SlowRetrievalError: A timeout occurs while receiving data.
tuf.exceptions.FetcherHTTPError: An HTTP error code is received.
Returns:
A bytes iterator
"""
raise NotImplementedError # pragma: no cover

3071
tuf/client/updater.py
View file

File diff suppressed because it is too large Load diff

1023
tuf/developer_tool.py
View file

File diff suppressed because it is too large Load diff

314
tuf/download.py
View file

@ -1,314 +0,0 @@
#!/usr/bin/env python
# Copyright 2012 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
download.py
<Started>
February 21, 2012. Based on previous version by Geremy Condra.
<Author>
Konstantin Andrianov
Vladimir Diaz <vladimir.v.diaz@gmail.com>
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Download metadata and target files and check their validity. The hash and
length of a downloaded file has to match the hash and length supplied by the
metadata of that file.
"""
import logging
import timeit
import tempfile
from urllib import parse
import securesystemslib # pylint: disable=unused-import
from securesystemslib import formats as sslib_formats
from tuf import exceptions
from tuf import formats
from tuf import settings
# See 'log.py' to learn how logging is handled in TUF.
logger = logging.getLogger(__name__)
def safe_download(url, required_length, fetcher):
"""
<Purpose>
Given the 'url' and 'required_length' of the desired file, open a connection
to 'url', download it, and return the contents of the file. Also ensure
the length of the downloaded file matches 'required_length' exactly.
download.unsafe_download() may be called if an upper download limit is
preferred.
<Arguments>
url:
A URL string that represents the location of the file.
required_length:
An integer value representing the length of the file. This is an exact
limit.
fetcher:
An object implementing FetcherInterface that performs the network IO
operations.
<Side Effects>
A file object is created on disk to store the contents of 'url'.
<Exceptions>
tuf.ssl_commons.exceptions.DownloadLengthMismatchError, if there was a
mismatch of observed vs expected lengths while downloading the file.
securesystemslib.exceptions.FormatError, if any of the arguments are
improperly formatted.
Any other unforeseen runtime exception.
<Returns>
A file object that points to the contents of 'url'.
"""
# Do all of the arguments have the appropriate format?
# Raise 'securesystemslib.exceptions.FormatError' if there is a mismatch.
sslib_formats.URL_SCHEMA.check_match(url)
formats.LENGTH_SCHEMA.check_match(required_length)
return _download_file(url, required_length, fetcher, STRICT_REQUIRED_LENGTH=True)
def unsafe_download(url, required_length, fetcher):
"""
<Purpose>
Given the 'url' and 'required_length' of the desired file, open a connection
to 'url', download it, and return the contents of the file. Also ensure
the length of the downloaded file is up to 'required_length', and no larger.
download.safe_download() may be called if an exact download limit is
preferred.
<Arguments>
url:
A URL string that represents the location of the file.
required_length:
An integer value representing the length of the file. This is an upper
limit.
fetcher:
An object implementing FetcherInterface that performs the network IO
operations.
<Side Effects>
A file object is created on disk to store the contents of 'url'.
<Exceptions>
tuf.ssl_commons.exceptions.DownloadLengthMismatchError, if there was a
mismatch of observed vs expected lengths while downloading the file.
securesystemslib.exceptions.FormatError, if any of the arguments are
improperly formatted.
Any other unforeseen runtime exception.
<Returns>
A file object that points to the contents of 'url'.
"""
# Do all of the arguments have the appropriate format?
# Raise 'securesystemslib.exceptions.FormatError' if there is a mismatch.
sslib_formats.URL_SCHEMA.check_match(url)
formats.LENGTH_SCHEMA.check_match(required_length)
return _download_file(url, required_length, fetcher, STRICT_REQUIRED_LENGTH=False)
def _download_file(url, required_length, fetcher, STRICT_REQUIRED_LENGTH=True):
"""
<Purpose>
Given the url and length of the desired file, this function opens a
connection to 'url' and downloads the file while ensuring its length
matches 'required_length' if 'STRICT_REQUIRED_LENGH' is True (If False,
the file's length is not checked and a slow retrieval exception is raised
if the downloaded rate falls below the acceptable rate).
<Arguments>
url:
A URL string that represents the location of the file.
required_length:
An integer value representing the length of the file.
STRICT_REQUIRED_LENGTH:
A Boolean indicator used to signal whether we should perform strict
checking of required_length. True by default. We explicitly set this to
False when we know that we want to turn this off for downloading the
timestamp metadata, which has no signed required_length.
<Side Effects>
A file object is created on disk to store the contents of 'url'.
<Exceptions>
tuf.exceptions.DownloadLengthMismatchError, if there was a
mismatch of observed vs expected lengths while downloading the file.
securesystemslib.exceptions.FormatError, if any of the arguments are
improperly formatted.
Any other unforeseen runtime exception.
<Returns>
A file object that points to the contents of 'url'.
"""
# 'url.replace('\\', '/')' is needed for compatibility with Windows-based
# systems, because they might use back-slashes in place of forward-slashes.
# This converts it to the common format. unquote() replaces %xx escapes in a
# url with their single-character equivalent. A back-slash may be encoded as
# %5c in the url, which should also be replaced with a forward slash.
url = parse.unquote(url).replace('\\', '/')
logger.info('Downloading: ' + repr(url))
# This is the temporary file that we will return to contain the contents of
# the downloaded file.
temp_file = tempfile.TemporaryFile()
average_download_speed = 0
number_of_bytes_received = 0
try:
chunks = fetcher.fetch(url, required_length)
start_time = timeit.default_timer()
for chunk in chunks:
stop_time = timeit.default_timer()
temp_file.write(chunk)
# Measure the average download speed.
number_of_bytes_received += len(chunk)
seconds_spent_receiving = stop_time - start_time
average_download_speed = number_of_bytes_received / seconds_spent_receiving
if average_download_speed < settings.MIN_AVERAGE_DOWNLOAD_SPEED:
logger.debug('The average download speed dropped below the minimum'
' average download speed set in settings. Stopping the download!.')
break
else:
logger.debug('The average download speed has not dipped below the'
' minimum average download speed set in settings.')
# Does the total number of downloaded bytes match the required length?
_check_downloaded_length(number_of_bytes_received, required_length,
STRICT_REQUIRED_LENGTH=STRICT_REQUIRED_LENGTH,
average_download_speed=average_download_speed)
except Exception:
# Close 'temp_file'. Any written data is lost.
temp_file.close()
logger.debug('Could not download URL: ' + repr(url))
raise
else:
return temp_file
def _check_downloaded_length(total_downloaded, required_length,
STRICT_REQUIRED_LENGTH=True,
average_download_speed=None):
"""
<Purpose>
A helper function which checks whether the total number of downloaded bytes
matches our expectation.
<Arguments>
total_downloaded:
The total number of bytes supposedly downloaded for the file in question.
required_length:
The total number of bytes expected of the file as seen from its metadata.
The Timestamp role is always downloaded without a known file length, and
the Root role when the client cannot download any of the required
top-level roles. In both cases, 'required_length' is actually an upper
limit on the length of the downloaded file.
STRICT_REQUIRED_LENGTH:
A Boolean indicator used to signal whether we should perform strict
checking of required_length. True by default. We explicitly set this to
False when we know that we want to turn this off for downloading the
timestamp metadata, which has no signed required_length.
average_download_speed:
The average download speed for the downloaded file.
<Side Effects>
None.
<Exceptions>
securesystemslib.exceptions.DownloadLengthMismatchError, if
STRICT_REQUIRED_LENGTH is True and total_downloaded is not equal
required_length.
tuf.exceptions.SlowRetrievalError, if the total downloaded was
done in less than the acceptable download speed (as set in
tuf.settings).
<Returns>
None.
"""
if total_downloaded == required_length:
logger.info('Downloaded ' + str(total_downloaded) + ' bytes out of the'
' expected ' + str(required_length) + ' bytes.')
else:
difference_in_bytes = abs(total_downloaded - required_length)
# What we downloaded is not equal to the required length, but did we ask
# for strict checking of required length?
if STRICT_REQUIRED_LENGTH:
logger.info('Downloaded ' + str(total_downloaded) + ' bytes, but'
' expected ' + str(required_length) + ' bytes. There is a difference'
' of ' + str(difference_in_bytes) + ' bytes.')
# If the average download speed is below a certain threshold, we flag
# this as a possible slow-retrieval attack.
logger.debug('Average download speed: ' + repr(average_download_speed))
logger.debug('Minimum average download speed: ' + repr(settings.MIN_AVERAGE_DOWNLOAD_SPEED))
if average_download_speed < settings.MIN_AVERAGE_DOWNLOAD_SPEED:
raise exceptions.SlowRetrievalError(average_download_speed)
else:
logger.debug('Good average download speed: ' +
repr(average_download_speed) + ' bytes per second')
raise exceptions.DownloadLengthMismatchError(required_length, total_downloaded)
else:
# We specifically disabled strict checking of required length, but we
# will log a warning anyway. This is useful when we wish to download the
# Timestamp or Root metadata, for which we have no signed metadata; so,
# we must guess a reasonable required_length for it.
if average_download_speed < settings.MIN_AVERAGE_DOWNLOAD_SPEED:
raise exceptions.SlowRetrievalError(average_download_speed)
else:
logger.debug('Good average download speed: ' +
repr(average_download_speed) + ' bytes per second')
logger.info('Downloaded ' + str(total_downloaded) + ' bytes out of an'
' upper limit of ' + str(required_length) + ' bytes.')

338
tuf/exceptions.py
View file

@ -1,338 +0,0 @@
#!/usr/bin/env python
# Copyright 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
exceptions.py
<Author>
Vladimir Diaz <vladimir.v.diaz@gmail.com>
<Started>
January 10, 2017
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Define TUF Exceptions.
The names chosen for TUF Exception classes should end in 'Error' except where
there is a good reason not to, and provide that reason in those cases.
"""
from urllib import parse
from typing import Any, Dict, Optional
import logging
logger = logging.getLogger(__name__)
class Error(Exception):
"""Indicate a generic error."""
class UnsupportedSpecificationError(Error):
"""
Metadata received claims to conform to a version of the specification that is
not supported by this client.
"""
class FormatError(Error):
"""Indicate an error while validating an object's format."""
class InvalidMetadataJSONError(FormatError):
"""Indicate that a metadata file is not valid JSON."""
def __init__(self, exception: BaseException):
super(InvalidMetadataJSONError, self).__init__()
# Store the original exception.
self.exception = exception
def __str__(self) -> str:
return repr(self)
def __repr__(self) -> str:
# Show the original exception.
return self.__class__.__name__ + ' : wraps error: ' + repr(self.exception)
# # Directly instance-reproducing:
# return self.__class__.__name__ + '(' + repr(self.exception) + ')'
class UnsupportedAlgorithmError(Error):
"""Indicate an error while trying to identify a user-specified algorithm."""
class LengthOrHashMismatchError(Error):
"""Indicate an error while checking the length and hash values of an object"""
class RepositoryError(Error):
"""Indicate an error with a repository's state, such as a missing file."""
class BadHashError(RepositoryError):
"""Indicate an error while checking the value of a hash object."""
def __init__(self, expected_hash: str, observed_hash: str):
super(BadHashError, self).__init__()
self.expected_hash = expected_hash
self.observed_hash = observed_hash
def __str__(self) -> str:
return (
'Observed hash (' + repr(self.observed_hash) + ') != expected hash (' +
repr(self.expected_hash) + ')')
def __repr__(self) -> str:
return self.__class__.__name__ + ' : ' + str(self)
# # Directly instance-reproducing:
# return (
# self.__class__.__name__ + '(' + repr(self.expected_hash) + ', ' +
# repr(self.observed_hash) + ')')
class BadPasswordError(Error):
"""Indicate an error after encountering an invalid password."""
class UnknownKeyError(Error):
"""Indicate an error while verifying key-like objects (e.g., keyids)."""
class BadVersionNumberError(RepositoryError):
"""Indicate an error for metadata that contains an invalid version number."""
class MissingLocalRepositoryError(RepositoryError):
"""Raised when a local repository could not be found."""
class InsufficientKeysError(Error):
"""Indicate that metadata role lacks a threshold of pubic or private keys."""
class ForbiddenTargetError(RepositoryError):
"""Indicate that a role signed for a target that it was not delegated to."""
class ExpiredMetadataError(RepositoryError):
"""Indicate that a TUF Metadata file has expired."""
class ReplayedMetadataError(RepositoryError):
"""Indicate that some metadata has been replayed to the client."""
def __init__(self, metadata_role: str, downloaded_version: int, current_version: int):
super(ReplayedMetadataError, self).__init__()
self.metadata_role = metadata_role
self.downloaded_version = downloaded_version
self.current_version = current_version
def __str__(self) -> str:
return (
'Downloaded ' + repr(self.metadata_role) + ' is older (' +
repr(self.downloaded_version) + ') than the version currently '
'installed (' + repr(self.current_version) + ').')
def __repr__(self) -> str:
return self.__class__.__name__ + ' : ' + str(self)
class CryptoError(Error):
"""Indicate any cryptography-related errors."""
class BadSignatureError(CryptoError):
"""Indicate that some metadata file has a bad signature."""
def __init__(self, metadata_role_name: str):
super(BadSignatureError, self).__init__()
self.metadata_role_name = metadata_role_name
def __str__(self) -> str:
return repr(self.metadata_role_name) + ' metadata has a bad signature.'
def __repr__(self) -> str:
return self.__class__.__name__ + ' : ' + str(self)
# # Directly instance-reproducing:
# return (
# self.__class__.__name__ + '(' + repr(self.metadata_role_name) + ')')
class UnknownMethodError(CryptoError):
"""Indicate that a user-specified cryptograpthic method is unknown."""
class UnsupportedLibraryError(Error):
"""Indicate that a supported library could not be located or imported."""
class DownloadError(Error):
"""Indicate an error occurred while attempting to download a file."""
class DownloadLengthMismatchError(DownloadError):
"""Indicate that a mismatch of lengths was seen while downloading a file."""
def __init__(self, expected_length: int, observed_length: int):
super(DownloadLengthMismatchError, self).__init__()
self.expected_length = expected_length #bytes
self.observed_length = observed_length #bytes
def __str__(self) -> str:
return (
'Observed length (' + repr(self.observed_length) +
') < expected length (' + repr(self.expected_length) + ').')
def __repr__(self) -> str:
return self.__class__.__name__ + ' : ' + str(self)
# # Directly instance-reproducing:
# return (
# self.__class__.__name__ + '(' + repr(self.expected_length) + ', ' +
# self.observed_length + ')')
class SlowRetrievalError(DownloadError):
""""Indicate that downloading a file took an unreasonably long time."""
def __init__(self, average_download_speed: Optional[int] = None):
super(SlowRetrievalError, self).__init__()
self.__average_download_speed = average_download_speed #bytes/second
def __str__(self) -> str:
msg = 'Download was too slow.'
if self.__average_download_speed is not None:
msg = ('Download was too slow. Average speed: ' +
repr(self.__average_download_speed) + ' bytes per second.')
return msg
def __repr__(self) -> str:
return self.__class__.__name__ + ' : ' + str(self)
# # Directly instance-reproducing:
# return (
# self.__class__.__name__ + '(' + repr(self.__average_download_speed + ')')
class KeyAlreadyExistsError(Error):
"""Indicate that a key already exists and cannot be added."""
class RoleAlreadyExistsError(Error):
"""Indicate that a role already exists and cannot be added."""
class UnknownRoleError(Error):
"""Indicate an error trying to locate or identify a specified TUF role."""
class UnknownTargetError(Error):
"""Indicate an error trying to locate or identify a specified target."""
class InvalidNameError(Error):
"""Indicate an error while trying to validate any type of named object."""
class UnsignedMetadataError(RepositoryError):
"""Indicate metadata object with insufficient threshold of signatures."""
# signable is not used but kept in method signature for backwards compat
def __init__(self, message: str, signable: Any = None):
super(UnsignedMetadataError, self).__init__()
self.exception_message = message
self.signable = signable
def __str__(self) -> str:
return self.exception_message
def __repr__(self) -> str:
return self.__class__.__name__ + ' : ' + str(self)
# # Directly instance-reproducing:
# return (
# self.__class__.__name__ + '(' + repr(self.exception_message) + ', ' +
# repr(self.signable) + ')')
class NoWorkingMirrorError(Error):
"""
An updater will throw this exception in case it could not download a
metadata or target file.
A dictionary of Exception instances indexed by every mirror URL will also be
provided.
"""
def __init__(self, mirror_errors: Dict[str, BaseException]):
super(NoWorkingMirrorError, self).__init__()
# Dictionary of URL strings to Exception instances
self.mirror_errors = mirror_errors
def __str__(self) -> str:
all_errors = 'No working mirror was found:'
for mirror_url, mirror_error in self.mirror_errors.items():
try:
# http://docs.python.org/2/library/urlparse.html#urlparse.urlparse
mirror_url_tokens = parse.urlparse(mirror_url)
except Exception:
logger.exception('Failed to parse mirror URL: ' + repr(mirror_url))
mirror_netloc = mirror_url
else:
mirror_netloc = mirror_url_tokens.netloc
all_errors += '\n ' + repr(mirror_netloc) + ': ' + repr(mirror_error)
return all_errors
def __repr__(self) -> str:
return self.__class__.__name__ + ' : ' + str(self)
# # Directly instance-reproducing:
# return (
# self.__class__.__name__ + '(' + repr(self.mirror_errors) + ')')
class NotFoundError(Error):
"""If a required configuration or resource is not found."""
class URLMatchesNoPatternError(Error):
"""If a URL does not match a user-specified regular expression."""
class URLParsingError(Error):
"""If we are unable to parse a URL -- for example, if a hostname element
cannot be isoalted."""
class InvalidConfigurationError(Error):
"""If a configuration object does not match the expected format."""
class FetcherHTTPError(Exception):
"""
Returned by FetcherInterface implementations for HTTP errors.
Args:
message (str): The HTTP error messsage
status_code (int): The HTTP status code
"""
def __init__(self, message: str, status_code: int):
super(FetcherHTTPError, self).__init__(message)
self.status_code = status_code

1009
tuf/formats.py
View file

File diff suppressed because it is too large Load diff

440
tuf/keydb.py
View file

@ -1,440 +0,0 @@
#!/usr/bin/env python
# Copyright 2012 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
keydb.py
<Author>
Vladimir Diaz <vladimir.v.diaz@gmail.com>
<Started>
March 21, 2012. Based on a previous version of this module by Geremy Condra.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Represent a collection of keys and their organization. This module ensures
the layout of the collection remain consistent and easily verifiable.
Provided are functions to add and delete keys from the database, retrieve a
single key, and assemble a collection from keys stored in TUF 'Root' Metadata.
The Update Framework process maintains a set of role info for multiple
repositories.
RSA keys are currently supported and a collection of keys is organized as a
dictionary indexed by key ID. Key IDs are used as identifiers for keys
(e.g., RSA key). They are the hexadecimal representations of the hash of key
objects (specifically, the key object containing only the public key). See
'rsa_key.py' and the '_get_keyid()' function to learn precisely how keyids
are generated. One may get the keyid of a key object by simply accessing the
dictionary's 'keyid' key (i.e., rsakey['keyid']).
"""
import logging
import copy
import securesystemslib # pylint: disable=unused-import
from securesystemslib import exceptions as sslib_exceptions
from securesystemslib import formats as sslib_formats
from securesystemslib import keys as sslib_keys
from tuf import exceptions
from tuf import formats
# List of strings representing the key types supported by TUF.
_SUPPORTED_KEY_TYPES = ['rsa', 'ed25519', 'ecdsa', 'ecdsa-sha2-nistp256']
# See 'log.py' to learn how logging is handled in TUF.
logger = logging.getLogger(__name__)
# The key database.
_keydb_dict = {}
_keydb_dict['default'] = {}
def create_keydb_from_root_metadata(root_metadata, repository_name='default'):
"""
<Purpose>
Populate the key database with the unique keys found in 'root_metadata'.
The database dictionary will conform to
'tuf.formats.KEYDB_SCHEMA' and have the form: {keyid: key,
...}. The 'keyid' conforms to 'securesystemslib.formats.KEYID_SCHEMA' and
'key' to its respective type. In the case of RSA keys, this object would
match 'RSAKEY_SCHEMA'.
<Arguments>
root_metadata:
A dictionary conformant to 'tuf.formats.ROOT_SCHEMA'. The keys found
in the 'keys' field of 'root_metadata' are needed by this function.
repository_name:
The name of the repository to store the key information. If not supplied,
the key database is populated for the 'default' repository.
<Exceptions>
securesystemslib.exceptions.FormatError, if 'root_metadata' does not have the correct format.
securesystemslib.exceptions.InvalidNameError, if 'repository_name' does not exist in the key
database.
<Side Effects>
A function to add the key to the database is called. In the case of RSA
keys, this function is add_key().
The old keydb key database is replaced.
<Returns>
None.
"""
# Does 'root_metadata' have the correct format?
# This check will ensure 'root_metadata' has the appropriate number of objects
# and object types, and that all dict keys are properly named.
# Raise 'securesystemslib.exceptions.FormatError' if the check fails.
formats.ROOT_SCHEMA.check_match(root_metadata)
# Does 'repository_name' have the correct format?
sslib_formats.NAME_SCHEMA.check_match(repository_name)
# Clear the key database for 'repository_name', or create it if non-existent.
if repository_name in _keydb_dict:
_keydb_dict[repository_name].clear()
else:
create_keydb(repository_name)
# Iterate the keys found in 'root_metadata' by converting them to
# 'RSAKEY_SCHEMA' if their type is 'rsa', and then adding them to the
# key database using the provided keyid.
for keyid, key_metadata in root_metadata['keys'].items():
if key_metadata['keytype'] in _SUPPORTED_KEY_TYPES:
# 'key_metadata' is stored in 'KEY_SCHEMA' format. Call
# create_from_metadata_format() to get the key in 'RSAKEY_SCHEMA' format,
# which is the format expected by 'add_key()'. Note: This call to
# format_metadata_to_key() uses the provided keyid as the default keyid.
# All other keyids returned are ignored.
key_dict, _ = sslib_keys.format_metadata_to_key(key_metadata,
keyid)
# Make sure to update key_dict['keyid'] to use one of the other valid
# keyids, otherwise add_key() will have no reference to it.
try:
add_key(key_dict, repository_name=repository_name)
# Although keyid duplicates should *not* occur (unique dict keys), log a
# warning and continue. However, 'key_dict' may have already been
# adding to the keydb elsewhere.
except exceptions.KeyAlreadyExistsError as e: # pragma: no cover
logger.warning(e)
continue
else:
logger.warning('Root Metadata file contains a key with an invalid keytype.')
def create_keydb(repository_name):
"""
<Purpose>
Create a key database for a non-default repository named 'repository_name'.
<Arguments>
repository_name:
The name of the repository. An empty key database is created, and keys
may be added to via add_key(keyid, repository_name).
<Exceptions>
securesystemslib.exceptions.FormatError, if 'repository_name' is improperly formatted.
securesystemslib.exceptions.InvalidNameError, if 'repository_name' already exists.
<Side Effects>
None.
<Returns>
None.
"""
# Is 'repository_name' properly formatted? Raise 'securesystemslib.exceptions.FormatError' if not.
sslib_formats.NAME_SCHEMA.check_match(repository_name)
if repository_name in _keydb_dict:
raise sslib_exceptions.InvalidNameError('Repository name already exists:'
' ' + repr(repository_name))
_keydb_dict[repository_name] = {}
def remove_keydb(repository_name):
"""
<Purpose>
Remove a key database for a non-default repository named 'repository_name'.
The 'default' repository cannot be removed.
<Arguments>
repository_name:
The name of the repository to remove. The 'default' repository should
not be removed, so 'repository_name' cannot be 'default'.
<Exceptions>
securesystemslib.exceptions.FormatError, if 'repository_name' is improperly formatted.
securesystemslib.exceptions.InvalidNameError, if 'repository_name' is 'default'.
<Side Effects>
None.
<Returns>
None.
"""
# Is 'repository_name' properly formatted? Raise 'securesystemslib.exceptions.FormatError' if not.
sslib_formats.NAME_SCHEMA.check_match(repository_name)
if repository_name not in _keydb_dict:
logger.warning('Repository name does not exist: ' + repr(repository_name))
return
if repository_name == 'default':
raise sslib_exceptions.InvalidNameError('Cannot remove the default repository:'
' ' + repr(repository_name))
del _keydb_dict[repository_name]
def add_key(key_dict, keyid=None, repository_name='default'):
"""
<Purpose>
Add 'rsakey_dict' to the key database while avoiding duplicates.
If keyid is provided, verify it is the correct keyid for 'rsakey_dict'
and raise an exception if it is not.
<Arguments>
key_dict:
A dictionary conformant to 'securesystemslib.formats.ANYKEY_SCHEMA'.
It has the form:
{'keytype': 'rsa',
'keyid': keyid,
'keyval': {'public': '-----BEGIN RSA PUBLIC KEY----- ...',
'private': '-----BEGIN RSA PRIVATE KEY----- ...'}}
keyid:
An object conformant to 'KEYID_SCHEMA'. It is used as an identifier
for RSA keys.
repository_name:
The name of the repository to add the key. If not supplied, the key is
added to the 'default' repository.
<Exceptions>
securesystemslib.exceptions.FormatError, if the arguments do not have the correct format.
securesystemslib.exceptions.Error, if 'keyid' does not match the keyid for 'rsakey_dict'.
tuf.exceptions.KeyAlreadyExistsError, if 'rsakey_dict' is found in the key database.
securesystemslib.exceptions.InvalidNameError, if 'repository_name' does not exist in the key
database.
<Side Effects>
The keydb key database is modified.
<Returns>
None.
"""
# Does 'key_dict' have the correct format?
# This check will ensure 'key_dict' has the appropriate number of objects
# and object types, and that all dict keys are properly named.
# Raise 'securesystemslib.exceptions.FormatError if the check fails.
sslib_formats.ANYKEY_SCHEMA.check_match(key_dict)
# Does 'repository_name' have the correct format?
sslib_formats.NAME_SCHEMA.check_match(repository_name)
# Does 'keyid' have the correct format?
if keyid is not None:
# Raise 'securesystemslib.exceptions.FormatError' if the check fails.
sslib_formats.KEYID_SCHEMA.check_match(keyid)
# Check if each keyid found in 'key_dict' matches 'keyid'.
if keyid != key_dict['keyid']:
raise sslib_exceptions.Error('Incorrect keyid. Got ' + key_dict['keyid'] + ' but expected ' + keyid)
# Ensure 'repository_name' is actually set in the key database.
if repository_name not in _keydb_dict:
raise sslib_exceptions.InvalidNameError('Repository name does not exist:'
' ' + repr(repository_name))
# Check if the keyid belonging to 'key_dict' is not already
# available in the key database before returning.
keyid = key_dict['keyid']
if keyid in _keydb_dict[repository_name]:
raise exceptions.KeyAlreadyExistsError('Key: ' + keyid)
_keydb_dict[repository_name][keyid] = copy.deepcopy(key_dict)
def get_key(keyid, repository_name='default'):
"""
<Purpose>
Return the key belonging to 'keyid'.
<Arguments>
keyid:
An object conformant to 'securesystemslib.formats.KEYID_SCHEMA'. It is used as an
identifier for keys.
repository_name:
The name of the repository to get the key. If not supplied, the key is
retrieved from the 'default' repository.
<Exceptions>
securesystemslib.exceptions.FormatError, if the arguments do not have the correct format.
tuf.exceptions.UnknownKeyError, if 'keyid' is not found in the keydb database.
securesystemslib.exceptions.InvalidNameError, if 'repository_name' does not exist in the key
database.
<Side Effects>
None.
<Returns>
The key matching 'keyid'. In the case of RSA keys, a dictionary conformant
to 'securesystemslib.formats.RSAKEY_SCHEMA' is returned.
"""
# Does 'keyid' have the correct format?
# This check will ensure 'keyid' has the appropriate number of objects
# and object types, and that all dict keys are properly named.
# Raise 'securesystemslib.exceptions.FormatError' is the match fails.
sslib_formats.KEYID_SCHEMA.check_match(keyid)
# Does 'repository_name' have the correct format?
sslib_formats.NAME_SCHEMA.check_match(repository_name)
if repository_name not in _keydb_dict:
raise sslib_exceptions.InvalidNameError('Repository name does not exist:'
' ' + repr(repository_name))
# Return the key belonging to 'keyid', if found in the key database.
try:
return copy.deepcopy(_keydb_dict[repository_name][keyid])
except KeyError as error:
raise exceptions.UnknownKeyError('Key: ' + keyid) from error
def remove_key(keyid, repository_name='default'):
"""
<Purpose>
Remove the key belonging to 'keyid'.
<Arguments>
keyid:
An object conformant to 'securesystemslib.formats.KEYID_SCHEMA'. It is used as an
identifier for keys.
repository_name:
The name of the repository to remove the key. If not supplied, the key
is removed from the 'default' repository.
<Exceptions>
securesystemslib.exceptions.FormatError, if the arguments do not have the correct format.
tuf.exceptions.UnknownKeyError, if 'keyid' is not found in key database.
securesystemslib.exceptions.InvalidNameError, if 'repository_name' does not exist in the key
database.
<Side Effects>
The key, identified by 'keyid', is deleted from the key database.
<Returns>
None.
"""
# Does 'keyid' have the correct format?
# This check will ensure 'keyid' has the appropriate number of objects
# and object types, and that all dict keys are properly named.
# Raise 'securesystemslib.exceptions.FormatError' is the match fails.
sslib_formats.KEYID_SCHEMA.check_match(keyid)
# Does 'repository_name' have the correct format?
sslib_formats.NAME_SCHEMA.check_match(repository_name)
if repository_name not in _keydb_dict:
raise sslib_exceptions.InvalidNameError('Repository name does not exist:'
' ' + repr(repository_name))
# Remove the key belonging to 'keyid' if found in the key database.
if keyid in _keydb_dict[repository_name]:
del _keydb_dict[repository_name][keyid]
else:
raise exceptions.UnknownKeyError('Key: ' + keyid)
def clear_keydb(repository_name='default', clear_all=False):
"""
<Purpose>
Clear the keydb key database.
<Arguments>
repository_name:
The name of the repository to clear the key database. If not supplied,
the key database is cleared for the 'default' repository.
clear_all:
Boolean indicating whether to clear the entire keydb.
<Exceptions>
securesystemslib.exceptions.FormatError, if 'repository_name' is improperly formatted.
securesystemslib.exceptions.InvalidNameError, if 'repository_name' does not exist in the key
database.
<Side Effects>
The keydb key database is reset.
<Returns>
None.
"""
# Do the arguments have the correct format? Raise 'securesystemslib.exceptions.FormatError' if
# 'repository_name' is improperly formatted.
sslib_formats.NAME_SCHEMA.check_match(repository_name)
sslib_formats.BOOLEAN_SCHEMA.check_match(clear_all)
if clear_all:
_keydb_dict.clear()
_keydb_dict['default'] = {}
if repository_name not in _keydb_dict:
raise sslib_exceptions.InvalidNameError('Repository name does not exist:'
' ' + repr(repository_name))
_keydb_dict[repository_name] = {}

448
tuf/log.py
View file

@ -1,448 +0,0 @@
#!/usr/bin/env python
# Copyright 2012 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
log.py
<Author>
Vladimir Diaz <vladimir.v.diaz@gmail.com>
<Started>
April 4, 2012. Based on a previous version of this module by Geremy Condra.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
A central location for all logging-related configuration. This module should
be imported once by the main program. If other modules wish to incorporate
'tuf' logging, they should do the following:
import logging
logger = logging.getLogger('tuf')
'logging' refers to the module name. logging.getLogger() is a function of
the module 'logging'. logging.getLogger(name) returns a Logger instance
associated with 'name'. Calling getLogger(name) will always return the same
instance. In this 'log.py' module, we perform the initial setup for the name
'tuf'. The 'log.py' module should only be imported once by the main program.
When any other module does a logging.getLogger('tuf'), it is referring to the
same 'tuf' instance, and its associated settings, set here in 'log.py'.
See http://docs.python.org/library/logging.html#logger-objects for more
information.
We use multiple handlers to process log messages in various ways and to
configure each one independently. Instead of using one single manner of
processing log messages, we can use two built-in handlers that have already
been configured for us. For example, the built-in FileHandler will catch
log messages and dump them to a file. If we wanted, we could set this file
handler to only catch CRITICAL (and greater) messages and save them to a
file. Other handlers (e.g., StreamHandler) could handle INFO-level
(and greater) messages.
Logging Levels:
--Level-- --Value--
logging.CRITICAL 50
logging.ERROR 40
logging.WARNING 30
logging.INFO 20
logging.DEBUG 10
logging.NOTSET 0
The logging module is thread-safe. Logging to a single file from
multiple threads in a single process is also thread-safe. The logging
module is NOT thread-safe when logging to a single file across multiple
processes:
http://docs.python.org/library/logging.html#thread-safety
http://docs.python.org/howto/logging-cookbook.html
"""
import logging
import time
from securesystemslib import exceptions as sslib_exceptions
from securesystemslib import formats as sslib_formats
from tuf import exceptions
from tuf import settings
# Setting a handler's log level filters only logging messages of that level
# (and above). For example, setting the built-in StreamHandler's log level to
# 'logging.WARNING' will cause the stream handler to only process messages
# of levels: WARNING, ERROR, and CRITICAL.
_DEFAULT_LOG_LEVEL = logging.DEBUG
_DEFAULT_CONSOLE_LOG_LEVEL = logging.INFO
_DEFAULT_FILE_LOG_LEVEL = logging.DEBUG
# Set the format for logging messages.
# Example format for '_FORMAT_STRING':
# [2013-08-13 15:21:18,068 localtime] [tuf]
# [INFO][_update_metadata:851@updater.py]
_FORMAT_STRING = '[%(asctime)s UTC] [%(name)s] [%(levelname)s] '+\
'[%(funcName)s:%(lineno)s@%(filename)s]\n%(message)s\n'
# Ask all Formatter instances to talk GMT. Set the 'converter' attribute of
# 'logging.Formatter' so that all formatters use Greenwich Mean Time.
# http://docs.python.org/library/logging.html#logging.Formatter.formatTime
# The 2nd paragraph in the link above contains the relevant information.
# GMT = UTC (Coordinated Universal Time). TUF metadata stores timestamps in UTC.
# We previously displayed the local time but this lead to confusion when
# visually comparing logger events and metadata information. Unix time stamps
# are fine but they may be less human-readable than UTC.
logging.Formatter.converter = time.gmtime
formatter = logging.Formatter(_FORMAT_STRING)
# Set the handlers for the logger. The console handler is unset by default. A
# module importing 'log.py' should explicitly set the console handler if
# outputting log messages to the screen is needed. Adding a console handler can
# be done with tuf.log.add_console_handler(). Logging messages to a file is not
# set by default.
console_handler = None
file_handler = None
# Set the logger and its settings.
# Note: we're configuring the top-level hierarchy for the tuf package,
# therefore we explicitly request the 'tuf' logger, rather than following
# the standard pattern of logging.getLogger(__name__)
logger = logging.getLogger('tuf')
logger.setLevel(_DEFAULT_LOG_LEVEL)
logger.addHandler(logging.NullHandler())
# Set the built-in file handler. Messages will be logged to
# 'settings.LOG_FILENAME', and only those messages with a log level of
# '_DEFAULT_LOG_LEVEL'. The log level of messages handled by 'file_handler'
# may be modified with 'set_filehandler_log_level()'. 'settings.LOG_FILENAME'
# will be opened in append mode.
if settings.ENABLE_FILE_LOGGING:
file_handler = logging.FileHandler(settings.LOG_FILENAME)
file_handler.setLevel(_DEFAULT_FILE_LOG_LEVEL)
file_handler.setFormatter(formatter)
logger.addHandler(file_handler)
else:
pass
# Silently ignore logger exceptions.
logging.raiseExceptions = False
class ConsoleFilter(logging.Filter):
def filter(self, record):
"""
<Purpose>
Use Vinay Sajip's recommendation from Python issue #6435 to modify a
LogRecord object. This is meant to be used with our console handler.
http://stackoverflow.com/q/6177520
http://stackoverflow.com/q/5875225
http://bugs.python.org/issue6435
http://docs.python.org/howto/logging-cookbook.html#filters-contextual
http://docs.python.org/library/logging.html#logrecord-attributes
<Arguments>
record:
A logging.LogRecord object.
<Exceptions>
None.
<Side Effects>
Replaces the LogRecord exception text attribute.
<Returns>
True.
"""
# If this LogRecord object has an exception, then we will replace its text.
if record.exc_info:
# We place the record's cached exception text (which usually contains the
# exception traceback) with much simpler exception information. This is
# most useful for the console handler, which we do not wish to deluge
# with too much data. Assuming that this filter is not applied to the
# file logging handler, the user may always consult the file log for the
# original exception traceback. The exc_info is explained here:
# http://docs.python.org/library/sys.html#sys.exc_info
exc_type, _, _ = record.exc_info
# Simply set the class name as the exception text.
record.exc_text = exc_type.__name__
# Always return True to signal that any given record must be formatted.
return True
def set_log_level(log_level: int=_DEFAULT_LOG_LEVEL):
"""
<Purpose>
Allow the default log level to be overridden. If 'log_level' is not
provided, log level defaults to 'logging.DEBUG'.
<Arguments>
log_level:
The log level to set for the 'log.py' file handler.
'log_level' examples: logging.INFO; logging.CRITICAL.
<Exceptions>
None.
<Side Effects>
Overrides the logging level for the 'log.py' file handler.
<Returns>
None.
"""
# Does 'log_level' have the correct format?
# Raise 'securesystems.exceptions.FormatError' if there is a mismatch.
sslib_formats.LOGLEVEL_SCHEMA.check_match(log_level)
logger.setLevel(log_level)
def set_filehandler_log_level(log_level=_DEFAULT_FILE_LOG_LEVEL):
"""
<Purpose>
Allow the default file handler log level to be overridden. If 'log_level'
is not provided, log level defaults to 'logging.DEBUG'.
<Arguments>
log_level:
The log level to set for the 'log.py' file handler.
'log_level' examples: logging.INFO; logging.CRITICAL.
<Exceptions>
None.
<Side Effects>
Overrides the logging level for the 'log.py' file handler.
<Returns>
None.
"""
# Does 'log_level' have the correct format?
# Raise 'securesystems.exceptions.FormatError' if there is a mismatch.
sslib_formats.LOGLEVEL_SCHEMA.check_match(log_level)
if file_handler:
file_handler.setLevel(log_level)
else:
raise exceptions.Error(
'File handler has not been set. Enable file logging'
' before attempting to set its log level')
def set_console_log_level(log_level=_DEFAULT_CONSOLE_LOG_LEVEL):
"""
<Purpose>
Allow the default log level for console messages to be overridden. If
'log_level' is not provided, log level defaults to 'logging.INFO'.
<Arguments>
log_level:
The log level to set for the console handler.
'log_level' examples: logging.INFO; logging.CRITICAL.
<Exceptions>
securesystemslib.exceptions.Error, if the 'log.py' console handler has not
been set yet with add_console_handler().
<Side Effects>
Overrides the logging level for the console handler.
<Returns>
None.
"""
# Does 'log_level' have the correct format?
# Raise 'securesystems.exceptions.FormatError' if there is a mismatch.
sslib_formats.LOGLEVEL_SCHEMA.check_match(log_level)
if console_handler is not None:
console_handler.setLevel(log_level)
else:
message = 'The console handler has not been set with add_console_handler().'
raise sslib_exceptions.Error(message)
def add_console_handler(log_level=_DEFAULT_CONSOLE_LOG_LEVEL):
"""
<Purpose>
Add a console handler and set its log level to 'log_level'.
<Arguments>
log_level:
The log level to set for the console handler.
'log_level' examples: logging.INFO; logging.CRITICAL.
<Exceptions>
None.
<Side Effects>
Adds a console handler to the 'log.py' logger and sets its logging level to
'log_level'.
<Returns>
None.
"""
# Does 'log_level' have the correct format?
# Raise 'securesystems.exceptions.FormatError' if there is a mismatch.
sslib_formats.LOGLEVEL_SCHEMA.check_match(log_level)
# Assign to the global console_handler object.
global console_handler
if not console_handler:
# Set the console handler for the logger. The built-in console handler will
# log messages to 'sys.stderr' and capture 'log_level' messages.
console_handler = logging.StreamHandler()
# Get our filter for the console handler.
console_filter = ConsoleFilter()
console_format_string = '%(message)s'
console_formatter = logging.Formatter(console_format_string)
console_handler.setLevel(log_level)
console_handler.setFormatter(console_formatter)
console_handler.addFilter(console_filter)
logger.addHandler(console_handler)
logger.debug('Added a console handler.')
else:
logger.warning('We already have a console handler.')
def remove_console_handler():
"""
<Purpose>
Remove the console handler from the logger in 'log.py', if previously added.
<Arguments>
None.
<Exceptions>
None.
<Side Effects>
A handler belonging to the console is removed from the 'log.py' logger
and the console handler is marked as unset.
<Returns>
None.
"""
# Assign to the global 'console_handler' object.
global console_handler
if console_handler:
logger.removeHandler(console_handler)
console_handler = None
logger.debug('Removed a console handler.')
else:
logger.warning('We do not have a console handler.')
def enable_file_logging(log_filename=settings.LOG_FILENAME):
"""
<Purpose>
Log messages to a file (i.e., 'log_filename'). The log level for the file
handler can be set with set_filehandler_log_level().
<Arguments>
log_filename:
Logging messages are saved to this file. If not provided, the log
filename specified in tuf.settings.LOG_FILENAME is used.
<Exceptions>
securesystemslib.exceptions.FormatError, if any of the arguments are
not the expected format.
tuf.exceptions.Error, if the file handler has already been set.
<Side Effects>
The global file handler is set.
<Returns>
None.
"""
# Are the arguments properly formatted?
sslib_formats.PATH_SCHEMA.check_match(log_filename)
global file_handler
# Add a file handler to the logger if not already set.
if not file_handler:
file_handler = logging.FileHandler(log_filename)
file_handler.setLevel(_DEFAULT_FILE_LOG_LEVEL)
file_handler.setFormatter(formatter)
logger.addHandler(file_handler)
else:
raise exceptions.Error(
'The file handler has already been been set. A new file handler'
' can be set by first calling disable_file_logging()')
def disable_file_logging():
"""
<Purpose>
Disable file logging by removing any previously set file handler.
A warning is logged if the file handler cannot be removed.
The file that was written to will not be deleted.
<Arguments>
None.
<Exceptions>
None.
<Side Effects>
The global file handler is unset.
<Returns>
None.
"""
# Assign to the global 'file_handler' object.
global file_handler
if file_handler:
logger.removeHandler(file_handler)
file_handler.close()
file_handler = None
logger.debug('Removed the file handler.')
else:
logger.warning('A file handler has not been set.')

122
tuf/mirrors.py
View file

@ -1,122 +0,0 @@
#!/usr/bin/env python
# Copyright 2012 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
mirrors.py
<Author>
Konstantin Andrianov.
Derived from original mirrors.py written by Geremy Condra.
<Started>
March 12, 2012.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Extract a list of mirror urls corresponding to the file type and the location
of the file with respect to the base url.
"""
import os
from urllib import parse
import securesystemslib # pylint: disable=unused-import
from securesystemslib import exceptions as sslib_exceptions
from securesystemslib import formats as sslib_formats
from securesystemslib.util import file_in_confined_directories
from tuf import formats
# The type of file to be downloaded from a repository. The
# 'get_list_of_mirrors' function supports these file types.
_SUPPORTED_FILE_TYPES = ['meta', 'target']
def get_list_of_mirrors(file_type, file_path, mirrors_dict):
"""
<Purpose>
Get a list of mirror urls from a mirrors dictionary, provided the type
and the path of the file with respect to the base url.
<Arguments>
file_type:
Type of data needed for download, must correspond to one of the strings
in the list ['meta', 'target']. 'meta' for metadata file type or
'target' for target file type. It should correspond to
NAME_SCHEMA format.
file_path:
A relative path to the file that corresponds to RELPATH_SCHEMA format.
Ex: 'http://url_prefix/targets_path/file_path'
mirrors_dict:
A mirrors_dict object that corresponds to MIRRORDICT_SCHEMA, where
keys are strings and values are MIRROR_SCHEMA. An example format
of MIRROR_SCHEMA:
{'url_prefix': 'http://localhost:8001',
'metadata_path': 'metadata/',
'targets_path': 'targets/',
'confined_target_dirs': ['targets/snapshot1/', ...],
'custom': {...}}
The 'custom' field is optional.
<Exceptions>
securesystemslib.exceptions.Error, on unsupported 'file_type'.
securesystemslib.exceptions.FormatError, on bad argument.
<Return>
List of mirror urls corresponding to the file_type and file_path. If no
match is found, empty list is returned.
"""
# Checking if all the arguments have appropriate format.
formats.RELPATH_SCHEMA.check_match(file_path)
formats.MIRRORDICT_SCHEMA.check_match(mirrors_dict)
sslib_formats.NAME_SCHEMA.check_match(file_type)
# Verify 'file_type' is supported.
if file_type not in _SUPPORTED_FILE_TYPES:
raise sslib_exceptions.Error('Invalid file_type argument.'
' Supported file types: ' + repr(_SUPPORTED_FILE_TYPES))
path_key = 'metadata_path' if file_type == 'meta' else 'targets_path'
list_of_mirrors = []
for junk, mirror_info in mirrors_dict.items():
# Does mirror serve this file type at all?
path = mirror_info.get(path_key)
if path is None:
continue
# for targets, ensure directory confinement
if path_key == 'targets_path':
full_filepath = os.path.join(path, file_path)
confined_target_dirs = mirror_info.get('confined_target_dirs')
# confined_target_dirs is optional and can used to confine the client to
# certain paths on a repository mirror when fetching target files.
if confined_target_dirs and not file_in_confined_directories(full_filepath,
confined_target_dirs):
continue
# parse.quote(string) replaces special characters in string using the %xx
# escape. This is done to avoid parsing issues of the URL on the server
# side. Do *NOT* pass URLs with Unicode characters without first encoding
# the URL as UTF-8. We need a long-term solution with #61.
# http://bugs.python.org/issue1712522
file_path = parse.quote(file_path)
url = os.path.join(mirror_info['url_prefix'], path, file_path)
# The above os.path.join() result as well as input file_path may be
# invalid on windows (might contain both separator types), see #1077.
# Make sure the URL doesn't contain backward slashes on Windows.
list_of_mirrors.append(url.replace('\\', '/'))
return list_of_mirrors

2306
tuf/repository_lib.py
View file

File diff suppressed because it is too large Load diff

3291
tuf/repository_tool.py
View file

File diff suppressed because it is too large Load diff

173
tuf/requests_fetcher.py
View file

@ -1,173 +0,0 @@
# Copyright 2021, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""Provides an implementation of FetcherInterface using the Requests HTTP
library.
"""
# Imports
import requests
import logging
import time
from urllib import parse
from urllib3.exceptions import ReadTimeoutError
import tuf
from tuf import exceptions
from tuf import settings
from tuf.client.fetcher import FetcherInterface
# Globals
logger = logging.getLogger(__name__)
# Classess
class RequestsFetcher(FetcherInterface):
"""A concrete implementation of FetcherInterface based on the Requests
library.
Attributes:
_sessions: A dictionary of Requests.Session objects storing a separate
session per scheme+hostname combination.
"""
def __init__(self):
# From http://docs.python-requests.org/en/master/user/advanced/#session-objects:
#
# "The Session object allows you to persist certain parameters across
# requests. It also persists cookies across all requests made from the
# Session instance, and will use urllib3's connection pooling. So if you're
# making several requests to the same host, the underlying TCP connection
# will be reused, which can result in a significant performance increase
# (see HTTP persistent connection)."
#
# NOTE: We use a separate requests.Session per scheme+hostname combination,
# in order to reuse connections to the same hostname to improve efficiency,
# but avoiding sharing state between different hosts-scheme combinations to
# minimize subtle security issues. Some cookies may not be HTTP-safe.
self._sessions = {}
def fetch(self, url, required_length):
"""Fetches the contents of HTTP/HTTPS url from a remote server.
Ensures the length of the downloaded data is up to 'required_length'.
Arguments:
url: A URL string that represents a file location.
required_length: An integer value representing the file length in bytes.
Raises:
tuf.exceptions.SlowRetrievalError: A timeout occurs while receiving data.
tuf.exceptions.FetcherHTTPError: An HTTP error code is received.
Returns:
A bytes iterator
"""
# Get a customized session for each new schema+hostname combination.
session = self._get_session(url)
# Get the requests.Response object for this URL.
#
# Defer downloading the response body with stream=True.
# Always set the timeout. This timeout value is interpreted by requests as:
# - connect timeout (max delay before first byte is received)
# - read (gap) timeout (max delay between bytes received)
response = session.get(url, stream=True,
timeout=settings.SOCKET_TIMEOUT)
# Check response status.
try:
response.raise_for_status()
except requests.HTTPError as e:
response.close()
status = e.response.status_code
raise exceptions.FetcherHTTPError(str(e), status)
# Define a generator function to be returned by fetch. This way the caller
# of fetch can differentiate between connection and actual data download
# and measure download times accordingly.
def chunks():
try:
bytes_received = 0
while True:
# We download a fixed chunk of data in every round. This is so that we
# can defend against slow retrieval attacks. Furthermore, we do not
# wish to download an extremely large file in one shot.
# Before beginning the round, sleep (if set) for a short amount of
# time so that the CPU is not hogged in the while loop.
if settings.SLEEP_BEFORE_ROUND:
time.sleep(settings.SLEEP_BEFORE_ROUND)
read_amount = min(
settings.CHUNK_SIZE, required_length - bytes_received)
# NOTE: This may not handle some servers adding a Content-Encoding
# header, which may cause urllib3 to misbehave:
# https://github.com/pypa/pip/blob/404838abcca467648180b358598c597b74d568c9/src/pip/_internal/download.py#L547-L582
data = response.raw.read(read_amount)
bytes_received += len(data)
# We might have no more data to read. Check number of bytes downloaded.
if not data:
logger.debug('Downloaded ' + repr(bytes_received) + '/' +
repr(required_length) + ' bytes.')
# Finally, we signal that the download is complete.
break
yield data
if bytes_received >= required_length:
break
except ReadTimeoutError as e:
raise exceptions.SlowRetrievalError(str(e))
finally:
response.close()
return chunks()
def _get_session(self, url):
"""Returns a different customized requests.Session per schema+hostname
combination.
"""
# Use a different requests.Session per schema+hostname combination, to
# reuse connections while minimizing subtle security issues.
parsed_url = parse.urlparse(url)
if not parsed_url.scheme or not parsed_url.hostname:
raise exceptions.URLParsingError(
'Could not get scheme and hostname from URL: ' + url)
session_index = parsed_url.scheme + '+' + parsed_url.hostname
logger.debug('url: ' + url)
logger.debug('session index: ' + session_index)
session = self._sessions.get(session_index)
if not session:
session = requests.Session()
self._sessions[session_index] = session
# Attach some default headers to every Session.
requests_user_agent = session.headers['User-Agent']
# Follows the RFC: https://tools.ietf.org/html/rfc7231#section-5.5.3
tuf_user_agent = 'tuf/' + tuf.__version__ + ' ' + requests_user_agent
session.headers.update({
# Tell the server not to compress or modify anything.
# https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding#Directives
'Accept-Encoding': 'identity',
# The TUF user agent.
'User-Agent': tuf_user_agent})
logger.debug('Made new session for ' + session_index)
else:
logger.debug('Reusing session for ' + session_index)
return session

1013
tuf/roledb.py
View file

File diff suppressed because it is too large Load diff

0
tuf/scripts/__init__.py
View file

236
tuf/scripts/client.py
View file

@ -1,236 +0,0 @@
#!/usr/bin/env python
# Copyright 2012 - 2018, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
client.py
<Author>
Vladimir Diaz <vladimir.v.diaz@gmail.com>
<Started>
September 2012.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Provide a basic TUF client that can update all of the metatada and target
files provided by the user-specified repository mirror. Updated files are
saved to the 'targets' directory in the current working directory. The
repository mirror is specified by the user through the '--repo' command-
line option.
Normally, a software updater integrating TUF will develop their own costum
client module by importing 'tuf.client.updater', instantiating the required
object, and calling the desired methods to perform an update. This basic
client is provided to users who wish to give TUF a quick test run without the
hassle of writing client code. This module can also used by updaters that do
not need the customization and only require their clients to perform an
update of all the files provided by their repository mirror(s).
For software updaters that DO require customization, see the
'example_client.py' script. The 'example_client.py' script provides an
outline of the client code that software updaters may develop and then tailor
to their specific software updater or package manager.
Additional tools for clients running legacy applications will also be made
available. These tools will allow secure software updates using The Update
Framework without the need to modify the original application.
<Usage>
$ client.py --repo http://localhost:8001 <target>
$ client.py --repo http://localhost:8001 --verbose 3 <target>
<Options>
--verbose:
Set the verbosity level of logging messages. Accepts values 1-5.
Example:
$ client.py --repo http://localhost:8001 --verbose 3 README.txt
--repo:
Set the repository mirror that will be responding to client requests.
E.g., 'http://localhost:8001'.
Example:
$ client.py --repo http://localhost:8001 README.txt
"""
import sys
import argparse
import logging
from tuf import exceptions
from tuf import log
from tuf import settings
from tuf.client.updater import Updater
# See 'log.py' to learn how logging is handled in TUF.
logger = logging.getLogger(__name__)
def update_client(parsed_arguments):
"""
<Purpose>
Perform an update of the metadata and target files located at
'repository_mirror'. Target files are saved to the 'targets' directory
in the current working directory. The current directory must already
include a 'metadata' directory, which in turn must contain the 'current'
and 'previous' directories. At a minimum, these two directories require
the 'root.json' metadata file.
<Arguments>
parsed_arguments:
An argparse Namespace object, containing the parsed arguments.
<Exceptions>
tuf.exceptions.Error, if 'parsed_arguments' is not a Namespace object.
<Side Effects>
Connects to a repository mirror and updates the local metadata files and
any target files. Obsolete, local targets are also removed.
<Returns>
None.
"""
if not isinstance(parsed_arguments, argparse.Namespace):
raise exceptions.Error('Invalid namespace object.')
else:
logger.debug('We have a valid argparse Namespace object.')
# Set the local repositories directory containing all of the metadata files.
settings.repositories_directory = '.'
# Set the repository mirrors. This dictionary is needed by the Updater
# class of updater.py.
repository_mirrors = {'mirror': {'url_prefix': parsed_arguments.repo,
'metadata_path': 'metadata', 'targets_path': 'targets'}}
# Create the repository object using the repository name 'repository'
# and the repository mirrors defined above.
updater = Updater('tufrepo', repository_mirrors)
# The local destination directory to save the target files.
destination_directory = './tuftargets'
# Refresh the repository's top-level roles...
updater.refresh(unsafely_update_root_if_necessary=False)
# ... and store the target information for the target file specified on the
# command line, and determine which of these targets have been updated.
target_fileinfo = []
for target in parsed_arguments.targets:
target_fileinfo.append(updater.get_one_valid_targetinfo(target))
updated_targets = updater.updated_targets(target_fileinfo, destination_directory)
# Retrieve each of these updated targets and save them to the destination
# directory.
for target in updated_targets:
try:
updater.download_target(target, destination_directory)
except exceptions.DownloadError:
pass
# Remove any files from the destination directory that are no longer being
# tracked.
updater.remove_obsolete_targets(destination_directory)
def parse_arguments():
"""
<Purpose>
Parse the command-line options and set the logging level
as specified by the user through the --verbose option.
'client' expects the '--repo' to be set by the user.
Example:
$ client.py --repo http://localhost:8001 LICENSE
If the required option is unset, a parser error is printed
and the scripts exits.
<Arguments>
None.
<Exceptions>
None.
<Side Effects>
Sets the logging level for TUF logging.
<Returns>
The parsed_arguments (i.e., a argparse Namespace object).
"""
parser = argparse.ArgumentParser(
description='Retrieve file from TUF repository.')
# Add the options supported by 'basic_client' to the option parser.
parser.add_argument('-v', '--verbose', type=int, default=2,
choices=range(0, 6), help='Set the verbosity level of logging messages.'
' The lower the setting, the greater the verbosity. Supported logging'
' levels: 0=UNSET, 1=DEBUG, 2=INFO, 3=WARNING, 4=ERROR,'
' 5=CRITICAL')
parser.add_argument('-r', '--repo', type=str, required=True, metavar='<URI>',
help='Specify the remote repository\'s URI'
' (e.g., http://www.example.com:8001/tuf/). The client retrieves'
' updates from the remote repository.')
parser.add_argument('targets', nargs='+', metavar='<file>', help='Specify'
' the target files to retrieve from the specified TUF repository.')
parsed_arguments = parser.parse_args()
# Set the logging level.
if parsed_arguments.verbose == 5:
log.set_log_level(logging.CRITICAL)
elif parsed_arguments.verbose == 4:
log.set_log_level(logging.ERROR)
elif parsed_arguments.verbose == 3:
log.set_log_level(logging.WARNING)
elif parsed_arguments.verbose == 2:
log.set_log_level(logging.INFO)
elif parsed_arguments.verbose == 1:
log.set_log_level(logging.DEBUG)
else:
log.set_log_level(logging.NOTSET)
# Return the repository mirror containing the metadata and target files.
return parsed_arguments
if __name__ == '__main__':
# Parse the command-line arguments and set the logging level.
arguments = parse_arguments()
# Perform an update of all the files in the 'targets' directory located in
# the current directory.
try:
update_client(arguments)
except (exceptions.NoWorkingMirrorError, exceptions.RepositoryError,
exceptions.FormatError, exceptions.Error) as e:
sys.stderr.write('Error: ' + str(e) + '\n')
sys.exit(1)
# Successfully updated the client's target files.
sys.exit(0)

1149
tuf/scripts/repo.py
View file

File diff suppressed because it is too large Load diff

100
tuf/settings.py
View file

@ -1,100 +0,0 @@
#!/usr/bin/env python
# Copyright 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
settings.py
<Author>
Vladimir Diaz <vladimir.v.diaz@gmail.com>
<Started>
January 11, 2017
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
A central location for TUF configuration settings. Example options include
setting the destination of temporary files and downloaded content, the maximum
length of downloaded metadata (unknown file attributes), and download
behavior.
"""
# Set a directory that should be used for all temporary files. If this
# is None, then the system default will be used. The system default
# will also be used if a directory path set here is invalid or
# unusable.
temporary_directory = None
# Set a local directory to store metadata that is requested from mirrors. This
# directory contains subdirectories for different repositories, where each
# subdirectory contains a different set of metadata. For example:
# tuf.settings.repositories_directory = /tmp/repositories. The root file for a
# repository named 'django_repo' can be found at:
# /tmp/repositories/django_repo/metadata/current/root.METADATA_EXTENSION
repositories_directory = None
# The 'log.py' module manages TUF's logging system. Users have the option to
# enable/disable logging to a file via 'ENABLE_FILE_LOGGING', or
# tuf.log.enable_file_logging() and tuf.log.disable_file_logging().
ENABLE_FILE_LOGGING = False
# If file logging is enabled via 'ENABLE_FILE_LOGGING', TUF log messages will
# be saved to 'LOG_FILENAME'
LOG_FILENAME = 'tuf.log'
# Since the timestamp role does not have signed metadata about itself, we set a
# default but sane upper bound for the number of bytes required to download it.
DEFAULT_TIMESTAMP_REQUIRED_LENGTH = 16384 #bytes
# The Root role may be updated without knowing its version if top-level
# metadata cannot be safely downloaded (e.g., keys may have been revoked, thus
# requiring a new Root file that includes the updated keys). Set a default
# upper bound for the maximum total bytes that may be downloaded for Root
# metadata.
DEFAULT_ROOT_REQUIRED_LENGTH = 512000 #bytes
# Set a default, but sane, upper bound for the number of bytes required to
# download Snapshot metadata.
DEFAULT_SNAPSHOT_REQUIRED_LENGTH = 2000000 #bytes
# Set a default, but sane, upper bound for the number of bytes required to
# download Targets metadata.
DEFAULT_TARGETS_REQUIRED_LENGTH = 5000000 #bytes
# Set a timeout value in seconds (float) for non-blocking socket operations.
SOCKET_TIMEOUT = 4 #seconds
# The maximum chunk of data, in bytes, we would download in every round.
CHUNK_SIZE = 400000 #bytes
# The minimum average download speed (bytes/second) that must be met to
# avoid being considered as a slow retrieval attack.
MIN_AVERAGE_DOWNLOAD_SPEED = 50 #bytes/second
# By default, limit number of delegatees we visit for any target.
MAX_NUMBER_OF_DELEGATIONS = 2**5
# A setting for the instances where a default hashing algorithm is needed.
# This setting is currently used to calculate the path hash prefixes of hashed
# bin delegations, and digests of targets filepaths. The other instances
# (e.g., digest of files) that require a hashing algorithm rely on settings in
# the securesystemslib external library.
DEFAULT_HASH_ALGORITHM = 'sha256'
# The hashing algorithms used to compute file hashes
FILE_HASH_ALGORITHMS = ['sha256', 'sha512']
# The client's update procedure (contained within a while-loop) can potentially
# hog the CPU. The following setting can be used to force the update sequence
# to suspend execution for a specified amount of time. See
# theupdateframework/tuf/issue#338.
SLEEP_BEFORE_ROUND = None
# Maximum number of root metadata file rotations we should perform in order to
# prevent a denial-of-service (DoS) attack.
MAX_NUMBER_ROOT_ROTATIONS = 2**5

395
tuf/sig.py
View file

@ -1,395 +0,0 @@
#!/usr/bin/env python
# Copyright 2012 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program Name>
sig.py
<Author>
Vladimir Diaz <vladimir.v.diaz@gmail.com>
<Started>
February 28, 2012. Based on a previous version by Geremy Condra.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Survivable key compromise is one feature of a secure update system
incorporated into TUF's design. Responsibility separation through
the use of multiple roles, multi-signature trust, and explicit and
implicit key revocation are some of the mechanisms employed towards
this goal of survivability. These mechanisms can all be seen in
play by the functions available in this module.
The signed metadata files utilized by TUF to download target files
securely are used and represented here as the 'signable' object.
More precisely, the signature structures contained within these metadata
files are packaged into 'signable' dictionaries. This module makes it
possible to capture the states of these signatures by organizing the
keys into different categories. As keys are added and removed, the
system must securely and efficiently verify the status of these signatures.
For instance, a bunch of keys have recently expired. How many valid keys
are now available to the Snapshot role? This question can be answered by
get_signature_status(), which will return a full 'status report' of these
'signable' dicts. This module also provides a convenient verify() function
that will determine if a role still has a sufficient number of valid keys.
If a caller needs to update the signatures of a 'signable' object, there
is also a function for that.
"""
import logging
import securesystemslib # pylint: disable=unused-import
from securesystemslib import exceptions as sslib_exceptions
from securesystemslib import formats as sslib_formats
from securesystemslib import keys as sslib_keys
from tuf import exceptions
from tuf import formats
from tuf import keydb
from tuf import roledb
# See 'log.py' to learn how logging is handled in TUF.
logger = logging.getLogger(__name__)
def get_signature_status(signable, role=None, repository_name='default',
threshold=None, keyids=None):
"""
<Purpose>
Return a dictionary representing the status of the signatures listed in
'signable'. Signatures in the returned dictionary are identified by the
signature keyid and can have a status of either:
* bad -- Invalid signature
* good -- Valid signature from key that is available in 'tuf.keydb', and is
authorized for the passed role as per 'roledb' (authorization may be
overwritten by passed 'keyids').
* unknown -- Signature from key that is not available in 'tuf.keydb', or if
'role' is None.
* unknown signing schemes -- Signature from key with unknown signing
scheme.
* untrusted -- Valid signature from key that is available in 'tuf.keydb',
but is not trusted for the passed role as per 'roledb' or the passed
'keyids'.
NOTE: The result may contain duplicate keyids or keyids that reference the
same key, if 'signable' lists multiple signatures from the same key.
<Arguments>
signable:
A dictionary containing a list of signatures and a 'signed' identifier.
signable = {'signed': 'signer',
'signatures': [{'keyid': keyid,
'sig': sig}]}
Conformant to tuf.formats.SIGNABLE_SCHEMA.
role:
TUF role string (e.g. 'root', 'targets', 'snapshot' or timestamp).
threshold:
Rather than reference the role's threshold as set in roledb, use
the given 'threshold' to calculate the signature status of 'signable'.
'threshold' is an integer value that sets the role's threshold value, or
the minimum number of signatures needed for metadata to be considered
fully signed.
keyids:
Similar to the 'threshold' argument, use the supplied list of 'keyids'
to calculate the signature status, instead of referencing the keyids
in roledb for 'role'.
<Exceptions>
securesystemslib.exceptions.FormatError, if 'signable' does not have the
correct format.
tuf.exceptions.UnknownRoleError, if 'role' is not recognized.
<Side Effects>
None.
<Returns>
A dictionary representing the status of the signatures in 'signable'.
Conformant to tuf.formats.SIGNATURESTATUS_SCHEMA.
"""
# Do the arguments have the correct format? This check will ensure that
# arguments have the appropriate number of objects and object types, and that
# all dict keys are properly named. Raise
# 'securesystemslib.exceptions.FormatError' if the check fails.
formats.SIGNABLE_SCHEMA.check_match(signable)
sslib_formats.NAME_SCHEMA.check_match(repository_name)
if role is not None:
formats.ROLENAME_SCHEMA.check_match(role)
if threshold is not None:
formats.THRESHOLD_SCHEMA.check_match(threshold)
if keyids is not None:
sslib_formats.KEYIDS_SCHEMA.check_match(keyids)
# The signature status dictionary returned.
signature_status = {}
good_sigs = []
bad_sigs = []
unknown_sigs = []
untrusted_sigs = []
unknown_signing_schemes = []
# Extract the relevant fields from 'signable' that will allow us to identify
# the different classes of keys (i.e., good_sigs, bad_sigs, etc.).
signed = sslib_formats.encode_canonical(signable['signed']).encode('utf-8')
signatures = signable['signatures']
# Iterate the signatures and enumerate the signature_status fields.
# (i.e., good_sigs, bad_sigs, etc.).
for signature in signatures:
keyid = signature['keyid']
# Does the signature use an unrecognized key?
try:
key = keydb.get_key(keyid, repository_name)
except exceptions.UnknownKeyError:
unknown_sigs.append(keyid)
continue
# Does the signature use an unknown/unsupported signing scheme?
try:
valid_sig = sslib_keys.verify_signature(key, signature, signed)
except sslib_exceptions.UnsupportedAlgorithmError:
unknown_signing_schemes.append(keyid)
continue
# We are now dealing with either a trusted or untrusted key...
if valid_sig:
if role is not None:
# Is this an unauthorized key? (a keyid associated with 'role')
# Note that if the role is not known, tuf.exceptions.UnknownRoleError
# is raised here.
if keyids is None:
keyids = roledb.get_role_keyids(role, repository_name)
if keyid not in keyids:
untrusted_sigs.append(keyid)
continue
# This is an unset role, thus an unknown signature.
else:
unknown_sigs.append(keyid)
continue
# Identify good/authorized key.
good_sigs.append(keyid)
else:
# This is a bad signature for a trusted key.
bad_sigs.append(keyid)
# Retrieve the threshold value for 'role'. Raise
# tuf.exceptions.UnknownRoleError if we were given an invalid role.
if role is not None:
if threshold is None:
# Note that if the role is not known, tuf.exceptions.UnknownRoleError is
# raised here.
threshold = roledb.get_role_threshold(
role, repository_name=repository_name)
else:
logger.debug('Not using roledb.py\'s threshold for ' + repr(role))
else:
threshold = 0
# Build the signature_status dict.
signature_status['threshold'] = threshold
signature_status['good_sigs'] = good_sigs
signature_status['bad_sigs'] = bad_sigs
signature_status['unknown_sigs'] = unknown_sigs
signature_status['untrusted_sigs'] = untrusted_sigs
signature_status['unknown_signing_schemes'] = unknown_signing_schemes
return signature_status
def verify(signable, role, repository_name='default', threshold=None,
keyids=None):
"""
<Purpose>
Verify that 'signable' has a valid threshold of authorized signatures
identified by unique keyids. The threshold and whether a keyid is
authorized is determined by querying the 'threshold' and 'keyids' info for
the passed 'role' in 'roledb'. Both values can be overwritten by
passing the 'threshold' or 'keyids' arguments.
NOTE:
- Signatures with identical authorized keyids only count towards the
threshold once.
- Signatures with the same key only count toward the threshold once.
<Arguments>
signable:
A dictionary containing a list of signatures and a 'signed' identifier
that conforms to SIGNABLE_SCHEMA, e.g.:
signable = {'signed':, 'signatures': [{'keyid':, 'method':, 'sig':}]}
role:
TUF role string (e.g. 'root', 'targets', 'snapshot' or timestamp).
threshold:
Rather than reference the role's threshold as set in roledb, use
the given 'threshold' to calculate the signature status of 'signable'.
'threshold' is an integer value that sets the role's threshold value, or
the minimum number of signatures needed for metadata to be considered
fully signed.
keyids:
Similar to the 'threshold' argument, use the supplied list of 'keyids'
to calculate the signature status, instead of referencing the keyids
in roledb for 'role'.
<Exceptions>
tuf.exceptions.UnknownRoleError, if 'role' is not recognized.
securesystemslib.exceptions.FormatError, if 'signable' is not formatted
correctly.
securesystemslib.exceptions.Error, if an invalid threshold is encountered.
<Side Effects>
tuf.sig.get_signature_status() called. Any exceptions thrown by
get_signature_status() will be caught here and re-raised.
<Returns>
Boolean. True if the number of good unique (by keyid) signatures >= the
role's threshold, False otherwise.
"""
formats.SIGNABLE_SCHEMA.check_match(signable)
formats.ROLENAME_SCHEMA.check_match(role)
sslib_formats.NAME_SCHEMA.check_match(repository_name)
# Retrieve the signature status. tuf.sig.get_signature_status() raises:
# tuf.exceptions.UnknownRoleError
# securesystemslib.exceptions.FormatError. 'threshold' and 'keyids' are also
# validated.
status = get_signature_status(signable, role, repository_name, threshold, keyids)
# Retrieve the role's threshold and the authorized keys of 'status'
threshold = status['threshold']
good_sigs = status['good_sigs']
# Does 'status' have the required threshold of signatures?
# First check for invalid threshold values before returning result.
# Note: get_signature_status() is expected to verify that 'threshold' is
# not None or <= 0.
if threshold is None or threshold <= 0: #pragma: no cover
raise sslib_exceptions.Error("Invalid threshold: " + repr(threshold))
unique_keys = set()
for keyid in good_sigs:
key = keydb.get_key(keyid, repository_name)
unique_keys.add(key['keyval']['public'])
return len(unique_keys) >= threshold
def may_need_new_keys(signature_status):
"""
<Purpose>
Return true iff downloading a new set of keys might tip this
signature status over to valid. This is determined by checking
if either the number of unknown or untrusted keys is > 0.
<Arguments>
signature_status:
The dictionary returned by tuf.sig.get_signature_status().
<Exceptions>
securesystemslib.exceptions.FormatError, if 'signature_status does not have
the correct format.
<Side Effects>
None.
<Returns>
Boolean.
"""
# Does 'signature_status' have the correct format?
# This check will ensure 'signature_status' has the appropriate number
# of objects and object types, and that all dict keys are properly named.
# Raise 'securesystemslib.exceptions.FormatError' if the check fails.
formats.SIGNATURESTATUS_SCHEMA.check_match(signature_status)
unknown = signature_status['unknown_sigs']
untrusted = signature_status['untrusted_sigs']
return len(unknown) or len(untrusted)
def generate_rsa_signature(signed, rsakey_dict):
"""
<Purpose>
Generate a new signature dict presumably to be added to the 'signatures'
field of 'signable'. The 'signable' dict is of the form:
{'signed': 'signer',
'signatures': [{'keyid': keyid,
'method': 'evp',
'sig': sig}]}
The 'signed' argument is needed here for the signing process.
The 'rsakey_dict' argument is used to generate 'keyid', 'method', and 'sig'.
The caller should ensure the returned signature is not already in
'signable'.
<Arguments>
signed:
The data used by 'securesystemslib.keys.create_signature()' to generate
signatures. It is stored in the 'signed' field of 'signable'.
rsakey_dict:
The RSA key, a 'securesystemslib.formats.RSAKEY_SCHEMA' dictionary.
Used here to produce 'keyid', 'method', and 'sig'.
<Exceptions>
securesystemslib.exceptions.FormatError, if 'rsakey_dict' does not have the
correct format.
TypeError, if a private key is not defined for 'rsakey_dict'.
<Side Effects>
None.
<Returns>
Signature dictionary conformant to securesystemslib.formats.SIGNATURE_SCHEMA.
Has the form:
{'keyid': keyid, 'method': 'evp', 'sig': sig}
"""
# We need 'signed' in canonical JSON format to generate
# the 'method' and 'sig' fields of the signature.
signed = sslib_formats.encode_canonical(signed).encode('utf-8')
# Generate the RSA signature.
# Raises securesystemslib.exceptions.FormatError and TypeError.
signature = sslib_keys.create_signature(rsakey_dict, signed)
return signature

151
tuf/unittest_toolbox.py
View file

@ -1,151 +0,0 @@
#!/usr/bin/env python
# Copyright 2012 - 2017, New York University and the TUF contributors
# SPDX-License-Identifier: MIT OR Apache-2.0
"""
<Program>
unittest_toolbox.py
<Author>
Konstantin Andrianov.
<Started>
March 26, 2012.
<Copyright>
See LICENSE-MIT OR LICENSE for licensing information.
<Purpose>
Provides an array of various methods for unit testing. Use it instead of
actual unittest module. This module builds on unittest module.
Specifically, Modified_TestCase is a derived class from unittest.TestCase.
"""
import os
import shutil
import unittest
import tempfile
import random
import string
from typing import Optional
class Modified_TestCase(unittest.TestCase):
"""
<Purpose>
Provide additional test-setup methods to make testing
of module's methods-under-test as independent as possible.
If you want to modify setUp()/tearDown() do:
class Your_Test_Class(modified_TestCase):
def setUp():
your setup modification
your setup modification
...
modified_TestCase.setUp(self)
<Methods>
make_temp_directory(self, directory=None):
Creates and returns an absolute path of a temporary directory.
make_temp_file(self, suffix='.txt', directory=None):
Creates and returns an absolute path of an empty temp file.
make_temp_data_file(self, suffix='', directory=None, data = junk_data):
Returns an absolute path of a temp file containing some data.
random_path(self, length = 7):
Generate a 'random' path consisting of n-length strings of random chars.
Static Methods:
--------------
Following methods are static because they technically don't operate
on any instances of the class, what they do is: they modify class variables
(dictionaries) that are shared among all instances of the class. So
it is possible to call them without instantiating the class.
random_string(length=7):
Generate a 'length' long string of random characters.
"""
def setUp(self) -> None:
self._cleanup = []
def tearDown(self) -> None:
for cleanup_function in self._cleanup:
# Perform clean up by executing clean-up functions.
try:
# OSError will occur if the directory was already removed.
cleanup_function()
except OSError:
pass
def make_temp_directory(self, directory: Optional[str]=None) -> str:
"""Creates and returns an absolute path of a directory."""
prefix = self.__class__.__name__+'_'
temp_directory = tempfile.mkdtemp(prefix=prefix, dir=directory)
def _destroy_temp_directory():
shutil.rmtree(temp_directory)
self._cleanup.append(_destroy_temp_directory)
return temp_directory
def make_temp_file(
self,suffix: str='.txt', directory: Optional[str]=None
) -> str:
"""Creates and returns an absolute path of an empty file."""
prefix='tmp_file_'+self.__class__.__name__+'_'
temp_file = tempfile.mkstemp(suffix=suffix, prefix=prefix, dir=directory)
def _destroy_temp_file():
os.unlink(temp_file[1])
self._cleanup.append(_destroy_temp_file)
return temp_file[1]
def make_temp_data_file(
self, suffix: str='', directory: Optional[str]=None, data: str = 'junk data'
) -> str:
"""Returns an absolute path of a temp file containing data."""
temp_file_path = self.make_temp_file(suffix=suffix, directory=directory)
temp_file = open(temp_file_path, 'wt', encoding='utf8')
temp_file.write(data)
temp_file.close()
return temp_file_path
def random_path(self, length: int = 7) -> str:
"""Generate a 'random' path consisting of random n-length strings."""
rand_path = '/' + self.random_string(length)
for junk in range(2):
rand_path = os.path.join(rand_path, self.random_string(length))
return rand_path
@staticmethod
def random_string(length: int=15) -> str:
"""Generate a random string of specified length."""
rand_str = ''
for junk in range(length):
rand_str += random.SystemRandom().choice('abcdefABCDEF' + string.digits)
return rand_str