Elgato_dark/python-tuf
1
0
Fork 0
mirror of https://github.com/theupdateframework/python-tuf synced 2026-05-24 10:08:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

doc: drop documentation for legacy tools

Browse source 
Remove documentation for legacy client, repository/developer tool
and command line tools, which will be removed in subsequent
commits.

See #1797 and #1798 for replacing ATTACKS.md and QUICKSTART.md.

Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
This commit is contained in:
Lukas Puehringer 2022-01-25 17:43:48 +01:00
parent 8c72588662
commit d498bc01c1
10 changed files with 0 additions and 2128 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

447
docs/CLI.md
View file

@ -1,447 +0,0 @@
# Command-Line Interface #
The TUF command-line interface (CLI) requires a full
[TUF installation](INSTALLATION.rst). Be sure to include the installation of
extra dependencies and C extensions (
```python3 -m pip install securesystemslib[crypto,pynacl]```).
The use of the CLI is documented with examples below.
----
# Basic Examples #
## Create a repository ##
Create a TUF repository in the current working directory. A cryptographic key
is created and set for each top-level role. The written Targets metadata does
not sign for any targets, nor does it delegate trust to any roles. The
`--init` call will also set up a client directory. By default, these
directories will be `./tufrepo` and `./tufclient`.
```Bash
$ repo.py --init
```
Optionally, the repository can be written to a specified location.
```Bash
$ repo.py --init --path </path/to/repo_dir>
```
The default top-level key files created with `--init` are saved to disk
encrypted, with a default password of 'pw'. Instead of using the default
password, the user can enter one on the command line for each top-level role.
These optional command-line options also work with other CLI actions (e.g.,
repo.py --add).
```Bash
$ repo.py --init [--targets_pw, --root_pw, --snapshot_pw, --timestamp_pw]
```
Create a bare TUF repository in the current working directory. A cryptographic
key is *not* created nor set for each top-level role.
```Bash
$ repo.py --init --bare
```
Create a TUF repository with [consistent
snapshots](https://github.com/theupdateframework/specification/blob/master/tuf-spec.md#7-consistent-snapshots)
enabled, where target filenames have their hash prepended (e.g.,
`<hash>.README.txt`), and metadata filenames have their version numbers
prepended (e.g., `<hash>.snapshot.json`).
```Bash
$ repo.py --init --consistent
```
## Add a target file ##
Copy a target file to the repo and add it to the Targets metadata (or the
Targets role specified in --role). More than one target file, or directory,
may be specified in --add. The --recursive option may be toggled to also
include files in subdirectories of a specified directory. The Snapshot
and Timestamp metadata are also updated and signed automatically, but this
behavior can be toggled off with --no_release.
```Bash
$ repo.py --add <foo.tar.gz> <bar.tar.gz>
$ repo.py --add </path/to/dir> [--recursive]
```
Similar to the --init case, the repository location can be chosen.
```Bash
$ repo.py --add <foo.tar.gz> --path </path/to/my_repo>
```
## Remove a target file ##
Remove a target file from the Targets metadata (or the Targets role specified
in --role). More than one target file or glob pattern may be specified in
--remove. The Snapshot and Timestamp metadata are also updated and signed
automatically, but this behavior can be toggled off with --no_release.
```Bash
$ repo.py --remove <glob_pattern> ...
```
Examples:
Remove all target files, that match `foo*.tgz,` from the Targets metadata.
```Bash
$ repo.py --remove "foo*.tgz"
```
Remove all target files from the `my_role` metadata.
```Bash
$ repo.py --remove "*" --role my_role --sign tufkeystore/my_role_key
```
## Generate key ##
Generate a cryptographic key. The generated key can later be used to sign
specific metadata with `--sign`. The supported key types are: `ecdsa`,
`ed25519`, and `rsa`. If a keytype is not given, an Ed25519 key is generated.
If adding a top-level key to a bare repo (i.e., repo.py --init --bare),
the filenames of the top-level keys must be "root_key," "targets_key,"
"snapshot_key," "timestamp_key." The filename can vary for any additional
top-level key.
```Bash
$ repo.py --key
$ repo.py --key <keytype>
$ repo.py --key <keytype> [--path </path/to/repo_dir> --pw [my_password],
--filename <key_filename>]
```
Instead of using a default password, the user can enter one on the command
line or be prompted for it via password masking.
```Bash
$ repo.py --key ecdsa --pw my_password
```
```Bash
$ repo.py --key rsa --pw
Enter a password for the RSA key (...):
Confirm:
```
## Sign metadata ##
Sign, with the specified key(s), the metadata of the role indicated in --role.
The Snapshot and Timestamp role are also automatically signed, if possible, but
this behavior can be disabled with --no_release.
```Bash
$ repo.py --sign </path/to/key> ... [--role <rolename>, --path </path/to/repo>]
```
For example, to sign the delegated `foo` metadata:
```Bash
$ repo.py --sign </path/to/foo_key> --role foo
```
## Trust keys ##
The Root role specifies the trusted keys of the top-level roles, including
itself. The --trust command-line option, in conjunction with --pubkeys and
--role, can be used to indicate the trusted keys of a role.
```Bash
$ repo.py --trust --pubkeys </path/to/foo_key.pub> --role <rolename>
```
For example:
```Bash
$ repo.py --init --bare
$ repo.py --trust --pubkeys tufkeystore/my_key.pub tufkeystore/my_key_too.pub
--role root
```
### Distrust keys ###
Conversely, the Root role can discontinue trust of specified key(s).
Example of how to discontinue trust of a key:
```Bash
$ repo.py --distrust --pubkeys tufkeystore/my_key_too.pub --role root
```
## Delegations ##
Delegate trust of target files from the Targets role (or the one specified in
--role) to some other role (--delegatee). --delegatee is trusted to sign for
target files that match the delegated glob pattern(s). The --delegate option
does not create metadata for the delegated role, rather it updates the
delegator's metadata to list the delegation to --delegatee. The Snapshot and
Timestamp metadata are also updated and signed automatically, but this behavior
can be toggled off with --no_release.
```Bash
$ repo.py --delegate <glob pattern> ... --delegatee <rolename> --pubkeys
</path/to/pubkey.pub> ... [--role <rolename> --terminating --threshold <X>
--sign </path/to/role_privkey>]
```
For example, to delegate trust of `foo*.gz` packages to the `foo` role:
```
$ repo.py --delegate "foo*.tgz" --delegatee foo --pubkeys tufkeystore/foo.pub
```
## Revocations ##
Revoke trust of target files from a delegated role (--delegatee). The
"targets" role performs the revocation if --role is not specified. The
--revoke option does not delete the metadata belonging to --delegatee, instead
it removes the delegation to it from the delegator's (or --role) metadata. The
Snapshot and Timestamp metadata are also updated and signed automatically, but
this behavior can be toggled off with --no_release.
```Bash
$ repo.py --revoke --delegatee <rolename> [--role <rolename>
--sign </path/to/role_privkey>]
```
## Verbosity ##
Set the verbosity of the logger (2, by default). The lower the number, the
greater the verbosity. Logger messages are saved to `tuf.log` in the current
working directory.
```Bash
$ repo.py --verbose <0-5>
```
## Clean ##
Delete the repo in the current working directory, or the one specified with
`--path`. Specifically, the `tufrepo`, `tufclient`, and `tufkeystore`
directories are deleted.
```Bash
$ repo.py --clean
$ repo.py --clean --path </path/to/dirty/repo>
```
----
# Further Examples #
## Basic Update Delivery ##
Steps:
(1) initialize a repo.
(2) delegate trust of target files to another role.
(3) add a trusted file to the delegated role.
(4) fetch the trusted file from the delegated role.
```Bash
Step (1)
$ repo.py --init
Step (2)
$ repo.py --key ed25519 --filename mykey
$ repo.py --delegate "README.*" --delegatee myrole --pubkeys tufkeystore/mykey.pub
$ repo.py --sign tufkeystore/mykey --role myrole
Enter a password for the encrypted key (tufkeystore/mykey):
$ echo "my readme text" > README.txt
Step (3)
$ repo.py --add README.txt --role myrole --sign tufkeystore/mykey
Enter a password for the encrypted key (tufkeystore/mykey):
```
Serve the repo
```Bash
$ python3 -m http.server 8001
```
```Bash
Step (4)
$ client.py --repo http://localhost:8001 README.txt
$ tree .
.
├── tuf.log
├── tufrepo
│   └── metadata
│   ├── current
│   │   ├── 1.root.json
│   │   ├── myrole.json
│   │   ├── root.json
│   │   ├── snapshot.json
│   │   ├── targets.json
│   │   └── timestamp.json
│   └── previous
│   ├── 1.root.json
│   ├── root.json
│   ├── snapshot.json
│   ├── targets.json
│   └── timestamp.json
└── tuftargets
└── README.txt
5 directories, 13 files
```
## Correcting a Key ##
The filename of the top-level keys must be "root_key," "targets_key,"
"snapshot_key," and "root_key." The filename can vary for any additional
top-level key.
Steps:
(1) initialize a repo containing default keys for the top-level roles.
(2) distrust the default key for the root role.
(3) create a new key and trust its use with the root role.
(4) sign the root metadata file.
```Bash
Step (1)
$ repo.py --init
Step (2)
$ repo.py --distrust --pubkeys tufkeystore/root_key.pub --role root
Step (3)
$ repo.py --key ed25519 --filename root_key
$ repo.py --trust --pubkeys tufkeystore/root_key.pub --role root
Step (4)
$ repo.py --sign tufkeystore/root_key --role root
Enter a password for the encrypted key (tufkeystore/root_key):
```
## More Update Delivery ##
Steps:
(1) create a bare repo.
(2) add keys to the top-level roles.
(3) delegate trust of particular target files to another role X, where role X
has a signature threshold 2 and is marked as a terminating delegation. The
keys for role X and Y should be created prior to performing the delegation.
(4) Delegate from role X to role Y.
(5) have role X sign for a file also signed by the Targets role, to demonstrate
the expected file that should be downloaded by the client.
(6) perform an update.
(7) halt the server, add README.txt to the Targets role, restart the server,
and fetch the Target's role README.txt.
(8) Add LICENSE to 'role_y' and demonstrate that the client must not fetch it
because 'role_x' is a terminating delegation (and hasn't signed for it).
```Bash
Steps (1) and (2)
$ repo.py --init --consistent --bare
$ repo.py --key ed25519 --filename root_key
$ repo.py --trust --pubkeys tufkeystore/root_key.pub --role root
$ repo.py --key ecdsa --filename targets_key
$ repo.py --trust --pubkeys tufkeystore/targets_key.pub --role targets
$ repo.py --key rsa --filename snapshot_key
$ repo.py --trust --pubkeys tufkeystore/snapshot_key.pub --role snapshot
$ repo.py --key ecdsa --filename timestamp_key
$ repo.py --trust --pubkeys tufkeystore/timestamp_key.pub --role timestamp
$ repo.py --sign tufkeystore/root_key --role root
Enter a password for the encrypted key (tufkeystore/root_key):
$ repo.py --sign tufkeystore/targets_key --role targets
Enter a password for the encrypted key (tufkeystore/targets_key):
```
```Bash
Steps (3) and (4)
$ repo.py --key ed25519 --filename key_x
$ repo.py --key ed25519 --filename key_x2
$ repo.py --delegate "README.*" "LICENSE" --delegatee role_x --pubkeys
tufkeystore/key_x.pub tufkeystore/key_x2.pub --threshold 2 --terminating
$ repo.py --sign tufkeystore/key_x tufkeystore/key_x2 --role role_x
$ repo.py --key ed25519 --filename key_y
$ repo.py --delegate "README.*" "LICENSE" --delegatee role_y --role role_x
--pubkeys tufkeystore/key_y.pub --sign tufkeystore/key_x tufkeystore/key_x2
$ repo.py --sign tufkeystore/key_y --role role_y
```
```Bash
Steps (5) and (6)
$ echo "role_x's readme" > README.txt
$ repo.py --add README.txt --role role_x --sign tufkeystore/key_x tufkeystore/key_x2
```
Serve the repo
```Bash
$ python3 -m http.server 8001
```
Fetch the role x's README.txt
```Bash
$ client.py --repo http://localhost:8001 README.txt
$ cat tuftargets/README.txt
role_x's readme
```
```Bash
Step (7)
halt the server...
$ echo "Target role's readme" > README.txt
$ repo.py --add README.txt
restart the server...
```
```Bash
$ rm -rf tuftargets/ tuf.log
$ client.py --repo http://localhost:8001 README.txt
$ cat tuftargets/README.txt
Target role's readme
```
```Bash
Step (8)
$ echo "role_y's license" > LICENSE
$ repo.py --add LICENSE --role role_y --sign tufkeystore/key_y
```
```Bash
$ rm -rf tuftargets/ tuf.log
$ client.py --repo http://localhost:8001 LICENSE
Error: 'LICENSE' not found.
```

10
docs/GETTING_STARTED.rst
View file

@ -1,10 +0,0 @@
Getting Started
---------------
- `Overview of TUF <https://theupdateframework.io/overview/>`_
- `Installation <INSTALLATION.rst>`_
- Beginner Tutorials (using the basic command-line interface):
- `Quickstart <QUICKSTART.md>`_
- `CLI Documentation and Examples <CLI.md>`_
- `Advanced Tutorial <TUTORIAL.md>`_
- `Guidelines for Contributors <CONTRIBUTORS.rst>`_

149
docs/QUICKSTART.md
View file

@ -1,149 +0,0 @@
# Quickstart #
In this quickstart tutorial, we'll use the basic TUF command-line interface
(CLI), which includes the `repo.py` script and the `client.py` script, to set
up a repository with an update and metadata about that update, then download
and verify that update as a client.
Unlike the underlying TUF modules that the CLI uses, the CLI itself is a bit
bare-bones. Using the CLI is the easiest way to familiarize yourself with
how TUF works, however. It will serve as a very basic update system.
----
**Step (0)** - Make sure TUF is installed.
Make sure that TUF is installed, along with some of the optional cryptographic
libraries and C extensions. Try this command to do that:
`python3 -m pip install securesystemslib[colors,crypto,pynacl] tuf`
If you run into errors during that pip command, please consult the more
detailed [TUF Installation Instructions](INSTALLATION.rst). (There are some
system libraries that you may need to install first.)
**Step (1)** - Create a basic repository and client.
The following command will set up a basic update repository and basic client
that knows about the repository. `tufrepo`, `tufkeystore`, and
`tufclient` directories will be created in the current directory.
```Bash
$ repo.py --init
```
Four sets of keys are created in the `tufkeystore` directory. Initial metadata
about the repository is created in the `tufrepo` directory, and also provided
to the client in the `tufclient` directory.
**Step (2)** - Add an update to the repository.
We'll create a target file that will later be delivered as an update to clients.
Metadata about that file will be created and signed, and added to the
repository's metadata.
```Bash
$ echo 'Test file' > testfile
$ repo.py --add testfile
$ tree tufrepo/
tufrepo/
├── metadata
│   ├── 1.root.json
│   ├── root.json
│   ├── snapshot.json
│   ├── targets.json
│   └── timestamp.json
├── metadata.staged
│   ├── 1.root.json
│   ├── root.json
│   ├── snapshot.json
│   ├── targets.json
│   └── timestamp.json
└── targets
└── testfile
3 directories, 11 files
```
The new file `testfile` is added to the repository, and metadata is updated in
the `tufrepo` directory. The Targets metadata (`targets.json`) now includes
the file size and hashes of the `testfile` target file, and this metadata is
signed by the Targets role's key, so that clients can verify that metadata
about `testfile` and then verify `testfile` itself.
**Step (3)** - Serve the repo.
We'll host a toy http server containing the `testfile` update and the
repository's metadata.
```Bash
$ cd "tufrepo/"
$ python3 -m http.server 8001
```
**Step (4)** - Obtain and verify the `testfile` update on a client.
The client can request the package `testfile` from the repository. TUF will
download and verify metadata from the repository as necessary to determine
what the trustworthy hashes and length of `testfile` are, then download
the target `testfile` from the repository and keep it only if it matches that
trustworthy metadata.
```Bash
$ cd "../tufclient/"
$ client.py --repo http://localhost:8001 testfile
$ tree
.
├── tufrepo
│   └── metadata
│   ├── current
│   │   ├── 1.root.json
│   │   ├── root.json
│   │   ├── snapshot.json
│   │   ├── targets.json
│   │   └── timestamp.json
│   └── previous
│   ├── 1.root.json
│   ├── root.json
│   ├── snapshot.json
│   ├── targets.json
│   └── timestamp.json
└── tuftargets
└── testfile
5 directories, 11 files
```
Now that a trustworthy update target has been obtained, an updater can proceed
however it normally would to install or use the update.
----
### Next Steps
TUF provides functionality for both ends of a software update system, the
**update provider** and the **update client**.
`repo.py` made use of `tuf.repository_tool`'s functionality for an update
provider, helping you produce and sign metadata about your updates.
`client.py` made use of `tuf.client.updater`'s client-side functionality,
performing download and the critical verification steps for metadata and the
update itself.
You can look at [CLI.md](CLI.md) to toy with the TUF CLI a bit more.
After that, try out using the underlying modules for a great deal more control.
The more detailed [Advanced Tutorial](TUTORIAL.md) shows you how to use the
underlying modules, `repository_tool` and `updater`.
Ultimately, a sophisticated update client will use or re-implement those
underlying modules. The TUF design is intended to play well with any update
workflow.
Please provide feedback or questions for this or other tutorials, or
TUF in general, by checking out
[our contact info](https://github.com/theupdateframework/python-tuf#contact), or
creating [issues](https://github.com/theupdateframework/python-tuf/issues) in this
repository!

696
docs/TUTORIAL.md
View file

@ -1,696 +0,0 @@
# Advanced Tutorial #
## Table of Contents ##
- [How to Create and Modify a TUF Repository](#how-to-create-and-modify-a-tuf-repository)
- [Overview](#overview)
- [Keys](#keys)
- [Create RSA Keys](#create-rsa-keys)
- [Import RSA Keys](#import-rsa-keys)
- [Create and Import Ed25519 Keys](#create-and-import-ed25519-keys)
- [Create Top-level Metadata](#create-top-level-metadata)
- [Create Root](#create-root)
- [Create Timestamp, Snapshot, Targets](#create-timestamp-snapshot-targets)
- [Targets](#targets)
- [Add Target Files](#add-target-files)
- [Remove Target Files](#remove-target-files)
- [Delegations](#delegations)
- [Revoke Delegated Role](#revoke-delegated-role)
- [Wrap-up](#wrap-up)
- [Delegate to Hashed Bins](#delegate-to-hashed-bins)
- [Consistent Snapshots](#consistent-snapshots)
- [How to Perform an Update](#how-to-perform-an-update)
## How to Create and Modify a TUF Repository ##
### Overview ###
A software update system must follow two steps to integrate The Update
Framework (TUF). First, it must add the framework to the client side of the
update system. The [tuf.client.updater](../tuf/client/README.md) module assists in
integrating TUF on the client side. Second, the software repository on the
server side must be modified to include a minimum of four top-level metadata
(root.json, targets.json, snapshot.json, and timestamp.json). No additional
software is required to convert a software repository to a TUF one. The
low-level repository tool that generates the required TUF metadata for a
software repository is the focus of this tutorial. There is also separate
document that [demonstrates how TUF protects against malicious
updates](../tuf/ATTACKS.md).
The [repository tool](../tuf/repository_tool.py) contains functions to generate
all of the files needed to populate and manage a TUF repository. The tool may
either be imported into a Python module, or used with the Python interpreter in
interactive mode.
A repository object that encapsulates the metadata files of the repository can
be created or loaded by the repository tool. Repository maintainers can modify
the repository object to manipulate the metadata files stored on the
repository. TUF clients use the metadata files to validate files requested and
downloaded. In addition to the repository object, where the majority of
changes are made, the repository tool provides functions to generate and
persist cryptographic keys. The framework utilizes cryptographic keys to sign
and verify metadata files.
To begin, cryptographic keys are generated with the repository tool. However,
before metadata files can be validated by clients and target files fetched in a
secure manner, public keys must be pinned to particular metadata roles and
metadata signed by role's private keys. After covering keys, the four required
top-level metadata are created next. Examples are given demonstrating the
expected work flow, where the metadata roles are created in a specific order,
keys imported and loaded, and metadata signed and written to disk. Lastly,
target files are added to the repository, and a custom delegation performed to
extend the default roles of the repository. By the end, a fully populated TUF
repository is generated that can be used by clients to securely download
updates.
### Keys ###
The repository tool supports multiple public-key algorithms, such as
[RSA](https://en.wikipedia.org/wiki/RSA_%28cryptosystem%29) and
[Ed25519](https://ed25519.cr.yp.to/), and multiple cryptography libraries.
Using [RSA-PSS](https://tools.ietf.org/html/rfc8017#section-8.1) or
[ECDSA](https://en.wikipedia.org/wiki/Elliptic_Curve_Digital_Signature_Algorithm)
signatures requires the [cryptography](https://cryptography.io/) library. If
generation of Ed25519 signatures is needed
[PyNaCl](https://github.com/pyca/pynacl) library should be installed. This
tutorial assumes both dependencies are installed: refer to
[Installation Instructions](INSTALLATION.rst#install-with-more-cryptographic-flexibility)
for details.
The Ed25519 and ECDSA keys are stored in JSON format and RSA keys are stored in PEM
format. Private keys are encrypted and passphrase-protected (strengthened with
PBKDF2-HMAC-SHA256.) Generating, importing, and loading cryptographic key
files can be done with functions available in the repository tool.
To start, a public and private RSA key pair is generated with the
`generate_and_write_rsa_keypair()` function. The keys generated next are
needed to sign the repository metadata files created in upcoming sub-sections.
Note: In the instructions below, lines that start with `>>>` denote commands
that should be entered by the reader, `#` begins the start of a comment, and
text without prepended symbols is the output of a command.
#### Create RSA Keys ####
```python
>>> from tuf.repository_tool import *
# Generate and write the first of two root keys for the TUF repository. The
# following function creates an RSA key pair, where the private key is saved to
# "root_key" and the public key to "root_key.pub" (both saved to the current
# working directory).
>>> generate_and_write_rsa_keypair(password="password", filepath="root_key", bits=2048)
# If the key length is unspecified, it defaults to 3072 bits. A length of less
# than 2048 bits raises an exception. A similar function is available to supply
# a password on the prompt. If an empty password is entered, the private key
# is saved unencrypted.
>>> generate_and_write_rsa_keypair_with_prompt(filepath="root_key2")
enter password to encrypt private key file '/path/to/root_key2'
(leave empty if key should not be encrypted):
Confirm:
```
The following four key files should now exist:
1. **root_key**
2. **root_key.pub**
3. **root_key2**
4. **root_key2.pub**
If a filepath is not given, the KEYID of the generated key is used as the
filename. The key files are written to the current working directory.
```python
# Continuing from the previous section . . .
>>> generate_and_write_rsa_keypair_with_prompt()
enter password to encrypt private key file '/path/to/KEYID'
(leave empty if key should not be encrypted):
Confirm:
```
### Import RSA Keys ###
```python
# Continuing from the previous section . . .
# Import an existing public key.
>>> public_root_key = import_rsa_publickey_from_file("root_key.pub")
# Import an existing private key. Importing a private key requires a password,
# whereas importing a public key does not.
>>> private_root_key = import_rsa_privatekey_from_file("root_key")
enter password to decrypt private key file '/path/to/root_key'
(leave empty if key not encrypted):
```
### Create and Import Ed25519 Keys ###
```Python
# Continuing from the previous section . . .
# The same generation and import functions as for rsa keys exist for ed25519
>>> generate_and_write_ed25519_keypair_with_prompt(filepath='ed25519_key')
enter password to encrypt private key file '/path/to/ed25519_key'
(leave empty if key should not be encrypted):
Confirm:
# Import the ed25519 public key just created . . .
>>> public_ed25519_key = import_ed25519_publickey_from_file('ed25519_key.pub')
# and its corresponding private key.
>>> private_ed25519_key = import_ed25519_privatekey_from_file('ed25519_key')
enter password to decrypt private key file '/path/to/ed25519_key'
(leave empty if key should not be encrypted):
```
Note: Methods are also available to generate and write keys from memory.
* generate_ed25519_key()
* generate_ecdsa_key()
* generate_rsa_key()
* import_ecdsakey_from_pem(pem)
* import_rsakey_from_pem(pem)
### Create Top-level Metadata ###
The [metadata document](METADATA.md) outlines the JSON files that must exist
on a TUF repository. The following sub-sections demonstrate the
`repository_tool.py` calls repository maintainers may issue to generate the
required roles. The top-level roles to be created are `root`, `timestamp`,
`snapshot`, and `target`.
We begin with `root`, the locus of trust that specifies the public keys of the
top-level roles, including itself.
#### Create Root ####
```python
# Continuing from the previous section . . .
# Create a new Repository object that holds the file path to the TUF repository
# and the four top-level role objects (Root, Targets, Snapshot, Timestamp).
# Metadata files are created when repository.writeall() or repository.write()
# are called. The repository directory is created if it does not exist. You
# may see log messages indicating any directories created.
>>> repository = create_new_repository("repository")
# The Repository instance, 'repository', initially contains top-level Metadata
# objects. Add one of the public keys, created in the previous section, to the
# root role. Metadata is considered valid if it is signed by the public key's
# corresponding private key.
>>> repository.root.add_verification_key(public_root_key)
# A role's verification key(s) (to be more precise, the verification key's
# keyid) may be queried. Other attributes include: signing_keys, version,
# signatures, expiration, threshold, and delegations (attribute available only
# to a Targets role).
>>> repository.root.keys
['b23514431a53676595922e955c2d547293da4a7917e3ca243a175e72bbf718df']
# Add a second public key to the root role. Although previously generated and
# saved to a file, the second public key must be imported before it can added
# to a role.
>>> public_root_key2 = import_rsa_publickey_from_file("root_key2.pub")
>>> repository.root.add_verification_key(public_root_key2)
# The threshold of each role defaults to 1. Maintainers may change the
# threshold value, but repository_tool.py validates thresholds and warns users.
# Set the threshold of the root role to 2, which means the root metadata file
# is considered valid if it's signed by at least two valid keys. We also load
# the second private key, which hasn't been imported yet.
>>> repository.root.threshold = 2
>>> private_root_key2 = import_rsa_privatekey_from_file("root_key2", password="password")
# Load the root signing keys to the repository, which writeall() or write()
# (write multiple roles, or a single role, to disk) use to sign the root
# metadata.
>>> repository.root.load_signing_key(private_root_key)
>>> repository.root.load_signing_key(private_root_key2)
# repository.status() shows missing verification and signing keys for the
# top-level roles, and whether signatures can be created (also see #955).
# This output shows that so far only the "root" role meets the key threshold and
# can successfully sign its metadata.
>>> repository.status()
'targets' role contains 0 / 1 public keys.
'snapshot' role contains 0 / 1 public keys.
'timestamp' role contains 0 / 1 public keys.
'root' role contains 2 / 2 signatures.
'targets' role contains 0 / 1 signatures.
# In the next section we update the other top-level roles and create a repository
# with valid metadata.
```
#### Create Timestamp, Snapshot, Targets
Now that `root.json` has been set, the other top-level roles may be created.
The signing keys added to these roles must correspond to the public keys
specified by the Root role.
On the client side, `root.json` must always exist. The other top-level roles,
created next, are requested by repository clients in (Root -> Timestamp ->
Snapshot -> Targets) order to ensure required metadata is downloaded in a
secure manner.
```python
# Continuing from the previous section . . .
# 'datetime' module needed to optionally set a role's expiration.
>>> import datetime
# Generate keys for the remaining top-level roles. The root keys have been set above.
>>> generate_and_write_rsa_keypair(password='password', filepath='targets_key')
>>> generate_and_write_rsa_keypair(password='password', filepath='snapshot_key')
>>> generate_and_write_rsa_keypair(password='password', filepath='timestamp_key')
# Add the verification keys of the remaining top-level roles.
>>> repository.targets.add_verification_key(import_rsa_publickey_from_file('targets_key.pub'))
>>> repository.snapshot.add_verification_key(import_rsa_publickey_from_file('snapshot_key.pub'))
>>> repository.timestamp.add_verification_key(import_rsa_publickey_from_file('timestamp_key.pub'))
# Import the signing keys of the remaining top-level roles.
>>> private_targets_key = import_rsa_privatekey_from_file('targets_key', password='password')
>>> private_snapshot_key = import_rsa_privatekey_from_file('snapshot_key', password='password')
>>> private_timestamp_key = import_rsa_privatekey_from_file('timestamp_key', password='password')
# Load the signing keys of the remaining roles so that valid signatures are
# generated when repository.writeall() is called.
>>> repository.targets.load_signing_key(private_targets_key)
>>> repository.snapshot.load_signing_key(private_snapshot_key)
>>> repository.timestamp.load_signing_key(private_timestamp_key)
# Optionally set the expiration date of the timestamp role. By default, roles
# are set to expire as follows: root(1 year), targets(3 months), snapshot(1
# week), timestamp(1 day).
>>> repository.timestamp.expiration = datetime.datetime(2080, 10, 28, 12, 8)
# Mark roles for metadata update (see #964, #958)
>>> repository.mark_dirty(['root', 'snapshot', 'targets', 'timestamp'])
# Write all metadata to "repository/metadata.staged/"
>>> repository.writeall()
```
### Targets ###
TUF makes it possible for clients to validate downloaded target files by
including a target file's length, hash(es), and filepath in metadata. The
filepaths are relative to a `targets/` directory on the software repository. A
TUF client can download a target file by first updating the latest copy of
metadata (and thus available targets), verifying that their length and hashes
are valid, and saving the target file(s) locally to complete the update
process.
In this section, the target files intended for clients are added to a
repository and listed in `targets.json` metadata.
#### Add Target Files ####
The repository maintainer adds target files to roles (e.g., `targets` and
`unclaimed`) by specifying their filepaths. The target files must exist at the
specified filepaths before the repository tool can generate and add their
(hash(es), length, and filepath) to metadata.
First, the actual target files are manually created and saved to the `targets/`
directory of the repository:
```Bash
# Create and save target files to the targets directory of the software
# repository.
$ cd repository/targets/
$ echo 'file1' > file1.txt
$ echo 'file2' > file2.txt
$ echo 'file3' > file3.txt
$ mkdir myproject; echo 'file4' > myproject/file4.txt
$ cd ../../
```
With the target files available on the `targets/` directory of the software
repository, the `add_targets()` method of a Targets role can be called to add
the target filepaths to metadata.
```python
# Continuing from the previous section . . .
# NOTE: If you exited the Python interactive interpreter above you need to
# re-import the repository_tool-functions and re-load the repository and
# signing keys.
>>> from tuf.repository_tool import *
# The 'os' module is needed to gather file attributes, which will be included
# in a custom field for some of the target files added to metadata.
>>> import os
# Load the repository created in the previous section. This repository so far
# contains metadata for the top-level roles, but no target paths are yet listed
# in targets metadata.
>>> repository = load_repository('repository')
# Create a list of all targets in the directory.
>>> list_of_targets = ['file1.txt', 'file2.txt', 'file3.txt']
# Add the list of target paths to the metadata of the top-level Targets role.
# Any target file paths that might already exist are NOT replaced, and
# add_targets() does not create or move target files on the file system. Any
# target paths added to a role must fall under the expected targets directory,
# otherwise an exception is raised. The targets added to a role should actually
# exist once writeall() or write() is called, so that the hash and size of
# these targets can be included in Targets metadata.
>>> repository.targets.add_targets(list_of_targets)
# Individual target files may also be added to roles, including custom data
# about the target. In the example below, file permissions of the target
# (octal number specifying file access for owner, group, others e.g., 0755) is
# added alongside the default fileinfo. All target objects in metadata include
# the target's filepath, hash, and length.
# Note: target path passed to add_target() method has to be relative
# to the targets directory or an exception is raised.
>>> target4_filepath = 'myproject/file4.txt'
>>> target4_abspath = os.path.abspath(os.path.join('repository', 'targets', target4_filepath))
>>> octal_file_permissions = oct(os.stat(target4_abspath).st_mode)[4:]
>>> custom_file_permissions = {'file_permissions': octal_file_permissions}
>>> repository.targets.add_target(target4_filepath, custom_file_permissions)
```
The private keys of roles affected by the changes above must now be imported and
loaded. `targets.json` must be signed because a target file was added to its
metadata. `snapshot.json` keys must be loaded and its metadata signed because
`targets.json` has changed. Similarly, since `snapshot.json` has changed, the
`timestamp.json` role must also be signed.
```Python
# Continuing from the previous section . . .
# The private key of the updated targets metadata must be re-loaded before it
# can be signed and written (Note the load_repository() call above).
>>> private_targets_key = import_rsa_privatekey_from_file('targets_key')
enter password to decrypt private key file '/path/to/targets_key'
(leave empty if key not encrypted):
>>> repository.targets.load_signing_key(private_targets_key)
# Due to the load_repository() and new versions of metadata, we must also load
# the private keys of Snapshot and Timestamp to generate a valid set of metadata.
>>> private_snapshot_key = import_rsa_privatekey_from_file('snapshot_key')
enter password to decrypt private key file '/path/to/snapshot_key'
(leave empty if key not encrypted):
>>> repository.snapshot.load_signing_key(private_snapshot_key)
>>> private_timestamp_key = import_rsa_privatekey_from_file('timestamp_key')
enter password to decrypt private key file '/path/to/timestamp_key'
(leave empty if key not encrypted):
>>> repository.timestamp.load_signing_key(private_timestamp_key)
# Mark roles for metadata update (see #964, #958)
>>> repository.mark_dirty(['snapshot', 'targets', 'timestamp'])
# Generate new versions of the modified top-level metadata (targets, snapshot,
# and timestamp).
>>> repository.writeall()
```
#### Remove Target Files ####
Target files previously added to roles may also be removed. Removing a target
file requires first removing the target from a role and then writing the
new metadata to disk.
```python
# Continuing from the previous section . . .
# Remove a target file listed in the "targets" metadata. The target file is
# not actually deleted from the file system.
>>> repository.targets.remove_target('myproject/file4.txt')
# Mark roles for metadata update (see #964, #958)
>>> repository.mark_dirty(['snapshot', 'targets', 'timestamp'])
>>> repository.writeall()
```
#### Excursion: Dump Metadata and Append Signature ####
The following two functions are intended for those that wish to independently
sign metadata. Repository maintainers can dump the portion of metadata that is
normally signed, sign it with an external signing tool, and append the
signature to already existing metadata.
First, the signable portion of metadata can be generated as follows:
```Python
>>> signable_content = dump_signable_metadata('repository/metadata.staged/timestamp.json')
```
Then, use a tool like securesystemslib to create a signature over the signable
portion. *Note, to make the signing key count towards the role's signature
threshold, it needs to be added to `root.json`, e.g. via
`repository.timestamp.add_verification_key(key)` (not shown in below snippet).*
```python
>>> from securesystemslib.formats import encode_canonical
>>> from securesystemslib.keys import create_signature
>>> private_ed25519_key = import_ed25519_privatekey_from_file('ed25519_key')
enter password to decrypt private key file '/path/to/ed25519_key'
>>> signature = create_signature(
... private_ed25519_key, encode_canonical(signable_content).encode())
```
Finally, append the signature to the metadata
```Python
>>> append_signature(signature, 'repository/metadata.staged/timestamp.json')
```
Note that the format of the signature is the format expected in metadata, which
is a dictionary that contains a KEYID, the signature itself, etc. See the
specification and [METADATA.md](METADATA.md) for a detailed example.
### Delegations ###
All of the target files available on the software repository created so far
have been added to one role (the top-level Targets role). However, what if
multiple developers are responsible for the files of a project? What if
responsibility separation is desired? Performing a delegation, where one role
delegates trust of some paths to another role, is an option for integrators
that require additional roles on top of the top-level roles available by
default.
In the next sub-section, the `unclaimed` role is delegated from the top-level
`targets` role. The `targets` role specifies the delegated role's public keys,
the paths it is trusted to provide, and its role name. <!--
TODO: Uncomment together with "Revoke Delegated Role" section below
Furthermore, the example
below demonstrates a nested delegation from `unclaimed` to `django`. Once a
role has delegated trust to another, the delegated role may independently add
targets and generate signed metadata.
-->
```python
# Continuing from the previous section . . .
# Generate a key for a new delegated role named "unclaimed".
>>> generate_and_write_rsa_keypair(password='password', filepath='unclaimed_key', bits=2048)
>>> public_unclaimed_key = import_rsa_publickey_from_file('unclaimed_key.pub')
# Make a delegation (delegate trust of 'myproject/*.txt' files) from "targets"
# to "unclaimed", where "unclaimed" initially contains zero targets.
>>> repository.targets.delegate('unclaimed', [public_unclaimed_key], ['myproject/*.txt'])
# Thereafter, we can access the delegated role by its name to e.g. add target
# files, just like we did with the top-level targets role.
>>> repository.targets("unclaimed").add_target("myproject/file4.txt")
# Load the private key of "unclaimed" so that unclaimed's metadata can be
# signed, and valid metadata created.
>>> private_unclaimed_key = import_rsa_privatekey_from_file('unclaimed_key', password='password')
>>> repository.targets("unclaimed").load_signing_key(private_unclaimed_key)
# Mark roles for metadata update (see #964, #958)
>>> repository.mark_dirty(['snapshot', 'targets','timestamp', 'unclaimed'])
>>> repository.writeall()
```
<!--
TODO: Integrate section with an updated delegation tutorial.
As it is now, it just messes up the state of the repository, i.e. marks
"unclaimed" as dirty, although there is nothing new to write.
#### Revoke Delegated Role ####
```python
# Continuing from the previous section . . .
# Create a delegated role that will be revoked in the next step...
>>> repository.targets('unclaimed').delegate("django", [public_unclaimed_key], ['bar*.tgz'])
# Revoke "django" and write the metadata of all remaining roles.
>>> repository.targets('unclaimed').revoke("django")
>>> repository.writeall()
```
-->
#### Wrap-up ####
In summary, the five steps a repository maintainer follows to create a TUF
repository are:
1. Create a directory for the software repository that holds the TUF metadata and the target files.
2. Create top-level roles (`root.json`, `snapshot.json`, `targets.json`, and `timestamp.json`.)
3. Add target files to the `targets` role.
4. Optionally, create delegated roles to distribute target files.
5. Write the changes.
The repository tool saves repository changes to a `metadata.staged` directory.
Repository maintainers may push finalized changes to the "live" repository by
copying the staged directory to its destination.
```Bash
# Copy the staged metadata directory changes to the live repository.
$ cp -r "repository/metadata.staged/" "repository/metadata/"
```
## Consistent Snapshots ##
The basic TUF repository we have generated above is adequate for repositories
that have some way of guaranteeing consistency of repository data. A community
software repository is one example where consistency of files and metadata can
become an issue. Repositories of this kind are continually updated by multiple
maintainers and software authors uploading their packages, increasing the
likelihood that a client downloading version X of a release unexpectedly
requests the target files of a version Y just released.
To guarantee consistency of metadata and target files, a repository may
optionally support multiple versions of `snapshot.json` simultaneously, where a
client with version 1 of `snapshot.json` can download `target_file.zip` and
another client with version 2 of `snapshot.json` can also download a different
`target_file.zip` (same file name, but different file digest.) If the
`consistent_snapshot` parameter of writeall() or write() are `True`, metadata
and target file names on the file system have their digests prepended (note:
target file names specified in metadata do not contain digests in their names.)
The repository maintainer is responsible for the duration of multiple versions
of metadata and target files available on a repository. Generating consistent
metadata and target files on the repository is enabled by setting the
`consistent_snapshot` argument of `writeall()` or `write()` . Note that
changing the consistent_snapshot setting involves writing a new version of
root.
<!--
TODO: Integrate section with an updated consistent snapshot tutorial.
As it is now, it just messes up the state of the repository, i.e. marks
"root" as dirty, although all other metadata needs to be re-written with
<VERSION> prefix and target files need to be re-written with <HASH> prefix in
their filenames.
```Python
# ----- Tutorial Section: Consistent Snapshots
>>> repository.root.load_signing_key(private_root_key)
>>> repository.root.load_signing_key(private_root_key2)
>>> repository.writeall(consistent_snapshot=True)
```
-->
## Delegate to Hashed Bins ##
Why use hashed bin delegations?
For software update systems with a large number of target files, delegating to
hashed bins (a special type of delegated role) might be an easier alternative
to manually performing the delegations. How many target files should each
delegated role contain? How will these delegations affect the number of
metadata that clients must additionally download in a typical update? Hashed
bin delegations are available to integrators that rather not deal with the
management of delegated roles and a great number of target files.
A large number of target files may be distributed to multiple hashed bins with
`delegate_hashed_bins()`. The metadata files of delegated roles will be nearly
equal in size (i.e., target file paths are uniformly distributed by calculating
the target filepath's digest and determining which bin it should reside in.)
The updater client will use "lazy bin walk" (visit and download the minimum
metadata required to find a target) to find a target file's hashed bin
destination. This method is intended for repositories with a large number of
target files, a way of easily distributing and managing the metadata that lists
the targets, and minimizing the number of metadata files (and size) downloaded
by the client.
The `delegate_hashed_bins()` method has the following form:
```Python
delegate_hashed_bins(list_of_targets, keys_of_hashed_bins, number_of_bins)
```
We next provide a complete example of retrieving target paths to add to hashed
bins, performing the hashed bin delegations, signing them, and delegating paths
to some role.
```Python
# Continuing from the previous section . . .
# Remove 'myproject/file4.txt' from unclaimed role and instead further delegate
# all targets in myproject/ to hashed bins.
>>> repository.targets('unclaimed').remove_target("myproject/file4.txt")
# Get a list of target paths for the hashed bins.
>>> targets = ['myproject/file4.txt']
# Delegate trust to 32 hashed bin roles. Each role is responsible for the set
# of target files, determined by the path hash prefix. TUF evenly distributes
# hexadecimal ranges over the chosen number of bins (see output).
# To initialize the bins we use one key, which TUF warns us about (see output).
# However, we can assign separate keys to each bin, with the method used in
# previous sections, accessing a bin by its hash prefix range name, e.g.:
# "repository.targets('00-07').add_verification_key('public_00-07_key')".
>>> repository.targets('unclaimed').delegate_hashed_bins(
... targets, [public_unclaimed_key], 32)
Creating hashed bin delegations.
1 total targets.
32 hashed bins.
256 total hash prefixes.
Each bin ranges over 8 hash prefixes.
Adding a verification key that has already been used. [repeated 32x]
# The hashed bin roles can also be accessed by iterating the "delegations"
# property of the delegating role, which we do here to load the signing key.
>>> for delegation in repository.targets('unclaimed').delegations:
... delegation.load_signing_key(private_unclaimed_key)
# Mark roles for metadata update (see #964, #958)
>>> repository.mark_dirty(['00-07', '08-0f', '10-17', '18-1f', '20-27', '28-2f',
... '30-37', '38-3f', '40-47', '48-4f', '50-57', '58-5f', '60-67', '68-6f',
... '70-77', '78-7f', '80-87', '88-8f', '90-97', '98-9f', 'a0-a7', 'a8-af',
... 'b0-b7', 'b8-bf', 'c0-c7', 'c8-cf', 'd0-d7', 'd8-df', 'e0-e7', 'e8-ef',
... 'f0-f7', 'f8-ff', 'snapshot', 'timestamp', 'unclaimed'])
>>> repository.writeall()
```
## How to Perform an Update ##
The following [repository tool](../tuf/repository_tool.py) function creates a directory
structure that a client downloading new software using TUF (via
[tuf/client/updater.py](../tuf/client/updater.py)) expects. The `root.json` metadata file must exist, and
also the directories that hold the metadata files downloaded from a repository.
Software updaters integrating TUF may use this directory to store TUF updates
saved on the client side.
```python
>>> from tuf.repository_tool import *
>>> create_tuf_client_directory("repository/", "client/tufrepo/")
```
`create_tuf_client_directory()` moves metadata from `repository/metadata` to
`client/` in this example. The repository in `repository/` may be the
repository example created earlier in this document.
## Test TUF Locally ##
Run the local TUF repository server.
```Bash
$ cd "repository/"; python3 -m http.server 8001
```
We next retrieve targets from the TUF repository and save them to `client/`.
The `client.py` script is available to download metadata and files from a
specified repository. In a different command-line prompt, where `tuf` is
installed . . .
```Bash
$ cd "client/"
$ ls
tufrepo/
$ client.py --repo http://localhost:8001 file1.txt
$ ls . tuftargets/
.:
tufrepo tuftargets
tuftargets/:
file1.txt
```

5
docs/conf.py
View file

@ -35,11 +35,6 @@
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ['GETTING_STARTED.rst']
# -- Options for HTML output -------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for

BIN
docs/images/repository_tool-diagram.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 90 KiB

323
tuf/ATTACKS.md
View file

@ -1,323 +0,0 @@
# Demonstrate protection against malicious updates
## Table of Contents ##
- [Blocking Malicious Updates](#blocking-malicious-updates)
- [Arbitrary Package Attack](#arbitrary-package-attack)
- [Rollback Attack](#rollback-attack)
- [Indefinite Freeze Attack](#indefinite-freeze-attack)
- [Endless Data Attack](#endless-data-attack)
- [Compromised Key Attack](#compromised-key-attack)
- [Slow Retrieval Attack](#slow-retrieval-attack)
- [Conclusion](#conclusion)
## Blocking Malicious Updates ##
TUF protects against a number of attacks, some of which include rollback,
arbitrary package, and mix and match attacks. We begin this document on
blocking malicious updates by demonstrating how the client rejects a target
file downloaded from the software repository that doesn't match what is listed
in TUF metadata.
The following demonstration requires and operates on the repository created in
the [repository management
tutorial](https://github.com/theupdateframework/python-tuf/blob/develop/tuf/README.md).
### Arbitrary Package Attack ###
In an arbitrary package attack, an attacker installs anything they want on the
client system. That is, an attacker can provide arbitrary files in response to
download requests and the files will not be detected as illegitimate. We
simulate an arbitrary package attack by creating a "malicious" target file
that our client attempts to fetch.
```Bash
$ mv 'repository/targets/file2.txt' 'repository/targets/file2.txt.backup'
$ echo 'bad_target' > 'repository/targets/file2.txt'
```
We next reset our local timestamp (so that a new update is prompted), and
the target files previously downloaded by the client.
```Bash
$ rm -rf "client/targets/" "client/metadata/current/timestamp.json"
```
The client now performs an update and should detect the invalid target file...
Note: The following command should be executed in the "client/" directory.
```Bash
$ python3 basic_client.py --repo http://localhost:8001
Error: No working mirror was found:
localhost:8001: BadHashError()
```
The log file (tuf.log) saved to the current working directory contains more
information on the update procedure and the cause of the BadHashError.
```Bash
...
BadHashError: Observed
hash ('f569179171c86aa9ed5e8b1d6c94dfd516123189568d239ed57d818946aaabe7') !=
expected hash (u'67ee5478eaadb034ba59944eb977797b49ca6aa8d3574587f36ebcbeeb65f70e')
[2016-10-20 19:45:16,079 UTC] [tuf.client.updater] [ERROR] [_get_file:1415@updater.py]
Failed to update /file2.txt from all mirrors: {u'http://localhost:8001/targets/file2.txt': BadHashError()}
```
Note: The "malicious" target file should be removed and the original file2.txt
restored, otherwise the following examples will fail with BadHashError
exceptions:
```Bash
$ mv 'repository/targets/file2.txt.backup' 'repository/targets/file2.txt'
```
### Indefinite Freeze Attack ###
In an indefinite freeze attack, an attacker continues to present a software
update system with the same files the client has already seen. The result is
that the client does not know that new files are available. Although the
client would be unable to prevent an attacker or compromised repository from
feeding it stale metadata, it can at least detect when an attacker is doing so
indefinitely. The signed metadata used by TUF contains an "expires" field that
indicates when metadata should no longer be trusted.
In the following simulation, the client first tries to perform an update.
```Bash
$ python3 basic_client.py --repo http://localhost:8001
```
According to the logger (`tuf.log` file in the current working directory),
everything appears to be up-to-date. The remote server should also show that
the client retrieved only the timestamp.json file. Let's suppose now that an
attacker continues to feed our client the same stale metadata. If we were to
move the time to a future date that would cause metadata to expire, the TUF
framework should raise an exception or error to indicate that the metadata
should no longer be trusted.
```Bash
$ sudo date -s '2080-12-25 12:34:56'
Wed Dec 25 12:34:56 EST 2080
$ python3 basic_client.py --repo http://localhost:8001
Error: No working mirror was found:
u'localhost:8001': ExpiredMetadataError(u"Metadata u'root' expired on Tue Jan 1 00:00:00 2030 (UTC).",)
```
Note: Reset the date to continue with the rest of the attacks.
### Rollback Attack ###
In a rollback attack, an attacker presents a software update system with older
files than those the client has already seen, causing the client to use files
older than those the client knows about. We begin this example by saving the
current version of the Timestamp file available on the repository. This saved
file will later be served to the client to see if it is rejected. The client
should not accept versions of metadata that is older than previously trusted.
Navigate to the directory containing the server's files and save the current
timestamp.json to a temporary location:
```Bash
$ cp repository/metadata/timestamp.json /tmp
```
We should next generate a new Timestamp file on the repository side.
```Bash
$ python3
>>> from tuf.repository_tool import *
>>> repository = load_repository('repository')
>>> repository.timestamp.version
1
>>> repository.timestamp.version = 2
>>> repository.dirty_roles()
Dirty roles: [u'timestamp']
>>> private_timestamp_key = import_rsa_privatekey_from_file("keystore/timestamp_key")
Enter a password for the encrypted RSA file (/path/to/keystore/timestamp_key):
>>> repository.timestamp.load_signing_key(private_timestamp_key)
>>> repository.write('timestamp')
$ cp repository/metadata.staged/* repository/metadata
```
Now start the HTTP server from the directory containing the 'repository'
subdirectory.
```Bash
$ python3 -m SimpleHTTPServer 8001
```
And perform an update so that the client retrieves the updated timestamp.json.
```Bash
$ python3 basic_client.py --repo http://localhost:8001
```
Finally, move the previous timestamp.json file to the current live repository
and have the client try to download the outdated version. The client should
reject it!
```Bash
$ cp /tmp/timestamp.json repository/metadata/
$ cd repository; python3 -m SimpleHTTPServer 8001
```
On the client side, perform an update...
```Bash
$ python3 basic_client.py --repo http://localhost:8001
Error: No working mirror was found:
u'localhost:8001': ReplayedMetadataError()
```
The tuf.log file contains more information about the ReplayedMetadataError
exception and update process. Please reset timestamp.json to the latest
version, which can be found in the 'repository/metadata.staged' subdirectory.
```Bash
$ cp repository/metadata.staged/timestamp.json repository/metadata
```
### Endless Data Attack ###
In an endless data attack, an attacker responds to a file download request with
an endless stream of data, causing harm to clients (e.g., a disk partition
filling up or memory exhaustion). In this simulated attack, we append extra
data to one of the target files available on the software repository. The
client should only download the exact number of bytes it expects for a
requested target file (according to what is listed in trusted TUF metadata).
```Bash
$ cp repository/targets/file1.txt /tmp
$ python3 -c "print 'a' * 1000" >> repository/targets/file1.txt
```
Now delete the local metadata and target files on the client side so
that remote metadata and target files are downloaded again.
```Bash
$ rm -rf client/targets/
$ rm client/metadata/current/snapshot.json* client/metadata/current/timestamp.json*
```
Lastly, perform an update to verify that the file1.txt is downloaded up to the
expected size, and no more. The target file available on the software
repository does contain more data than expected, though.
```Bash
$ python3 basic_client.py --repo http://localhost:8001
```
At this point, part of the "file1.txt" file should have been fetched. That is,
up to 31 bytes of it should have been downloaded, and the rest of the maliciously
appended data ignored. If we inspect the logger, we'd discover the following:
```Bash
[2016-10-06 21:37:39,092 UTC] [tuf.download] [INFO] [_download_file:235@download.py]
Downloading: u'http://localhost:8001/targets/file1.txt'
[2016-10-06 21:37:39,145 UTC] [tuf.download] [INFO] [_check_downloaded_length:610@download.py]
Downloaded 31 bytes out of the expected 31 bytes.
[2016-10-06 21:37:39,145 UTC] [tuf.client.updater] [INFO] [_get_file:1372@updater.py]
Not decompressing http://localhost:8001/targets/file1.txt
[2016-10-06 21:37:39,145 UTC] [tuf.client.updater] [INFO] [_check_hashes:778@updater.py]
The file's sha256 hash is correct: 65b8c67f51c993d898250f40aa57a317d854900b3a04895464313e48785440da
```
Indeed, the sha256 sum of the first 31 bytes of the "file1.txt" available
on the repository should match to what is trusted. The client did not
downloaded the appended data.
Note: Restore file1.txt
```Bash
$ cp /tmp/file1.txt repository/targets/
```
### Compromised Key Attack ###
An attacker who compromise less than a given threshold of keys is limited in
scope. This includes relying on a single online key (such as only being
protected by SSL) or a single offline key (such as most software update systems
use to sign files). In this example, we attempt to sign a role file with
less-than-a-threshold number of keys. A single key (suppose this is a
compromised key) is used to demonstrate that roles must be signed with the
total number of keys required for the role. In order to compromise a role, an
attacker would have to compromise a threshold of keys. This approach of
requiring a threshold number of signatures provides compromise resilience.
Let's attempt to sign a new snapshot file with a less-than-threshold number of
keys. The client should reject the partially signed snapshot file served by
the repository (or imagine that it is a compromised software repository).
```Bash
$ python3
>>> from tuf.repository_tool import *
>>> repository = load_repository('repository')
>>> version = repository.root.version
>>> repository.root.version = version + 1
>>> private_root_key = import_rsa_privatekey_from_file("keystore/root_key", password="password")
>>> repository.root.load_signing_key(private_root_key)
>>> private_root_key2 = import_rsa_privatekey_from_file("keystore/root_key2", password="password")
>>> repository.root.load_signing_key(private_root_key2)
>>> repository.snapshot.version = 8
>>> repository.snapshot.threshold = 2
>>> private_snapshot_key = import_rsa_privatekey_from_file("keystore/snapshot_key", password="password")
>>> repository.snapshot.load_signing_key(private_snapshot_key)
>>> repository.timestamp.version = 8
>>> private_timestamp_key = import_rsa_privatekey_from_file("keystore/timestamp_key", password="password")
>>> repository.timestamp.load_signing_key(private_timestamp_key)
>>> repository.write('root')
>>> repository.write('snapshot')
>>> repository.write('timestamp')
$ cp repository/metadata.staged/* repository/metadata
```
The client now attempts to refresh the top-level metadata and the
partially written snapshot.json, which should be rejected.
```Bash
$ python3 basic_client.py --repo http://localhost:8001
Error: No working mirror was found:
u'localhost:8001': BadSignatureError()
```
### Slow Retrieval Attack ###
In a slow retrieval attack, an attacker responds to clients with a very slow
stream of data that essentially results in the client never continuing the
update process. In this example, we simulate a slow retrieval attack by
spawning a server that serves data at a slow rate to our update client data.
TUF should not be vulnerable to this attack, and the framework should raise an
exception or error when it detects that a malicious server is serving it data
at a slow enough rate.
We first spawn the server that slowly streams data to the client. The
'slow_retrieval_server_old.py' module (can be found in the tests/ directory of the
source code) should be copied over to the server's 'repository/' directory from
which to launch it.
```Bash
# Before launching the slow retrieval server, copy 'slow_retrieval_server_old.py'
# to the 'repository/' directory and run it from that directory as follows:
$ python3 slow_retrieval_server_old.py 8002 mode_2
```
The client may now make a request to the slow retrieval server on port 8002.
However, before doing so, we'll reduce (for the purposes of this demo) the
minimum average download rate allowed and download chunk size. Open the
'settings.py' module and set MIN_AVERAGE_DOWNLOAD_SPEED = 5 and CHUNK_SIZE = 1.
This should make it so that the client detects the slow retrieval server's
delayed streaming.
```Bash
$ python3 basic_client.py --verbose 1 --repo http://localhost:8002
Error: No working mirror was found:
u'localhost:8002': SlowRetrievalError()
```
The framework should detect the slow retrieval attack and raise a
SlowRetrievalError exception to the client application.
## Conclusion ##
These are just some of the attacks that TUF provides protection against. For
more attacks and updater weaknesses, please see the
[Security](https://theupdateframework.io/security/)
page.

342
tuf/README-developer-tools.md
View file

@ -1,342 +0,0 @@
# The Update Framework Developer Tool: How to Update your Project Securely on a TUF Repository
## Table of Contents
- [Overview](#overview)
- [Creating a Simple Project](#creating_a_simple_project)
- [Generating a Key](#generating_a_key)
- [The Project Class](#the_project_class)
- [Signing and Writing the Metadata](#signing_and_writing_the_metadata)
- [Loading an Existing Project](#loading_an_existing_project)
- [Delegations](#delegations)
- [Managing Keys](#managing_keys)
- [Managing Targets](#managing_targets)
<a name="overview">
## Overview
The Update Framework (TUF) is a Python-based security system for software
updates. In order to prevent your users from downloading vulnerable or malicious
code disguised as updates to your software, TUF requires that each update you
release include certain metadata verifying your authorship of the files.
The TUF developer tools are a Python Library that enables you to create and
maintain the required metadata for files hosted on a TUF Repository. (We call
these files “targets,” to distinguish them from the metadata associated with
them. Both of these together comprise a complete “project”.) You will use these
tools to generate the keys and metadata you need to claim and secure your files
on the repository, and to update the metadata and sign it with those keys
whenever you upload a new version of those files.
This document will teach you how to use these tools in two parts. The first
part walks through the creation of a minimal-complexity TUF project, which is
all you need to get started, and can be expanded later. The second part details
the full functionality of the tools, which offer a finer degree of control in
securing your project.
<a name="creating_a_simple_project">
## Creating a Simple Project
This section walks through the creation of a small example project with just
one target. Once created, this project will be fully functional, and can be
modified as needed.
<a name="generating_a_key">
### Generating a Key
First, we will need to generate a key to sign the metadata. Keys are generated
in pairs: one public and the other private. The private key is
password-protected and is used to sign metadata. The public key can be shared
freely, and is used to verify signatures made by the private key. You will need
to share your public key with the repository hosting your project so they can
verify your metadata is signed by the right person.
The generate\_and\_write\_rsa\_keypair function will create two key files named
"path/to/key.pub", which is the public key and "path/to/key", which
is the private key.
```
>>> from tuf.developer_tool import *
>>> generate_and_write_rsa_keypair_with_prompt(filepath="path/to/key")
enter password to encrypt private key file 'path/to/key'
(leave empty if key should not be encrypted):
Confirm:
>>>
```
We can also use the bits parameter to set a different key length (the default
is 3072). We can also `generate_and_write_rsa_keypair` with a `password`
parameter if a prompt is not desired.
In this example we will be using rsa keys, but ed25519 keys are also supported.
Now we have a key for our project, we can proceed to create our project.
<a name="the_project_class">
### The Project Class
The TUF developer tool is built around the Project class, which is used to
organize groups of targets associated with a single set of metadata. A single
Project instance is used to keep track of all the target files and metadata
files in one project. The Project also keeps track of the keys and signatures,
so that it can update all the metadata with the correct changes and signatures
on a single command.
Before creating a project, you must know where it will be located in the TUF
Repository. In the following example, we will create a project to be hosted as
"repo/unclaimed/example_project" within the repository, and store a local copy
of the metadata at "path/to/metadata". The project will comprise a single
target file, "local/path/to/example\_project/target\_1" locally, and we will
secure it with the key generated above.
First, we must import the generated keys. We can do that by issuing the
following command:
```
>>> public_key = import_rsa_publickey_from_file("path/to/keys.pub")
```
After importing the key, we can generate a new project with the following
command:
```
>>> project = create_new_project(project_name="example_project",
... metadata_directory="local/path/to/metadata/",
... targets_directory="local/path/to/example_project",
... location_in_repository="repo/unclaimed", key=public_key)
```
Let's list the arguments and make sense out of this rather long function call:
- create a project named example_project: the name of the metadata file will match this name
- the metadata will be located in "local/path/to/metadata", this means all of the generated files
for this project will be located here
- the targets are located in local/path/to/example project. If your targets are located in some other
place, you can point the targets directory there. Files must reside under the path local/path/to/example_project or else it won't be possible to add them.
- location\_in\_repository points to repo/unclaimed, this will be prepended to the paths in the generated metadata so the signatures all match.
Now the project is in memory and we can do different operations on it such as
adding and removing targets, delegating files, changing signatures and keys,
etc. For the moment we are interested in adding our one and only target inside
the project.
To add a target, we issue the following method:
```
>>> project.add_target("local/path/to/example_project/target_1")
```
Note that the file "target\_1" should be located in
"local/path/to/example\_project", or this method will throw an
error.
At this point, the metadata is not valid. We have assigned a key to the
project, but we have not *signed* it with that key. Signing is the process of
generating a signature with our private key so it can be verified with the
public key by the server (upon uploading) and by the clients (when updating).
<a name="signing_and_writing_the_metadata">
### Signing and Writing the Metadata ###
In order to sign the metadata, we need to import the private key corresponding
to the public key we added to the project. One the key is loaded to the project,
it will automatically be used to sign the metadata whenever it is written.
```
>>> private_key = import_rsa_privatekey_from_file("path/to/key")
Enter password for the RSA key:
>>> project.load_signing_key(private_key)
>>> project.write()
```
When all changes to the project have been written, the metadata is ready to be
uploaded to the repository, and it is safe to exit the Python interpreter, or
to delete the Project instance.
The project can be loaded later to update changes to the project. The metadata
contains checksums that have to match the actual files or else it won't be
accepted by the upstream repository.
At this point, if you have followed all the steps in this document so far
(substituting appropriate names and filepaths) you will have created a basic
TUF project, which can be expanded as needed. The simplest way to get your
project secured is to add all your files using add\_target() (or see [Managing
Keys](#managing_keys) on how to add whole directories). If your project has
several contributors, you may want to consider adding
[delegations](#delegations) to your project.
<a name="loading_an_existing_project">
## Loading an Existing Project
To make changes to existing metadata, we will need the Project again. We can
restore it with the load_project() function.
```
>>> from tuf.developer_tool import *
>>> project = load_project("local/path/to/metadata")
```
Each time the project is loaded anew, the necessary private keys must also be
loaded in order to sign metadata.
```
>>> private_key = import_rsa_privatekey_from_file("path/to/key")
Enter a password for the RSA key:
>>> project.load_signing_key(private_key)
>>> project.write()
```
If your project does not use any delegations, the five commands above are all
you need to update your project's metadata.
<a name="delegations">
## Delegations
The project we created above is secured entirely by one key. If you want to
allow someone else to update part of your project independently, you will need
to delegate a new role for them. For example, we can do the following:
```
>>> other_key = import_rsa_publickey_from_file(“another_public_key.pub”)
>>> targets = ['local/path/to/newtarget']
>>> project.delegate(“newrole”, [other_key], targets)
```
The new role is now an attribute of the Project instance, and contains the same
methods as Project. For example, we can add targets in the same way as before:
```
>>> project(“newrole”).add_target(“delegated_1”)
```
Recall that we input the other person’s key as part of a list. That list can
contain any number of public keys. We can also add keys to the role after
creating it using the [add\_verification\_key()](#adding_a_key_to_a_delegation)
method.
### Delegated Paths
By default, a delegated role is permitted to add and modify targets anywhere in
the Project's targets directory. We can delegate trust of paths to a role to
limit this permission.
```
>>> project.add_paths(["delegated/filepath"], "newrole")
```
This will prevent the delegated role from signing targets whose local filepaths
do not begin with "delegated/filepath". We can delegate several filepaths to a
role by adding them to the list in the first parameter, or by invoking the
method again. A role with multiple delegated paths can add targets to any of
them.
Note that this method is invoked from the parent role (in this case, the Project)
and takes the delegated role name as an argument.
### Nested Delegations
It is possible for a delegated role to have delegations of its own. We can do
this by calling delegate() on a delegated role:
```
>>> project("newrole").delegate(“nestedrole”, [key], targets)
```
Nested delegations function no differently than first-order delegations. to
demonstrate, adding a target to nested delegation looks like this:
```
>>> project("newrole")("nestedrole").add_target("foo")
```
### Revoking Delegations
Delegations can be revoked, removing the delegated role from the project.
```
>>> project.revoke("newrole")
```
<a name="managing_keys">
## Managing Keys
This section describes the key-related functions and parameters not covered in
the [Creating a Simple Project](#creating_a_simple_project) section.
### Additional Parameters for Key Generation
When generating keys, it is possible to specify the length of the key in bits
and its password as parameters:
```
>>> generate_and_write_rsa_keypair(password="pw", filepath="path/to/key", bits=2048)
```
The bits parameter defaults to 3072, and values below 2048 will raise an error.
The password parameter is only intended to be used in scripts.
<a name="adding_a_key_to_a_delegation">
### Adding a Key to a Delegation
New verifications keys can be added to an existing delegation using
add\_verification\_key():
```
>>> project("rolename").add_verification_key(pubkey)
```
A delegation can have several verification keys at once. By default, a
delegated role with multiple keys can be written using any one of their
corresponding signing keys. To modify this behavior, you can change the
delegated role's [threshold](#delegation_thrsholds).
### Removing a Key from a Delegation
Verification keys can also be removed, like this:
```
>>> project("rolename").remove_verification_key(pubkey)
```
Remember that a project can only have one key, so this method will return an
error if there is already a key assigned to it. In order to replace a key we
must first delete the existing one and then add the new one. It is possible to
omit the key parameter in the create\_new\_project() function, and add the key
later.
### Changing the Project Key
Each Project instance can only have one verification key. This key can be
replaced by removing it and adding a new key, in that order.
```
>>> project.remove_verification_key(oldkey)
>>> project.add_verification_key(new)
```
<a name="delegation_thresholds">
### Delegation Thresholds
Every delegated role has a threshold, which determines how many of its signing
keys need to be loaded to write the role. The threshold defaults to 1, and
should not exceed the number of verification keys assigned to the role. The
threshold can be accessed as a property of a delegated role.
```
>>> project("rolename").threshold = 2
```
The above line will set the "rolename" role's threshold to 2.
<a name="managing_targets">
## Managing Targets
There are supporting functions of the targets library to make the project
maintenance easier. These functions are described in this section.
### Adding Targets by Directory
This function is especially useful when creating a new project to add all the
files contained in the targets directory. The following code block illustrates
the usage of this function:
```
>>> list_of_targets = \
... project.get_filepaths_in_directory(“path/within/targets/folder”,
... recursive_walk=False, follow_links=False)
>>> project.add_targets(list_of_targets)
```
### Deleting Targets from a Project
It is possible that we want to delete existing targets inside our project. To
stop the developer tool from tracking this file we can issue the following
command:
```
>>> project.remove_target(“target_1”)
```
Now the target file won't be part of the metadata.

5
tuf/README.md
View file

@ -1,5 +0,0 @@
[Quickstart](../docs/QUICKSTART.md)
[CLI](../docs/CLI.md)
[Tutorial](../docs/TUTORIAL.md)

151
tuf/client/README.md
View file

@ -1,151 +0,0 @@
# updater.py
**updater.py** is intended as the only TUF module that software update
systems need to utilize for a low-level integration. It provides a single
class representing an updater that includes methods to download, install, and
verify metadata or target files in a secure manner. Importing
**tuf.client.updater** and instantiating its main class is all that is
required by the client prior to a TUF update request. The importation and
instantiation steps allow TUF to load all of the required metadata files
and set the repository mirror information.
The **tuf.repository_tool** module can be used to create a TUF repository. See
[tuf/README](../README.md) for more information on creating TUF repositories.
## Overview of the Update Process
1. The software update system instructs TUF to check for updates.
2. TUF downloads and verifies timestamp.json.
3. If timestamp.json indicates that snapshot.json has changed, TUF downloads and
verifies snapshot.json.
4. TUF determines which metadata files listed in snapshot.json differ from those
described in the last snapshot.json that TUF has seen. If root.json has changed,
the update process starts over using the new root.json.
5. TUF provides the software update system with a list of available files
according to targets.json.
6. The software update system instructs TUF to download a specific target
file.
7. TUF downloads and verifies the file and then makes the file available to
the software update system.
If at any point in the above procedure there is a problem (i.e., if unexpired,
signed, valid metadata cannot be retrieved from the repository), the Root file
is downloaded and the process is retried once more (and only once to avoid an
infinite loop). Optionally, the software update system using the framework
can decide how to proceed rather than automatically downloading a new Root file.
## Example Client
### Refresh TUF Metadata
```Python
# The client first imports the 'updater.py' module, the only module the
# client is required to import. The client will utilize a single class
# from this module.
import tuf.client.updater
import tuf.settings
# The only other module the client interacts with is 'settings'. The
# client accesses this module solely to set the repository directory.
# This directory will hold the files downloaded from a remote repository.
tuf.settings.repositories_directory = 'path/to/local_repository'
# Next, the client creates a dictionary object containing the repository
# mirrors. The client may download content from any one of these mirrors.
# In the example below, a single mirror named 'mirror1' is defined. The
# mirror is located at 'http://localhost:8001', and all of the metadata
# and targets files can be found in the 'metadata' and 'targets' directory,
# respectively. If the client wishes to only download target files from
# specific directories on the mirror, the 'confined_target_dirs' field
# should be set. In this example, the client hasn't set confined_target_dirs,
# which is interpreted as no confinement. In other words, the client can download
# targets from any directory or subdirectories. If the client had chosen
# 'targets1/', they would have been confined to the '/targets/targets1/'
# directory on the 'http://localhost:8001' mirror.
repository_mirrors = {'mirror1': {'url_prefix': 'http://localhost:8001',
'metadata_path': 'metadata',
'targets_path': 'targets'}}
# The updater may now be instantiated. The Updater class of 'updater.py'
# is called with two arguments. The first argument assigns a name to this
# particular updater and the second argument the repository mirrors defined
# above.
updater = tuf.client.updater.Updater('updater', repository_mirrors)
# The client calls the refresh() method to ensure it has the latest
# copies of the top-level metadata files (i.e., Root, Targets, Snapshot,
# Timestamp).
updater.refresh()
```
### Download Specific Target File
```Python
# Example demonstrating an update that downloads a specific target.
# Refresh the metadata of the top-level roles (i.e., Root, Targets, Snapshot, Timestamp).
updater.refresh()
# get_one_valid_targetinfo() updates role metadata when required. In other
# words, if the client doesn't possess the metadata that lists 'LICENSE.txt',
# get_one_valid_targetinfo() will try to fetch / update it.
target = updater.get_one_valid_targetinfo('LICENSE.txt')
updated_target = updater.updated_targets([target], destination_directory)
for target in updated_target:
updater.download_target(target, destination_directory)
# Client code here may also reference target information (including 'custom')
# by directly accessing the dictionary entries of the target. The 'custom'
# entry is additional file information explicitly set by the remote repository.
target_path = target['filepath']
target_length = target['fileinfo']['length']
target_hashes = target['fileinfo']['hashes']
target_custom_data = target['fileinfo']['custom']
# Remove any files from the destination directory that are no longer being
# tracked. For example, a target file from a previous snapshot that has since
# been removed on the remote repository.
updater.remove_obsolete_targets(destination_directory)
```
### A Simple Integration Example with client.py
``` Bash
# Assume a simple TUF repository has been setup with 'repo.py'.
$ client.py --repo http://localhost:8001
# Metadata and target files are silently updated. An exception is only raised if an error,
# or attack, is detected. Inspect 'tuf.log' for the outcome of the update process.
$ cat tuf.log
[2013-12-16 16:17:05,267 UTC] [tuf.download] [INFO][_download_file:726@download.py]
Downloading: http://localhost:8001/metadata/timestamp.json
[2013-12-16 16:17:05,269 UTC] [tuf.download] [WARNING][_check_content_length:589@download.py]
reported_length (545) < required_length (2048)
[2013-12-16 16:17:05,269 UTC] [tuf.download] [WARNING][_check_downloaded_length:656@download.py]
Downloaded 545 bytes, but expected 2048 bytes. There is a difference of 1503 bytes!
[2013-12-16 16:17:05,611 UTC] [tuf.download] [INFO][_download_file:726@download.py]
Downloading: http://localhost:8001/metadata/snapshot.json
[2013-12-16 16:17:05,612 UTC] [tuf.client.updater] [INFO][_check_hashes:636@updater.py]
The file's sha256 hash is correct: 782675fadd650eeb2926d33c401b5896caacf4fd6766498baf2bce2f3b739db4
[2013-12-16 16:17:05,951 UTC] [tuf.download] [INFO][_download_file:726@download.py]
Downloading: http://localhost:8001/metadata/targets.json
[2013-12-16 16:17:05,952 UTC] [tuf.client.updater] [INFO][_check_hashes:636@updater.py]
The file's sha256 hash is correct: a5019c28a1595c43a14cad2b6252c4d1db472dd6412a9204181ad6d61b1dd69a
[2013-12-16 16:17:06,299 UTC] [tuf.download] [INFO][_download_file:726@download.py]
Downloading: http://localhost:8001/targets/file1.txt
[2013-12-16 16:17:06,303 UTC] [tuf.client.updater] [INFO][_check_hashes:636@updater.py]
The file's sha256 hash is correct: ecdc5536f73bdae8816f0ea40726ef5e9b810d914493075903bb90623d97b1d8