Elgato_dark/python-tuf
1
0
Fork 0
mirror of https://github.com/theupdateframework/python-tuf synced 2026-05-24 10:08:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge branch 'develop' of https://github.com/theupdateframework/tuf into develop

Browse source
This commit is contained in:
Vladimir Diaz 2014-07-03 09:20:37 -04:00
commit c874abc408
6 changed files with 158 additions and 118 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

52
README.md
View file

@ -83,32 +83,54 @@ TUF specification document is also available:
##Installation
```Bash
pip - installing and managing Python packages (recommended):
pip - installing and managing Python packages (recommended)
# Installing from Python Package Index (https://pypi.python.org/pypi).
Installing from Python Package Index (https://pypi.python.org/pypi).
$ pip install tuf
# Installing from local source archive.
Installing from local source archive.
$ pip install <path to archive>
# Or from the root directory of the unpacked archive.
Or from the root directory of the unpacked archive.
$ pip install .
```
### Installing optional requirements (i.e., after installing tuf).
```Bash
# The optional `tuf[tools]` can be installed by users that wish to generate
# TUF repository files, such as metadata, cryptographic keys, and signatures.
# Whereas the basic install can only verify ed25519 signatures and is intended
# for sofware updater clients, `tuf[tools]` provides repository maintainers
# secure ed25519 key and signature generation with PyNaCl / libsodium.
### Installation of Optional Requirements (after minimal install)
The optional `tuf[tools]` can be installed by repository maintainers that need to generate TUF repository files, such as metadata, cryptographic keys, and signatures. Whereas the minimal install can only verify ed25519 signatures and is intended for sofware updater clients, `tuf[tools]` provides repository maintainers secure ed25519 key and signature generation with PyNaCl / libsodium.
# The TUF tools also enable general-purpose cryptography with PyCrypto. Software
# updaters that want to support verification of RSASSA-PSS signatures must require
# their clients to install `tuf[tools]`.
TUF tools also enable general-purpose cryptography with PyCrypto. Software updaters that want to support verification of RSASSA-PSS signatures should require their clients to install `tuf[tools]`.
```Bash
$ pip install tuf[tools]
```
### Instructions for Contributors
[Virtualenv](https://virtualenv.pypa.io/en/latest/virtualenv.html#introduction) is a tool to create isolated Python environments. It also includes `pip` and `setuptools`, Python packages used to install TUF and its dependencies. All installation methods of virtualenv are outlined in the [installation section](https://virtualenv.pypa.io/en/latest/virtualenv.html#installation) and instructions for installing locally from source here:
```Bash
$ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.11.6.tar.gz
$ tar xvfz virtualenv-1.11.6.tar.gz
$ cd virtualenv-1.11.6
$ python virtualenv.py myVE
```
PyCrypto and PyNaCl (third-party dependencies needed by the repository tools) require
Python and FFI (Foreign Function Interface) development header files. Debian-based
distributions can install these header libraries with apt (Advanced Package Tool.)
```Bash
$ apt-get install python-dev
$ apt-get install libffi-dev
```
Installation of minimal, optional, development, and testing requirements can then be accomplished with one command:
```Bash
$ pip install -r dev-requirements.txt
```
The Update Framework's unit tests can be executed by invoking [tox](https://testrun.org/tox/). All supported Python versions are tested, but must already be installed locally.
```Bash
$ tox
```
##Using TUF
TUF has four major classes of users: clients, for whom TUF is largely transparent; mirrors, who will (in most cases) have nothing at all to do with TUF; upstream servers, who will largely be responsible for care and feeding of repositories; and integrators, who do the work of putting TUF into existing projects.

119
tests/test_developer_tool.py
View file

@ -5,7 +5,7 @@
test_developer_tool.py.
<Authors>
Santiago Torres Arias <torresariass@gmail.com
Santiago Torres Arias <torresariass@gmail.com>
Zane Fisher <zanefisher@gmail.com>
<Copyright>
@ -33,7 +33,7 @@
from tuf.developer_tool import METADATA_DIRECTORY_NAME
from tuf.developer_tool import TARGETS_DIRECTORY_NAME
logger = logging.getLogger("tuf.test_developer_tool")
logger = logging.getLogger('tuf.test_developer_tool')
developer_tool.disable_console_log_messages()
@ -71,7 +71,7 @@ def test_create_new_project(self):
# These are the usual values we will be throwing to the function, however
# we will swap these for nulls or malformed values every now and then to
# test input.
project_name = "test_suite"
project_name = 'test_suite'
metadata_directory = local_tmp
location_in_repository = '/prefix'
targets_directory = None
@ -184,19 +184,19 @@ def test_load_project(self):
local_tmp = tempfile.mkdtemp(dir = self.tmp_dir)
# Test non-existent project filepath.
nonexistent_path = os.path.join(local_tmp, "nonexistent")
nonexistent_path = os.path.join(local_tmp, 'nonexistent')
self.assertRaises(IOError, developer_tool.load_project, nonexistent_path)
# Copy the pregenerated metadata.
project_data_filepath = os.path.join('repository_data', 'project')
target_project_data_filepath = os.path.join(local_tmp, 'project')
shutil.copytree("repository_data/project", target_project_data_filepath)
shutil.copytree('repository_data/project', target_project_data_filepath)
# Properly load a project.
repo_filepath = os.path.join(local_tmp, 'project', 'test-repo')
project = developer_tool.load_project(repo_filepath)
self.assertTrue(project.layout_type == "repo-like")
self.assertTrue(project.layout_type == 'repo-like')
repo_filepath = os.path.join(local_tmp, 'project', 'test-flat')
new_targets_path = os.path.join(local_tmp, 'project', 'targets')
@ -213,7 +213,7 @@ def test_load_project(self):
# Load a project with a file missing.
file_to_corrupt = os.path.join(repo_filepath, 'test-flat', 'role1.json')
with open(file_to_corrupt, 'wt') as fp:
fp.write("this is not a json file")
fp.write('this is not a json file')
self.assertRaises(tuf.Error, developer_tool.load_project, repo_filepath)
@ -221,14 +221,14 @@ def test_load_project(self):
def test_add_verification_keys(self):
# create a new project instance
project = developer_tool.Project("test_verification_keys", "somepath",
"someotherpath", "prefix")
# Create a new project instance.
project = developer_tool.Project('test_verification_keys', 'somepath',
'someotherpath', 'prefix')
# add invalid verification key
self.assertRaises(tuf.FormatError, project.add_verification_key, "invalid")
# Add invalid verification key.
self.assertRaises(tuf.FormatError, project.add_verification_key, 'invalid')
# add verification key
# Add verification key.
# - load it first
keystore_path = os.path.join('repository_data','keystore')
first_verification_key_path = os.path.join(keystore_path,'root_key.pub')
@ -238,7 +238,7 @@ def test_add_verification_keys(self):
project.add_verification_key(first_verification_key)
# add another verification key (should expect exception)
# Add another verification key (should expect exception.)
second_verification_key_path = os.path.join(keystore_path,'snapshot_key.pub')
second_verification_key = \
developer_tool.import_rsa_publickey_from_file(second_verification_key_path)
@ -248,72 +248,73 @@ def test_add_verification_keys(self):
# add a verification key for the delegation
project.delegate("somedelegation", [], [])
project("somedelegation").add_verification_key(first_verification_key)
project("somedelegation").add_verification_key(second_verification_key)
# Add a verification key for the delegation.
project.delegate('somedelegation', [], [])
project('somedelegation').add_verification_key(first_verification_key)
project('somedelegation').add_verification_key(second_verification_key)
# add another delegation of the delegation
project("somedelegation").delegate("somesubdelegation", [], [])
project("somedelegation")("somesubdelegation").add_verification_key(
# Add another delegation of the delegation.
project('somedelegation').delegate('somesubdelegation', [], [])
project('somedelegation')('somesubdelegation').add_verification_key(
first_verification_key)
project("somedelegation")("somesubdelegation").add_verification_key(
project('somedelegation')('somesubdelegation').add_verification_key(
second_verification_key)
def test_write(self):
# create tmp directory
# Create tmp directory.
local_tmp = tempfile.mkdtemp(dir=self.tmp_dir)
# create new project inside tmp directory
project = developer_tool.create_new_project("test_write", local_tmp,
"prefix");
# Create new project inside tmp directory.
project = developer_tool.create_new_project('test_write', local_tmp,
'prefix');
# create some target files inside the tmp directory
target_filepath = os.path.join(local_tmp, "targets", "test_target")
with open(target_filepath, "wt") as fp:
fp.write("testing file")
# Create some target files inside the tmp directory.
target_filepath = os.path.join(local_tmp, 'targets', 'test_target')
with open(target_filepath, 'wt') as fp:
fp.write('testing file')
# add the targets
# Add the targets.
project.add_target(target_filepath)
# add verification keys
keystore_path = os.path.join('repository_data','keystore')
project_key_path = os.path.join(keystore_path,'root_key.pub')
# Add verification keys.
keystore_path = os.path.join('repository_data', 'keystore')
project_key_path = os.path.join(keystore_path, 'root_key.pub')
project_key = \
developer_tool.import_rsa_publickey_from_file(project_key_path)
# call status (for the sake of doing it)
# Call status (for the sake of doing it and to improve test coverage by
# executing its statements.)
project.status()
project.add_verification_key(project_key)
# add another verification key (should expect exception)
delegation_key_path = os.path.join(keystore_path,'snapshot_key.pub')
# Add another verification key (should expect exception.)
delegation_key_path = os.path.join(keystore_path, 'snapshot_key.pub')
delegation_key = \
developer_tool.import_rsa_publickey_from_file(delegation_key_path)
# add a subdelegation
subdelegation_key_path = os.path.join(keystore_path,'timestamp_key.pub')
# Add a subdelegation.
subdelegation_key_path = os.path.join(keystore_path, 'timestamp_key.pub')
subdelegation_key = \
developer_tool.import_rsa_publickey_from_file(subdelegation_key_path)
# add a delegation
project.delegate("delegation", [delegation_key], [])
project("delegation").delegate("subdelegation", [subdelegation_key], [])
# Add a delegation.
project.delegate('delegation', [delegation_key], [])
project('delegation').delegate('subdelegation', [subdelegation_key], [])
# call write (except)
self.assertRaises(tuf.Error, project.write, ())
# call status (for the sake of doing it)
# Call status (for the sake of doing it and executing its statements.)
project.status()
# load private keys
# Load private keys.
project_private_key_path = os.path.join(keystore_path, 'root_key')
project_private_key = \
developer_tool.import_rsa_privatekey_from_file(project_private_key_path,
@ -330,21 +331,21 @@ def test_write(self):
developer_tool.import_rsa_privatekey_from_file(subdelegation_private_key_path,
'password')
# Test partial write
# Test partial write.
# backup everything (again)
# + backup targets:
# + backup targets.
targets_backup = project.target_files
# + backup delegations
# + backup delegations.
delegations_backup = \
tuf.roledb.get_delegated_rolenames(project._project_name)
# + backup layout type
# + backup layout type.
layout_type_backup = project.layout_type
# + backup keyids
# + backup keyids.
keys_backup = project.keys
delegation_keys_backup = project("delegation").keys
delegation_keys_backup = project('delegation').keys
# + backup the prefix.
prefix_backup = project._prefix
@ -352,7 +353,7 @@ def test_write(self):
# + backup the name.
name_backup = project._project_name
# Set the compressions, we will be checking this part here too
# Set the compressions. We will be checking this part here too.
project.compressions = ['gz']
project('delegation').compressions = project.compressions
@ -363,12 +364,12 @@ def test_write(self):
project = developer_tool.load_project(local_tmp)
# Check against backup.
self.assertEqual(project.target_files, list(targets_backup.keys()))
self.assertEqual(list(project.target_files.keys()), list(targets_backup.keys()))
new_delegations = tuf.roledb.get_delegated_rolenames(project._project_name)
self.assertEqual(new_delegations, delegations_backup)
self.assertEqual(project.layout_type, layout_type_backup)
self.assertEqual(project.keys, keys_backup)
self.assertEqual(project("delegation").keys, delegation_keys_backup)
self.assertEqual(project('delegation').keys, delegation_keys_backup)
self.assertEqual(project._prefix, prefix_backup)
self.assertEqual(project._project_name, name_backup)
@ -381,11 +382,11 @@ def test_write(self):
# Load_signing_keys.
project("delegation").load_signing_key(delegation_private_key)
project('delegation').load_signing_key(delegation_private_key)
project.status()
project("delegation")("subdelegation").load_signing_key(
project('delegation')('subdelegation').load_signing_key(
subdelegation_private_key)
project.status()
@ -405,7 +406,7 @@ def test_write(self):
# + backup keyids
keys_backup = project.keys
delegation_keys_backup = project("delegation").keys
delegation_keys_backup = project('delegation').keys
# + backup the prefix.
prefix_backup = project._prefix
@ -424,19 +425,17 @@ def test_write(self):
# Check against backup.
self.assertEqual(project.target_files, targets_backup)
self.assertEqual(list(project.target_files.keys()), list(targets_backup.keys()))
new_delegations = tuf.roledb.get_delegated_rolenames(project._project_name)
self.assertEqual(new_delegations, delegations_backup)
self.assertEqual(project.layout_type, layout_type_backup)
self.assertEqual(project.keys, keys_backup)
self.assertEqual(project("delegation").keys, delegation_keys_backup)
self.assertEqual(project('delegation').keys, delegation_keys_backup)
self.assertEqual(project._prefix, prefix_backup)
self.assertEqual(project._project_name, name_backup)
if __name__ == '__main__':
unittest.main()

68
tuf/README-developer-tools.md
View file

@ -1,6 +1,6 @@
# Developing for a TUF repository #
# The Update Framework Developer Tool: How to Update your Project Securely on a TUF Repository
## Table of Contents ##
## Table of Contents
- [Overview](#overview)
- [Creating a Simple Project](#creating_a_simple_project)
- [Generating a Key](#generating_a_key)
@ -13,29 +13,39 @@
<a name="overview">
## Overview
The TUF developer tool is a Python library that enables developers to create
and maintain the required metadata for files hosted in a TUF Repository. The
main concern when generating metadata for a TUF repository is generating
information that matches the future location of the files in the repository. We
use the developer tools to generate valid information so that the project and
its metadata can be applied to the TUF project transparently.
The Update Framework (TUF) is a Python-based security system for software
updates. In order to prevent your users from downloading vulnerable or malicious
code disguised as updates to your software, TUF requires that each update you
release include certain metadata verifying your authorship of the files.
This document has two parts. The first part walks through the creation of a
prototypal TUF project. The second part demonstrates the full capabilities of
the TUF developer tool, which can be used to expand the project from the first
part to meet the developer's needs.
The TUF developer tools are a Python Library that enables you to create and
maintain the required metadata for files hosted on a TUF Repository. (We call
these files “targets,” to distinguish them from the metadata associated with
them. Both of these together comprise a complete “project”.) You will use these
tools to generate the keys and metadata you need to claim and secure your files
on the repository, and to update the metadata and sign it with those keys
whenever you upload a new version of those files.
This document will teach you how to use these tools in two parts. The first
part walks through the creation of a minimal-complexity TUF project, which is
all you need to get started, and can be expanded later. The second part details
the full functionality of the tools, which offer a finer degree of control in
securing your project.
<a name="creating_a_simple_project">
## Creating a Simple project
The following section describes a very basic example usage of the developer
tools with a one-file project.
## Creating a Simple Project
This section walks through the creation of a small example project with just
one target. Once created, this project will be fully functional, and can be
modified as needed.
<a name="generating_a_key">
### Generating a Key
First, we will need to generate a key to sign the metadata. Keys are generated
in pairs: one public and the other private. The private key is
password-protected and is used to sign metadata. The public key can be shared
freely, and is used to verify signatures made by the private key.
freely, and is used to verify signatures made by the private key. You will need
too share your public key with the repository hosting your project so they can
verify your metadata is signed by the right person.
The generate\_and\_write\_rsa\_keypair function will create two key files named
"path/to/key.pub", which is the public key and "path/to/key", which
@ -60,11 +70,11 @@ Now we have a key for our project, we can proceed to create our project.
<a name="the_project_class">
### The Project Class
The TUF developer tool is built around the Project class, which is used to
organize groups of targets associated with a single set of metadata. Each
Project instance keeps track of which target files are associated with a single
set of metadata. Each Project instance keeps track of which target files are
signed and which need signing, which keys are used to sign metadata. It also
keeps track of delegated roles, which are covered later.
organize groups of targets associated with a single set of metadata. A single
Project instance is used to keep track of all the target files and metadata
files in one project. The Project also keeps track of the keys and signatures,
so that it can update all the metadata with the correct changes and signatures
on a single command.
Before creating a project, you must know where it will be located in the TUF
Repository. In the following example, we will create a project to be hosted as
@ -74,7 +84,7 @@ target file, "local/path/to/example\_project/target\_1" locally, and we will
secure it with the key generated above.
First, we must import the generated keys. We can do that by issuing the
following:
following command:
```
>>> public_key = import_rsa_publickey_from_file("path/to/keys.pub")
@ -110,8 +120,8 @@ To add a target, we issue the following method:
>>> project.add_target("target_1")
```
Have in mind the file "target\_1" should be located in
"local/path/to/example\_project" or else the adding procedure will throw an
Note that the file "target\_1" should be located in
"local/path/to/example\_project", or this method will throw an
error.
At this point, the metadata is not valid. We have assigned a key to the
@ -132,8 +142,9 @@ Enter password for the RSA key:
>>> project.write()
```
When all changes to a project have been written, the Project instance can
safely be deleted.
When all changes to the project have been written, the metadata is ready to be
uploaded to the repository, and it is safe to exit the Python interpreter, or
to delete the Project instance.
The project can be loaded later to update changes to the project. The metadata
contains checksums that have to match the actual files or else it won't be
@ -166,6 +177,9 @@ Enter a password for the RSA key:
>>> project.write()
```
If your project does not use any delegations, the five commands above are all
you need to update your project's metadata.
<a name="delegations">
## Delegations
@ -206,7 +220,7 @@ filepaths to a role by adding them to the list in the first parameter, or by
invoking the method again. A role with multiple restricted paths can add
targets to any of them.
Note that this method is invoked the parent role (in this case, the Project)
Note that this method is invoked from the parent role (in this case, the Project)
and takes the delegated role name as an argument.
### Nested Delegations

5
tuf/README.md
View file

@ -250,9 +250,10 @@ $ mkdir django; echo 'file4' > django/file4.txt
# In the example below, file permissions of the target (octal number specifying file
# access for owner, group, others (e.g., 0755) is added alongside the default fileinfo.
# All target objects in metadata include the target's filepath, hash, and length.
>>> octal_file_permissions = oct(os.stat(target2_filepath).st_mode)[4:]
>>> target3_filepath = "path/to/repository/targets/file3.txt"
>>> octal_file_permissions = oct(os.stat(target3_filepath).st_mode)[4:]
>>> custom_file_permissions = {'file_permissions': octal_file_permissions}
>>> repository.targets.add_target("path/to/repository/targets/file3.txt", custom_file_permissions)
>>> repository.targets.add_target(target3_filepath, custom_file_permissions)
# The private key of the updated targets metadata must be loaded before it can be signed and
# written (Note the load_repository() call above).

31
tuf/developer_tool.py
View file

@ -46,6 +46,7 @@
import tuf.log
import tuf.conf
import tuf.repository_tool
import tuf._vendor.six as six
# These imports provide the interface for 'developer_tool.py', since the imports
# are made there.
@ -397,7 +398,7 @@ def status(self):
targets_directory,
metadata_directory,
False)
self._print_status(delegated_role, signable[0])
self._log_status(delegated_role, signable[0])
except tuf.Error:
insufficient_signatures.append(delegated_role)
@ -405,13 +406,13 @@ def status(self):
if len(insufficient_keys):
message = 'Delegated roles with insufficient keys: ' +\
repr(insufficient_keys)
print(message)
logger.info(message)
return
if len(insufficient_signatures):
message = 'Delegated roles with insufficient signatures: ' +\
repr(insufficient_signatures)
print(message)
logger.info(message)
return
# Targets role.
@ -419,7 +420,7 @@ def status(self):
_check_role_keys(self.rolename)
except tuf.InsufficientKeysError as e:
print(str(e))
logger.info(str(e))
return
try:
@ -428,11 +429,11 @@ def status(self):
targets_directory,
metadata_directory,
False)
self._print_status(self._project_name, signable)
self._log_status(self._project_name, signable)
except tuf.Error as e:
signable = e[1]
self._print_status(self._project_name, signable)
self._log_status(self._project_name, signable)
return
finally:
@ -442,7 +443,7 @@ def status(self):
def _print_status(self, rolename, signable):
def _log_status(self, rolename, signable):
"""
Non-public function prints the number of (good/threshold) signatures of
'rolename'.
@ -453,7 +454,7 @@ def _print_status(self, rolename, signable):
message = repr(rolename) + ' role contains ' +\
repr(len(status['good_sigs'])) + ' / ' + repr(status['threshold']) +\
' signatures.'
print(message)
logger.info(message)
@ -486,8 +487,8 @@ def _generate_and_write_metadata(rolename, metadata_filename, write_partial,
roleinfo['delegations'],
False)
# Preprend the prefix to the project's filepath to avoid signature errors
# in upstream.
# Prepend the prefix to the project's filepath to avoid signature errors in
# upstream.
target_filepaths = metadata['targets'].items()
for element in list(metadata['targets']):
junk_path, relative_target = os.path.split(element)
@ -877,7 +878,7 @@ def load_project(project_directory, prefix='', new_targets_location=None):
roleinfo = tuf.roledb.get_roleinfo(project_name)
roleinfo['signatures'].extend(signable['signatures'])
roleinfo['version'] = targets_metadata['version']
roleinfo['paths'] = list(targets_metadata['targets'])
roleinfo['paths'] = targets_metadata['targets']
roleinfo['delegations'] = targets_metadata['delegations']
roleinfo['partial_loaded'] = False
@ -922,8 +923,8 @@ def load_project(project_directory, prefix='', new_targets_location=None):
metadata_name = \
metadata_path[len(metadata_directory):].lstrip(os.path.sep)
# Strip the extension. The roledb does not include '.json' role
# extensions
# Strip the extension. The roledb does not include an appended '.json'
# extensions for each role.
if metadata_name.endswith(METADATA_EXTENSION):
extension_length = len(METADATA_EXTENSION)
metadata_name = metadata_name[:-extension_length]
@ -948,7 +949,9 @@ def load_project(project_directory, prefix='', new_targets_location=None):
roleinfo['signatures'].extend(signable['signatures'])
roleinfo['version'] = metadata_object['version']
roleinfo['expires'] = metadata_object['expires']
roleinfo['paths'] = list(metadata_object['targets'])
roleinfo['paths'] = {}
for filepath, fileinfo in six.iteritems(metadata_object['targets']):
roleinfo['paths'].update({filepath: fileinfo.get('custom', {})})
roleinfo['delegations'] = metadata_object['delegations']
roleinfo['partial_loaded'] = False

1
tuf/repository_lib.py
View file

@ -1,3 +1,4 @@
#!/usr/bin/env python
"""
<Program Name>