Commit graph

3284 commits

Author SHA1 Message Date
Sebastien Awwad
0589666f47
ASN.1: DOC: Mark some more lines as debug-only. Squash me.
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:52 -05:00
Sebastien Awwad
963cc7316c
ASN.1: DOC: squash me. mark a comment as untrustworthy; add check
Adds a check to old code that confirms that arg is a hex string.
This will likely be deleted code anyway.

Marks a comment that will probably be deleted as untrustworthy.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:51 -05:00
Sebastien Awwad
ac2d8f92f0
ASN.1: WIP! First sketch of asn1crypto use for conversion
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:51 -05:00
Sebastien Awwad
a0719ddddb
ASN.1: minor improvements to conversion docstrings
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:51 -05:00
Sebastien Awwad
20ec4d15da
ASN.1: Use new error class relevant to ASN.1 conversions
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:51 -05:00
Sebastien Awwad
c7687564a9
ASN.1: make expiration data string, not int, to match TUF metadata
While int is preferable for efficiency, the initial goal here is
total compatibility.  The TUF-internal metadata format defined in
the TUF specification uses strings for expiration dates, and so
should the ASN.1 definitions used in the reference implementation
for now.

This also facilitates automated conversion between the formats.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:51 -05:00
Sebastien Awwad
3bdc58dfc8
ASN.1: switch definitions from pyasn1 to asn1crypto
and adjust the documentation.

This commit does not update asn1_convert.py.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:51 -05:00
Sebastien Awwad
d26db7433e
ASN.1: test: refine timestamp metadata conversion testing
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:21 -05:00
Sebastien Awwad
51bdaf936d
ASN.1: test: use conversion_check to perform better tests
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:21 -05:00
Sebastien Awwad
1cf5e3e94b
ASN.1: testing: expand conversion_check to handle generic converters
(to_pyasn1 and from_pyasn1 require additional arguments to specify
the datatype they're to convert to/from. conversion_check did not
previously have an argument to provide those. Now it does and so it
can test those functions.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:21 -05:00
Sebastien Awwad
21bfe06920
ASN.1: squash me: uncomment some temporarily commented test code
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:21 -05:00
Sebastien Awwad
f2fcf4bdc5
ASN.1: draft generic pyasn1->native metadata functions
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:21 -05:00
Sebastien Awwad
2dbb065c2c
ASN.1: WIP: amend/squash: remove erroneous _ to - swap:
The keys the - and _ swap was being attempted in in this spot were
not structural fields in the ASN.1 definitions, so they should not
be tinkered with.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:21 -05:00
Sebastien Awwad
444d3329cb
ASN.1: improve _list_to_pyasn1 loop
setup once before the loop; no need to repeat in every iteration

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:21 -05:00
Sebastien Awwad
6e16662640
ASN.1: expect even number of chars in hex strings
and generate a helpful error if an odd number is received. (Do not
implicitly pad with terminal zero.)

Also slightly improve readability of nearby error lines.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:21 -05:00
Sebastien Awwad
61fbd0901c
ASN.1: WIP: squash/amend me. add more metadata docstrings
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:21 -05:00
Sebastien Awwad
5bf9bf6a20
WIP: DO NOT MERGE: better comment to come later
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:20 -05:00
Sebastien Awwad
0d4bd1deea
ASN.1: partial draft of test for timestamp conversion
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:20 -05:00
Sebastien Awwad
a1d69e165a
ASN.1: ABORTIVE, do not merge: handle num- args in list-likes
See comment in code.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:20 -05:00
Sebastien Awwad
a5a8dd2644
ASN.1: bugfix: add missing return val for _list_to_pyasn1
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:20 -05:00
Sebastien Awwad
19f73b1766
ASN.1: better way to count elements in a struct-like ASN.1 class
Use len() on the NamedTypes object we assume is the componentType for
a struct-like pyasn1 class, instead of iterating over each element.

Also renames element_name_underscores variable to element_name_python
(ditto relevant_element_name_underscores).

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:20 -05:00
Sebastien Awwad
2349ef0e3a
ASN.1: better docstring for _structlike_dict_to_pyasn1
and some better comments in it, too. Also fixes an indentation
oversight (not impactful, just style)

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:20 -05:00
Sebastien Awwad
a7c753053a
ASN.1: Add some more debug statements (DO NOT MERGE)
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:20 -05:00
Sebastien Awwad
1885194aa5
ASN.1: fix Hashes definition, add HashesOfSnapshot def
These will both hopefully not be necessary in the end, since they're
used in only one way each and the definitions should just directly
say things like
   meta: SetOf(componentType=HashOfSnapshot())
instead of having an extra level, but for now this is easier to
debug.

Separately, the Hashes definition was incorrectly Set() instead of
SetOf() Hash

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:20 -05:00
Sebastien Awwad
1fda798d40
ASN.1: deal with empty private values in public key dicts
by removing them. This is potentially problematic as it might change
the keyid when converting to ASN.1 and back, but it'll have to do for
now, and we shouldn't have these things in there in the first place.
It's an edge case, and this is a compromise.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:20 -05:00
Sebastien Awwad
da726fed76
ASN.1: RSA key compat: key vals OctetString -> VisibleString
Switch to using VisibleString to encode key values, since RSA keys
are ASCII-prefixed Base64, while ed25519 key values are hex strings.
This is inefficient, but this reference implementation profits from
being simple. May reconsider later and add specialized code. :/

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:20 -05:00
Sebastien Awwad
427dac3f55
WIP: Revise me. Draft ASN.1 conversion and testing
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:36:19 -05:00
Sebastien Awwad
365aeb9c68
ASN.1: add question about optional field Custom
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:35:30 -05:00
Sebastien Awwad
c4b27ccf01
ASN.1: Use Set instead of Sequence for some unordered lists:
Note that the natural translation of a Python dictionary (which has no implicit
order of elements) into ASN.1 is to a Set (unordered) of objects, and not a
Sequence (ordered). For example, you can think of {'key_a': ..., 'key_b': ...}
as (key_a_obj, key_b_obj), where the order is irrelevant.
Despite this natural interpretation, there are some cases where we impose an
order here in the translation to ASN.1. We do this when the dictionary is
essentially a translation of an object/struct that contains disparate
conceptual types. For example: {'keyid': ..., 'sig': ...} is translated as
[keyid_object, sig_object], not (keyid_object, sig_object). This is to make the
structure of objects in the wire format predictable. RootMetadata below is a
good example: in the ASN.1/DER wire format, root metadata will always begin
with a 'type' element, followed by 'expires', then 'version', etc.

Already-ordered components like lists are, of course, always retained as
ordered lists (Sequence). (e.g. the 'signatures' element of metadata is always
ordered in both ASN.1 and the JSON-compatible metadata format.)

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:35:30 -05:00
Sebastien Awwad
0d26e37300
ASN.1: add necessary import for optional named types
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:35:29 -05:00
Sebastien Awwad
eb728881f3
ASN.1: Remove small, unnecessary custom types per Joey
Definitions like 'Filename' that are just renames of VisibleString
etc. are removed for simplicity. This may be slightly harder to read.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:35:29 -05:00
Sebastien Awwad
2d8070cd75
ASN.1 Correctly reorder root metadata type definitions
so that TopLevelDelegation class declaration appears before the
RootMetadata class definition that uses it.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:35:29 -05:00
Sebastien Awwad
9ba9eea8e5
ASN.1 rename role hash fields to match spec ('meta')
even though 'meta' is not a particularly expressive name, it's what
the spec uses, and looking like the JSON-compatible metadata is
important for the ASN.1 metadata.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:35:29 -05:00
Sebastien Awwad
aadf0add41
ASN.1 minor bugfix: fix name of element in TimestampMetadata
RoleHash() does not exist; it's named HashOfSnapshot currently.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:35:29 -05:00
Sebastien Awwad
f3a39cec17
ASN.1: correct definition of Custom field in Targets
It had an additional layer of structure unnecessarily.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:35:29 -05:00
Sebastien Awwad
79703abae3
ASN.1: Add TAP 5 metadata placeholder (urls in root)
Once TAP 5 is accepted and implemented,	root role metadata will
allow for an optional URL list element for each top-level role.
See TAP 5 for more details.

The placeholders here are commented-out.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:35:29 -05:00
Sebastien Awwad
ce066f26d9
ASN.1: Initial commit of asn1 metadata definitions
both in abstract ASN.1 definition language and a format compatible
with pyasn1.

These will be used for conversion of TUF metadata between the
JSON-compatible internal TUF metadata format and ASN.1/DER.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-08 16:35:29 -05:00
Sebastien Awwad
dad9f9f11b
test: extend sleep in testing after spawning server processes
This adds to PR #799; please see that for details.  In short,
tests sometimes fail on slow test systems (primarily on AppVeyor)
if we don't have long enough delays.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-06 09:59:24 -05:00
Sebastien Awwad
e99ff85b2c
test: extend a delay in testing after spawning server processes
This adds to PR #799; please see that for details.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-05 16:28:59 -05:00
Sebastien Awwad
0380b652bd
Merge pull request #799 from theupdateframework/longer_testing_delays_for_appveyor
test: Delay more when spawning test servers to avoid rare failures
2018-11-05 16:18:53 -05:00
Sebastien Awwad
375ef4bd0d
test: Delay more when spawning test servers to avoid rare failures
on AppVeyor during automatic testing.  Also explains a bit better
the reason for those delays, in the comments adjacent to them.

AppVeyor was occasionally laggy enough that spawning a separate
server process didn't happen fast enough for the included delays,
so connection attempts in the tests occasionally failed.

This lengthens a few 0.3s delays that I've seen pop up in test
failures to 2s delays, along with a few others for good measure.
Sadly, this slows testing a bit.... I'll keep an eye out for more
of these.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-05 16:14:33 -05:00
Justin Cappos
a372410944
Adopters update. 2018-11-05 16:14:30 -05:00
Sebastien Awwad
5fe0a8312f
Merge pull request #798 from theupdateframework/dependencies_update
Dependencies update
2018-11-05 16:11:16 -05:00
Sebastien Awwad
23cfcbe783
Bump version to 0.11.2.dev2 from 0.11.2.dev1
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-05 15:57:05 -05:00
pyup-bot
4c7ee1d5ab
Update urllib3 from 1.23 to 1.24.1
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-05 15:52:20 -05:00
pyup-bot
32f1acfa5b
Update certifi from 2018.8.24 to 2018.10.15
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-05 15:52:17 -05:00
pyup-bot
8e5c2a1567
Update virtualenv from 16.0.0 to 16.1.0
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-05 15:52:13 -05:00
pyup-bot
dde55697a8
Update tox from 3.4.0 to 3.5.3
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-05 15:52:08 -05:00
pyup-bot
fd9b225e2a
Update stevedore from 1.29.0 to 1.30.0
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-05 15:52:04 -05:00
pyup-bot
496d3dccb5
Update smmap2 from 2.0.4 to 2.0.5
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-11-05 15:52:00 -05:00