Commit graph

96 commits

Author SHA1 Message Date
Sebastien Awwad
264186fa51
Session index: hostname-indexed --> scheme+hostname-indexed
In order to avoid re-using session data from an HTTPS connection
in an HTTP connection. Some cookies may not be HTTP-safe.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-09-10 16:56:02 -04:00
Sebastien Awwad
4595ab839a
Remove unused os import in tuf.download
It is not longer used, and pylint complains if it's left in.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-09-10 16:30:32 -04:00
Sebastien Awwad
a5416d4baa
Raise helpful error in download.py if cannot parse URL
to extract hostname. After commit
"use a different session per hostname",
the code no longer raises MissingSchema if a URL is malformed in
certain cases. Since it parses URLs to extract the hostname and
would have raised securesystemslib.exceptions.FormatError, so the
test would have to check for that error instead of requests's
MissingSchema.

However, it's best to use a different error type, since while that
would be, true enough, a formatting error, FormatError is customarily
reserved for the automatic detection based on schemas in formats.py
(using <SCHEMA>.check_match()), and in any case it is not a
securesystemslib error.

So this commit adds error type tuf.exceptions.URLParsingError and
raises it if the hostname cannot be isolated in a URL, and checks
for it in test_download.py.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-09-10 16:30:32 -04:00
Sebastien Awwad
d199610f94
DOC: Clarify and correct comments on download timeouts
and call out need for more testing in a comment.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-09-10 16:30:32 -04:00
Sebastien Awwad
8951e8b9a8
Style fixes in download.py within PR 781 changes
- Remove trailing whitespace
- Fix indent sizes (4 if continuing line, else 2)
- Fix line continuation to match PEP 8 and lab code guidelines

Also fixes one minor typo.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-09-10 16:30:32 -04:00
Sebastien Awwad
cc4628735a
Raise ReadTimeoutError from requests as TUF SlowRetrievalError
so as to preserve 'API' of expected errors.

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-09-10 16:30:32 -04:00
Trishank K Kuppusamy
f29622b2c6
add debug statements
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-09-10 16:30:27 -04:00
Trishank K Kuppusamy
c25ce7c3be
use a different session per hostname
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-09-10 16:30:06 -04:00
Trishank K Kuppusamy
34e0ec7c62
Add TUF version number, and user agent
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-09-10 16:29:48 -04:00
Sebastien Awwad
1e97275b9a
minor: undo some import/dependency re-ordering
(sorry -- just to keep things simple)

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-08-31 13:28:43 -04:00
Trishank K Kuppusamy
b9bc8602c2
Update TUF to handle HTTPS proxies
Signed-off-by: Trishank K Kuppusamy <trishank.kuppusamy@datadoghq.com>
2018-08-31 13:18:35 -04:00
Sebastien Awwad
d98152bc9d
Remove some unnecessary try-reraise constructions; thanks, pylint!
These changes simplify logic, removing some try/except structures
that were unnecessary and potentially confusing, and get us back
to passing pylint's test.

pylint 2.0.0 adds try-except-raise tests, to catch immediate
re-raising after catching an exception, outside of some special
cases. See this GitHub comment for more info:
https://github.com/theupdateframework/tuf/pull/770#issuecomment-412993992

Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
2018-08-21 14:59:29 -04:00
Vladimir Diaz
0713648697
Explain via comment how to deal with function call that might raise socket exceptions
Signed-off-by: Vladimir Diaz <vladimir.v.diaz@gmail.com>
2018-05-30 11:18:17 -04:00
Vladimir Diaz
ff120795fb
Replace %xx escapes in urls
Signed-off-by: Vladimir Diaz <vladimir.v.diaz@gmail.com>
2018-05-25 10:54:53 -04:00
Vladimir Diaz
17dba0d830
Implement settings.SLEEP_BEFORE_ROUND in download.py
Signed-off-by: Vladimir Diaz <vladimir.v.diaz@gmail.com>
2018-03-27 14:42:16 -04:00
Vladimir Diaz
8f65fa4154
Rename license files in modules
Signed-off-by: Vladimir Diaz <vladimir.v.diaz@gmail.com>
2018-02-05 11:31:19 -05:00
Vladimir Diaz
3894bc4e31
Add copyright and license info to module files
Signed-off-by: Vladimir Diaz <vladimir.v.diaz@gmail.com>
2017-11-30 13:23:38 -05:00
Vladimir Diaz
d74f890954
Resolve remaining Pylint warnings for download.py
* Rename variable that was treated as an attribute (should only be defined in __init__()).

Signed-off-by: Vladimir Diaz <vladimir.v.diaz@gmail.com>
2017-11-27 14:36:58 -05:00
Vladimir Diaz
082b8bda74
Remove unused CertificateError import
Signed-off-by: Vladimir Diaz <vladimir.v.diaz@gmail.com>
2017-11-20 14:10:24 -05:00
Vladimir Diaz
9a6b4b95cc
Fix typos: 'tuf.exception' and 'PROTOCOL_SSLv23'
Signed-off-by: Vladimir Diaz <vladimir.v.diaz@gmail.com>
2017-11-17 13:15:07 -05:00
Vladimir Diaz
0380a3ea8b
Fix issues identified by Bandit
Signed-off-by: Vladimir Diaz <vladimir.v.diaz@gmail.com>
2017-11-16 17:37:40 -05:00
Vladimir Diaz
bb8729487d
Resolve Pylint's identified [E]rrors in the codebase
Signed-off-by: Vladimir Diaz <vladimir.v.diaz@gmail.com>
2017-11-15 16:34:34 -05:00
Vladimir Diaz
72d7ebfd8d
Remove py26 conditional import statements 2017-09-21 17:16:29 -04:00
Vladimir Diaz
97eccf5fb8
Remove chech for > py2.5 in download.py
implementation doesn't support < py2.6
2017-07-14 15:28:02 -04:00
Vladimir Diaz
301965ff2e Fix import statements for download.py and test_download.py 2017-01-11 10:46:22 -05:00
Vladimir Diaz
eb56b150ac Replace import statements in download.py 2017-01-10 10:43:03 -05:00
Vladimir Diaz
cbca00baea Address issue #409.
- _download_file() no longer verifies hashes, only the required length of the file.  Update the docstring to say this.
- clean up extraneous/trailing whitespace
2016-11-30 11:18:01 -05:00
Vladimir Diaz
bea840f4ab Ensure the expected formats.py is imported in the remaining modules 2016-11-09 15:35:36 -05:00
Artiom Baloian
b0156944bb Moved util.py, now it is common 2016-11-02 09:49:37 -04:00
Artiom Baloian
b793c81739 Added simple_settings, now it is a requirement of TuF.
Rename conf.py into the settings.py an changed import call.
2016-11-01 15:38:40 -04:00
Artiom Baloian
b21ad871b9 Now ed25519_keys.py and hash.py modules are common for
TuF and in-toto
2016-11-01 10:55:23 -04:00
Artiom Baloian
b07d9a5d0f Move some part of tufformats into the common formats 2016-10-31 19:04:14 -04:00
Artiom Baloian
8247a02737 Moved TuF exceptions and schem into the ssl_commons 2016-10-31 11:58:05 -04:00
Artiom Baloian
7098cdbc3a Move _vendor dir and split formats into the formats and tufformats 2016-10-28 18:37:28 -04:00
Vladimir Diaz
07b4b24343 Finish edits to repository tool doc and update code to fix issues with slow retrieval errors 2016-10-21 11:36:48 -04:00
Vladimir Diaz
b03deee874 Modify the way slow retrieval errors are raised and detected.
Potential slow retrieval errors are raised for safe downloads (strict check on file size) and not for unsafe (a file size less than an upper limit).  Slow retrieval errors should always be prevented
2016-10-10 17:34:00 -04:00
Vladimir Diaz
d72f733862 Do not raise tuf.SlowRetrievalError in download.py
(at least until the tests are updated).
2016-10-07 17:24:54 -04:00
Vladimir Diaz
0a84a4e7d6 Throw an explicit exception if streamed data occurs at a slow enough rate 2016-10-07 11:34:20 -04:00
Vladimir Diaz
6ec2f8a7a4 Merge branch 'develop' of github.com:theupdateframework/tuf into remaining-1.0-tasks
Conflicts:
	tuf/download.py
2016-07-27 16:56:05 -04:00
Vladimir Diaz
6936432790 Update download.py 2016-07-05 17:20:57 -04:00
Vladimir Diaz
b52876ac9f Sleep for a short amount of time in download.py's while loop to release CPU 2016-06-28 14:12:10 -04:00
Vladimir Diaz
b435618878 Remove extra whitespace around argument 2016-01-27 16:34:13 -05:00
Vladimir Diaz
1c8f41b0c0 Update import statements of unvendored dependencies 2015-06-02 08:29:22 -04:00
Vladimir Diaz
16685d2ff3 Add configuration option for supported URI schemes
Support only 'http' and 'https' by default.  Thanks Marcin W.
2015-05-04 15:34:22 -04:00
Vladimir Diaz
003847e73f Improve code coverage for download.py 2015-04-02 16:37:47 -04:00
Vladimir Diaz
b7cf7e6588 Improve each module's code coverage to at least 90%
Update unit tests for download.py, repository_lib.py, repository_tool.py, and updater.py.
Re-generate repository data so that compressed metadata may be tested.
2015-02-11 13:22:46 -05:00
Vladimir Diaz
49a3a4990b Fix comment / typo in unsafe_download() test condition 2014-06-18 09:06:49 -04:00
Vladimir Diaz
6dafddc10f Add test case for https connection. 2014-06-17 10:28:56 -04:00
Vladimir Diaz
e4c98d38ba Add missing test cases for download.py and and updater.py. 2014-06-12 09:33:12 -04:00
vladdd
15592b2e15 Add missing docstrings. 2014-06-09 12:34:58 -04:00