Thanks go to @lukpueh for this helpful tip.
After merge, stdout should only appear if a test has failed. This
functionality is provided by `unittest.TextTestRunner` argument `buffer=True`.
This functions like the `--buffer` command line argument listed here:
https://docs.python.org/3/library/unittest.html#command-line-options
std out is discarded if a test succeeds.
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
This adds to PR #799; please see that for details. In short,
tests sometimes fail on slow test systems (primarily on AppVeyor)
if we don't have long enough delays.
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
on AppVeyor during automatic testing. Also explains a bit better
the reason for those delays, in the comments adjacent to them.
AppVeyor was occasionally laggy enough that spawning a separate
server process didn't happen fast enough for the included delays,
so connection attempts in the tests occasionally failed.
This lengthens a few 0.3s delays that I've seen pop up in test
failures to 2s delays, along with a few others for good measure.
Sadly, this slows testing a bit.... I'll keep an eye out for more
of these.
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
After seeing some AppVeyor failures, I've increased the wait after
starting test HTTP, HTTPS, and proxy servers from 0.5s to 1s, to make
it less likely that tests will fail because the servers weren't done
starting up yet.
After some review comments by @aaaaalbert, I've tightened the logic
in aggregate_tests.py around which tests to skip unless a certain
Python version is running, and added some consistency checks.
This also involved a bit of clarification of comments and variable
names.
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
to make sure that the test uses the intended certificate. (There's some
indirect indication that the updated environment variable might not always
have been used.)
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
that draws from sys.executable (the currently running Python interpreter)
instead of assuming 'python' is correct. Use this function instead of having
many individual subprocess calls written out. Slightly simplifies code, too.
This should eventually be moved to a common test module instead of appearing
in two places in the test code.
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
- two reversions to unnecessary changes
- some typo fixes
- capitalization of HTTP/S where reasonable
- commenting out code section with ''' rather than #
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
in test_download.py. In the process, added another test cert
and generalized the server process killer in test_download.py.
Additionally, I added another expected-to-be-good certificate
that was generated in the same way as the new bad certificates
(but for their individual flaws of course). This is because
the new certs aren't exactly like the old good cert, so that
we have another cert to test against in case the way the
certs were generated turns out to matter at some point in the
future.
Also slightly increased a start-servers delay in the test
in response to one test system taking too long and seeing
connection issues. Probably not helped by the number of
processes. Clarified a related comment in the test code.
Also made a note that environment variable cleanup would be
good to add to test_download.py, either copied from or moved
somewhere accessible from test_proxy_use.py
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
- if it is provided, don't require the certificate filename to be
provided as an absolute path
- raise an error if the provided certificate filename does not
point to an existing file, rather than just printing and
ignoring (to avoid possible future diagnostic headaches)
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
Added Python version checking and changed behavior
in Python2.7.9+ to use custom certificate for target server
inherited from command line argument.
In Python versions < 2.7.9, proxy_server.py does not perform certificate
validation of the target server. As that is not part of what the current
tests using this script require, that is currently OK. In Python
versions > 2.7.9 (SSLContext was added in 2.7.9), the same code actually does
check the certificate, using the system's trusted CAs. As a result, since we
are using custom certificates, we need to either disable certificate
checking in 2.7.9 or load the specific CA for target test server, using the
SSLContext and create_default_context functionality also added in 2.7.9. It
is easier to do the latter, so the behavior in 2.7.9+ is to check the cert
and below 2.7.9 is not to. Note that we do not support Python < 2.7.
SSLContext is also available in all Python3 versions that we support.
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
as that set of tests is now redundant, and depended on Twisted,
which we need not depend on.
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
add more explanatory comments in test_proxy.use,
and prepare for rename of proxy2.py to proxy_server.py in next
commit (separate so that the rename can be seen as such).
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
- move some configuration values up to the module level (proxy certs
dir, proxy ca key and cert, general certs dir).
- add explanatory comments for these values
- create the proxy's host-specific client certificates directory
if it does not yet exist.
- note that the module is not thread-safe
- fix a Windows-incompatible line (explicit path separator)
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
- client makes HTTPS connection to proxy; proxy makes HTTP
connection to target server
- client makes HTTPS connection to proxy; proxy makes HTTPS
connection to target server
Added functionality to the proxy2.py script to allow it to
take and use a certificate to use to validate the target
server.
Also added clarifying comments in test_proxy_use.py.
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
Fixes a typo in arg processing for test script proxy2.py.
Also removes an outdated comment and clarifies another.
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
for proxy testing. Also update the test scripts to point to
the new location of ssl certificates and ssl keys.
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
in proxy2. And use it to run both relaying and intercepting proxies.
True: normal HTTP proxy. Support HTTP & HTTPS connections to target server
False: intercepting MITM transparent HTTPS proxy. Makes own TLS connections
and has its own cert; must be trusted by the client and is able to
modify requests.
Also perform some cleanup of test_proxy_use.py
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
- adds inaz2/proxy2.py, copied code.
- adds dev dependency on twisted for a simple proxy test
- adds a new test module, test_proxy_use, and runs those tests only in
Python2.7 (as proxy2 only supports Python2.7) using new logic in
aggregate_tests.py.
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
Rewrite test_https_connection to do a more thorough test, including
the use of an unknown certificate and the use of a good certificate
which lists a hostname not matching that expected in the connection.
In the process, made some small changes to the simple_https_server
module used in tests (takes an extra argument: certificate file to
use). Given the extent of the changes to test_https_connection, I
also made some style adjustments to better match our code style
guidelines.
I also reduced the length of a delay after the https servers
started from 1s to 0.2s, as part of a general campaign to speed up
the TUF tests. 200ms should do to start the servers, and if not,
I'll adjust it upward.
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
to extract hostname. After commit
"use a different session per hostname",
the code no longer raises MissingSchema if a URL is malformed in
certain cases. Since it parses URLs to extract the hostname and
would have raised securesystemslib.exceptions.FormatError, so the
test would have to check for that error instead of requests's
MissingSchema.
However, it's best to use a different error type, since while that
would be, true enough, a formatting error, FormatError is customarily
reserved for the automatic detection based on schemas in formats.py
(using <SCHEMA>.check_match()), and in any case it is not a
securesystemslib error.
So this commit adds error type tuf.exceptions.URLParsingError and
raises it if the hostname cannot be isolated in a URL, and checks
for it in test_download.py.
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
and also clarify setup workflow in test_slow_retrieval_attack.
Because the test was written incorrectly, the test was failing
with a bad hash error after 31 seconds, when it should instead
have been failing because the slow retrieval attack was not
averted.
In particular, a target file was updated, but metadata was not
correctly updated on the repository and, further, the client's
copy of the metadata was never updated. (The client continued
to expect 31 bytes of target file instead of 400 or 800.)
The way the test used to run, the target file change
previously had no purpose.
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
Also halves the duration of a failed test (from 800s to 400s).
Otherwise, this is a code-style-only change.
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
The test for repository_tool.get_filepaths_in_directory now expects
absolute paths, and also now tests the *results* of the function's use when
the recursive flag is on.
Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
