This commit includes manual fixes for a lot of mypy warnings.
When there were warnings that we are calling non-annotated function
in annotated context I decided to add annotations instead of ignoring
those warnings.
That's how I end up adding annotations in the whole tests/utils.py
module.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Generalize the decorator used in test_metadata_serialization.py and
move it inside tests/utils.py, so it can be reused in other similar
situations.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Use a common test level constant for defining
the host address forming the download URL on
the client side.
Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
This quote from the Google Python style guide made me realize
why empty list as a default value for an argument could be
dangerous:
"Default arguments are evaluated once at module load time.
This may cause problems if the argument is a mutable object
such as a list or a dictionary. If the function modifies the object
(e.g., by appending an item to a list), the default value is modified."
Read more here:
https://google.github.io/styleguide/pyguide.html#2123-cons
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
The current implementation for server startup in TestServerProcess
relies on the fact that "bind successful..." is the first message
sent by the server process.
Make sure that this is true and leave a comment about this.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
These changes can be summarized with the following bullets:
- Delegate generation of ports used for the tests to the OS
- Use thread-safe Queue for processes communication
instead of temporary files
- Remove all instances of port generation or hardcoded ports
- Make test_slow_retrieval.py fully conform with TestServerProcess
Delegate generation of ports used for the tests to the OS is much
better than if we manually generate them, because there is always
a chance that the port we have randomly pick turns out to be taken.
By giving 0 to the port argument we ask the OS to give us
an arbitrary unused port.
Use thread-safe Queue for processes communication instead of temporary
files became a necessity because of findings made by Jussi Kukkonen.
With the latest changes made in pr 1192 we were rapidly reading
from the temporary files and Jussi found that it happened rarely
the successful message "bind succeded..." to be corrupted.
It seems, this is a thread issue related to the thread redirecting
the subprocess stdout to the temp file and our thread rapidly
reading from the file.
By using a thread-safe Queue we eliminate this possibility.
For reference read:
https://github.com/theupdateframework/tuf/issues/1196
Lastly, test_slow_retrieval.py and slow_retrieval.py were refactored.
Until now, slow_retrieval.py couldn't use the TestServerProcess class
from utils.py for a port generation because of a bug related to
httpd.handle_request().
Now, when we use httpd.serve_forever() we can refactor both of those
files and fully conform with TestServerProcess.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Filter out:
* DeprecationWarnings for updater module when we are on purpose
testing deprecated methods from updater
* SubjectAltNameWarning for connections to our test server
These warnings are visible with e.g.
python3 test_updater.py
The large change in test_download.py is just indentation into with-block.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Logging the stdout and stderr from the test subprocesses into
temporary files clean the console from unnecessary messages from
the server-side such as "code 404, message File not found" or
"GET" queries.
I have decided to create TestServerProcess class that will handle
the server subprocess creation and redirection to a temporary file
object. That way that code can be reused in more than 10 files.
Also, I have cleaned some parts of the unit test to make them more
readable and efficient with the new abstraction.
The unit tests are executed in sequential order and that's why
we can reuse one temporary file object for multiple tests.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
We want the tests own log output visible as well, and they are not
under the "tuf" logger. Set root level to the same value as "tuf".
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
all test_*.py files now accept zero or more '-v' to increase tuf
logging level. The default is now ERROR.
default: ERROR
"-v": ERROR, but unittest prints test names
"-vv": WARNING
"-vvv": INFO
"-vvvv": DEBUG
Example to run a single test with DEBUG level:
python3 test_updater.py -vvvv TestUpdater.test_4_refresh
Also make test_log.py restore the log level it modifies during test.
Fixes#1093
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
Travis managed to still timeout with 5 seconds: increasing the timeout
(now that it's not a sleep) doesn't really hurt normal non-VM use cases
so let's bump it to 10 seconds.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
* Add utility function to wait on a socket until it responds
* Use the function instead of sleeping in tests that need to wait for
the server to start
* Increase the max timeout to 5 seconds by default (as appveyor builds
still seem to hit the 3 second mark sometimes)
wait_for_server() functions quite differently depending on OS: Windows
can take 2 seconds to respond with ECONNREFUSED whereas Linux is almost
instant. There might be tricks to be faster on Windows (like setting
a shorter socket timeout) but this was not done here.
This makes a full Linux test run almost 40% faster and should be more
reproducible on every platform.
Signed-off-by: Jussi Kukkonen <jkukkonen@vmware.com>
