Improve test setup

[blueyed]

The test setup is quite bad: do you plan to improve this, e.g. by using tox as a first class citizen, and not just wrapping setup.py test there?

[beniwohli]

Hi @blueyed. We mostly rely on Travis to run our test matrix. There are some improvements that we plan (e.g. separating unit tests from integration tests more clearly, and rewrite the nose tests into proper pytest tests). Do you have any other suggestions?

[beniwohli]

Oh yeah, there is also the issue of isort being finicky as hell. I plan to move isort into its own check. Or remove it completely, it's debatable if the benefit is worth the constant pain it generates...

[blueyed]

I see.
I think it would be nice to make more use of tox's features, instead of the custom script for Travis, so that you could run the tests easily using tox -e.
There is e.g. https://github.com/ryanhiebert/tox-travis which might be useful for integration on Travis.

isort

Yes, just having a single qa job (e.g. via make check_qa) would also help with finishing the test builds faster.

Regarding the test build duration on Travis, it might also make sense to trim the matrix down a bit?!