Hello everyone,
one of the discussion we had during connect was to find (and use) a common testing framework for unit (and maybe beyond) tests. What we should use is probably a framework that still supports all the unittest based tests we already have in our projects.
Following also other people suggestions, I looked around and did some initial tests. What follows is a list of what we could use:
- pytest: http://pytest.org/latest/ - nose: https://nose.readthedocs.org/en/latest/ - stick with Python provided unittest: no need to install anything else
Personally I do not dislike unittest even if of the three is the most verbose, but pytest is a powerful handy tool and easier to use (you don't even need to inherit from TestCase).
I didn't play with pytest and Django tests, but looks like it is possible to easily integrate it:
http://pytest-django.readthedocs.org/en/latest/
Tools:
These kind of tools are more targeted at mocking/patching objects or behavior, should we avoid them or use them? (no flame wars please! :-) Personally I find that sometimes (probably too often) I need them. If we need them we should suggest which one to use, and stick with it:
- mock: http://www.voidspace.org.uk/python/mock/ - mocker: https://pypi.python.org/pypi/mocker
There are more, but these two are the ones I know about or have been using. Other suggestions?
Ciao!
-- Milo Casagrande | Automation Engineer Linaro.org <www.linaro.org> │ Open source software for ARM SoCs
Hi
Mocker and mock are totally different. Mock is in stdlib since 3.3 so it is likely the future but I found mocker easier to use and understand (I'm also the current maintainer for mocker, if inactive a bit).
As for nose/py.test: both are a bit non standard. I would strongly recommend that you use unittest2, the python 2.x backport of updated stdlib test stuff. It has a lot of compatible extensions. Python.test and nose have those too but 1) they are not compatible 2) IMHO there is no advantage over stock stuff _anymore_
Lastly Django is a bit of a different story but it gets healthier lately. Strongly recommend to track what key upstream devs are doing there and why. It will likely be in next django release and if you choose badly you'll drift apart from other projects.
Quick question: do you have a list of requirements?
Thanks ZK El 16/07/2013 11:54, "Milo Casagrande" milo.casagrande@linaro.org va escriure:
Hello everyone,
one of the discussion we had during connect was to find (and use) a common testing framework for unit (and maybe beyond) tests. What we should use is probably a framework that still supports all the unittest based tests we already have in our projects.
Following also other people suggestions, I looked around and did some initial tests. What follows is a list of what we could use:
- pytest: http://pytest.org/latest/
- nose: https://nose.readthedocs.org/en/latest/
- stick with Python provided unittest: no need to install anything else
Personally I do not dislike unittest even if of the three is the most verbose, but pytest is a powerful handy tool and easier to use (you don't even need to inherit from TestCase).
I didn't play with pytest and Django tests, but looks like it is possible to easily integrate it:
http://pytest-django.readthedocs.org/en/latest/
Tools:
These kind of tools are more targeted at mocking/patching objects or behavior, should we avoid them or use them? (no flame wars please! :-) Personally I find that sometimes (probably too often) I need them. If we need them we should suggest which one to use, and stick with it:
There are more, but these two are the ones I know about or have been using. Other suggestions?
Ciao!
-- Milo Casagrande | Automation Engineer Linaro.org <www.linaro.org> │ Open source software for ARM SoCs
linaro-validation mailing list linaro-validation@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-validation
On Tue, Jul 16, 2013 at 12:52 PM, Zygmunt Bazyli Krynicki zkrynicki@gmail.com wrote:
Mocker and mock are totally different. Mock is in stdlib since 3.3 so it is likely the future but I found mocker easier to use and understand (I'm also the current maintainer for mocker, if inactive a bit).
As for nose/py.test: both are a bit non standard. I would strongly recommend that you use unittest2, the python 2.x backport of updated stdlib test stuff. It has a lot of compatible extensions. Python.test and nose have those too but 1) they are not compatible 2) IMHO there is no advantage over stock stuff _anymore_
Lastly Django is a bit of a different story but it gets healthier lately. Strongly recommend to track what key upstream devs are doing there and why. It will likely be in next django release and if you choose badly you'll drift apart from other projects.
Quick question: do you have a list of requirements?
No, nothing special I would say or that pops out of my mind at this moment.
I would prefer to use what is available in Python by "default" - unittest and mock. I never considered unittest2 though, that is a good catch.
Ciao.
-- Milo Casagrande | Automation Engineer Linaro.org <www.linaro.org> │ Open source software for ARM SoCs
Hello,
On Tue, 16 Jul 2013 12:52:59 +0200 Zygmunt Bazyli Krynicki zkrynicki@gmail.com wrote:
Hi
Mocker and mock are totally different. Mock is in stdlib since 3.3 so it is likely the future but I found mocker easier to use and understand (I'm also the current maintainer for mocker, if inactive a bit).
As for nose/py.test: both are a bit non standard.
Please define "standard". Let's read http://docs.python.org/2/library/unittest.html : "The Python unit testing framework, sometimes referred to as “PyUnit,” is a Python language version of JUnit". So, what we read is that Python somehow includes Java's standard test facility, but what's standard for Java is not natural for Python (or we could just stick with Java in the first place). "Standard" is what follows language philosophy, and nosetests is truly adheres to that - easy and intuitive start, learn extra and more powerful features as you progress and need them.
I would strongly recommend that you use unittest2, the python 2.x backport of updated stdlib test stuff. It has a lot of compatible extensions.
Looking at https://pypi.python.org/pypi/unittest2 : "The new features in unittest for Python 2.7 backported to Python 2.3+." So, it appears it's confusion: there's no unittest2 really (well, maybe there's in Python2.4 world, but where's that?), it's just unittest of Python2.7.
Python.test and nose have those too but 1) they are not compatible 2) IMHO there is no advantage over stock stuff _anymore_
Looking at http://www.voidspace.org.uk/python/articles/unittest2.shtml , it starts with advertizing alphabet soup like:
assertIDontKnownHowItsWrittenLessOrEqualOrEqualOrLess()
So, how there's "no advantage over stock stuff _anymore_", can I just create a dir "tests", put "test_it.py" file there, write in it:
=== def test(): assert foo <= 5 ===
Run single command w/o params and get tests run?
Hello,
On Tue, 16 Jul 2013 11:53:17 +0200 Milo Casagrande milo.casagrande@linaro.org wrote:
Hello everyone,
one of the discussion we had during connect was to find (and use) a common testing framework for unit (and maybe beyond) tests. What we should use is probably a framework that still supports all the unittest based tests we already have in our projects.
Following also other people suggestions, I looked around and did some initial tests. What follows is a list of what we could use:
+10
Nose is truly Pythonic tool, intended to do testing pleasant and unobtrusive (while all the usual feature supported). There's no obscure APIs to learn, everything just intuitive and natural:
def test_obvious(): assert 2 + 2 == 4
def test_exc(): try: 1/0 assert False, "Math doomed" exception: pass
To run tests, just run "nosetests" - it will collected all the test from intuitive, convention-over-configuration places.
*Lot* of (truly Pythonic) projects use nosetest.
- pytest: http://pytest.org/latest/
Didn't use that, http://pytest.org/latest/getting-started.html#getstarted shows ability to write test with asserts, that's good, but then, there's nosetest already ;-).
- stick with Python provided unittest: no need to install anything
else
But very verbose, alien to Python, RSI-provoking syntax. Also, no decent test runner anyway, so something needs to be installed. nosetest can handle unittest of course.
Personally I do not dislike unittest even if of the three is the most verbose, but pytest is a powerful handy tool and easier to use (you don't even need to inherit from TestCase).
I didn't play with pytest and Django tests, but looks like it is possible to easily integrate it:
http://pytest-django.readthedocs.org/en/latest/
Tools:
These kind of tools are more targeted at mocking/patching objects or behavior, should we avoid them or use them? (no flame wars please! :-) Personally I find that sometimes (probably too often) I need them.
Sure, you can't do complete testing without them (mock objects), and without patching, can't do that comfortable (inversion of control/dependency injection is not the natural Python patterns after all).
If we need them we should suggest which one to use, and stick with it:
Classy tool, has its corner cases, but that's because it's rather powerful. Just as with nosetest, truly Python - start is easy, learn more as you need.
There are more, but these two are the ones I know about or have been using. Other suggestions?
Ciao!
linaro-validation@lists.linaro.org
-
Milo Casagrande -
Paul Sokolovsky -
Zygmunt Bazyli Krynicki