670 lines
18 KiB
Plaintext
670 lines
18 KiB
Plaintext
Metadata-Version: 2.1
|
|
Name: parameterized
|
|
Version: 0.9.0
|
|
Summary: Parameterized testing with any Python test framework
|
|
Author-email: David Wolever <david@wolever.net>
|
|
License: FreeBSD
|
|
Project-URL: Homepage, https://github.com/wolever/parameterized
|
|
Classifier: Programming Language :: Python :: 3
|
|
Classifier: License :: OSI Approved :: BSD License
|
|
Requires-Python: >=3.7
|
|
Description-Content-Type: text/x-rst
|
|
License-File: LICENSE.txt
|
|
Provides-Extra: dev
|
|
Requires-Dist: jinja2 ; extra == 'dev'
|
|
|
|
Parameterized testing with any Python test framework
|
|
====================================================
|
|
|
|
.. image:: https://img.shields.io/pypi/v/parameterized
|
|
:alt: PyPI
|
|
:target: https://pypi.org/project/parameterized/
|
|
|
|
.. image:: https://img.shields.io/pypi/dm/parameterized
|
|
:alt: PyPI - Downloads
|
|
:target: https://pypi.org/project/parameterized/
|
|
|
|
.. image:: https://circleci.com/gh/wolever/parameterized.svg?style=svg
|
|
:alt: Circle CI
|
|
:target: https://circleci.com/gh/wolever/parameterized
|
|
|
|
|
|
Parameterized testing in Python sucks.
|
|
|
|
``parameterized`` fixes that. For everything. Parameterized testing for nose,
|
|
parameterized testing for py.test, parameterized testing for unittest.
|
|
|
|
.. code:: python
|
|
|
|
# test_math.py
|
|
from nose.tools import assert_equal
|
|
from parameterized import parameterized, parameterized_class
|
|
|
|
import unittest
|
|
import math
|
|
|
|
@parameterized([
|
|
(2, 2, 4),
|
|
(2, 3, 8),
|
|
(1, 9, 1),
|
|
(0, 9, 0),
|
|
])
|
|
def test_pow(base, exponent, expected):
|
|
assert_equal(math.pow(base, exponent), expected)
|
|
|
|
class TestMathUnitTest(unittest.TestCase):
|
|
@parameterized.expand([
|
|
("negative", -1.5, -2.0),
|
|
("integer", 1, 1.0),
|
|
("large fraction", 1.6, 1),
|
|
])
|
|
def test_floor(self, name, input, expected):
|
|
assert_equal(math.floor(input), expected)
|
|
|
|
@parameterized_class(('a', 'b', 'expected_sum', 'expected_product'), [
|
|
(1, 2, 3, 2),
|
|
(5, 5, 10, 25),
|
|
])
|
|
class TestMathClass(unittest.TestCase):
|
|
def test_add(self):
|
|
assert_equal(self.a + self.b, self.expected_sum)
|
|
|
|
def test_multiply(self):
|
|
assert_equal(self.a * self.b, self.expected_product)
|
|
|
|
@parameterized_class([
|
|
{ "a": 3, "expected": 2 },
|
|
{ "b": 5, "expected": -4 },
|
|
])
|
|
class TestMathClassDict(unittest.TestCase):
|
|
a = 1
|
|
b = 1
|
|
|
|
def test_subtract(self):
|
|
assert_equal(self.a - self.b, self.expected)
|
|
|
|
|
|
With nose (and nose2)::
|
|
|
|
$ nosetests -v test_math.py
|
|
test_floor_0_negative (test_math.TestMathUnitTest) ... ok
|
|
test_floor_1_integer (test_math.TestMathUnitTest) ... ok
|
|
test_floor_2_large_fraction (test_math.TestMathUnitTest) ... ok
|
|
test_math.test_pow(2, 2, 4, {}) ... ok
|
|
test_math.test_pow(2, 3, 8, {}) ... ok
|
|
test_math.test_pow(1, 9, 1, {}) ... ok
|
|
test_math.test_pow(0, 9, 0, {}) ... ok
|
|
test_add (test_math.TestMathClass_0) ... ok
|
|
test_multiply (test_math.TestMathClass_0) ... ok
|
|
test_add (test_math.TestMathClass_1) ... ok
|
|
test_multiply (test_math.TestMathClass_1) ... ok
|
|
test_subtract (test_math.TestMathClassDict_0) ... ok
|
|
|
|
----------------------------------------------------------------------
|
|
Ran 12 tests in 0.015s
|
|
|
|
OK
|
|
|
|
As the package name suggests, nose is best supported and will be used for all
|
|
further examples.
|
|
|
|
|
|
With py.test (version 2.0 and above)::
|
|
|
|
$ py.test -v test_math.py
|
|
============================= test session starts ==============================
|
|
platform darwin -- Python 3.6.1, pytest-3.1.3, py-1.4.34, pluggy-0.4.0
|
|
collecting ... collected 13 items
|
|
|
|
test_math.py::test_pow::[0] PASSED
|
|
test_math.py::test_pow::[1] PASSED
|
|
test_math.py::test_pow::[2] PASSED
|
|
test_math.py::test_pow::[3] PASSED
|
|
test_math.py::TestMathUnitTest::test_floor_0_negative PASSED
|
|
test_math.py::TestMathUnitTest::test_floor_1_integer PASSED
|
|
test_math.py::TestMathUnitTest::test_floor_2_large_fraction PASSED
|
|
test_math.py::TestMathClass_0::test_add PASSED
|
|
test_math.py::TestMathClass_0::test_multiply PASSED
|
|
test_math.py::TestMathClass_1::test_add PASSED
|
|
test_math.py::TestMathClass_1::test_multiply PASSED
|
|
test_math.py::TestMathClassDict_0::test_subtract PASSED
|
|
==================== 12 passed, 4 warnings in 0.16 seconds =====================
|
|
|
|
With unittest (and unittest2)::
|
|
|
|
$ python -m unittest -v test_math
|
|
test_floor_0_negative (test_math.TestMathUnitTest) ... ok
|
|
test_floor_1_integer (test_math.TestMathUnitTest) ... ok
|
|
test_floor_2_large_fraction (test_math.TestMathUnitTest) ... ok
|
|
test_add (test_math.TestMathClass_0) ... ok
|
|
test_multiply (test_math.TestMathClass_0) ... ok
|
|
test_add (test_math.TestMathClass_1) ... ok
|
|
test_multiply (test_math.TestMathClass_1) ... ok
|
|
test_subtract (test_math.TestMathClassDict_0) ... ok
|
|
|
|
----------------------------------------------------------------------
|
|
Ran 8 tests in 0.001s
|
|
|
|
OK
|
|
|
|
(note: because unittest does not support test decorators, only tests created
|
|
with ``@parameterized.expand`` will be executed)
|
|
|
|
With green::
|
|
|
|
$ green test_math.py -vvv
|
|
test_math
|
|
TestMathClass_1
|
|
. test_method_a
|
|
. test_method_b
|
|
TestMathClass_2
|
|
. test_method_a
|
|
. test_method_b
|
|
TestMathClass_3
|
|
. test_method_a
|
|
. test_method_b
|
|
TestMathUnitTest
|
|
. test_floor_0_negative
|
|
. test_floor_1_integer
|
|
. test_floor_2_large_fraction
|
|
TestMathClass_0
|
|
. test_add
|
|
. test_multiply
|
|
TestMathClass_1
|
|
. test_add
|
|
. test_multiply
|
|
TestMathClassDict_0
|
|
. test_subtract
|
|
|
|
Ran 12 tests in 0.121s
|
|
|
|
OK (passes=9)
|
|
|
|
|
|
Installation
|
|
------------
|
|
|
|
::
|
|
|
|
$ pip install parameterized
|
|
|
|
|
|
Compatibility
|
|
-------------
|
|
|
|
`Yes`__ (mostly).
|
|
|
|
__ https://app.circleci.com/pipelines/github/wolever/parameterized?branch=master
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
:stub-columns: 1
|
|
|
|
* -
|
|
- Py3.7
|
|
- Py3.8
|
|
- Py3.9
|
|
- Py3.10
|
|
- Py3.11
|
|
- PyPy3
|
|
- ``@mock.patch``
|
|
* - nose
|
|
- yes
|
|
- yes
|
|
- yes
|
|
- yes
|
|
- no§
|
|
- no§
|
|
- yes
|
|
* - nose2
|
|
- yes
|
|
- yes
|
|
- yes
|
|
- yes
|
|
- yes
|
|
- yes
|
|
- yes
|
|
* - py.test 2
|
|
- no*
|
|
- no*
|
|
- no*
|
|
- no*
|
|
- no*
|
|
- no*
|
|
- no*
|
|
* - py.test 3
|
|
- yes
|
|
- yes
|
|
- yes
|
|
- yes
|
|
- no*
|
|
- no*
|
|
- yes
|
|
* - py.test 4
|
|
- no**
|
|
- no**
|
|
- no**
|
|
- no**
|
|
- no**
|
|
- no**
|
|
- no**
|
|
* - py.test fixtures
|
|
- no†
|
|
- no†
|
|
- no†
|
|
- no†
|
|
- no†
|
|
- no†
|
|
- no†
|
|
* - | unittest
|
|
| (``@parameterized.expand``)
|
|
- yes
|
|
- yes
|
|
- yes
|
|
- yes
|
|
- yes
|
|
- yes
|
|
- yes
|
|
* - | unittest2
|
|
| (``@parameterized.expand``)
|
|
- yes
|
|
- yes
|
|
- yes
|
|
- yes
|
|
- no§
|
|
- no§
|
|
- yes
|
|
|
|
§: nose and unittest2 - both of which were last updated in 2015 - sadly do not
|
|
appear to support Python 3.10 or 3.11.
|
|
|
|
\*: `py.test 2 does not appear to work under Python 3 (#71)`__, and
|
|
`py.test 3 does not appear to work under Python 3.10 or 3.11 (#154)`__.
|
|
|
|
\*\*: py.test 4 is not yet supported (but coming!) in `issue #34`__
|
|
|
|
†: py.test fixture support is documented in `issue #81`__
|
|
|
|
|
|
__ https://github.com/wolever/parameterized/issues/71
|
|
__ https://github.com/wolever/parameterized/issues/154
|
|
__ https://github.com/wolever/parameterized/issues/34
|
|
__ https://github.com/wolever/parameterized/issues/81
|
|
|
|
Dependencies
|
|
------------
|
|
|
|
(this section left intentionally blank)
|
|
|
|
|
|
Exhaustive Usage Examples
|
|
--------------------------
|
|
|
|
The ``@parameterized`` and ``@parameterized.expand`` decorators accept a list
|
|
or iterable of tuples or ``param(...)``, or a callable which returns a list or
|
|
iterable:
|
|
|
|
.. code:: python
|
|
|
|
from parameterized import parameterized, param
|
|
|
|
# A list of tuples
|
|
@parameterized([
|
|
(2, 3, 5),
|
|
(3, 5, 8),
|
|
])
|
|
def test_add(a, b, expected):
|
|
assert_equal(a + b, expected)
|
|
|
|
# A list of params
|
|
@parameterized([
|
|
param("10", 10),
|
|
param("10", 16, base=16),
|
|
])
|
|
def test_int(str_val, expected, base=10):
|
|
assert_equal(int(str_val, base=base), expected)
|
|
|
|
# An iterable of params
|
|
@parameterized(
|
|
param.explicit(*json.loads(line))
|
|
for line in open("testcases.jsons")
|
|
)
|
|
def test_from_json_file(...):
|
|
...
|
|
|
|
# A callable which returns a list of tuples
|
|
def load_test_cases():
|
|
return [
|
|
("test1", ),
|
|
("test2", ),
|
|
]
|
|
@parameterized(load_test_cases)
|
|
def test_from_function(name):
|
|
...
|
|
|
|
.. **
|
|
|
|
Note that, when using an iterator or a generator, all the items will be loaded
|
|
into memory before the start of the test run (we do this explicitly to ensure
|
|
that generators are exhausted exactly once in multi-process or multi-threaded
|
|
testing environments).
|
|
|
|
The ``@parameterized`` decorator can be used test class methods, and standalone
|
|
functions:
|
|
|
|
.. code:: python
|
|
|
|
from parameterized import parameterized
|
|
|
|
class AddTest(object):
|
|
@parameterized([
|
|
(2, 3, 5),
|
|
])
|
|
def test_add(self, a, b, expected):
|
|
assert_equal(a + b, expected)
|
|
|
|
@parameterized([
|
|
(2, 3, 5),
|
|
])
|
|
def test_add(a, b, expected):
|
|
assert_equal(a + b, expected)
|
|
|
|
|
|
And ``@parameterized.expand`` can be used to generate test methods in
|
|
situations where test generators cannot be used (for example, when the test
|
|
class is a subclass of ``unittest.TestCase``):
|
|
|
|
.. code:: python
|
|
|
|
import unittest
|
|
from parameterized import parameterized
|
|
|
|
class AddTestCase(unittest.TestCase):
|
|
@parameterized.expand([
|
|
("2 and 3", 2, 3, 5),
|
|
("3 and 5", 3, 5, 8),
|
|
])
|
|
def test_add(self, _, a, b, expected):
|
|
assert_equal(a + b, expected)
|
|
|
|
Will create the test cases::
|
|
|
|
$ nosetests example.py
|
|
test_add_0_2_and_3 (example.AddTestCase) ... ok
|
|
test_add_1_3_and_5 (example.AddTestCase) ... ok
|
|
|
|
----------------------------------------------------------------------
|
|
Ran 2 tests in 0.001s
|
|
|
|
OK
|
|
|
|
Note that ``@parameterized.expand`` works by creating new methods on the test
|
|
class. If the first parameter is a string, that string will be added to the end
|
|
of the method name. For example, the test case above will generate the methods
|
|
``test_add_0_2_and_3`` and ``test_add_1_3_and_5``.
|
|
|
|
The names of the test cases generated by ``@parameterized.expand`` can be
|
|
customized using the ``name_func`` keyword argument. The value should
|
|
be a function which accepts three arguments: ``testcase_func``, ``param_num``,
|
|
and ``params``, and it should return the name of the test case.
|
|
``testcase_func`` will be the function to be tested, ``param_num`` will be the
|
|
index of the test case parameters in the list of parameters, and ``param``
|
|
(an instance of ``param``) will be the parameters which will be used.
|
|
|
|
.. code:: python
|
|
|
|
import unittest
|
|
from parameterized import parameterized
|
|
|
|
def custom_name_func(testcase_func, param_num, param):
|
|
return "%s_%s" %(
|
|
testcase_func.__name__,
|
|
parameterized.to_safe_name("_".join(str(x) for x in param.args)),
|
|
)
|
|
|
|
class AddTestCase(unittest.TestCase):
|
|
@parameterized.expand([
|
|
(2, 3, 5),
|
|
(2, 3, 5),
|
|
], name_func=custom_name_func)
|
|
def test_add(self, a, b, expected):
|
|
assert_equal(a + b, expected)
|
|
|
|
Will create the test cases::
|
|
|
|
$ nosetests example.py
|
|
test_add_1_2_3 (example.AddTestCase) ... ok
|
|
test_add_2_3_5 (example.AddTestCase) ... ok
|
|
|
|
----------------------------------------------------------------------
|
|
Ran 2 tests in 0.001s
|
|
|
|
OK
|
|
|
|
|
|
The ``param(...)`` helper class stores the parameters for one specific test
|
|
case. It can be used to pass keyword arguments to test cases:
|
|
|
|
.. code:: python
|
|
|
|
from parameterized import parameterized, param
|
|
|
|
@parameterized([
|
|
param("10", 10),
|
|
param("10", 16, base=16),
|
|
])
|
|
def test_int(str_val, expected, base=10):
|
|
assert_equal(int(str_val, base=base), expected)
|
|
|
|
|
|
If test cases have a docstring, the parameters for that test case will be
|
|
appended to the first line of the docstring. This behavior can be controlled
|
|
with the ``doc_func`` argument:
|
|
|
|
.. code:: python
|
|
|
|
from parameterized import parameterized
|
|
|
|
@parameterized([
|
|
(1, 2, 3),
|
|
(4, 5, 9),
|
|
])
|
|
def test_add(a, b, expected):
|
|
""" Test addition. """
|
|
assert_equal(a + b, expected)
|
|
|
|
def my_doc_func(func, num, param):
|
|
return "%s: %s with %s" %(num, func.__name__, param)
|
|
|
|
@parameterized([
|
|
(5, 4, 1),
|
|
(9, 6, 3),
|
|
], doc_func=my_doc_func)
|
|
def test_subtraction(a, b, expected):
|
|
assert_equal(a - b, expected)
|
|
|
|
::
|
|
|
|
$ nosetests example.py
|
|
Test addition. [with a=1, b=2, expected=3] ... ok
|
|
Test addition. [with a=4, b=5, expected=9] ... ok
|
|
0: test_subtraction with param(*(5, 4, 1)) ... ok
|
|
1: test_subtraction with param(*(9, 6, 3)) ... ok
|
|
|
|
----------------------------------------------------------------------
|
|
Ran 4 tests in 0.001s
|
|
|
|
OK
|
|
|
|
Finally ``@parameterized_class`` parameterizes an entire class, using
|
|
either a list of attributes, or a list of dicts that will be applied to the
|
|
class:
|
|
|
|
.. code:: python
|
|
|
|
from yourapp.models import User
|
|
from parameterized import parameterized_class
|
|
|
|
@parameterized_class([
|
|
{ "username": "user_1", "access_level": 1 },
|
|
{ "username": "user_2", "access_level": 2, "expected_status_code": 404 },
|
|
])
|
|
class TestUserAccessLevel(TestCase):
|
|
expected_status_code = 200
|
|
|
|
def setUp(self):
|
|
self.client.force_login(User.objects.get(username=self.username)[0])
|
|
|
|
def test_url_a(self):
|
|
response = self.client.get('/url')
|
|
self.assertEqual(response.status_code, self.expected_status_code)
|
|
|
|
def tearDown(self):
|
|
self.client.logout()
|
|
|
|
|
|
@parameterized_class(("username", "access_level", "expected_status_code"), [
|
|
("user_1", 1, 200),
|
|
("user_2", 2, 404)
|
|
])
|
|
class TestUserAccessLevel(TestCase):
|
|
def setUp(self):
|
|
self.client.force_login(User.objects.get(username=self.username)[0])
|
|
|
|
def test_url_a(self):
|
|
response = self.client.get("/url")
|
|
self.assertEqual(response.status_code, self.expected_status_code)
|
|
|
|
def tearDown(self):
|
|
self.client.logout()
|
|
|
|
|
|
The ``@parameterized_class`` decorator accepts a ``class_name_func`` argument,
|
|
which controls the name of the parameterized classes generated by
|
|
``@parameterized_class``:
|
|
|
|
.. code:: python
|
|
|
|
from parameterized import parameterized, parameterized_class
|
|
|
|
def get_class_name(cls, num, params_dict):
|
|
# By default the generated class named includes either the "name"
|
|
# parameter (if present), or the first string value. This example shows
|
|
# multiple parameters being included in the generated class name:
|
|
return "%s_%s_%s%s" %(
|
|
cls.__name__,
|
|
num,
|
|
parameterized.to_safe_name(params_dict['a']),
|
|
parameterized.to_safe_name(params_dict['b']),
|
|
)
|
|
|
|
@parameterized_class([
|
|
{ "a": "hello", "b": " world!", "expected": "hello world!" },
|
|
{ "a": "say ", "b": " cheese :)", "expected": "say cheese :)" },
|
|
], class_name_func=get_class_name)
|
|
class TestConcatenation(TestCase):
|
|
def test_concat(self):
|
|
self.assertEqual(self.a + self.b, self.expected)
|
|
|
|
::
|
|
|
|
$ nosetests -v test_math.py
|
|
test_concat (test_concat.TestConcatenation_0_hello_world_) ... ok
|
|
test_concat (test_concat.TestConcatenation_0_say_cheese__) ... ok
|
|
|
|
|
|
|
|
Using with Single Parameters
|
|
............................
|
|
|
|
If a test function only accepts one parameter and the value is not iterable,
|
|
then it is possible to supply a list of values without wrapping each one in a
|
|
tuple:
|
|
|
|
.. code:: python
|
|
|
|
@parameterized([1, 2, 3])
|
|
def test_greater_than_zero(value):
|
|
assert value > 0
|
|
|
|
Note, however, that if the single parameter *is* iterable (such as a list or
|
|
tuple), then it *must* be wrapped in a tuple, list, or the ``param(...)``
|
|
helper:
|
|
|
|
.. code:: python
|
|
|
|
@parameterized([
|
|
([1, 2, 3], ),
|
|
([3, 3], ),
|
|
([6], ),
|
|
])
|
|
def test_sums_to_6(numbers):
|
|
assert sum(numbers) == 6
|
|
|
|
(note, also, that Python requires single element tuples to be defined with a
|
|
trailing comma: ``(foo, )``)
|
|
|
|
|
|
Using with ``@mock.patch``
|
|
..........................
|
|
|
|
``parameterized`` can be used with ``mock.patch``, but the argument ordering
|
|
can be confusing. The ``@mock.patch(...)`` decorator must come *below* the
|
|
``@parameterized(...)``, and the mocked parameters must come *last*:
|
|
|
|
.. code:: python
|
|
|
|
@mock.patch("os.getpid")
|
|
class TestOS(object):
|
|
@parameterized(...)
|
|
@mock.patch("os.fdopen")
|
|
@mock.patch("os.umask")
|
|
def test_method(self, param1, param2, ..., mock_umask, mock_fdopen, mock_getpid):
|
|
...
|
|
|
|
Note: the same holds true when using ``@parameterized.expand``.
|
|
|
|
|
|
Migrating from ``nose-parameterized`` to ``parameterized``
|
|
----------------------------------------------------------
|
|
|
|
To migrate a codebase from ``nose-parameterized`` to ``parameterized``:
|
|
|
|
1. Update your requirements file, replacing ``nose-parameterized`` with
|
|
``parameterized``.
|
|
|
|
2. Replace all references to ``nose_parameterized`` with ``parameterized``::
|
|
|
|
$ perl -pi -e 's/nose_parameterized/parameterized/g' your-codebase/
|
|
|
|
3. You're done!
|
|
|
|
|
|
FAQ
|
|
---
|
|
|
|
What happened to Python 2.X, 3.5, and 3.6 support?
|
|
As of version 0.9.0, ``parameterized`` no longer supports Python 2.X, 3.5,
|
|
or 3.6. Previous versions of ``parameterized`` - 0.8.1 being the latest -
|
|
will continue to work, but will not receive any new features or bug fixes.
|
|
|
|
What do you mean when you say "nose is best supported"?
|
|
There are small caveates with ``py.test`` and ``unittest``: ``py.test``
|
|
does not show the parameter values (ex, it will show ``test_add[0]``
|
|
instead of ``test_add[1, 2, 3]``), and ``unittest``/``unittest2`` do not
|
|
support test generators so ``@parameterized.expand`` must be used.
|
|
|
|
Why not use ``@pytest.mark.parametrize``?
|
|
Because spelling is difficult. Also, ``parameterized`` doesn't require you
|
|
to repeat argument names, and (using ``param``) it supports optional
|
|
keyword arguments.
|
|
|
|
Why do I get an ``AttributeError: 'function' object has no attribute 'expand'`` with ``@parameterized.expand``?
|
|
You've likely installed the ``parametrized`` (note the missing *e*)
|
|
package. Use ``parameterized`` (with the *e*) instead and you'll be all
|
|
set.
|
|
|
|
What happened to ``nose-parameterized``?
|
|
Originally only nose was supported. But now everything is supported, and it
|
|
only made sense to change the name!
|