Using pytest to play with data-driven testing framework

pytest What is architecture ？

First , Come and have a look at pytest Example ：

def test_a():

  print(123)

collected 1 item

test_a.py .                                                                                                            [100%]

============ 1 passed in 0.02s =======================

The output is simple ： Collect to 1 A use case , And the test case execution passed .
Now think about two questions ：
1.pytest How to collect the information of use cases ？
2.pytest How to make python Code , convert to pytest The test case （ also called item） ？

pytest How to collect the use cases ？

This is very simple , Traverse the execution directory , If a module in the directory is found to match “ pytest Required by the test case python object ”, Turn it into pytest The test case .
For example, write the following hook function ：

def pytest_collect_file(path, parent):

    print("hello", path)

hello C:\Users\yuruo\Desktop\tmp\tmp123\tmp\testcase\__init__.py

hello C:\Users\yuruo\Desktop\tmp\tmp123\tmp\testcase\conftest.py

hello C:\Users\yuruo\Desktop\tmp\tmp123\tmp\testcase\test_a.py

You will see the contents of all the files .

pytest Like a box , take python The objects are wrapped up , For example, below ：

When you write well python Code ：

def test_a:

    print(123)

Will be wrapped up in Function ：

<Function test_a>

It can be downloaded from hook Function to see the details ：

def pytest_collection_modifyitems(session, config, items):

    pass

therefore , Understanding the package process is the key to solving the puzzle .pytest How to package python Object's ？
The following code has only two lines , It seems simple , But there's a mystery ！

def test_a:

    print(123)

Take a screenshot of the code location , as follows ：

We can say , The above code is in “testcase package ” Under the “test_a.py modular ” Of “test_a function ”, pytest The generated test cases should also have this information ：
be in “testcase package ” Under the “test_a.py modular ” Of “test_a The test case ：
Translate the above expression into the following figure ：
pytest Use parent Attribute represents the upper layer level relationship , such as Module yes Function My superior , Function Of parent Properties are as follows ：

<Function test_a>:

  parent: <Module test_parse.py>

Of course Module Of parent Namely Package:

<Module test_parse.py>:

  parent: <Package tests>

Let's popularize science here ,python Of package and module Are real objects , You can start your obj Property , such as Module Of obj Properties are as follows ：

If you understand pytest The purpose of the package , very nice ！ Let's move on to the next step ： How to construct pytest Of item ？

Take the following code for example ：

def test_a:

    print(123)

structure pytest Of item , need ：
3. structure Package
4. structure Module
5. structure Function
In order to build Function For example , It needs to be called from_parent() Build methods , The process is as follows ：

, You can guess ,“ structure Function” It must be different from parent There is no small connection ！ Again because Function Of parent yes Module ：
According to the following Function Part of the code （ be located python.py file ）：

class Function(PyobjMixin, nodes.Item):

    #  Used to create test cases 

    @classmethod

    def from_parent(cls, parent, **kw):

        """The public constructor."""

        return super().from_parent(parent=parent, **kw)

    #  Get instance 

    def _getobj(self):

        assert self.parent is not None

        return getattr(self.parent.obj, self.originalname)  # type: ignore[attr-defined]

    #  Run test cases 

    def runtest(self) -> None:

        """Execute the underlying test function."""

        self.ihook.pytest_pyfunc_call(pyfuncitem=self)

Come to the conclusion , You can use Module structure Function！ The calling pseudo code is as follows ：

Function.from_parent(Module)

Since you can use Module structure Function, How to build Module ？
Use, of course Package structure Module！

Module.from_parent(Package)

Since you can use Package structure Module How to build Package ？
Don't ask. , It's almost a doll , See the following figure for the calling relationship ：

pytest from Config Start , Layer by layer construction , until Function ！Function yes pytest The smallest execution unit of .
Build... Manually item It's simulation pytest structure Function The process of . in other words , Need to create Config , And then use it Config establish Session , And then use it Session establish Package ,…, Finally create Function.

It's not that complicated , pytest Will automatically create Config, Session and Package , These three do not need to be created manually .

For example, write the following hook Code , Break the point to see its parent Parameters ：

def pytest_collect_file(path, parent):

    pass

If the traversal path is a package （ Can be obtained from path View the specific path in the parameter ）, For example, the package in the figure below ：

Its parent The parameter is Package , You can use this Package establish Module ：

Write the following code to build pytest Of Module , If it turns out to be yaml file , According to yaml Dynamic creation of file content Module and module ：

from _pytest.python import Module, Package

def pytest_collect_file(path, parent):

    if path.ext == ".yaml":

        pytest_module = Module.from_parent(parent, fspath=path)

        #  Return the self-defined  python module

        pytest_module._getobj = lambda : MyModule

        return pytest_module

We need to pay attention to , The above code is rewritten with monkey patch _getobj Method , Why do you do this ？
Module utilize _getobj Method to find and import （import sentence ） path Under bag module , The source code is as follows ：

# _pytest/python.py Module

class Module(nodes.File, PyCollector):

    def _getobj(self):

        return self._importtestmodule()

def _importtestmodule(self):

    # We assume we are only called once per module.

    importmode = self.config.getoption("--import-mode")

    try:

        #  Key code ： Import from path  module

        mod = import_path(self.fspath, mode=importmode) 

    except SyntaxError as e:

        raise self.CollectError(

            ExceptionInfo.from_current().getrepr(style="short")

        ) from e

        #  Omitted code ...

however , If you use data driven , That is, the data file created by the user test_parse.yaml , It is not .py file , It won't be python Identify a module （ Only .py Documents can be recognized as module）.
At this time , You can't let pytest Import （import sentence ） test_parse.yaml , Need dynamic rewriting _getobj , Returns a custom module ！
therefore , Can use lambda The expression returns a custom module ：

lambda : MyModule

This involves metaprogramming Technology ： Build dynamically python Of module , And to module Dynamically add classes or functions to ：

import types

#  Dynamically create  module

module = types.ModuleType(name)

def function_template(*args, **kwargs):

    print(123)

#  towards  module  Add a function to 

setattr(module, "test_abc", function_template)

Sum up , Define yourself as module Put in pytest Of Module You can generate item ：

# conftest.py

import types

from _pytest.python import Module

def pytest_collect_file(path, parent):

    if path.ext == ".yaml":

        pytest_module = Module.from_parent(parent, fspath=path)

        #  Dynamically create  module

        module = types.ModuleType(path.purebasename)

        def function_template(*args, **kwargs):

            print(123)

        #  towards  module  Add a function to 

        setattr(module, "test_abc", function_template)

        pytest_module._getobj = lambda: module

        return pytest_module

Create a yaml file , Use pytest function ：

======= test session starts ====

platform win32 -- Python 3.8.1, pytest-6.2.4, py-1.10.0, pluggy-0.13.1

rootdir: C:\Users\yuruo\Desktop\tmp

plugins: allure-pytest-2.8.11, forked-1.3.0, rerunfailures-9.1.1, timeout-1.4.2, xdist-2.2.1

collected 1 item

test_a.yaml 123

.

======= 1 passed in 0.02s =====

PS C:\Users\yuruo\Desktop\tmp>

Now stop , Take a look back. , What we did ？
To borrow pytest hook , take .yaml File conversion to python module.

As a data-driven testing framework , We didn't do anything ？
No resolution yaml The contents of the document ! The above generated module , The functions in it are as follows ：

def function_template(*args, **kwargs):

    print(123)

Just a simple print 123 . The data-driven test framework needs to be parsed yaml Content , Dynamically generate functions or classes according to the content . For example, below yaml Content ：

test_abc:

  - print: 123

The meaning of the expression is “ Defined function test_abc, This function prints 123”.
You can use yaml.safe_load load yaml Content , And carry out keyword analysis , among path.strpath representative yaml File address ：

import types

import yaml

from _pytest.python import Module

def pytest_collect_file(path, parent):

    if path.ext == ".yaml":

        pytest_module = Module.from_parent(parent, fspath=path)

        #  Dynamically create  module

        module = types.ModuleType(path.purebasename)

        #  analysis  yaml  Content 

        with open(path.strpath) as f:

            yam_content = yaml.safe_load(f)

            for function_name, steps in yam_content.items():



                def function_template(*args, **kwargs):

                    """

                     Function modules 

                    """

                    #  Traverse multiple test steps  [print: 123, print: 456]

                    for step_dic in steps:

                        #  Parse a test step  print: 123

                        for step_key, step_value in step_dic.items():

                            if step_key == "print":

                                print(step_value)



                #  towards  module  Add a function to 

                setattr(module, function_name, function_template)

        pytest_module._getobj = lambda: module

        return pytest_module

The running results of the above test cases are as follows ：

=== test session starts ===

platform win32 -- Python 3.8.1, pytest-6.2.4, py-1.10.0, pluggy-0.13.1

rootdir: C:\Users\yuruo\Desktop\tmp

plugins: allure-pytest-2.8.11, forked-1.3.0, rerunfailures-9.1.1, timeout-1.4.2, xdist-2.2.1

collected 1 item

test_a.yaml 123

.

=== 1 passed in 0.02s ====

Of course , It also supports more complex test cases ：

test_abc:

  - print: 123

  - print: 456

test_abd:

  - print: 123

  - print: 456

The result is as follows ：

== test session starts ==

platform win32 -- Python 3.8.1, pytest-6.2.4, py-1.10.0, pluggy-0.13.1

rootdir: C:\Users\yuruo\Desktop\tmp

plugins: allure-pytest-2.8.11, forked-1.3.0, rerunfailures-9.1.1, timeout-1.4.2, xdist-2.2.1

collected 2 items

test_a.yaml 123

456

.123

456

.

== 2 passed in 0.02s ==

More technical articles