buy the book ribbon

3: A Brief Interlude: On Coupling and Abstractions3:简短的插曲:关于耦合与抽象

Allow us a brief digression on the subject of abstractions, dear reader. We’ve talked about abstractions quite a lot. The Repository pattern is an abstraction over permanent storage, for example. But what makes a good abstraction? What do we want from abstractions? And how do they relate to testing?亲爱的读者,请允许我们在这里简单地谈谈抽象这个话题。我们已经讨论过很多次抽象了。例如,Repository 模式就是对持久化存储的一种抽象。但什么样的抽象才是好的抽象?我们希望从抽象中得到什么?它们与测试之间又有什么联系?

Tip

The code for this chapter is in the chapter_03_abstractions branch on GitHub:

git clone https://github.com/cosmicpython/code.git
git checkout chapter_03_abstractions

A key theme in this book, hidden among the fancy patterns, is that we can use simple abstractions to hide messy details. When we’re writing code for fun, or in a kata,[1] we get to play with ideas freely, hammering things out and refactoring aggressively. In a large-scale system, though, we become constrained by the decisions made elsewhere in the system.本书的一个核心主题,隐藏在那些花哨的模式之下,就是我们可以使用简单的抽象来隐藏复杂的细节。当我们为了乐趣编写代码或进行代码练习(kata)[1] 时,可以自由地尝试想法、反复推敲并积极重构。然而,在大型系统中,我们往往会受到系统中其他地方所做决策的约束。

When we’re unable to change component A for fear of breaking component B, we say that the components have become coupled. Locally, coupling is a good thing: it’s a sign that our code is working together, each component supporting the others, all of them fitting in place like the gears of a watch. In jargon, we say this works when there is high cohesion between the coupled elements.当我们因为担心破坏组件 B 而无法修改组件 A 时,就称这两个组件发生了耦合。局部来看,耦合是一件好事:它表明我们的代码正在协同工作,每个组件都在支持其他组件,就像钟表的齿轮一样各司其职。用行话来说,当耦合元素之间具有高内聚性时,这种方式是有效的。

Globally, coupling is a nuisance: it increases the risk and the cost of changing our code, sometimes to the point where we feel unable to make any changes at all. This is the problem with the Ball of Mud pattern: as the application grows, if we’re unable to prevent coupling between elements that have no cohesion, that coupling increases superlinearly until we are no longer able to effectively change our systems.从全局来看,耦合则是一种麻烦:它增加了修改代码的风险和成本,有时甚至让我们感到无法进行任何修改。这就是“泥球”(Ball of Mud)模式的问题所在:随着应用程序的增长,如果我们无法防止那些缺乏内聚性的元素之间产生耦合,这种耦合就会以超线性的速度增加,直到我们无法再有效地对系统进行任何改动。

We can reduce the degree of coupling within a system (Lots of coupling) by abstracting away the details (Less coupling).我们可以通过抽象掉细节(降低耦合)来减少系统内部的耦合程度(高耦合)。

apwp 0301
Figure 1. Lots of coupling图 1. 高耦合
[ditaa, apwp_0301]
+--------+      +--------+
| System | ---> | System |
|   A    | ---> |   B    |
|        | ---> |        |
|        | ---> |        |
|        | ---> |        |
+--------+      +--------+
apwp 0302
Figure 2. Less coupling图 2. 低耦合
[ditaa, apwp_0302]
+--------+                           +--------+
| System |      /-------------\      | System |
|   A    | ---> |             | ---> |   B    |
|        | ---> | Abstraction | ---> |        |
|        |      |             | ---> |        |
|        |      \-------------/      |        |
+--------+                           +--------+

In both diagrams, we have a pair of subsystems, with one dependent on the other. In Lots of coupling, there is a high degree of coupling between the two; the number of arrows indicates lots of kinds of dependencies between the two. If we need to change system B, there’s a good chance that the change will ripple through to system A.在这两幅图中,我们都有两个子系统,其中一个依赖于另一个。在“高耦合”图中,两者之间存在高度耦合;箭头的数量表示两者之间存在多种依赖关系。如果我们需要修改系统 B,那么这个变化很有可能会波及到系统 A。

In Less coupling, though, we have reduced the degree of coupling by inserting a new, simpler abstraction. Because it is simpler, system A has fewer kinds of dependencies on the abstraction. The abstraction serves to protect us from change by hiding away the complex details of whatever system B does—we can change the arrows on the right without changing the ones on the left.而在“低耦合”图中,我们通过插入一个新的、更简单的抽象层降低了耦合度。由于抽象层更简单,系统 A 对抽象的依赖类型也更少。抽象层通过隐藏系统 B 内部复杂的实现细节,保护我们免受变化的影响——我们可以修改右侧的箭头,而无需改动左侧的箭头。

Abstracting State Aids Testability抽象状态有助于提高可测试性

Let’s see an example. Imagine we want to write code for synchronizing two file directories, which we’ll call the source and the destination:让我们看一个例子。想象一下,我们要编写代码来同步两个目录,我们称之为源目录和目标目录:

  • If a file exists in the source but not in the destination, copy the file over.如果文件存在于源目录但不存在于目标目录中,则将该文件复制过去。

  • If a file exists in the source, but it has a different name than in the destination, rename the destination file to match.如果文件存在于源目录中,但文件名与目标目录中的不同,则重命名目标文件以匹配。

  • If a file exists in the destination but not in the source, remove it.如果文件存在于目标目录中但不存在于源目录中,则将其删除。

Our first and third requirements are simple enough: we can just compare two lists of paths. Our second is trickier, though. To detect renames, we’ll have to inspect the content of files. For this, we can use a hashing function like MD5 or SHA-1. The code to generate a SHA-1 hash from a file is simple enough:我们的第一和第三个需求很简单:只需比较两个路径列表即可。但第二个需求比较棘手。为了检测重命名,我们必须检查文件的内容。为此,我们可以使用 MD5 或 SHA-1 等哈希函数。从文件生成 SHA-1 哈希的代码很简单:

Hashing a file (sync.py)对文件进行哈希处理 (sync.py)
BLOCKSIZE = 65536


def hash_file(path):
    hasher = hashlib.sha1()
    with path.open("rb") as file:
        buf = file.read(BLOCKSIZE)
        while buf:
            hasher.update(buf)
            buf = file.read(BLOCKSIZE)
    return hasher.hexdigest()

Now we need to write the bit that makes decisions about what to do—the business logic, if you will.现在我们需要编写负责决策的部分——也就是所谓的业务逻辑。

When we have to tackle a problem from first principles, we usually try to write a simple implementation and then refactor toward better design. We’ll use this approach throughout the book, because it’s how we write code in the real world: start with a solution to the smallest part of the problem, and then iteratively make the solution richer and better designed.当我们必须从零开始解决一个问题时,通常会先写一个简单的实现,然后再通过重构来优化设计。在本书中,我们将始终采用这种方法,因为这也是我们在现实世界中编写代码的方式:从解决问题中最微小的部分开始,然后通过迭代使解决方案变得更丰富、设计更完善。

Our first hackish approach looks something like this:我们最初的草率实现大致如下:

Basic sync algorithm (sync.py)基础同步算法 (sync.py)
import hashlib
import os
import shutil
from pathlib import Path


def sync(source, dest):
    # Walk the source folder and build a dict of filenames and their hashes
    source_hashes = {}
    for folder, _, files in os.walk(source):
        for fn in files:
            source_hashes[hash_file(Path(folder) / fn)] = fn

    seen = set()  # Keep track of the files we've found in the target

    # Walk the target folder and get the filenames and hashes
    for folder, _, files in os.walk(dest):
        for fn in files:
            dest_path = Path(folder) / fn
            dest_hash = hash_file(dest_path)
            seen.add(dest_hash)

            # if there's a file in target that's not in source, delete it
            if dest_hash not in source_hashes:
                dest_path.remove()

            # if there's a file in target that has a different path in source,
            # move it to the correct path
            elif dest_hash in source_hashes and fn != source_hashes[dest_hash]:
                shutil.move(dest_path, Path(folder) / source_hashes[dest_hash])

    # for every file that appears in source but not target, copy the file to
    # the target
    for source_hash, fn in source_hashes.items():
        if source_hash not in seen:
            shutil.copy(Path(source) / fn, Path(dest) / fn)

Fantastic! We have some code and it looks OK, but before we run it on our hard drive, maybe we should test it. How do we go about testing this sort of thing?太棒了!我们有了一些代码,看起来还不错,但在将其运行在硬盘上之前,也许我们应该先测试一下。我们该如何测试这类东西呢?

Some end-to-end tests (test_sync.py)一些端到端测试 (test_sync.py)
def test_when_a_file_exists_in_the_source_but_not_the_destination():
    try:
        source = tempfile.mkdtemp()
        dest = tempfile.mkdtemp()

        content = "I am a very useful file"
        (Path(source) / "my-file").write_text(content)

        sync(source, dest)

        expected_path = Path(dest) / "my-file"
        assert expected_path.exists()
        assert expected_path.read_text() == content

    finally:
        shutil.rmtree(source)
        shutil.rmtree(dest)


def test_when_a_file_has_been_renamed_in_the_source():
    try:
        source = tempfile.mkdtemp()
        dest = tempfile.mkdtemp()

        content = "I am a file that was renamed"
        source_path = Path(source) / "source-filename"
        old_dest_path = Path(dest) / "dest-filename"
        expected_dest_path = Path(dest) / "source-filename"
        source_path.write_text(content)
        old_dest_path.write_text(content)

        sync(source, dest)

        assert old_dest_path.exists() is False
        assert expected_dest_path.read_text() == content

    finally:
        shutil.rmtree(source)
        shutil.rmtree(dest)

Wowsers, that’s a lot of setup for two simple cases! The problem is that our domain logic, "figure out the difference between two directories," is tightly coupled to the I/O code. We can’t run our difference algorithm without calling the pathlib, shutil, and hashlib modules.天哪,仅仅为了两个简单的案例就需要这么多的设置工作!问题在于我们的领域逻辑(“找出两个目录之间的差异”)与 I/O 代码紧密耦合在一起了。如果不调用 pathlib、shutil 和 hashlib 模块,我们就无法运行我们的差异算法。

And the trouble is, even with our current requirements, we haven’t written enough tests: the current implementation has several bugs (the shutil.move() is wrong, for example). Getting decent coverage and revealing these bugs means writing more tests, but if they’re all as unwieldy as the preceding ones, that’s going to get real painful real quickly.麻烦的是,即使满足了当前的需求,我们编写的测试仍然不够:当前的实现存在几个 bug(例如,shutil.move() 的用法是错误的)。要获得足够的覆盖率并发现这些 bug,意味着要编写更多的测试,但如果它们都像前面那样笨重,很快就会变得非常痛苦。

On top of that, our code isn’t very extensible. Imagine trying to implement a --dry-run flag that gets our code to just print out what it’s going to do, rather than actually do it. Or what if we wanted to sync to a remote server, or to cloud storage?此外,我们的代码扩展性也不好。试想一下,如果想实现一个 --dry-run 标志,让代码只打印出它将要做什么,而不是真正去执行,该怎么办?或者如果我们想同步到远程服务器或云存储呢?

Our high-level code is coupled to low-level details, and it’s making life hard. As the scenarios we consider get more complex, our tests will get more unwieldy. We can definitely refactor these tests (some of the cleanup could go into pytest fixtures, for example) but as long as we’re doing filesystem operations, they’re going to stay slow and be hard to read and write.我们的高层代码与底层细节耦合在一起,这让生活变得很艰难。随着我们考虑的场景变得越来越复杂,测试也会变得越来越笨重。我们当然可以重构这些测试(例如,将一些清理工作放入 pytest fixture 中),但只要我们还在执行文件系统操作,它们就注定运行缓慢且难以阅读和编写。

Choosing the Right Abstraction(s)选择正确的抽象

What could we do to rewrite our code to make it more testable?我们该如何重写代码以使其更具可测试性呢?

First, we need to think about what our code needs from the filesystem. Reading through the code, we can see that three distinct things are happening. We can think of these as three distinct responsibilities that the code has:首先,我们需要思考代码到底需要从文件系统中获取什么。通读代码,可以看到发生了三件不同的事情。我们可以将它们视为代码的三项不同职责:

  1. We interrogate the filesystem by using os.walk and determine hashes for a series of paths. This is similar in both the source and the destination cases.我们通过使用 os.walk 查询文件系统,并确定一系列路径的哈希值。这在源目录和目标目录的情况下是相似的。

  2. We decide whether a file is new, renamed, or redundant.我们判断文件是新的、重命名的还是冗余的。

  3. We copy, move, or delete files to match the source.我们复制、移动或删除文件以与源目录匹配。

Remember that we want to find simplifying abstractions for each of these responsibilities. That will let us hide the messy details so we can focus on the interesting logic.[2]记住,我们希望为这些职责中的每一项找到简化的抽象。这将使我们能够隐藏复杂的细节,从而专注于有趣的逻辑。[2]

Note
In this chapter, we’re refactoring some gnarly code into a more testable structure by identifying the separate tasks that need to be done and giving each task to a clearly defined actor, along similar lines to the duckduckgo example.

For steps 1 and 2, we’ve already intuitively started using an abstraction, a dictionary of hashes to paths. You may already have been thinking, "Why not build up a dictionary for the destination folder as well as the source, and then we just compare two dicts?" That seems like a nice way to abstract the current state of the filesystem:对于第 1 步和第 2 步,我们已经在直觉上开始使用一种抽象,即哈希到路径的字典。你可能已经在想:“为什么不为目标文件夹和源文件夹都构建一个字典,然后直接比较两个字典呢?”这似乎是抽象文件系统当前状态的一种很好的方式:

source_files = {'hash1': 'path1', 'hash2': 'path2'}
dest_files = {'hash1': 'path1', 'hash2': 'pathX'}

What about moving from step 2 to step 3? How can we abstract out the actual move/copy/delete filesystem interaction?那么从第 2 步到第 3 步该如何转换呢?我们如何抽象出实际的移动/复制/删除文件系统的交互?

We’ll apply a trick here that we’ll employ on a grand scale later in the book. We’re going to separate what we want to do from how to do it. We’re going to make our program output a list of commands that look like this:我们将在这里应用一个技巧,稍后在本书中会大规模使用它。我们将把“想做什么”与“如何做”分离开来。我们将使程序输出一个命令列表,看起来像这样:

("COPY", "sourcepath", "destpath"),
("MOVE", "old", "new"),

Now we could write tests that just use two filesystem dicts as inputs, and we would expect lists of tuples of strings representing actions as outputs.现在,我们可以编写测试,只需使用两个文件系统字典作为输入,并期望得到表示操作的字符串元组列表作为输出。

Instead of saying, "Given this actual filesystem, when I run my function, check what actions have happened," we say, "Given this abstraction of a filesystem, what abstraction of filesystem actions will happen?"我们不再说:“给定这个真实的文件系统,当我运行函数时,检查发生了什么操作”,而是说:“给定这个文件系统的抽象,会发生什么文件系统操作的抽象?”

Simplified inputs and outputs in our tests (test_sync.py)测试中简化的输入和输出 (test_sync.py)

Implementing Our Chosen Abstractions实现我们选择的抽象

That’s all very well, but how do we actually write those new tests, and how do we change our implementation to make it all work?这听起来很不错,但我们实际上该如何编写这些新测试,又该如何修改实现来让一切正常工作呢?

Our goal is to isolate the clever part of our system, and to be able to test it thoroughly without needing to set up a real filesystem. We’ll create a "core" of code that has no dependencies on external state and then see how it responds when we give it input from the outside world (this kind of approach was characterized by Gary Bernhardt as Functional Core, Imperative Shell, or FCIS).我们的目标是隔离系统中聪明的部分,并在不需要设置真实文件系统的情况下对其进行彻底测试。我们将创建一个没有外部状态依赖的“核心”代码,然后观察它在接收来自外部世界的输入时如何响应(这种方法被 Gary Bernhardt 称为“函数式核心,命令式外壳”,即 FCIS)。

Let’s start off by splitting the code to separate the stateful parts from the logic.让我们从拆分代码开始,将有状态的部分与逻辑部分分离开来。

And our top-level function will contain almost no logic at all; it’s just an imperative series of steps: gather inputs, call our logic, apply outputs:我们的顶层函数将几乎不包含任何逻辑;它只是一系列命令式的步骤:收集输入、调用逻辑、应用输出:

Split our code into three (sync.py)将代码拆分为三部分 (sync.py)
def sync(source, dest):
    # imperative shell step 1, gather inputs
    source_hashes = read_paths_and_hashes(source)  #(1)
    dest_hashes = read_paths_and_hashes(dest)  #(1)

    # step 2: call functional core
    actions = determine_actions(source_hashes, dest_hashes, source, dest)  #(2)

    # imperative shell step 3, apply outputs
    for action, *paths in actions:
        if action == "COPY":
            shutil.copyfile(*paths)
        if action == "MOVE":
            shutil.move(*paths)
        if action == "DELETE":
            os.remove(paths[0])
  1. Here’s the first function we factor out, read_paths_and_hashes(), which isolates the I/O part of our application.这是我们提取出来的第一个函数 read_paths_and_hashes(),它隔离了应用程序的 I/O 部分。

  2. Here is where we carve out the functional core, the business logic.这是我们构建函数式核心(业务逻辑)的地方。

The code to build up the dictionary of paths and hashes is now trivially easy to write:构建路径和哈希字典的代码现在写起来非常简单:

A function that just does I/O (sync.py)一个只负责 I/O 的函数 (sync.py)
def read_paths_and_hashes(root):
    hashes = {}
    for folder, _, files in os.walk(root):
        for fn in files:
            hashes[hash_file(Path(folder) / fn)] = fn
    return hashes

The determine_actions() function will be the core of our business logic, which says, "Given these two sets of hashes and filenames, what should we copy/move/delete?". It takes simple data structures and returns simple data structures:determine_actions() 函数将成为我们业务逻辑的核心,它负责回答:“给定这两组哈希值和文件名,我们应该复制/移动/删除什么?”它接收简单的数据结构并返回简单的数据结构:

A function that just does business logic (sync.py)一个只负责业务逻辑的函数 (sync.py)
def determine_actions(source_hashes, dest_hashes, source_folder, dest_folder):
    for sha, filename in source_hashes.items():
        if sha not in dest_hashes:
            sourcepath = Path(source_folder) / filename
            destpath = Path(dest_folder) / filename
            yield "COPY", sourcepath, destpath

        elif dest_hashes[sha] != filename:
            olddestpath = Path(dest_folder) / dest_hashes[sha]
            newdestpath = Path(dest_folder) / filename
            yield "MOVE", olddestpath, newdestpath

    for sha, filename in dest_hashes.items():
        if sha not in source_hashes:
            yield "DELETE", dest_folder / filename

Our tests now act directly on the determine_actions() function:我们的测试现在直接作用于 determine_actions() 函数:

Nicer-looking tests (test_sync.py)看起来更整洁的测试 (test_sync.py)
def test_when_a_file_exists_in_the_source_but_not_the_destination():
    source_hashes = {"hash1": "fn1"}
    dest_hashes = {}
    actions = determine_actions(source_hashes, dest_hashes, Path("/src"), Path("/dst"))
    assert list(actions) == [("COPY", Path("/src/fn1"), Path("/dst/fn1"))]


def test_when_a_file_has_been_renamed_in_the_source():
    source_hashes = {"hash1": "fn1"}
    dest_hashes = {"hash1": "fn2"}
    actions = determine_actions(source_hashes, dest_hashes, Path("/src"), Path("/dst"))
    assert list(actions) == [("MOVE", Path("/dst/fn2"), Path("/dst/fn1"))]

Because we’ve disentangled the logic of our program—​the code for identifying changes—​from the low-level details of I/O, we can easily test the core of our code.因为我们将程序的逻辑(识别变化的代码)与底层的 I/O 细节解耦了,所以可以轻松地测试代码的核心部分。

With this approach, we’ve switched from testing our main entrypoint function, sync(), to testing a lower-level function, determine_actions(). You might decide that’s fine because sync() is now so simple. Or you might decide to keep some integration/acceptance tests to test that sync(). But there’s another option, which is to modify the sync() function so it can be unit tested and end-to-end tested; it’s an approach Bob calls edge-to-edge testing.通过这种方法,我们已经从测试主入口函数 sync() 转变为测试更底层的函数 determine_actions()。你可能认为这没问题,因为 sync() 现在已经非常简单了。或者你也可以选择保留一些集成/验收测试来测试 sync()。但还有另一种选择,那就是修改 sync() 函数,使其既能进行单元测试,又能进行端到端测试;这是一种 Bob 称之为“端到端测试”(edge-to-edge testing)的方法。

Testing Edge to Edge with Fakes and Dependency Injection使用伪对象(Fakes)和依赖注入进行端到端测试

When we start writing a new system, we often focus on the core logic first, driving it with direct unit tests. At some point, though, we want to test bigger chunks of the system together.当我们开始编写一个新系统时,通常会先专注于核心逻辑,通过直接的单元测试来驱动它。然而,在某个阶段,我们希望一起测试系统中更大的部分。

We could return to our end-to-end tests, but those are still as tricky to write and maintain as before. Instead, we often write tests that invoke a whole system together but fake the I/O, sort of edge to edge:我们可以回到端到端测试,但它们仍然像以前一样难以编写和维护。相反,我们通常编写测试来调用整个系统,但伪造 I/O,这有点像端到端测试:

Explicit dependencies (sync.py)显式依赖 (sync.py)
  1. Our top-level function now exposes a new dependency, a FileSystem.我们的顶层函数现在暴露了一个新的依赖项:FileSystem。

  2. We invoke filesystem.read() to produce our files dict.我们调用 filesystem.read() 来生成文件字典。

  3. We invoke the FileSystem's .copy(), .move() and .delete() methods to apply the changes we detect.我们调用 FileSystem 的 .copy()、.move() 和 .delete() 方法来应用我们检测到的变化。

Tip
Although we’re using dependency injection, there is no need to define an abstract base class or any kind of explicit interface. In this book, we often show ABCs because we hope they help you understand what the abstraction is, but they’re not necessary. Python’s dynamic nature means we can always rely on duck typing.

The real (default) implementation of our FileSystem abstraction does real I/O:FileSystem 抽象的真实(默认)实现会执行真正的 I/O 操作:

The real dependency (sync.py)真实的依赖 (sync.py)

But the fake one is a wrapper around our chosen abstractions, rather than doing real I/O:但伪造的实现是我们所选抽象的一个包装器,它不会执行真正的 I/O 操作:

Tests using DI使用依赖注入(DI)的测试
  1. We initialize our fake filesysem using the abstraction we chose to represent filesystem state: dictionaries of hashes to paths.我们使用选择用来表示文件系统状态的抽象(哈希到路径的字典)来初始化伪文件系统。

  2. The action methods in our FakeFileSystem just appends a record to an list of .actions so we can inspect it later. This means our test double is both a "fake" and a "spy". FakeFileSystem 中的操作方法只是将记录追加到 .actions 列表中,以便我们稍后进行检查。这意味着我们的测试替身既是“伪对象”(fake),也是“间谍对象”(spy)。

So now our tests can act on the real, top-level sync() entrypoint, but they do so using the FakeFilesystem(). In terms of their setup and assertions, they end up looking quite similar to the ones we wrote when testing directly against the functional core determine_actions() function:所以现在我们的测试可以直接作用于真实的顶层 sync() 入口点,但它们使用的是 FakeFileSystem()。在设置和断言方面,它们最终看起来与我们直接针对函数式核心 determine_actions() 函数进行测试时编写的测试非常相似:

Tests using DI

The advantage of this approach is that our tests act on the exact same function that’s used by our production code. The disadvantage is that we have to make our stateful components explicit and pass them around. David Heinemeier Hansson, the creator of Ruby on Rails, famously described this as "test-induced design damage."这种方法的优点是我们的测试作用于生产代码所使用的完全相同的函数。缺点是我们必须使有状态组件显式化,并将它们传递下去。Ruby on Rails 的创建者 David Heinemeier Hansson 曾著名地将其描述为“测试导致的架构损伤”(test-induced design damage)。

In either case, we can now work on fixing all the bugs in our implementation; enumerating tests for all the edge cases is now much easier.无论哪种情况,我们现在都可以着手修复实现中的所有 bug;枚举所有边界情况的测试现在容易多了。

Why Not Just Patch It Out?为什么不直接打补丁(Patch)?

At this point you may be scratching your head and thinking, "Why don’t you just use mock.patch and save yourself the effort?"此时你可能会抓耳挠腮,心想:“为什么不直接用 mock.patch 来省去这些麻烦呢?”

We avoid using mocks in this book and in our production code too. We’re not going to enter into a Holy War, but our instinct is that mocking frameworks, particularly monkeypatching, are a code smell.我们在本书和生产代码中都避免使用 mocks。我们不想引发一场“圣战”,但我们的直觉是,mock 框架,尤其是猴子补丁(monkeypatching),是一种代码异味。

Instead, we like to clearly identify the responsibilities in our codebase, and to separate those responsibilities into small, focused objects that are easy to replace with a test double.相反,我们喜欢清楚地识别代码库中的职责,并将这些职责分离成小的、专注的对象,以便于用测试替身进行替换。

Note
You can see an example in [chapter_08_events_and_message_bus], where we mock.patch() out an email-sending module, but eventually we replace that with an explicit bit of dependency injection in [chapter_13_dependency_injection].

We have three closely related reasons for our preference:我们有三个密切相关的理由支持这种偏好:

  • Patching out the dependency you’re using makes it possible to unit test the code, but it does nothing to improve the design. Using mock.patch won’t let your code work with a --dry-run flag, nor will it help you run against an FTP server. For that, you’ll need to introduce abstractions.打补丁虽然可以对代码进行单元测试,但对改进设计毫无帮助。使用 mock.patch 不会让你的代码支持 --dry-run 标志,也不会帮助你在 FTP 服务器上运行。为此,你需要引入抽象。

  • Tests that use mocks tend to be more coupled to the implementation details of the codebase. That’s because mock tests verify the interactions between things: did we call shutil.copy with the right arguments? This coupling between code and test tends to make tests more brittle, in our experience. 使用 mocks 的测试往往与代码库的实现细节耦合更紧密。这是因为 mock 测试验证的是事物之间的交互:我们是否用正确的参数调用了 shutil.copy?根据我们的经验,代码与测试之间的这种耦合往往会使测试变得更加脆弱。

  • Overuse of mocks leads to complicated test suites that fail to explain the code.过度使用 mocks 会导致复杂的测试套件,反而无法解释代码的意图。

Note
Designing for testability really means designing for extensibility. We trade off a little more complexity for a cleaner design that admits novel use cases.
Mocks Versus Fakes; Classic-Style Versus London-School TDDMocks 与 Fakes;经典风格与伦敦学派 TDD

Here’s a short and somewhat simplistic definition of the difference between mocks and fakes:以下是对 mocks 和 fakes 之间区别的一个简短且略显简单的定义:

  • Mocks are used to verify how something gets used; they have methods like assert_called_once_with(). They’re associated with London-school TDD.Mocks 用于验证某物是如何被使用的;它们拥有像 assert_called_once_with() 这样的方法。它们与伦敦学派的 TDD 相关联。

  • Fakes are working implementations of the thing they’re replacing, but they’re designed for use only in tests. They wouldn’t work "in real life"; our in-memory repository is a good example. But you can use them to make assertions about the end state of a system rather than the behaviors along the way, so they’re associated with classic-style TDD.Fakes 是它们所替代事物的有效实现,但它们仅设计用于测试。它们无法在“现实生活中”工作;我们的内存存储库就是一个很好的例子。但你可以利用它们对系统的最终状态进行断言,而不是对过程中的行为进行断言,因此它们与经典风格的 TDD 相关联。

We’re slightly conflating mocks with spies and fakes with stubs here, and you can read the long, correct answer in Martin Fowler’s classic essay on the subject called "Mocks Aren’t Stubs".我们在这里稍微混淆了 mocks 与 spies,以及 fakes 与 stubs 的概念,你可以在 Martin Fowler 关于该主题的经典文章《Mocks Aren’t Stubs》中阅读到详尽且正确的答案。

It also probably doesn’t help that the MagicMock objects provided by unittest.mock aren’t, strictly speaking, mocks; they’re spies, if anything. But they’re also often used as stubs or dummies. There, we promise we’re done with the test double terminology nitpicks now.此外,unittest.mock 提供的 MagicMock 对象严格来说并不是 mocks,如果非要说的话,它们是 spies。但它们也经常被用作 stubs 或 dummies。好了,我们保证关于测试替身术语的吹毛求疵到此为止。

What about London-school versus classic-style TDD? You can read more about those two in Martin Fowler’s article that we just cited, as well as on the Software Engineering Stack Exchange site, but in this book we’re pretty firmly in the classicist camp. We like to build our tests around state both in setup and in assertions, and we like to work at the highest level of abstraction possible rather than doing checks on the behavior of intermediary collaborators.[3]那么伦敦学派与经典风格的 TDD 呢?你可以在我们刚才引用的 Martin Fowler 的文章以及 Software Engineering Stack Exchange 网站上阅读更多关于这两者的信息,但在本书中,我们坚定地站在经典学派这一边。我们喜欢围绕状态来构建测试(包括设置和断言),并且喜欢在尽可能高的抽象级别上工作,而不是去检查中间协作者的行为。[3]

Read more on this in [kinds_of_tests].阅读 [kinds_of_tests] 以了解更多信息。

We view TDD as a design practice first and a testing practice second. The tests act as a record of our design choices and serve to explain the system to us when we return to the code after a long absence.我们将 TDD 视为一种设计实践,其次才是测试实践。测试充当了我们设计选择的记录,并在我们长时间离开代码后重新回归时,起到解释系统的作用。

Tests that use too many mocks get overwhelmed with setup code that hides the story we care about.使用过多 mocks 的测试会被大量的设置代码所淹没,从而掩盖了我们真正关心的逻辑故事。

Steve Freeman has a great example of overmocked tests in his talk "Test-Driven Development". You should also check out this PyCon talk, "Mocking and Patching Pitfalls", by our esteemed tech reviewer, Ed Jung, which also addresses mocking and its alternatives.Steve Freeman 在他的演讲“测试驱动开发”(Test-Driven Development)中有一个关于过度 mock 测试的绝佳例子。你还应该看看我们尊敬的技术审稿人 Ed Jung 的 PyCon 演讲“Mocking and Patching Pitfalls”,其中也讨论了 mocking 及其替代方案。

And while we’re recommending talks, check out the wonderful Brandon Rhodes in "Hoisting Your I/O". It’s not actually about mocks, but is instead about the general issue of decoupling business logic from I/O, in which he uses a wonderfully simple illustrative example. 在推荐演讲的同时,也请看看 Brandon Rhodes 的精彩演讲“Hoisting Your I/O”。它实际上不是关于 mocks 的,而是关于将业务逻辑与 I/O 解耦这一普遍问题的,他在演讲中使用了非常简单的说明性例子。

Tip
In this chapter, we’ve spent a lot of time replacing end-to-end tests with unit tests. That doesn’t mean we think you should never use E2E tests! In this book we’re showing techniques to get you to a decent test pyramid with as many unit tests as possible, and with the minimum number of E2E tests you need to feel confident. Read on to [types_of_test_rules_of_thumb] for more details.
So Which Do We Use In This Book? Functional or Object-Oriented Composition?那么我们在本书中使用哪种方法?函数式组合还是面向对象组合?

Both. Our domain model is entirely free of dependencies and side effects, so that’s our functional core. The service layer that we build around it (in [chapter_04_service_layer]) allows us to drive the system edge to edge, and we use dependency injection to provide those services with stateful components, so we can still unit test them.两者都用。我们的领域模型完全没有依赖项和副作用,这就是我们的函数式核心。我们围绕它构建的服务层(在 [chapter_04_service_layer] 中)允许我们进行端到端的系统驱动,并且我们使用依赖注入为这些服务提供有状态组件,从而仍然可以对它们进行单元测试。

See [chapter_13_dependency_injection] for more exploration of making our dependency injection more explicit and centralized.参见 [chapter_13_dependency_injection] 以进一步探讨如何使我们的依赖注入更加显式和集中。

Wrap-Up总结

We’ll see this idea come up again and again in the book: we can make our systems easier to test and maintain by simplifying the interface between our business logic and messy I/O. Finding the right abstraction is tricky, but here are a few heuristics and questions to ask yourself:我们会在本书中反复看到这个想法:通过简化业务逻辑与复杂 I/O 之间的接口,我们可以使系统更易于测试和维护。找到正确的抽象很棘手,但这里有一些启发式方法和问题可以问自己:

  • Can I choose a familiar Python data structure to represent the state of the messy system and then try to imagine a single function that can return that state?我能否选择一种熟悉的 Python 数据结构来表示复杂系统的状态,然后尝试构思一个可以返回该状态的函数?

  • Separate the what from the how: can I use a data structure or DSL to represent the external effects I want to happen, independently of how I plan to make them happen?将“做什么”与“如何做”分离开来:我能否使用数据结构或 DSL 来表示我希望发生的外部影响,而不依赖于我计划如何实现它们?

  • Where can I draw a line between my systems, where can I carve out a seam to stick that abstraction in? 我可以在系统之间画出一条界线吗?我可以在哪里开辟一个接缝(seam)来插入那个抽象?

  • What is a sensible way of dividing things into components with different responsibilities? What implicit concepts can I make explicit?将事物划分为具有不同职责的组件,有什么合理的方法吗?我可以把哪些隐式概念显式化?

  • What are the dependencies, and what is the core business logic?什么是依赖项,什么是核心业务逻辑?

Practice makes less imperfect! And now back to our regular programming…​熟能生巧!现在回到我们的常规编程中……


1. A code kata is a small, contained programming challenge often used to practice TDD. See "Kata—The Only Way to Learn TDD" by Peter Provost.
2. If you’re used to thinking in terms of interfaces, that’s what we’re trying to define here.
3. Which is not to say that we think the London school people are wrong. Some insanely smart people work that way. It’s just not what we’re used to.