James Shore's "Testing without Mocks"

James Shore released an update of this article “Testing without Mocks: A Pattern Language”. The article is about 40 pages long and it is accompanied by a Mastodon thread to explain it all. This note is my one-page summary of that thread.

Shore wants to avoid mocks as they have a tend to have “lots of detail about the interactions the code”. This locks in in your implementation and makes bigger refactorings really hard.

One way to avoid mocks is by using an architectural pattern that separate logic from infrastructure. Such a pattern allows you to fully test your logic without mocks but tend to ignore the infrastructure code. The latter is something he wants to avoid with his approach.

The gist of his approach is this:

1. Tests don’t test classes (or methods, or concepts) in isolation of their dependencies: a test of also executes code in its dependencies.
2. Each class has a variant that doesn’t depend on an outside state that is hard to manage. This variant is called a Nullable and it’s behavior can be configured on instantiation.
3. Each class can instantiate its dependencies. It can create its production dependencies but also create their Nullable versions.
4. Tests verify return values and state changes, they don’t verify interactions.

Step 3) is what’s new in his approach, at least to me. This step allows you to test a class without a complex setup. For example, to test class UnderTest, you create a Nullable instance. That instance instance creates Nullable instances of its dependencies and this goes on until the lowest-level dependency. As each Nullable comes with its default behavior, or has configurable behavior, the UnderTest instance is ready for testing.

Shore formalizes his approach by adding factory methods create and createNull to each class. The former method creates the class with production dependencies and the latter with Nullable ones. This gives you a consistent way to create and configure the object tree required for a class under test. It also allows each test to cover large parts of your production code without the need to rely on test doubles. Of course, you will need them for the lowest-level dependencies that are difficult or even outside of your control.

Let’s go back to the infrastructure tests. Shore distinguishes two levels of infrastructure (code): low-level infrastructure, which directly interfaces to the outside world and high-level infrastructure, which builds upon the low-level infrastructure. You wrap low-level infrastructure code in a dedicated class and test the high-level infrastructure using the Nullable versions of the wrapper classes. You test the wrapper class separately, for example against the actual infrastructure.

My take

The last couple of years I moved away from testing entities¹ in isolation to testing entities together with their dependencies. I realized that tests that fake the complete environment of the entity under test, tend to become a straitjacket when a refactoring crosses the boundaries of that entity. A lot of puzzle pieces fell into place when I read “Unit Testing - Principles, Practices, and Patterns” by Vladimir Khorikov last year². One of the things I got out of that book is to try to reserve the use test doubles for the lower-level dependencies of your system under test³.

So some of the things that Shore proposes and describes thread familiar territory. What is new to me is the use of factory methods create and createNull. They allow you to quickly create a testable object with all its testable dependencies. The problem I have with this approach is that I don’t see how to use these methods when you need non-default behavior for one of your lower-level dependencies.

Say you want to test an instance of class A, which depends on an instance of class B, which depends on an instance of class C. To create a testable instance of A

• you call A.createNull(),
• which implicitly calls B.createNull(),
• which implicitly calls C.createNull().

The last call sets up some fake data, but what if you need other data to test a different scenario? Of course you can extend factory method C.createNull to accept parameters with the fake data, but I don’t see an elegant way to pass that data from the level of A, through to the level of B to the level of C.

One thing I didn’t mention above, Shore names the different test and development patterns he describes and this makes it easy to refer to them. For example, a test that also exercizes the code of the dependencies is a Sociable Test. A test that verifies the wrapper around a third-party dependency by actually using that third-party dependency, is a Narrow Integration Test. The latter already seems to be known under another name, viz. Contract Test, but a widely-accepted common vocabulary for these patters would indeed improve communication.