What is testability? When we look at a test subject what is the criteria it that we use to judge it? What makes us say that needs to be refactored? What makes us say that is easy to test? I am sure that there are a finite set of rules that we use to measure testability. I would like to raise them out of instinctive and subconscious thought and into conscious thought. To do this we need to back all the way up to sense and control. These are the two most basic tools of testing. To test something you must be able to control it and sense what it did. This implies that there are two main axis that we judge a subject on for testability: controllability and sensabiltiy.
Control
There are two main areas of control: test subject creation and execution. To control a test subject we must be able to create it, get it ready for use in the test case, and execute it. The difficulty of the execute part increases when the test subject contains conditionals. The questions below should be asked of any test subject when trying to determine its testability.
What makes a test subject easy/hard to create?
- Is the constructor visabiltiy public?
- If not is there a member in the test subject that is a substitute for the constructor?
- Is there a default constructor?
- If not is there a constructor that accepts few parameters?
- Is the constructor loosely coupled to other types?
- Is the constructor’s behaviour independent of other types?
What makes a test subject easy/hard to execute?
- After construction is it ready to execute?
- Are there no parameters?
- Is the test subject loosely coupled to other types?
- Is the test subject’s behaviour independent of other types?
An answer of no to any of those questions indicates a more difficult test subject to control.
Sense
There are two main areas of sensation as well: interaction with other types and value assignment (returning a value is included in value assignment). For both types of sensation we must have a reference to the object the test subject is using. If we do then it is very easy to sense if values were assigned, interaction on the other hand can get difficult. Lets call the objects that the objects that the test subject interacts with actors, like the supporting cast in a play. So if an actor is easy to create, setup, and it allows us to easily sense that the interaction between it and the test subject occurred correctly then it is easy to use in the test. This is often not the case. Often actors are difficult to create, setup, and or they hide what interaction did occur between it and the test subject. This difficulty can be over come easily with test doubles, again with the acting analogy, think of a stand in or a stunt double. The use of test doubles requires the test subject to know of actors by an abstraction; a base class or an interface. The questions below should be asked of any test subject when trying to determine its testability. Keep in mind that part of the test has already been written.
What makes a value assignment easy/hard to sense?
- Do we already have a reference to the object the assignment was made to?
- If not can we get a reference to the assignment object from an object we already have a reference to?
- Can there be multiple instances of the assignment object? (if it is a singleton this complicates things)
What makes an interaction easy/hard to sense?
- Is the test subject introverted (not interacting with other types)?
- If not is there just one actor?
- Do we already have a reference to each actor?
- If not can we get a reference to the actor from an object that we already have a reference to?
- Is each actor easy to control and does it expose evidence of interaction?
- If not does the test subject know of the actor by an abstraction?
Again an answer of no to any of those questions indicates a more difficult test subject to sense.
Application
Lets play with this a little and say that an answer of no to a question earns a point. So the ideal score is 0. That means that the ideal test subject has these characteristics:
- The constructor is public.
- The constructor accepts no parameters, a default constructor.
- The constructor interacts with no other types, its behaviour is independent.
- The test subject is ready to execute after construction.
- The test subject accepts no parameters.
- The test subject interacts with no other types, its behaviour is independent.
- The test subject returns a value and makes no other assignments.
- The test subject is introverted, it interacts with no other types.
public class Ideal
{
public Ideal(){}
public String Method()
{
return “Hello World!”;
}
}
[TestFixture]
public class TestsIdeal
{
[Test]
public void TestsMethod()
{
Ideal TestSubject = new Ideal();
String ReturnValue;
ReturnValue = TestSubject.Method();
Assert.AreEqual("Hello World!", ReturnValue);
}
}
So lets apply this to Jeremy’s post on State vs. Interaction Testing:
public class InvoiceProcessor
{
public void ProcessInvoices(Invoice[] invoices)
{
InvoiceErrorReport report = new InvoiceErrorReport();
foreach (Invoice invoice in invoices)
{
// Execute the validation business logic
InvoiceError[] invoiceErrors = this.validateQuantities(invoice);
report.StoreErrors(invoice, invoiceErrors);
}
// determine if the critical error threshold has been exceeded
if (this.hasTooManyCriticalErrors())
{
MailMessage message = this.createErrorSummaryMessage(report);
// Call to the local (or default) SMTP service to send the email
SmtpMail.Send(message);
}
// determine if the warning threshold for the error count has been reached
else if (this.hasTooManyErrors())
{
MailMessage message = this.createDetailsMessage(report);
SmtpMail.Send(message);
}
else
{
// If the invoices are okay, send the invoices on to the next process via MSMQ
MessageQueue queue = new MessageQueue(this.getMessageQueuePath());
queue.Send(invoices);
}
}
}
The constructor gives us a score of 0 so we can move on to controlling the execution. The test subject is not ready to execute after construction. The MSMQ needs to be created and or purged. The email server needs to be checked to see if it is ready for the test as well. So there is 1 point. There is one parameter so there is an other point: total 2. The test subject is tightly coupled to several objects: total 3. Its behaviour is independent of other types. It does not perform any value assignments. It does interact with several other types: total 5 (1 for interaction and 1 for several). We do not already have a reference to any of the actors nor is one exposed to us: total 7. The SmtpMail type does not show evidence of interaction nor does the test subject know of it by abstraction: total 9. The MessageQueue type does offer state based evidence of interaction (as long as there is no contention/competition for the message). So we end up with a total of 9. Jeremy goes on to show an improved design for testability. Lets see the score produced, I hope it goes to show that these are the right questions to be asking.
public class InvoiceProcessor
{
// “Dependency Inversion Principle” — All dependencies are now to an abstracted interface
private readonly IEmailGateway _emailGateway;
private readonly IMessagingGateway _messagingGateway;
private readonly IUserInformationStore _userStore;
// Use “Constructor Injection” to push dependencies into InvoiceProcessor
public InvoiceProcessor(
IEmailGateway emailGateway,
IMessagingGateway messagingGateway,
IUserInformationStore userStore)
{
_emailGateway = emailGateway;
_messagingGateway = messagingGateway;
_userStore = userStore;
}
public void ProcessInvoices(Invoice[] invoices)
{
InvoiceErrorReport report = new InvoiceErrorReport();
foreach (Invoice invoice in invoices)
{
// Execute the validation business logic
InvoiceError[] invoiceErrors = this.validateQuantities(invoice);
report.StoreErrors(invoice, invoiceErrors);
}
// determine if the critical error threshold has been exceeded
if (this.hasTooManyCriticalErrors())
{
MailMessage message = this.createErrorSummaryMessage(report);
_emailGateway.SendMail(message);
}
// determine if the warning threshold for the error count has been reached
else if (this.hasTooManyErrors())
{
MailMessage message = this.createDetailsMessage(report);
_emailGateway.SendMail(message);
}
else
{
// no longer responsible for knowing where the MSMQ path is. Or even if this
// is an MSMQ
_messagingGateway.SendInvoices(invoices);
}
}
}
It now has a constructor with parameters so we start of with a score of 2. The test subject can be ready to execute immediately after construction if test doubles are used so we don’t get dinged for that any more. The test subject still accepts one parameter: total 3. It has improved in that it is loosely coupled to other types. It does interact with other types, 1 point, and there is more than one: total 5. We now have references to the types being interacted with, we had to pass them to the constructor. The SmtpMail type still hides interactions for a point but the test subject now knows of it by an abstraction: total 6. So the grand total after refactoring is a 6.
Tuning
I don’t mean the points part as stated here as a metric that should be used on any project. It was just a means to illustrate, though I do think that with some tuning to the questions and the assignment of points a metric could be produced. The answers to groups of the defined questions can even indicate particular design sicknesses and possible refactorings.
