Testing Asynchronous Functionality...

I encountered an excellent question today on the asunit-users mailing list. AliasRob asked:

>> Just wondering if anyone has had any interesting stories or
>> experiences regarding unit testing asynchronous functions, such as
>> loading a .swf, or waiting for a database or remoting call to return.
>> I'm wondering if there's a particular strategy that's becoming
>> popular.

The testing of asynchronous functionality has been the subject of deliberation by many folks that are smarter and more experienced than I. Following, I'll do my best to describe this problem and it's potential solutions as I have understood them from these other people...

The Problem:
You have a bit of functionality that does not execute in a single thread of execution that you would like to test and verify.


The Information:
There are some fundamental tidbits related to Unit Testing that ought to receive some consideration before we go much further.

1. There should be no dependency from one test method to another. Test methods should be decoupled.


2. There should be no dependency from one test case to another. Test cases should be decoupled.


3. As with applications, test fixtures should attempt, at all times, to avoid code duplication and all other code smells (as outlined in "Refactoring").


4. A Unit Test should be designed to test the smallest bit of functionality, and nothing else.


5. A complete application test fixture should execute as quickly as possible in order to ensure that we continue to run it.


6. The setUp and tearDown methods of the abstract TestCase are overridden and implemented to perform (and clean up from) the complete configuration that needs to be in place for each test method in a test case. These methods will be executed exactly one time before and after (respectively) calls to each method whose declaration matches the expression "public function test*".


7. One can use Mock objects for a number of reasons, one of which is as a stub or faux representation of an actually-networked resource.


8. Test cases should never (the emphasis here is mine, most books I have read never say never) require access to the internet in order to successfully execute.

The Solution:
With all that said, the answer to your question is that it depends on exactly what you're testing. If you're attempting to test the backend of a system using it's client, you should stop. The server side of a system should be unit tested independently of any client software. These two halves of an application should not ever be tightly-coupled. The developer or developers responsible for the server should write unit tests that ensure the server is working as expected in whatever language and using whatever technology the server was written in. Likewise, the developers responsible for the client should test its features independently.

How does one test the interface between these two systems?

That's an excellent question. It is done by building a "mock" or "stub" of the server that doesn't actually connect to anything, but instead returns consistent, known results to API queries.

Then, the server developer can build herself a "mock" or "stub" client that behaves exactly the same way in order to test her side of the connection.

Neither of these tests should require network connections.

The biggest issue with this approach occurs if your server has an extensive, large set of complicated APIs. I have often found that the server side of a system should usually have only a small handful of interface methods, and when I have encountered systems that actually did have an extensive interface, it seems that it was usually a code smell where we should have been reading the book, "Enterprise Integration Patterns", instead of over-engineering the connection between our efforts.

In another scenario, you may want to test some piece of functionality like an "xml parser" of sorts or some such other feature that is much easier to test if the data does not have to be managed in ActionScript.

For example, the following is a hideous thing to uncover and later have to manage:


private function getData():XML {
    var str:String = "";
    str += "<Application id='foo'>";
    str += " <child id='bar' />";
    str += "</Application>";

    return new XML(str);
}


For this use case, we have added full support for a truly asynchronous test case. Following is an example concrete TestCase that will execute asynchronously.

The first method called by the TestCase constructor is the "run()" method. In the following example, we override run(), and instead of doing other work in the override, we simply create and request an XML document that resides in our test folders next to the test case in question. In this case, we are using the AsUnit-provided TestCaseXml class which will, upon load completion, call "onXmlLoaded" on whatever object was passed as the second argument of it's constructor.

You'll notice that in that handler, we finally call super.run(). This begins execution of the test case only after a successful load of the xml document. This process can be similarly implemented to load and verify external assets such as swfs or images as well.

import org.yourdomain.data.XmlParser;
import com.asunit.framework.*;

class org.lifebin.data.XmlParserTest extends TestCase {
    private var className:String = "org.lifebin.data.XmlParserTest";
    private var parser:XmlParser;
    private var xmlData:TestCaseXml;
    private var xmlPath:String = "com/yourdomain/data/XmlParserData.xml";

    // This method begins execution of the abstract TestCase
    // We are overriding it here to effectively pause that execution
    // until our xml data is loaded.
    public function run():Void {
        xmlData = new XmlTransport(xmlPath, this);
    }

    public function onXmlLoaded(data:XMLNode):Void {
        super.run();
    }

    public function setUp():Void {
        parser = new XmlParser(xmlData.cloneNode(true));

    }

    public function tearDown():Void {
        delete parser;
    }

    public function testInstantiated():Void {
        assertTrue("XmlParser instantiated", parser instanceof XmlParser);
    }

    public function testParser():Void {
        assertTrue(false); // failing test
    }
}

In closing, I just want to make the point that the asynchronous test case is really only valuable when we want to store "configuration" data in a more easily manageable form. EG, if I have an image parser in ActionScript 3 that will examine each pixel of an image as a ByteArray, it is much easier to load an external, known image than it is to write out some 300k ByteArray in a setUp method. Similarly, it is often easier to test various Xml parsing implementations by actually using a separate xml example document. Keep in mind that the test for these parsers are not intended to also test the "network" or the "server".

One should also note that in this example setUp method, we use the "cloneNode" method of the flash XML object. We also pass in "true" as an argument. This performs a "deep clone" on the loaded XML data structure so that we have a clean copy of the data exactly as it was loaded for each testMethod that is executed.

Thanks,


Luke Bayes
www.asunit.org