Doctest.js: The Reference

HTML page

Your HTML page needs just a couple things to set it up for a test:

<html>
 <head>
  <script src="doctestjs/doctest.js"></script>
  <link href="doctestjs/doctest.css" rel="stylesheet">
 </head>
 <body class="autodoctest">

  <pre class="test / doctest">
    test
  </pre>
 </body>
</html>

Notably you need <body class="autodoctest"> to get the tests to run automatically on page load. If you were invoking doctest explicitly (like is done on the Try It page) then you might leave this off.

External code

Often you won't want to write your test code inside the test itself. Instead you'll want to put it in its own .js file. Especially with test the code is valid Javascript, and you will probably want syntax highlighting and all that.

To do this, use an href attribtue on your <pre> element, like:

<pre class="test" href="./tests.js"></pre>

The contents of the element don't matter. The test will be loaded from that location (which could be a full URL but you'll probably have cross-origin errors if you tried that — this doesn't use <script> tags to load those scripts, it uses XMLHttpRequest). Note that the class is still required!

You can also load it from a variable location, using query string parameters to find the file. The most common pattern would be like this:

<pre class="test" data-href-pattern="./{test-name|default.js}"></pre>

Any URL parameters (like {test-name}) get filled in by the query string (?test-name=example.js). By default these values can only contain letters, numbers, _, - and . — this is to protect against loading scripts from unexpected locations via an implicitly unsafe parameters in the query string.

If you want to use a different restriction on a variable name, use {variable-name:regular_expression} — and use ^ and $ to make sure to match the entire string.

If you want a default (if the parameter isn't set or is empty) use | to separate out the default. The default should come after the regular expression. In the example the default is default.js. Don't use extra spaces around |!

Note this works well with relative addresses like <a href="?test-name=foo.js">foo.js</a>

Format of test

There are two formats that a doctest can take. You've probably seen the test format, there is also the more traditional doctest format.

`doctest` format

This format is used like:

<pre class="doctest">
$ first_line();
> continuation line
output
</pre>

The first line of course starts wtih $ and a space. Think of it like a Unix command line. And also similar to a command line shell the continuation line is >. Any line without a leading $ or > is considered expected output.

Note you can still have multiple statements using a continuation line. The only difference between two lines with $ and one with a $ followed by > is that in the latter case the two lines are executed together and the output from both is combined into what is expected.

`test` format

In this format the expected output is in a comment, like:

<pre class="test">
statement_1();
statement_2();
// Some other comment
// => expected output

More statements

/* =>
expected output
*/

Basically the test is split up by using the // => comments. Each chunk is executed independently, and the test may be paused at the point where the expected output is found.

Test sections

If you are using external test code (and test) you can include section headers like:

// == SECTION A section header

You can have one or more ='s. The text A section header will become a header. Each section header turns into a new <pre> element.

Compact `<pre>`'s

Sometimes you'll have boilerplate code that sets up the test environment, and you'll be uninterested in that code (unless it fails). This might define helper functions, or run some really routine sanity tests. You can use this to make those test blocks small:

<pre class="test expand-on-failure">
  ...
</pre>

This will make the test 3em tall unless there's a failure, at which point it expands to its full size.

Printing/writing

The print() function is pretty important, of course. (Note it also was called writeln(), a name which is still supported).

print() will print out any strings given to it, and the repr() of any other objects, with a space between each argument.

You can make a custom repr() for any object by adding a .repr() method. It should look like:

MyObject.prototype.repr = function () {
  return '[MyObject attr: ' + repr(this.attr) + ']';
};

If you can't add a method to your object, you can also add a stringifier using repr.register().

You might do this like:

repr.register(
  function (o) {
    return o instanceof MyClass;
  },
  function (o, indentString) {
    return '[MyClass attr: ' + repr(o.attr) + ']';
  }
);

The first function is a test that is applied to objects. If it returns true, then the second function is called to stringify the object. If your representation uses multiple lines, then you should indent subsequent lines with the string, like return '[Beginning\n' + indentString + ' end\n' + indentString + ']';. The beginning is never indented.

Some of the objects that have custom representations:

XML and DOM objects: These are displayed as the normal serialization, e.g., <input type="text" />. XML-style endings for empty tags are always used. Attributes are alphabetized. HTML tags are upppercase and attributes are lowercase, because the DOM seems to like that.
XMLHttpRequest: These are displayed like [XMLHttpRequest STATE [STATUS]]. The STATE is one of UNSENT, OPENED, HEADERS_RECEIVED, LOADING and DONE. The [STATUS] is only displayed if the request has finished.

Arrays and objects are displayed like you'd think. This might include objects that you might not think of as "plain" objects. Also the object prototype is not displayed. You'll have to use something like print(o.constructor) if you want to be specific about classes.

`console.log()`

console.log() works a lot like print(). But unlike print() it doesn't make a test pass or fail, it's purely informative.

All the methods on console should work, like console.warn(). The underlying normal console function is called, but in addition on a test-by-test basis these are collected and displayed.

`printResolved()` for Deferred and Promises

There's special support for the jQuery Deferred object, and generally Promises/A.

This support is through the printResolved() function, which is basically equivalent to print() except any promise arguments will be waited on to resolve (proper resolution or an error). You can use it like this:

var def = $.Deferred();
setTimeout(function () {
  def.resolve("Resolved!");
});
printResolved(def);
// => Resolved!

Errors are printed out with Error: before the value, and multiple arguments are printed out with spaces between them, or a placeholder if there's no arguments. For instance:

var def = $.Deferred();
def.reject({code: 1}, "a message");
printResolved(def);
// => Error: {code: 1} a message
// Or you might not have any value at all:
def = $.Deferred();
def.resolve();
printResolved(def);
// => (resolved)

Output matching

The expected output is compared with the output you actually got (received).

First all whitespace is normalized. Empty lines are removed from both sides. Leading spaces are removed (i.e., indentation does not matter). Multiple spaces are normalized to a single space.

There are two wildcard patterns. Ellipsis — ... — means "match anything". This will match zero character, or multiple lines. You should be careful about matching too much.

The shorter wildcard is ?. This matches letters, numbers, underscore, period, and question mark. Note if you want to match a string with such characters you might have to use "?".

Also a special case, " matches ' and vice versa. Since in most contexts these mean the same thing, this lets you be agnostic.

When you have a large expected text and got a lot of text, and that text differs just a bit, you'll see a line-by-line comparison of the two, to help you identify exactly where the problem is. Note if you use wildcards the line-by-line comparison might be very inaccurate.

`wait()`, async code, and pausing the tests

Often you'll want to let code run for a while on its own before you are done with testing a section of code. I.e., you want to let all the requests complete, DOM elements update, and so forth.

Each section of code can be paused at the end. Code cannot be paused in the middle of a section. So before the output (i.e., before // =>) the test runner can wait and collect output.

Anytime you call wait() inside a section of code it tells the test runner to wait at the end of the test, either until some condition is true or until some time has passed. If the condition doesn't complete an error/timeout message is print()'d.

A half baked version of what happens is this:

var printed = [], waiting = null;
function print(arg) {
  printed.push(arg);
}
function wait(condition) {
  waiting = condition;
}
function checkOutput() {
  if (printed.join('\n') != expectedOutput) {
     fail();
  }
}
eval(exampleCode);
hardTimeout = 5000; // 5 seconds
checkTime = 100; // check every 0.1 seconds
if (waiting === null) {
  checkOutput();
} else if (typeof waiting == "number") {
  setTimeout(checkOutput, waiting);
} else {
  var now = Date.now();
  function checker() {
    if (waiting()) {
      checkOutput();
    } else if (Date.now() - now > hardTimeout) {
      print("Error: timed out");
      checkOutput();
    } else {
      setTimeout(checker, hardTimeout);
    }
  }
  setTimeout(checker, 0);
}

Now you practically know how to write doctest yourself!

Specifically this is how you can run wait():

wait(): This makes the test pause just for a moment. It's the same as wait(0).
wait(milliseconds): This forces the test to pause for the given number of milliseconds. Everything is always in milliseconds.
wait(condition): This calls condition() frequently until it returns true.
wait(condition, timeout): This calls condition up until timeout milliseconds. You can use this to extend the timeout. The default timeout is 5 seconds (5000).

Spy, mocking, and watching functions

Spy is used to create a mock object/function that can be used to track calls and inspect call order and arguments.

The basic use is like this:

func = Spy('func');
func(1, 2, 3);

obj = {a: 1, func: func};
obj.func();

/* =>
func(1, 2, 3)
{a: 1, func: Spy('func')}.func()
*/

That is, every time the Spy is called it will print out the call, all its arguments, and if there was a bound this (as in the obj.func() example) then that value will be displayed as well.

Each spy is named, and if you call Spy(name) with a name that has been used before you will get the same Spy object back.

Spy can be invoked in a couple ways:

Spy(name): Just creates/gets the Spy with the given name.
Spy(name, {options}): Create the Spy with some options (as described below)
Spy(name, function () {...}): Creates a Spy that wraps another function that you provide. The Spy will be called first, and will print out the call, and then it will call the sub-function with the same arguments and this.
Spy(name, function () {...}, {options}): Create a Spy that wraps a function and has extra options.

Note that your function can raise an exception, and the Spy will pass it through (though also note the exception using console.log()). You can use this to inspect how a library reacts to exceptions in callbacks.

Spy options

The options available:

applies: function () {...}: This is the function that will be called when the Spy is called. It's the same as passing in the function as the second argument.
writes: false (default true): If this is false (default true) then it will not print out the call.
returns: value (default undefined): This is what the Spy returns when called (assuming you did not use applies). By default it simply returns undefined, which is what a function returns when you have no explicit return statement.
throwError: exceptionObject: If given, when the Spy is called it'll do throw exceptionObject
ignoreThis: true (default false): If true, then this won't be printed out regardless of whether it is bound. This is useful when a library binds this carelessly.
wrapArgs: true (default false): If true then wrapping will be forced when the arguments are printed out. Otherwise wrapping is only applied if an argument is longer than 80 columns (the default for printing generally).
wait: true (default false): This is equivalent to calling Spy(name).wait(). You can also pass in a number, which will be the millisecond timeout, e.g., Spy(name, {wait: 10000}) to wait for 10 seconds for the Spy to be called.
methods: {...}: Equivalent to calling Spy.methods({...}). See below for details.

You can also change Spy.defaultOptions if you want to override one option by default, for instance to turn off printing or ignore this.

Spy methods

Several methods are available:

Spy().wait([timeout])

This makes the test pause until the Spy has been called. Sometimes you must use the method instead of Spy(name, {wait: true}). An example:

SomeAPI.onload = Spy('SomeAPI.onload', function (data) {
  SomeOtherAPI.save(data, Spy('SomeOtherAPI.save'));
};
Spy('SomeOtherAPI.save').wait();
/* =>
SomeAPI.onlaod({...})
SomeOtherAPI.save()
*/

In this case the Spy is created inside another method, and that method is not called right away. wait: true doesn't work in this case. Instead you should call Spy(name).wait() later. Since names are unique, this will be the same Spy object as referenced earlier.

Spy.on("obj.attr", [applies/options])

This replaces the attribute attr on the object obj with a Spy. The object must be defined at the top level (i.e., eval("obj") must return the object). This is basically the same as:

obj = eval("obj");
spy = Spy("obj.attr", [applies/options]);
obj.attr = spy;

Spy.on(obj, "obj.attr", [applies/options])

This is the same as the previous form, except for use when obj is not a global variable.

aSpy.formatCall()

When the Spy is called, generally this does:

print(aSpy.formatCall());

If you use writes: false then this might be helpful.

aSpy.method("methodName", [applies/options])

This creates an attribute aSpy.methodName and assigns a Spy to that attribute. You may give the normal constructor arguments.

aSpy.methods({methodName: [true or options], ...})

This creates multiple attributes at once. You may use {methodName: true} if you have no options to pass in.

Spy attributes

Spies have several attributes to inspect how they have been called:

aSpy.self and aSpy.selfList: This is the value of this as the spy was called. As the spy is called multiple times each this value is appended to selfList, forming a history.
aSpy.args and aSpy.argList: This is the list of arguments that the function was called with. .argList gives the history of past calls.

Aborting your tests

Tests often has prerequesites. Perhaps some browsers aren't supported. Maybe you need a server setup. Normally doctest will run through all the tests regardless of failures, but when basic prerequesites are missing this creates lots of chatter and failures with no purpose.

To stop the tests from running call Abort(). This will still run the rest of the test block (up until the next // =>).

jshint

A helper is provided to help you run JSHint regularly on your code. Just do this:

jshint("source-filename.js", [jshint options]);

You can give a full URL, but you can also just give the filename. When given a filename then all the <script> tags are searched for that filename. The source is fetched and JSHint is run on that source.

You may give options to suppress or enforce checks. In addition you may list the known issues that you wish to ignore: issues are printed out in order, and are matched like any other text. You might want to use this to simply see the errors without checking them for anything in paticular:

jshint("source-filename");
// => ...

`NosyXMLHttpRequest`

Sometimes you may want to watch the progress of XMLHttpRequest requests — both how the request is constructed and its result. You can use NosyXMLHttpRequest to wrap request objects.

You probably want to use it like:

XMLHttpRequest = NosyXMLHttpRequest.factory("request");

The name will be used when showing output (e.g., request.setRequestHeader('X-Something', 'value')).

Node.js

Doctest.js has some Node support. You must use the comment-based test format in stand-alone Javascript files. Then:

$ npm install -g doctestjs
$ doctest test.js

This will print a success or failure message, and will exit with a code (the number of failures) if the test does not pass.

If you do not wish to install the package globally, do:

$ npm install doctestjs
$ node_modules/.bin/doctest test.js