Testing NetLogo model code

Abstract

In simulation modeling, good code testing practices provide (a) efficiency, by reducing time wasted using erroneous software; (b) credibility, by providing evidence that important errors are unlikely; and (c) understanding, by showing how complex model results arose. However, code testing receives little attention in agent-based modeling, and many modeling projects have been derailed by undetected code errors. We present code testing methods appropriate for models implemented in NetLogo by novice programmers. An efficient cycle for implementing new models includes: drafting a written model description and testing the “ submodels ” that each represent a separate process, coding the model while updating the written description to correct ambiguities and errors, and then thorough code testing. As the model is then used and revised, the written description, code, and code tests are all updated. We identify eight kinds of errors that are especially common in NetLogo software. Logic errors are the hardest to detect; doing so typically requires producing detailed test output for key sub-models and then analyzing that output for unexpected results. We provide methods for producing test output, with example NetLogo code. Test output can be analyzed graphically, statistically, by tracing the fate of individual agents, or by comparing it to an independent implementation of the submodel. When unexpected results are found, a variety of “ debugging ” methods help determine whether they are caused by software errors and, if so, where the errors are. Documenting code testing methods and results is also key to model credibility and efficiency: documented tests provide evidence that the code is reasonably error-free and makes it easier to repeat tests as models are revised. While NetLogo currently lacks code-testing tools such as stepwise debuggers and unit testers, its powerful commands make it easy to add similar capabilities to models.