Testing recommendations

The recommended tooling topic is excellent and particularly valuable for newcomers but I couldn’t find anything similar for testing in PureScript. I’ve pulled together some links to libraries and examples below and hope that together with some comments we might stumble our way toward some newcomer-friendly testing recommendations for PureScript.

Libraries

QuickCheck is a port of Haskell’s QuickCheck and offers property-based testing that works by generating random data with which to test your project. A guide is available.

PureScript Spec is a testing framework, inspired by Haskell’s hspec, that provides a DSL for writing write synchronous and asynchronous tests. A guide is available. It can be used to run tests defined by QuickCheck.

Test-Unit is an asynchronous test runner which can also run QuickCheck tests.

Assert is a basic assertions library for low level testing. It is primarily for testing the core libraries that cannot use QuickCheck without resulting in circular dependencies.

Proxy is not a testing library but can be useful in “passing types” into functions, which is often helpful when using QuickCheck.

All five libraries are in the standard package set and hence can be installed via Spago.

Books

Chapter 13 of the excellent Purescript by Example contains a chapter on testing, using QuickCheck. An version updated for Purescript syntax and best practices is available.

Examples

Jordan Martinez’s repo Jordan’s reference contains excellent examples and notes on testing. Had I known about it before I may not have bothered with this post. Start with the readme files in 21-Hello-World/07-Testing and 21-Hello-World/07-Testing/test.

The library ordered-collections contains a good test suite which uses QuickCheck. The library lists contains a good test suite using Assert.

Provisional recommendations

  • Use QuickCheck. If you can think of a property to test, test it with QuickCheck
  • Use PureScript Spec for simple assertions, testing against external APIs and running QuickCheck tests
  • Whenever you define a type, instantiate the Arbitrary typeclass, if possible
  • If a type is constrained:
    • Document in comments which values are acceptable and which are not
    • Do not export its constructor but provide safe ways to construct it
    • Provide a isValid :: a -> Boolean function that can be used to check that a value of that type meets the constraints
    • Write a test that ensures that Arbitrary generates valid values
    • Test that each function that returns a value of this type returns a valid value

Comments welcome

This document exists mostly to extract knowledge from the PureScript community via comments. All comments, especially on the provisional recommendations, are welcome.

7 Likes

Just fyi. My learning repo has a folder on Testing: https://github.com/JordanMartinez/purescript-jordans-reference/tree/latestRelease/21-Hello-World/07-Testing

Thanks Jordan. I’ve put a quick update in to point to your repo. I’ll take a closer look and will probably either replace this post with just a pointer to your repo or crib a bunch of stuff from there.

I have this library that implements quickcheck-based law tests for the super common classes: https://github.com/garyb/purescript-quickcheck-laws.

It may also be worth mentioning https://github.com/purescript/purescript-gen, which allows you to provide generators for types without tying the library directly to quickcheck.

1 Like

I found argonaut-core as a good example on how to configure continuous integration with Travis CI.

Do any of these tools support coverage reporting?

I should’ve googled first. It looks like you can use nyc along with source maps according to https://github.com/istanbuljs/nyc/issues/1081#issuecomment-488133529.