- Unit Tests: tests involving specific functions. The function is given a specific input, and the output is compared to the expected output. This allows us to have guarantees about the behavior of code even after we refactor it, or even before we write the code itself.
- Property Tests: tests which involve specifying the general behavior of a function - given random input, what property should its output have - rather than choosing specific input and output combinations, as we do in unit testing.
- Expect tests: a form of unit testing. Rather than coming up with specific inputs and their respective outputs, the programmer tries different inputs and gets the dumped state of the function output, which is incorporated into the test itself. This requires printers for every type involved, but allows for exploration and regression testing. See this article.
- Cram Tests: a form of unit testing that involves using some shell, such as the OCaml toplevel. All input and output is included in the test itself, and the test plays back the input and compares the output to what is expected.
Unit Testing Frameworks
OCaml has many unit testing frameworks to choose from:
has a useful page
explaining how to integrate all unit test options.
dunealso supports its own simplified version of expect tests using
.expectfiles, which may be enough for many users.
dune promoteare usually all that’s needed to run and update expect tests (including
dunealso includes support for running Cram tests.
expect tests utilising the ppx system.
As discussed here.
Tightly coupled with
dune. Tests can be in the same files as the code itself or placed separately.
- ppx_inline_test: Inline unit tests using the ppx system. Works well in concert with ppx_expect.
- Alcotest: a lightweight, easy to use, and popular test framework. Nice colored output for results.
- OUnit: an older but still competitive unit test framework for OCaml.
a library that allows you to perform property tests.
Sub-libraries are provided to integrate with both
write simple inline pragmas within your code to generate unit tests.
qtestcan also integrate
QChecktests to have random testing, too.
- mdx: Execute OCaml code blocks inside Markdown documentation files. Also supports Cram tests.
Less Popular Options
- Kaputt: a comprehensive testing framework.
- TestSimple: a lightweight unit testing framework compatible with the Test Anything Protocol.
- Cucumber.ml: Behavior Driven Development using Cucumber.
State machine framework for testing imperative code.
QCheck. Paper Video
- Monolith: Specify the behavior of a program and perform random or fuzz testing on it automatically. Paper
Fuzzing involves hitting your program with random or adversarially optimized input until you find ill-defined behavior.
- Crowbar: Quickcheck tests + fuzzing courtesy of afl-fuzz.
bun: Integrate fuzzing into your CI.
- Article about fuzzing with Crowbar, bun and afl-fuzz
- Monolith can also be used for fuzzing.