We usually write tests that check that our code behaves a certain way when we run it. However, we can use tests to check not just the behavior of programs, but the “behavior” of the entire project and all of its code files.
I learned about this idea of “testing everything” from Alex Kladov, aka matklad, and his post How to Test. It’s a great post, and I’m focusing on just one part of it in this post of mine.
In my project Millet, a language server for Standard ML written in Rust, I have some repository tests, which ensure the repository itself must be a certain way.
One feature of Rust that makes these kinds of tests easier to write is the include_str macro. This built-in macro includes the contents of a file into the program at compile time as a string literal. As you’ll see, many of the tests include some file and then assert that the file satisfies some condition.
For every rule in the statics for the Definition of Standard ML, there must be exactly one place in the code that references that rule.
Sometimes there are places in the code that must be kept “in sync” with some other place. In this case, there must be exactly two comments that name the thing that must be kept in sync, to link them together.
There are comments above interesting or unintuitive bits of the code that make reference to tests that specifically use that interesting or unintuitive bit. These comments must refer to actual tests.
All of the following Node versions must be the same:
package.json for the VS Code extensionEvery Rust file must have a //! documentation comment at the start of the file.
Every crate in crates/ must be mentioned in the architecture documentation. The idea of “architecture documentation” comes again from a post from matklad.
Every diagnostic that Millet can emit must have a diagnostic code. The code must be unique, and it must also be documented.
The VS Code extension settings must be documented in the manual exactly as they appear in the package.json for the extension.
Every git tag must have a corresponding entry in the changelog.
Writing tests for the repository itself is a nifty trick with a lot of possible benefits:
And so on.