We usually write tests that check that our code behaves a certain way when we run it. However, we can use tests to check not just the behavior of programs, but the “behavior” of the entire project and all of its code files.
I learned about this idea of “testing everything” from Alex Kladov, aka matklad, and his post How to Test. It’s a great post, and I’m focusing on just one part of it in this post of mine.
Examples
In my project Millet, a language server for Standard ML written in Rust, I have some repository tests, which ensure the repository itself must be a certain way.
One feature of Rust that makes these kinds of tests easier to write is the include_str macro. This built-in macro includes the contents of a file into the program at compile time as a string literal. As you’ll see, many of the tests include some file and then assert that the file satisfies some condition.
Statics rule references
For every rule in the statics for the Definition of Standard ML, there must be exactly one place in the code that references that rule.
Manual sync
Sometimes there are places in the code that must be kept “in sync” with some other place. In this case, there must be exactly two comments that name the thing that must be kept in sync, to link them together.
Test references
There are comments above interesting or unintuitive bits of the code that make reference to tests that specifically use that interesting or unintuitive bit. These comments must refer to actual tests.
Node version
All of the following Node versions must be the same:
- The one mentioned in the readme under the directions for building the project
- The one we use for building the project in CI
- The one we use in the npm scripts in the
package.jsonfor the VS Code extension
Doc comments
Every Rust file must have a //! documentation comment at the start of the file.
Architecture
Every crate in crates/ must be mentioned in the architecture documentation. The idea of “architecture documentation” comes again from a post from matklad.
Diagnostics
Every diagnostic that Millet can emit must have a diagnostic code. The code must be unique, and it must also be documented.
Settings
The VS Code extension settings must be documented in the manual exactly as they appear in the package.json for the extension.
Changelog
Every git tag must have a corresponding entry in the changelog.
Conclusion
Writing tests for the repository itself is a nifty trick with a lot of possible benefits:
- If there is a test that everything must be documented, it can nudge you to write more documentation.
- If there is a test that things must reference one another with special comments, it is less likely those things will drift out of sync with one another.
And so on.