Brendan Higgins recently proposed adding unit tests to the Linux kernel,
\nsupplementing other development infrastructure such as
\nperf, autotest and
\nkselftest. The whole issue of testing is very dear to kernel developers’
\nhearts, because Linux sits at the core of the system and often has a very
\nstrong stability\/security requirement. Hosts of automated tests regularly
\nchurn through kernel source code, reporting any oddities to the mailing
\nlist.<\/p>\n
Unit tests, Brendan said, specialize in testing standalone code snippets.
\nIt was not necessary to run a whole kernel, or even to compile the kernel
\nsource tree, in order to perform unit tests. The code to be tested could be
\ncompletely extracted from the tree and tested independently. Among other
\nbenefits, this meant that dozens of unit tests could be performed in less
\nthan a second, he explained.<\/p>\n
Giving credit where credit was due, Brendan identified
\nJUnit, Python’s
\nunittest.mock and Googletest\/Googlemock
\nfor C++ as the inspirations for
\nthis new KUnit testing idea.<\/p>\n
Brendan also pointed out that since all code being unit-tested is
\nstandalone and has no dependencies, this meant the tests also
\nwere deterministic. Unlike on a running Linux system, where any number of pieces
\nof the running system might be responsible for a given problem, unit tests
\nwould identify problem code with repeatable certainty.<\/p>\n
Daniel Vetter replied extremely enthusiastically to Brendan’s work. In
\nparticular, he said, “Having proper and standardized infrastructure for
\nkernel unit tests sounds terrific. In other words: I want.” He added that
\nhe and some others already had been working on a much more specialized set
\nof unit tests for the Direct Rendering Manager (DRM) driver. Brendan’s
\napproach, he said, would be much more convenient than his own more
\nlocalized efforts.<\/p>\n
Dan Williams was also very excited about Brendan’s work,
\nand he said he had
\nbeen doing a half-way job of unit tests on the libnvdimm (non-volatile
\ndevice) project code. He felt Brendan’s work was much more general-purpose,
\nand he wanted to convert his own tests to use KUnit.<\/p>\n
Tim Bird replied to Brendan’s initial email as well, saying he thought unit
\ntests could be useful, but he wanted to make sure the behaviors were
\ncorrect. In particular, he wanted clarification on just how it was possible
\nto test standalone code. If the code were to be compiled independently,
\nwould it then run on the local system? What if the local system had a
\ndifferent hardware architecture from the system for which the code was
\nintended?
\nAlso, who would maintain unit tests, and where would the tests live, within
\nthe source tree? Would they clutter up the directory being tested, or would
\nthey live
\nfar away in a special directory reserved for test code? And finally, would
\ntest code be easier to write than the code being tested? In other words,
\ncould new developers cut their teeth on a project by writing test code, as
\na gateway to helping work on a given driver or subsystem? Or would unit
\ntests have to be written by people who had total expertise in the area
\nalready?<\/p>\n
Brendan attempted to address each of those issues in turn. To start, he
\nconfirmed that the test code was indeed extracted and compiled on the local
\nsystem. Eventually, he said, each test would compile into its own
\ncompletely independent test binary, although for the moment, they were all
\nlumped together into a single user-mode-linux (UML) binary.<\/p>\n
In terms of cross-compiling test code for other architectures, Brendan felt
\nthis would be hard to maintain and had decided not to support it. Tests
\nwould run locally and would not depend on architecture-specific
\ncharacteristics.<\/p>\n
In terms of where the unit tests would live, Brendan said they would be in
\nthe same directory as the code being tested. So every directory would have
\nits own set of unit tests readily available and visible. The same person
\nmaintaining the code being tested would maintain the tests themselves. The
\nunit tests, essentially, would become an additional element of every
\nproject. That maintainer would then presumably require that all patches to
\nthat driver or subsystem pass all the unit tests before they could be
\naccepted into the tree.<\/p>\n
In terms of who was qualified to write unit tests for a given project,
\nBrendan explained:<\/p>\n
In order to write a unit test, the person who writes
\nthe test must understand what the code they are testing is supposed to do.
\nTo some extent that will probably require someone with some expertise to
\nensure that the test makes sense, and indeed a change that breaks a test
\nshould be accompanied by an update to the test. On the other hand, I think
\nunderstanding what pre-existing code does and is supposed to do is much
\neasier than writing new code from scratch, and probably doesn’t require too
\nmuch expertise.<\/p><\/blockquote>\nBrendan added that unit tests would probably reduce, rather than increase,
\na maintainer’s workload. In spite of representing more code overall:<\/p>\nCode
\nwith unit tests is usually cleaner, the tests tell me exactly what the code
\nis supposed to do, and I can run the tests (or ideally have an automated
\nservice run the tests) that tell me that the code actually does what the
\ntests say it should. Even when it comes to writing code, I find that writing
\ncode with unit tests ends up saving me time.<\/p><\/blockquote>\nOverall, Brendan was very pleased by all the positive interest, and said he
\nplanned to do additional releases to address the various technical
\nsuggestions that came up during the course of discussion.
\nNo voices really were raised in opposition to any of Brendan’s ideas. It
\nappears that unit tests may soon become a standard part of many drivers and
\nsubsystems.<\/p>\nNote: if you’re mentioned above and want to post a response above the comment section, send a message with your response text to ljeditor@linuxjournal.com.<\/em><\/p>\n