Spring cleaning

March 28, 2024

Looks like it’s that time of year again: I’m ripping apart what I have in JeeH 5.3 and reconstructing it in a different way. Perhaps it’s just madness, but I have two reasons to do this: 1) the task/thread design is too messy and 2) the way I can add and run tests is too tedious.

Tasks vs threads #

The first issue was unavoidable, once I figured out that tasks should not be an add-on to a threaded system, but exactly the other way around: threads are a special kind of task. Threads are tasks with their own stack, which can therefore be suspended and resumed. Tasks can’t: they either run to completion or they act on their “owning thread’s” behalf, i.e. they suspend their thread with them, if needed. This change puts tasks first, and optionally adds threads and context switching when needed. Then again, the reality is a bit more complex: there is in fact always one thread, the main() app code. It’s just that there’s no context switching involved until at least a second thread is created (with sys::fork(), as before). To support this, JeeH must be given an extra stack for interrupts, exceptions, and system calls, so that it can use ARM’s PSP + MSP dual stack approach (as before) and use PendSV for context switching.

The point is, that as long as only one (main) thread is running, no task switching or messing around with stack contexts is involved, and none of that code even needs to be linked in.

I’ve set up a new dev branch in the JeeH project (at https://git.sr.ht/~jcw/jeeh/tree/dev) with all the JeeH library code in include/ and src/, following PlatformIO’s normal library conventions. It’s early days and there’s still a lot of code which needs to be brought in from the 5.3 main branch.

Running tests #

The second issue is a biggie for me: I want a setup whereby I can push a button, walk away, and then check that nothing got messed up after the latest changes (yeah, it happens … a lot!). I can’t claim to be a fanatic Test-Driven Design person, but I do want to develop things in small steps and leave behind test code which can easily be run again to make sure it still works. Anything less quickly becomes unsustainable.

I tried an approach whereby I keep adding to a single application, and then use the console to run specific tests, often by entering just one character. But that’s still manual repetition, and bound to wear me out to the point that I’ll start skipping these checks. I would prefer to have lots and lots of tiny apps, each one checking one aspect of the codebase, and sorted in such a way that the simplest details are checked before more complex code runs which relies on it. But running lots of little apps on an embedded µC means: lots of uploads, memory flash cycles, and some way to automate the verification of each result.

In the end, I came up with a new approach based on explorations from a while back: 1) run tests in RAM (loads faster & avoids flash writes), 2) send output through the ITM/SWO debug channel (supported by ST-Link), and 3) automate the test verification (extending PIO/SCons using Python).

The result is a new build configuration. It’s not meant to be the only way to build and test things in JeeH, but it’s a nice self-contained setup which seems to get the job done so far. All it took was ≈100 lines of Python code, a simple openocd.cfg file to enable ITM/SWO, and a modified ram.ld linker script to place the “firmware” in RAM instead of in flash. Oh, and a small Makefile to drive it.

The code for this new test setup, with a README file describing it all, lives in that dev branch I mentioned above, in the builds/g431k/ area. It’s still not as snappy as I’d like (openocd is giving me some grief), but I seem to get about 10 tests in 15s for a fresh build, and about 8s when the source changes do not affect any test builds. When working on a single test, that corresponds to a 1..2s edit/run/test cycle. On real hardware, i.e. no need to mock or simulate anything.

The only faster option I can think of would be to emulate an STM32-alike on Qemu. It has been done, but I don’t like the amount of tooling (and upgrade brittleness) this brings with it.

Anyway, it’s all work in progress, as always …