For many years I’ve been interested in dataflow as a way to structure software at a high level. Starting out from Flow-based programming (“FBP”), I was really impressed by the open-source Pure Data (“Pd”) project, which is used to create interactive computer music and all sorts of audio-video setups. In my opinion, this is the “right” way to tie modular code together in applications.
What if this approach could be scaled down to fit on a microcontroller?
Wikipedia’s page on dataflow programming is a good introduction, but here’s my slightly different take on it: the core idea is that the call graph in procedural code is turned into a data-centric mechanism: instead a functions calling other functions, you set up a datastructure which represents calls as “wires”, and then perform those same calls by sending messages over those wires. It’s very easy to adjust such config data.
In the music-oriented Pd application (and its successor, called Purr-data), there is a visual layer built on top which makes the wiring explicit and trivial to create and modify. Software modules are called “objects” in Pd and they are combined into “patches”. Here is one of many such built-in objects, called “moses”:
The above is a screenshot of a live patch in Pd. You can change the
numbers and see the results come out. It starts off splitting on < 10 and ≥ 10, but that split can be changed by adjusting the top-right value.
In this demo, “moses 10” is the object being described and the other boxes are
used to send and receive messages. In Pd, messages move from top to bottom
inside each object. The top and bottom connection points are called “inlets” and
“outlets”, respectively.
But … the moses object is itself also a patch. IOW, Pd lets you compose /
nest patches inside each other. The first diagram was the “moses-help” patch,
here’s the “moses” patch itself:
Note that inlets can receive data from multiple sources and outlets can send their messages to multiple destinations - very similar to “publish-subscribe” (the pub/sub pattern) in message-oriented designs.
The above diagram looks complicated, but just as with functions in text-based code, you don’t need to know the details just to use its functionality: the moses-help patch acts as documentation and live demo. Best of all, this sort of help is just a mouse-click away for most Pd objects. Here’s a more elaborate help patch for the “spigot” object used inside moses:
Let’s do one more example, the “alternate-help” patch:
And the underlying “alternate” patch, also part of the (large) built-in Pd patch collection:
These “patches” are all very basic and only scratch the surface of Pure-Data. One key aspect of Pd which I’ve not mentioned so far, is that it also manages complete audio samples using this dataflow approach. There can be extensive digital signal processing (DSP) going on behind the scenes of Pd and Purr-data (oscillators, filters, mixers, FFTs, graphs …).
So what does this have to do with software in embedded microcontrollers?
Imagine being able to connect modules together in this way: sensors, displays, buttons, timers, communication interfaces. But not exclusively: the ability to tie “standard” code into this should remain as before, but at the top level it could be all objects, messages, and wires. Ideally also visually presented in a web browser during design, and perhaps also for application-specific dashboards.
This may sound like the Node-RED project, but what I’m after is dataflow inside low-power microcontrollers.
I think it’s feasible: there is now a first proof-of-concept design working which implements the core dataflow engine in C++ in some 6 kilobytes of code (excluding all the “base objects” needed to make it all practical).
Note that this is neither a new language, nor an interpreter: the objects themselves are compiled and run at full speed - only the construction of messages and sending them around will have some overhead.
Onwards …