Plugin Road-map

Plugin Development
1

Here are the bigger features/changes for the plugin API and SDK that are planned in the short/medium term future, and roughly where we’re putting them on a timeline. However if anyone wants to help on any particular new features (either by testing the API on their own plugins, or helping with code), we can re-arrange priorities.

Edits in Italics

v1.3:

  • X Done! Text Display API. The API is already done (thanks @hemmer) and merged into the main firmware and SDK branches. I’m adding support for plugins to be able to use their own font files.

v2.x:

See the v2.0-dev branch.
This will be first ABI compatibility break, that is, all plugins will need to be recompiled with the v2.x SDK in order to run on firmware v2.x.

  • In progress on v2.0-dev: Add MIDI raw streams. This is still experimental. The idea is to pass raw MIDI packets to modules. The idea I’m working with now would not require an API break, so it could be part of v1 SDK.
  • X Done in v2.0-dev: Context menus. These show up in the elements list on the Module view page. In most cases there are no changes needed to the VCV plugin code. So far the only changes needed were purely for UX (updating the text of the menu live as you select items, which does not usually happen in VCV Context menus).
  • SDK helpers for adding AltParams. These are regular params but they can’t be mapped and don’t show up on the panel (just at the end of the list of elements in ModuleView). I’d love to have a way to automatically convert rack::Menu items to AltParams, but I haven’t found a way to do this automatically (yet?). Update: This is not necessary now that we have automatic adding of Context Menus.
  • X Done v2.0-dev: Support for loading Factory Presets: The presets/ dir containing json files are included in the assets/ dir. Backported into v1.x branch.
  • Support for loading User Presets: This means scanning for and loading .vcvm files from SD Card or USB drive.

Changes to CoreProcessor (base class for all modules):

  • X Done Adding a CoreProcessor function to get a param value from the module. This allows the GUI to update the visual state of module if the module changes its own parameters.
  • In progress Support bypass for modules. Adds bool bypass to the CoreProcessor base class. The engine ends up calling processBypass() instead of process() on VCV-ported modules when bypass is true. Native modules may have BypassRoutes in their info structs.

Changes to core-interface:

  • X Done Add min/max value, display base/mult/offset, units, to all ParamElements. This allows the human-readable values such as “440Hz” instead of just 0-100%.
  • X Done Add min/max angle to Pots so they are drawn more correctly.

Changes to rack-interface:

  • X Done Remove dependency on MetaModule::Elements by rack::Widget. Doing this means that future changes to Element types (such as the two items above) would not be an ABI break.

Next up (v2.x?)

  • In progress: Filesystem access: This is more-or-less worked out. It adds two features: modules may run background tasks which are allowed to block (and do not block the audio task); and an API for accessing filesystems. The FS API is based on FatFs API, but a posix adaptor is easy. In progress and may be back-ported to v1.x: fopen, fread, fclose, lseek/fseek which allows for plugins to read from their own assets/ dirs
  • In progress Direct Draw Elements: I wrote a back-end renderer for nanovg which ties into LVGL. It supports TTF fonts, and most NVG vector commands. This lets VCV modules display graphics and text, without any code modification.

Future directions:

In general I want to see less changes required to VCV-based plugins. Some easy ones are:

  • Support exceptions, and support streams (iostream, stringstream). This is quite far along, but I have it at a low priority. Fixing this will save quite a few #ifdef blocks…
  • Integrate building MM plugins with building VCV plugins. Ideally a MM-compatible VCV plugin could have minimal modifications made to the Makefile or CmakeLists file, so that something like this:
export RACK_DIR=/path/to/Rack-SDK
export MM_SDK_DIR=/path/to/mm-sdk
make dep
make dist
make install-mm-sdk
make mm-plugin

… builds both the .vcvplugin and the .mmplugin files. There is an example of this working in our CVFunk fork

  • Use SVGs instead of PNGs. This isn’t a huge deal, but it would mean we could “zoom” in on part of module at better resolution. A quick check I did showed that compressed SVGs (.svgz) were close in size to 240-px high PNGs.
5 Likes
2

sounds great !

DirectDraw - yeah, a callback passing a framebuffer, then access a graphics api for drawing to it, lvgl would be fine. (assuming you can blit fb efficiently )

if its a callback w/ frame buffer, this implies the ‘host’ will control the frame rate - which I think is not a bad thing. ofc, the plugin could reduce load by only drawing every N frames. (using double buffering)

if its not full screen, not sure how efficient this will be. i.e. if its a component backed by frame buffer.

overall, I suspect that if we are creating a plugin that requires direct draw, then that code is likely to be specific to the MM, due to dimensions etc - so I don’t mind using a specific api, and implementing specific callbacks etc.

for me, this implies:

a) mm specific ui implementation
specific code in a plugin , that will differ from mmplugin to vcvplugin,
so I’d have separate implementations which Id pull in depending on if im building for vcv or mm.
(so if you were porting an existing vcvplugin, you’d write a new implementation)

Id use vcv (desktop) mainly for initial development and things like dsp processing/midi etc.

b) simulator
to test the UI on my Mac, Id want to run within the mm simulator on the Mac.
I think, you mentioned, currently this requires copying plugin into simulator?
it would be useful if this could be streamlined e.g. dlopen plugins from a directory?
(can be simplified compared to MM, e.g. load all plugins)

idea is we load in test patches, so we can MM specific features (incl UI) in simulator.

note: generally, Id not care about building vcv and mm plugin at the same time, as I’ll be focused on a specific dev task - so either in vcv OR in mm simulator/mm hardware.

one things to bare in mind, you’ll want to also support ‘enumerated types’, also non-linear numerics (*)
the way Ive done this in the past (not vcv) , is a callback on param displays, so then there can be a default implementation for simple numerics, but custom types can be implemented.
not sure, how this relates to vcv. or if this is part of what you were doing with text display api?

( * ) unfortunately, doing things like frequency / dB with linear scaling is not that useful :frowning:
(and which exp/log scale they use is not consistent either)

edit:: oops, longer post that I expected… feel free to move it to a separate post in dev, if you want to keep this one ‘clean’

3

The preset loading is HUGE. With just one modules you can make a bunch of different sound and cv control the sounds or use the same modules and swap sound mid set…

2 Likes
4

I guess there are actually two features here:

  • Rending graphical widgets (e.g. VCV Free/Fundamental WTVCO), more or less like how text displays work in v1.3. So, basically allowing VCV-ported modules to look more like they do on the computer.
  • Full-screen module mode, or running a full-screen “app”

For graphical widgets (small screens inlayed on a module), it would need work within the framework of the GUI backend (LVGL), so the screen itself would probably need to be an LVGL canvas object. I’m not sure the best way. One idea is that the GUI engine passes an LVGL object to the module which uses vector drawing (draw_line, draw_arc, etc)… Another idea is to pass a partial frame buffer (that the widget specfies the X/Y dimensions) and the module just fills the pixels any way it wants (we could still provide basic vector drawing routines, but the module could use its own drawing library if it wanted)

For full-screen module mode, I would first try just passing a framebuffer and see if it works out decently. If not, a more streamlined way would be to do what LVGL does (so we can dovetail right into that): requesting a partial framebuffer with the option to repeat. So the callback fills a rectangular area with pixels and specifies the coordinates and width/height of the rectangle. Then it returns true if it needs to render another rect, or false if its done. This would bypass all of the current GUI rendering, so would be about as efficient as possible.

That’d be great to compile plugins locally as shared object and have the simulator truly load them in. All the plugin file loading/unloading and scanning dirs for plugin files, is already running in the simulator.

The feature request is just to mimick the way rack::ParamQuantity works: you can set the base value for log/expo, along with an offset, etc… At least this much gets us display compatibility with VCV-ported modules.

A callback for adjusting params would be of course the most general purpose. Along these lines, having a lot of hooks or callbacks for various events would give us a lot of flexibility.

3 Likes
5

I just edited the first post, adding in some status reports for what’s done or in-progress

1 Like
6

Also – One thing not really on the API roadmap, but will help with plugin development is enabling USB console logging.
I’m working on a driver right now, it’s fairly trivial.

What that means is when debugging a plugin you can use printf or to send data over USB to a console window on your computer.

Right now it’s possible to do this, but requires a $20 dongle and you have to connect to certain pins on the back… so this just makes it a bit more accessible.

2 Likes
7

Edited the first post again:
We have part of the filesystem feature implemented now: various standard library filesystem functions like fopen(), fread(), fclose() are implemented and allow a module to read (but not write) files in their assets/ dir. This does not allow for reading directly from the SD Card or USB drive, but only for reading files which are have been unpacked from the .mmplugin file into the internal ram disk. Reading is fast, though I still would not do it in the audio loop. Best practice is to read files in the plugin initialization or module constructor. Here is the project we’re using to test the API:

6 Likes