I've done a bit of an overhaul on the multiprocessing_plugin. The work's not quite done yet, and I haven't added docs, but I think the changes are pretty useful and would be happy to clean it up and provide some docs and support scripts if you'd like it in master. Eh, I also need to add support for handling events and spikes, but that should be pretty easy. Oh yeah, and for writing events back to the pipe. Anyways:
Everything's in my fork of this repo, under the 'blp' branch, in the plugin 'blp_trace'. The two key files are:
base classes: multiproc_base.pyx
example usage: blp_trace.pyx
In short:
- Most of the ugly stuff, and all of the stuff related to threads, processes, and pipes, happens in abstract base classes that users shouldn't have to touch.
- To make a new multiprocessing plugin, you can just make a plugin subclass and a processor subclass
- I've added a couple of thread-safe circular buffer types
- I've added a "pipe cleaner" thread that cleans out the input pipe rapidly and copies incoming data to buffers that get passed to preprocessing stages (or directly to the plotting stage)
- I've added a base class (and an example usage) for handling preprocessing on background (non-plotting) threads.
- The coordination of pipe reads, preprocessing, and plotting threads is all handled in the base classes
- none of the base classes depend on the use of Matplotlib timers: the user is free to select whatever plotting or processing framework they like
- with a little bit of work, this would be easy to generalize to handle not just plotting, but any kind of work people might want to do in a separate process... which could be pretty useful
I'm forgetting something, I'm sure. And I'm still working out the kinks, but the current version runs and plots.
I've done a bit of an overhaul on the multiprocessing_plugin. The work's not quite done yet, and I haven't added docs, but I think the changes are pretty useful and would be happy to clean it up and provide some docs and support scripts if you'd like it in master. Eh, I also need to add support for handling events and spikes, but that should be pretty easy. Oh yeah, and for writing events back to the pipe. Anyways:
Everything's in my fork of this repo, under the 'blp' branch, in the plugin 'blp_trace'. The two key files are:
base classes: multiproc_base.pyx
example usage: blp_trace.pyx
In short:
I'm forgetting something, I'm sure. And I'm still working out the kinks, but the current version runs and plots.