a topic to discuss python 2 and eigenD, and the way forward…
1 Like
I’m wondering what your thoughts are on the Python 2 end of life on Jan 1, 2020. I’m wondering how this might effect older installations of EigenD and it’s tools on Mac and Windows. Specifically I’m wondering how it might impact users that still rely on Commander. I use Linux and Windows and am familiar with having multiple versions of python running on my machines, but new users, especially on OSX might end up with conflicts depending on how newer versions of the OS install packages.
I use a tau, and because there is no easy access to entering belcanto directly from the keyboard with classic and factory setups, not having a stable Commander is an issue for me. I can run scripts from command line, but this is a bit of a hack, and not something some folk will be happy with, especially I suspect OSX users.
I personally think it might be time to look at rewriting commander in a more up to date python, but not sure what the implications of that are for the other tools that may be relying on older python interpreters. Haven’t looked too deeply into it, but I don’t see any major hurdles at the outset. Be interested to see what someone who’s taken a much deeper dive than I has to say about this.
Any comment?
ok, so this is worthy of its own discussion … partly so I can also ‘document’ what Ive already considered and found out… and detail some of the issues and idas so far.
disclaimer:
I should preface this with one difficulty I face is that I am no ‘python expert’ ,
I know how to code a bit of python, but as will become evident eigenD integration is a bit more involved than that - so digging in EigenD has been extending my knowledge a lot in this area.
(also I did the last ‘dig’ a quite a few months back, so some of this is a bit ‘rusty’)
Python 2 ‘end of life’ , yes im aware of it, but really I’ve no idea how quickly the end-of-life would materially affect EigenD, we are already using an older version of 2.7.
I’d say its not a short term issue, more mid-term (2-5 years?)
some fact-iods to scope the ‘problem’:
EigenD uses python in two areas:
- tying eigenD modules to the architecture
- as a UI for Eigen Commander, Eigen Browser
EigenD module integration
This is done with the low level python C api, it is critical to EigenD - and would be a massive amount of work to remove (i.e. nigh impossible!)
the good news is, it does not depend upon any other python modules - its is very ‘vanilla’.
I have made attempts to replace this with Python 3, and had some success BUT had some issues with the threading model causing a block - this is not entirely surprising, as python and multithreading is generally problematic - however, I believe mid-term i can resolve this - but its going to take a reasonable amount of playing/hacking to do
Python UI (commander/browser)
we are already facing issues with these on macOS, and they will not work with the 64 bit version of python. the reason for this, is they are dependent on a UI python module (or two?) which is no longer maintained (?).
these UI are not ‘plain’ python, they also rely upon EigenD libs for communication (discussed a bit more below)
the plan has been to rewrite these python UI in C++ using Juce (as is the case with Workbench) - this would not be a one for one rewrite - rather looking at the requirements, and deciding what the app should look like. (e.g. it might be some functionality might be migrated into Workbench?)
once we achieve this, we dramatically reduce EigenD dependency on Python which I believe is very important for long term support.
unfortunately, the re-write is not just a bit of C++ code (otherwise id have done it a long time ago!)
the issue is, these applications do not communicate with EigenD via a simple binary protocol (e.g. osc, sockets) , rather than have to interface via a python interface to talk to the eigenD agents within EigenD.
this requires a much deeper understanding of EigenD architecture, and also the low level Python interface.
so, the first step was to write a very simple C++ app that could communicate with EigenD via this architecture. this was ‘no mean feat’ but I did actually manage to get working. so in principle I have an idea how to get this working …
Im really not worried about the C++ / Juce code and building the UI, that is an area Im very comfortable with.
Obviously, there is still quite a lot of work to go from ‘proof of concept’ to building one or two full applications - and also it is likely i would (at least initially) scale back the functionality to try to reduce this workload.
so, the plan…
I don’t believe the end of life will cause us any short term issues ( probably be fine for a couple of years?)
but we have two pieces of work…
the most important is to get EigenD to Python 3, so that the main application continues to work.
Its hard to say how much effort this will take me - my best guess is , it’ll be one of those tasks where I’ll spend days banging my head against a brick wall, and then suddenly it will ‘all fall into place’ 
Id then turn to the UI, basically, I will attempt to sort out the architectural parts, and then turn to writing the actually UI. (my general strategy is always to tackle the ‘unknowns first’)
I don’t think this is too problematic…but its quite a lot of work.
(there is no point starting this till Ive moved over to python 3, so there may be some gotchas in that too)
Collaboration/spreading the workload
Id love it if others want to help out in these area - esp. if they have deep python experience.
unfortunately the lower level stuff needs a pretty good understanding of the EigenD codebase, so I guess I will tackle that.
however, once I get the main ‘framework’ for the UI apps in place, hopefully there would be opportunities for others with C++ skills to get involved.
Other options:
I really appreciate EigenD and its functionality, and for many users its suits their needs perfectly,
so Im committed to keeping it working.
however, what I personally have found, and noticed from other longer term users, is that many would like a much lighter way to interface Eigenharps to their other instruments (software and hardware). They are used to using midi/osc and dont really need the Eigenharp to provide much more than this.
this was why MEC was born,
the primary goal of MEC, is to provide an application and framework, that can take many forms and can live on many different types of hardware.
It is based on pure C++ and cmake (a modern build system) so is not only portable but the longevity of the toolchain is guaranteed, and being much simpler (focused) - it can be developed/maintained by developers which a far reduced skillset (compared to eigenD).
a second part of the MEC ‘vision’ is running on smaller/cheaper devices.
the idea here is simple, musicians could afford to dedicate something like a raspberry PI to being a Eigenharp to Midi/OSC/CV converter - the rPI essentially becomes a ‘dongle’ , so can be treated like hardware.
we could then have a standard disk image that eigenharp users could then use… plug n’ play.
we would never have to even update this if we didnt want to, it would work for as long as the rPI lives
this is very much do-able today - Ive already got MEC working on a number of platforms (rPI/organelle/bela),
we are very fortunately that this second part is ‘coming towards us’ very rapidly, technology in this area is still improving … e.g. the new raspberry PI 4 is considerably more powerful than the PI3, and we are seeing adoption of this technology in many areas (e.g. Organelle/Norns using raspberryPI and Bela using Beaglebone) … also the technology is getting smaller (PI Zero)
at the last 3eee event in Portugal, I was able to use the Eigenharp without a laptop on an Organelle-m , both as standalone, and also for sending CV and midi to a modular - so its possible today 
my hope is in 2020, I’ll have MEC covering the use-cases required for many more users, and also to perhaps have a ‘standardised’ distribution that is cheap enough that anyone can use it,
at that point Eigenharp uses will be able to choose to use Eigenharps ‘standalone’ or to use EigenD on a laptop (etc) when they require more features etc.
side note:
for completeness (but unrelated to this topic really) , I should talk about Stage.
so Stage is written in C++/Juce and it actually talks to EigenD via OSC,
however, the OSC messages implemented only cover the limited functionality of Stage - it does not cover the full functionality of EigenD - so could not be used for an Eigen Commander or Browser replacement.
(the reason this approach was take for stage, is due to the iOS, I believe Eigenlabs would not have been able to ship the necessary libraries required to link Stage via the python runtime)
6 Likes
apologies for the ‘wall of text’ above…
but I wanted to get down some ideas, as much for myself to remind me of what is ‘to do’ and what has already been done - and now we have a forum, its much easier to spread information for discussion, and also that i can update as we ‘move forwards’
but to even it up a bit, a picture of my 3eee setup
Eigenharp Alpha -> Organelle MEC/Orac -> (sound and CV via ES-8) -> Eurorack
Soundplane is connected to Eurorack directly to Bela Salt to emit CV.
this also works for Eigenharp Pico, but for some reason I have issues doing the same with the Alpha which I need to investigate more.
1 Like
I have made a promise to myself to familiarize myself with the MEC/EigenLite source code once my current projects are done (hopefully around Jan - Feb). If I am able to grasp what is what and feel that I might actually be able to contribute something of value I’ll check in with you if/when I have a general overview.
I mostly write web applications and C# stuff these days, so my C++ skills aren’t all that great. But I really like the concept of a light EigenD alternative and would like to help move that along if I’m able to.
3 Likes
the main focus of MEC is to be headless, to act like an embedded device - but I could imagine adding a web front end to it - to help with configuration etc - so your knowledge in this area could be very useful
i realised i did not answer @amplogik directly …
tl;dr - this should not be an issue
background…
macOS can also have many version of Python installed, and EigenD hardcodes (annoyingly at times ;)) the path to the python to use. this is one reason why we have been able to run the 32 bit and 64 bit version of EigenD side by side for quite a while now.
furthermore, EigenD factory install ships with its own version of python (its the main contents of the ‘EigenD runtime’ package), which would also be an option going forward - but something Ive avoided for the 64bit version so far.
my experience is not that many people are writing belcanto directly, mostly adding to talkers in workbench. EigenCommander has not been working on the latest version of macOS for quite a while, and few people have noticed, much less commented on it
what I did to get around this was to actually interface a text editor to the command line tools, which was ‘good enough’ for my purpose.
but thats just a ‘guess’, perhaps others are sorely missing EigenCommander.
the good news is a new EigenCommander (unlike EigenCommander) could be built just on the rpc interface (like the command line tools) if you just want to quickly enter belcanto.
without the ‘agent integration’ (which is the problematic bit) you’d lack a few features of the current commander - but I suspect it would be still very useful!
( this is one reason, I considered extending workbench to have belcanto/commander functionality)
all that said i would not want to re-integrate any python UI code back into the main EigenD repository. I think strategically with EigenD we need to reduce its toolchain and dependancies.
so, having some UIs written in python (EB/EC) and others in C++ (workbench/stage) adds confusion and dependancies - I suspect Eigenlabs came to realise this, and thats why Stage and Workbench (the newer elements) were written in C++.
honestly, Id love to take the whole python dependancy out of EigenD, but its just impractical for the agent side, not only is it a huge amount of work, but also require the every agent to have its python code re-written in C++ - so reducing down to vanilla python with no modules/ui dependancies is the best we can achieve.(imho)
2 Likes