EigenD - installers/mac os version

[thetechnobear]

Im working on the next release of EigenD for macos, and have stumbled on juce requiring a newer macos version - than was previously required.

so I wanted to throw a few thoughts out there about future supported versions etc, and the installer

my current thinking…

installation

I want to get rid of the resources and runtime package.

I do not have the source for these, and they are basically full of old/unsupported software anyway.
also the fact they are massive in size, compare to the actual Eigen package which is a few MBs !

to do this, will require two things…

getting rid of runtime package

the main component of runtime packages is python.
I think instead we will come up with instructions on how to source Python (if necessary), and also how to install python packages etc.

however I think this might currently be a moot point - it appears on most version of macOS we can use the system python 2 - without any packages

except for EigenCommander/Browser (which need python packages) but these are not working anyway!

IF we resurrect EigenCommand/Browser, and so need python packages - I’ll take input from you python experts about how best we install these (site-packages/environments) without distributing an entire python install.

getting rid of resource package

these mainly was VSTs which were basically freeware, and are now defunct. and some samples and loops.

this was nice, since it meant the Eigenharp worked ‘out of the box’ - which is something I think we still want.

however… i disagree with distributing binaries and vsts… users can install these themselves.

samples/loops - Im not sure of the size of these, but Id like to include some of these in the basic factory install - so that EigenD works ‘out of the box’ with just one install.

however, I do not want to ‘bloat’ the installer - so we may need to pick n’ choose for his
(we may need to update factory setups to reflect any changes)

also @keymanpal has a great idea (not uncommon), it would be great to have the ‘modular synth’ included in some form in the Pico factory setup.
the advantage of this,this synth will always work for a new user - wihout any dependancies, so is a good ‘first test’

MAC OS versions supported.

(EDITED)

Min OSX version for EigenD 2.2.0 will be : Mac OSX 10.13 (High Sierra) 64 bit only (no 32 bit)

with Apple silicon its another target to build, so Im dropping 32 bit.
also juce requires us to move min os spec to 10.7 at least, but we are going to 10.13.
given we are 64 bit only now

really, given processing requirements of audio software and eigend, I dont see a need for old OSs.

also, the existing released versions of eigenD on my github, will stay there for older machines.

generally new releases are focused on support for new OS/machines anway, so they are not missing much

for now, Im keeping windows/linux support as is.

thoughts?

[Kai]

I’d say anything that increases the odds of EigenD working forward in time trumphs backwards compatibility. It is the future that needs worrying about! So my 2c is leave 32bit and old OSes even if the reason to do so is trivial.

One install and ditching resources and runtime also sounds sensible, if possible.

[NothanUmber]

I think the directory is hardcoded into the packages inside the site.packages folder when installing Python wheels. So as long as the directory stays the same one might be able to get away with zipping it up and uncompressing it again at the target.
Or it might be possible to turn EigenCommander and EigenBrowser and finally EigenD itself into Python wheels, then a pip install EigenD would install EigenD to any desired folder.

Are EigenBrowser and EigenCommander pure Python or is there still some C++ involved?

And — isn‘t the main entry into EigenD also in Python, so it could be considered as a Python application with native extensions? In that case py2app or py2exe might be an idea to get a portable installation that feels like a usual standalone app: py2app - Create standalone Mac OS X applications with Python — py2app 0.24 documentation

Alternative to py2app: https://www.pyinstaller.org/

[thetechnobear]

we dont need any packages for EigenD.

EigenCommander/Browser are I suspect going to pretty much need to be re-written. (*)
ths issue is much of the comms layer between EigenD and EigenCommander/Browser is in python, unwrapping that is an absolutely nightmare - that was my first idea… to basically ditch python entirely for the clients. yes they all use extensions.

the only except is Stage which uses xml - however that is a completely different protocol which is nowhere near as complete as the one used by EigenCommander/Browser.

I have zero desire to make EigenD more dependent on python, in fact completely the opposite - python is the whole cause of the mess and complications within EigenD … not its solution

---

(*) the thing Ive learnt from looking at EigenBrowser, and then later other python apps is , devs love using python because its quick n’ easy - since they just grab a few other libraries and build your app on top.
the problem is this creates a really fragile system, where libraries are dependent on other libraries, and python versions, and operating system versions … its a dependancy nightmare.

this is why these apps have been broken on macOS for so long, because I cannot upgrade to a later version of wxPython, without breaking other stuff… or it not hanging …
let alone the issues with python 2.7 → 3.0

I was pretty neutral to Python when I started messing with EigenD, now Im a hater
(actually, I started liking it a bit more on the C&G organelle/etc… but thats different again!)

[NothanUmber]

With dependency managers like poetry hell got a little more bearable. The libraries particularly in the scientific field are outstanding though. Perhaps Julia might be a replacement some day, but currently not using Python makes life a lot harder than needed in that area.
But for music software it‘s probably ok to hate it

[thetechnobear]

yeah, I think for fast prototyping/dev stuff… things like python are really good.

but the thing is, if you look at the (modules) python code in eigend,
its not readable, its not easy to develop and 99% of the time has to be interfaced to c++.
so all it does is require more knowledge from a developer (python/c++/ python interfacing)

I dont see it provides any benefit from doing it soley in c++… (modern c++ can be suprisingly readable)

Im guessing, like other projects - the idea was including a scripting language will make it easier for non-developers to write modules - but this is often a real fallacy, languages are often not the issue, its the knowledge of the api/system architecture that is off putting (esp. as its often not documented )

heres an example:

    # resolve: resolve an impulse belcanto name into an impulse cookie
    def rpc_resolve_ideal(self,arg):
        (type,arg) = action.unmarshal(arg)
        # only support 'impulse' type
        if type=='impulse':
            return self[4].resolve_name(' '.join(arg))
        return action.marshal(())

is that easier to read because its in python

I could write something almost identical in c++, the language would be as easy to read - but like python, the issue is the meanig… whats an ideal, why are we resolving it - to what, and when?

anyway… we are going a bit off-topic … but I do like to rant about misuse of scripting languages

---

really the main topic, for me at least, is trying to determine how we can easily/quickly get from where we are to a better place.

the question about python as I said is a bit of a moot point for now, as EigenBrowser and co arent working.

but my thoughts around these are. EigenLabs partly had a local python install, so as not to ‘pollute’ the system one with its ‘packages’
however, I remember when looking at Python a few months back - python already supports this, you can create some kind of separate environment for different python apps. (because unsurprisingly this is a problem for all python apps )
so… my plan would be to use these mechanisms , rather than clone the python installs.

[GoneCaving]

My 2c:

• Runtime: I think we should ditch EigenBrowser/Commander. Workbench is a better tool (albeit that it doesn’t hide the complexity of what’s going on under the hood). If there are strong use cases for these tools, then perhaps it might be better to redsign ground up than try to port them.
• Runtime: I would suggest splitting this up to much smaller packages. I agree with @keymanpal, the synth, clarinet and cello should work out of the box. I wonder if a small subset of the loops should also be included to demostrate the capability/ensure it works, with the majority in a separate download.
• OS Version: Dropping 32bit and bumping the min OSX version seems reasonable at this point.

To the comment about wxPython and the Python/C++ issue. The issue isn’t really with python per se, but rather that this is a multi-threaded app with lots of calls that cross that boundary. The architecture is pretty complex to start with, not particularly well documented. As a consequence, trying to debug the issue is really hard.

[thetechnobear]

can someone check that the following exists on older versions of macos X (say 10.14?)

/System/Library/Frameworks/Python.framework/Versions/2.7/bin/python --version

please onfirm what it reports

(my mojave mbp is reporting 2.7.16, which I think is what most should - but my machines get a lot of dev installations - so I never know for sure )

as stated, idea is to move to EigenD using this system Python, rather than requiring python install.

I guess for windows , I will ‘leave as is’ , as Im assuming Windows does not come with a python install by default - later I will look into changing it to use whatever the default installer from python.org uses.

yeah, we are staying on 2.7 for now… I’ll look into python 3 later.

---

@GoneCaving
yeah, that pretty much echos my thoughts…

my hope is to get rid of resources (I think you meant this in your second bullet point) , by simply including some minimal samples that are used by the factory setup in the default install package.
(my only hope is the piano sample isnt too big… I just need to go check that )

… at the end of the day, if someone wants all the original loops/samples EigenLabs supplied, then they can go and down the resources package from EigenLabs.
(I dont have anywhere to store such a large package anyway - so its a bit of a moot point)

EigenCommander/Browser - well Ive been kicking that can down the road for a while now, I guess really Im saying, Im just going to stop building/releasing (*)- and pretending its working.

EigenD/Workbench/Stage are the focus , so the priority is to get these to python 3 - and basically ensure they are using a modern toolchain (**) - we are looking pretty good except python 2.7

once we are there, then we can review EigenCommander/Browser

---

(*) hmm, not sure if I should leave in Windows… given I believe its still working there…
Im tempted to remove it, for consistency…

(**) I just really hope Scons doesnt die… that would be a major pain to update to something like CMake, as much as Id love too.

[keymanpal]

@thetechnobear

please onfirm what it reports

iMac = High Sierra
/System/Library/Frameworks/Python.framework/Versions/2.7/bin/python --version

Python 2.7.16

edit: same on Mojave (MBP2017)

[thetechnobear]

your iMac on high sierra… are you already running the 64 bit build of EigenD?
if so perhaps I’ll see if I can change min mac version to 10.13 (high sierra)

[keymanpal]

@thetechnobear

are you already running the 64 bit build of EigenD?

Yes!
If I may say it, high sierra got many fans during many years?!?

[GoneCaving]

@thetechnobear

my hope is to get rid of resources (I think you meant this in your second bullet point) , by simply including some minimal samples that are used by the factory setup in the default install package.
(my only hope is the piano sample isnt too big… I just need to go check that )

Sorry, yes. Meant the resources (not enough coffee!)

[thetechnobear]

ok, just for you - 64 bit 10.13 is now min spec for mac osx
(only joking, 10.14 was also pretty arbitary, because its what Ive got currently on my MBP )

[BobTheDog]

I have Juce stuff running here 32/64 10.7.

What issues are you seeing supporting 32 and pre 10.13?

Edit: I had to fix up the Juce code for 32 bit plugins but it is a simple fix if that is the problem?

[thetechnobear]

no technical reasons.

Juce could support 10.7 but I have no way of running mac os of that era.
so if someone reports a bug on it - all i can say is ‘sorry, dont know’ , and Id prefer not to do that - I have a machine running 10.14, so was going to go for that but 10.13 is ok.

32 bit vs 64 bit , similar , except it takes a while for EigenD to build, so why push out a release that no-one is going to use… its a bit of a waste of my time.

I totally recognise, others might disagree - but I do work on a lot of open source projects in my spare time - so at some point I have to decide where my boundaries are due to time constraints.

of course, this is the great thing about open source, anyone that needs it - can simply grab the source and compile for 32bit/10.7 - I’m not removing the code, just not supporting it going forward… if you need to make changes in the future you can also fork it.

finally, as stated - most of the work im doing on EigenD at the moment is maintenance.
so if you want a 32 bit build, or something to run on even 10.6 (iirc) - you can use my build of 2.1.7 … you wont really lose anything.

[BobTheDog]

Ah, ok. I thought it was a technical thing.

Totally agree with you, supporting 32 bit is becoming near impossible on OSX and if you haven’t got VMs with the all the versions you are suckered for support, and with Open Source you don’t get paid to have all this set up.

[cmc]

@NothanUmber

I think the directory is hardcoded into the packages inside the site.packages folder when installing Python wheels. So as long as the directory stays the same one might be able to get away with zipping it up and uncompressing it again at the target.

Yes, it is. I was working on updating the current version to get it to use a different Python,* but I haven’t been able to compile the packages successfully. (On Catalina; haven’t tried on Big Sur yet, but I’d expect the same issues.)

It should also be possible to rebuild the existing packages so they install in a different location, but any hardcoded paths are going to fail.

* Python2 from MacPorts with py27-wxpython-4.0, which installs a few other packages and seems to work fine if you do a bunch of symlinking to match what EigenD is expecting to find.

[thetechnobear]

ive had newer versions of wxpython running ok, packaging was not the issue.
the issue was the eigend code broke with the newer versions.

I personally do not want to invest time into getting these python apps working, esp, when the (inevitable) move to python 3 will no doubt break them again.

I also feel that EigenD weakess point from a dev point of view, is its complexity in its toolchain. more technologies. more sdks - not only mean new OS versions will cause incompatiblities , its also increases the skill set that developers need to contribute to EigenD. ( * )
reducing tech used as much as possible needs to be a goal.

C++ obviously is the backbone and Juce is a very mature (cross-platform) framework (not saying its my favourite, but its fine for the job!) - so those stay.

Python, id love to rip out, but it would be a huge task to remove from the module definitions.
so the goal is to minimise it JUST to the module definition. ( ** )
for this one task, we do not need any python packages… so that reduces dependancies.

yes this means the ui’s have to be in C++/Juce, but that is ok… we already have Workbench and Stage in C++/Juce - so we can concentrate on that approach.
(its pretty clear thats was the direction EigenLabs was heading anyway)

Id also love to get rid of Scons, really CMake would be a much better choice, but thats a bit harder.
Similarly, I wonder if moving to swig might be better for the python/c++ integration.

but I’ll admit its hard to feel motivated to do these kind of improvements as they don’t seem to be a pressing problem at the moment.

for me feels like priorities are
a) sorting out windows, move away from eigend driver, as so allow 64 bit builds
b) python 3. (python 2 is around a long while, but it is EOL)

---

( * ) this is a mistake a lot of projects make… really open source projects are best when you don’t need a ton of different skills… its hard enough finding decent C++ programmers, but then trying to find one that knows the internals of python, and various python packages is nigh impossible.
… its a non issue in companies, since you just hire a bunch of devs with different skill sets, a luxury open source projects rarely have!