EigenD - Apple Silicon - It lives!

Status:

EARLY PRE-RELEAE DEV TESTING ALPHA of MAC OS versions

this is NOT the official release, which will be released later to github
read included release notes, and below for status/known issues

Apple Silicon

Intel 64 bit

Note : Please only report bugs/issues which are new to this release, at this time Im not focused on ‘existing’ issues.

Requirements

  • MacOs or Mac OSX 10.13+
  • Apple Silicon or Intel 64 bit.

32 bit and versions of OSX prior to 10.13 are no longer supported.

Known Issues:

  • limited testing :wink:
  • convolution disabled for apple silicon - so cello will not work

Limitations

  • can only load plugins for same architecture
  • resources not installed - incl samples/convolution responses

To Do

  • convolution for apple silicon
  • drummer/sampler - decide how to ship samples?
  • test alpha mic input (*)
  • create release
  • windows build

(*) someone else needs to do this, as I dont have the hardware

Tested and working

  • Alpha, Tau and Pico
  • Midi Ouput
  • Vst/AU host
  • Synth module
  • t3d module
  • mpe
  • sampler (but you need to copy samples)
  • clarinet
  • Workbench
  • Stage
  • Release packing
  • headphone output (tau/alpha)
  • latest version of Juce (6.0.8)

EDIT: I’ve updated this as the main post abotu EigenD on Apple Siliton

initial post - kind of redundant now :wink:

just a heads up… dont get excited … its not (even close to) working yet.

Apple Silicon native build of EigenD

so today Ive done a lot of the ground work for this, I can build many (most?) of the modules of EigenD on my Mac Mini M1.

however, theres still quite a lot of hurdles…
modules - most are compiling, plg_convolver is using an fft lib that has sse / intel code in it.

pico decoder - dylib is now available for arm64

juce - ive hacked the build, but likely this is going to need to be updated to the latest juce version, so it properly supports apple silicon/big sur.

pip - there seems to be an issue with modern compilers and the pip code, which is preventing me building a few things…e.g. host and midi … this is going to take a bit of thinking times.

python - so there is 2.7.16 of python installed on big sur, Ive no idea how compatible this is going to be , 2.7.10 was ok, when I did the 64 bit intel build BUT I know, the EigenD code was very fragile to changing python version. I tried python 3, and it all fell apart very quickly :wink:
my main concern here is wxpython, a graphics library thats used… as I again, had alot of issues with this to get the 64 bit intel build working
the general takeaway here is … python and eigend are tightly bound, and efforts Ive tried to upgrade to newer versions have been unsucessful causing locks in the bowels of python (I think GIL is an issue) - I got the 64 bit version, but finding a combo of versions that work BUT with apple silicon, we wont really have that choice!

the upshot of all of this is… even when I can get it all to compile (which Im pretty close to), its likely to still be quite alot of work to get it to actual function (e.g. new juce version, figuring out python)

you’ll find what Im up to on this in the applesilicon branch of my EigenD repo.

Eigenharp Hardware on Apple Silicon

my basic tests with MEC, were promising on M1 - but I need to finish this off, and check the underlying usb /eigenharp comms is working - if it works on MEC, it’ll likely work on EigenD.

EigenD existing 64 bit build under Rosetta 2?

I’ve no idea if the EigenD 64 bit build will work under Rosetta 2 or not.
whilst Ive been really impressed with Rosetta generally, I think EigenD has a bit of a unique problem… its using python…

really it goes back to issue above with Python … can we use the Python 2.7 present in BigSur, with the added compilcation of an Intel/Rosetta binary, working with that arm python install.

honestly, at the moment, Im not looking at this… for the simple reason, to get it to work might require me to change the ‘default python install’ and generally mess around with my M1.
and I think for now, Id prefer to keep the M1 ‘clean’ , so that if the ‘native build’ works, I know its not because Ive been hacking it … i.e. that build will work on others M1 without changes.

if someone else wants to give it a go then that would be cool - as it might contribute useful information that can be used for the native build.

note: id advise, install the 64 bit build but do NOT install python as indicated in release notes, rather try to use the python thats already installed in /System/Library/Frameworks/Python (or similar)

7 Likes

Hey, as of yesterday, I now have access to a M1 MPB Air !! Let’s say… after midnight :wink:

Sure! I’ll give it a try, I’m checking many other things also …

Happy you have some time dedicated to this!

3 Likes

It is interesting that Apple is usually more on the pushing side when it comes to introducing new technologies but is still staying at Python 2.7 (which is unsupported by the OS community since january 2020 now).
Well - good for us(?), at least for the moment. Probably the necessity to migrate to Python 3 is still more a question of time. Might be a good point then to make some fundamental decisions in which direction to go.

ooh, nice that M1 Air is tasty :slight_smile:

Im a bit busy at the moment, but I’m guessing/assuming at the apple dev conference Jun 7 '21 we will likely see the bigger Mac released with apple silicon, likely the iMac and bigger mac book pro. … which’ll intensify the interest in native apps - so want to get some kind of solution under way …

though I have to admit Im not totally sure what that is !

attempting to run EigenD under Rosetta 2.

as said, Im not entirely sure how to try this.

heres my suggestions…

IMPORTANT EDIT: we do not need the python packages (site-packages) for this experiement

do NOT follow the 64 bit install instructions - in particular , lets NOT install python
only install the main EigenD 64 bit package, so no runtime or resources.
we are aiming for a bare bones install…

so try to install the main EigenD package - this should work, and install to /usr/local/pi

however, it will not work, as I believe it will look for python in
/Library/Frameworks/Python.framework/Versions/2.7

this is where we install it usually, but I think we should try to use the macOS install

to do this we could try to ‘redirect it’

to do this

sudo ln -s /System/Library/Frameworks/Python.framework /Library/Frameworks

at this point, Id try to run EigenD to see what kind of errors we get…

next step would be the same as we do on the 64 bit install, so getting the python site-packages from another machine and moving them here.

its possible we might get it to work under Rosetta , but I do want to limit ‘expectations’, as Ive been down this road with the 64 bit install, and it was honestly quite frustrating.

Rolling back… so to a clear BigSur setup

now to be clear, we can ‘roll back’ from these changes at any time

then remove the link we create in /Library

sudo rm /Library/Frameworks/Python.framework

yeah, I was amazed 2.7 was included…
I guess just goes to show how many python 2 apps are still out there.

from my (limited) experience, the issue is that many python apps depend on python packages, so if you have an older app, dependent on older python packages you may well struggle.

this is kind of the problem we have with EigenD.

I found last time, even upgrading Python to another 2.7.x version broke wxPython.
I spent hours trying to debug it, but it was really unclear what had changed and how to fix it.

I also tried upgrading EigenD to use Python 3.0 , which I hoped would work with the new wxPython, but in that case EigenD would just hang - some kind of mutex race condition, I suspect caused by python GIL.

so… this is the underlying problem with EigenD/Python.
I’m ok with writing python, however, EigenD is integrating python and c++, at a simple level Im sure I could cope with that too, but Eigenlabs created a whole ‘pip’ layer which auto generates the C++ binding layer… so it not using one of the common python/c++ binding solutions - which means zero documentation.
then on top of this, as with any kind of language binding like this, you have to be really careful with threading - so what you c++ code does in the interpreter thread…
this is where is seems to hang.

Here is Apples stance:

Scripting Language Runtimes

Deprecations

  • Scripting language runtimes such as Python, Ruby, and Perl are included in macOS for compatibility with legacy software. Future versions of macOS won’t include scripting language runtimes by default, and might require you to install additional packages. If your software depends on scripting languages, it’s recommended that you bundle the runtime within the app. (49764202)
  • Use of Python 2.7 isn’t recommended as this version is included in macOS for compatibility with legacy software. Future versions of macOS won’t include Python 2.7. Instead, it’s recommended that you run python3 from within Terminal. (51097165)

https://developer.apple.com/documentation/macos-release-notes/macos-catalina-10_15-release-notes

So - perhaps it might not be worth to spend too much effort into trying to use the on-board Python but bring the Python that we need with us in an installer? Then it would be in our hands what to use.

If I understand this page correctly, then 3.9.1 is the first Python that runs natively on M1 silicon and thus 2.7 runs in Rosetta anyways? Which would mean for a native build we’d need to get it to run with Python 3 anyways(?)

yeah, thats all expected…
but lets face it, the reason Apple are including 2.7 is they know, lots of bigger customers are using it.
my suspicion is… they will continue to include it until suddently they hit a technical issue that means they can no longer do it (without a big engineering effort)

but for sure, you never know with Apple,
they routinely will just annouce ‘this version no longers supports X’ :slight_smile:

anyway, as above, its not that I have not looked at moving to Python 3, or even at removing Python all together - but unfortunately its not that simple.

every module includes a python interface, which has a significant amount of code in it,
code that cannot just be automatically ported, but would have to be understood and re-written.

it also goes deeper than that, many parts of EigenD code only exist in Python
(e.g. even for MEC, I had to re-write some python code that uploads firmware into c++)

I was actually looking at this 2 days ago… and it really became clear, Python is here to say in EigenD…
it would be easier to re-write everything, than to extract it :frowning:

my belief as stated previously is really two long-term directions…

a) EigenD
remove any UI dependancy on python, leaving only the module interfacing
move to python 3
hopefully this means EigenD is not dependent on any other python packages.

b) MEC / EigenLite / Similar
pure C++ no dependancies, modern tool chain, that can be easily maintained going forward by anyone.
(this is partly why MEC and EigenLite are separate - to give multiple solutions)

1 Like

Architectures in the fat file: /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2 are: x86_64 arm64e

anyway…

kind of the point of this thread is to move away from (my own) speculation/assumption,
and actually start trying things out so we can get to some solid experience of what is needed for this to work.

obviously the goal is to try to do this in baby steps if possible.
its easy to just say , convert it to python 3, compile natively.

but I already know that move to python3 is problematic, and will take a lot of dev effort.

2 Likes

Moved my posting to Some future Eigenharp software thoughts/fantasies to keep this thread on topic.

2 Likes

that seems like sometihng to discuss on this topic…
https://community.polyexpression.com/t/some-future-eigenharp-software-thoughts-fantasies

you are totally correct, we all have limited time - so I have to allocate my ‘dev’ time to whatever will be most fruitful to most users.
that is part of my hope is that MEC/EigenLite is much simpler for other devs to get into, thus reducing the dependancy on one single developer (me) - that is something we can explore in the above topic.

however, I do think that EigenD has alot of functionality, and that some users would really miss it. so, Id really like to not ‘abandon’ it unless it becomes obvious its a huge time sync - which im hopeful its not. (*)

anyway, perhaps lets use this topic to focus on EigenD on Apple Silicon, either via Rosetta or Native :slight_smile:


(*) again, Im trying to manage expectations here… there are complications with EigenD/Python versions - its not trivial. however its not all doom n’ gloom, its quite possible that I will have a ‘break-through’ and suddenly things will fall into place, and then progess will be rapid.

so, my main issue is… just its a bit of an unknown time sink… last time, I spend a few days and didnt get far - but who knows, I could have just been a few hours, or days (or weeks!) off the breakthrough I need.

(my main mistake was leaving it, I should have just thrown a few hours at it , here and there… thus limiting frustration, but still made progress)

2 Likes

ok, first … Ive edited the ‘try in Rosetta’ post…
as Ive realised for this experiment we do not need any python packages…
(they really are only needed for EigenBrowser/Commander which wont work anyway)

BUT… the better news is this…

I’ve managed to compile ALL of EigenD natively on Apple Silicon. (*)
everything seems to run, arm vsts are even loaded !
logs actually look surprisingly healthy.

Ive not plugged in an Eigenharp yet… thats my next step, but looking hopeful given my mec tests previously.

you can see a few graphic anamolies in the UI.
not sure, if this is big sur/m1 related, or possibly even my monitor
(Im using a very wide screen resolution… might try something in normal aspect ratio)
my suspiscioin is a new version of Juce will probably fix that (**)


(*) the only module that is not compiling is the convolution engine, and due to to needing a new version of the fftw library, which supports NEON as well as SSE,
I’ve done this on linux, so it should not be a massive move, to change this on macOS.
really its just about building the lib, since on linux, I just make it a dependency, as it always installed anyway.

(**) its quite a ‘jump’ from where we are - to the latest juce version… so that might cause other issues, so its a bit on the back burner, I’ll look at it once Ive tested hardware is working!

2 Likes

holy smoke…

its working… and running exceptionally too…

Ive got midi coming out, and the synth rig is playing with its reverb - very little cpu!

vst host is not working yet, but given they scanned - Ive a suspicion i know what the issue is…

simple performance test …loading alpha 3 setup

mac mini m1 - 1 min 43 seconds
mac book pro i7 2.5ghz - 3 min 26 seconds

twice as fast !!!

lets remember this is really not showing off the M1, since the M1 does NOT shine in single core performance - its the fact it has 8 cores (4 performance, 4 eco)

in practical terms, this means the Mac Mini M1 will really shine if you want to run EigenD , and then have a Daw/GigPerformer also running on the same machine - since it has a ton of resources left over!

@keymanpal , given this is basically working… there is no real need to test under Rosetta.
I need to check the packaging/install process is working , and fix the vst hosting.
once thats done I’ll get a copy of this native version over to you

6 Likes

Great awesome news!!

1 Like

Im on a roll today…

VSTs are now working, and also fixed the packaging… so feels fully functional.
so its ready for those brave enough to play :slight_smile:

Ive a feeling, I will need to update Juce before this is ‘released’ in any kind of public form, but if you are familar with EigenD and have an M1 to test on send me a DM and I’ll supply a link (sorry this is not the release to ‘learn eigend with’) - just about to send it to you @keymanpal !
you do not need to do anything, other than install the package - hopefully it should ‘just work’.

vsts - we can only host native arm plugins.
amusingly, the only ones I had installed were the ones I had developed :slight_smile:
but I then went and found that u-he, vahalla and modartt (pianoteq) , all have eleases or betas of thier plugings…

all of these appear to work when I tried :slight_smile:

known issues:

  • I think the graphics library we use (juce) is struggling a little , probably more with Big Sur than the M1.
  • convolver missing - so thing like cello will not work

limitations: (aka unlikely to change)

  • can only host ARM plugins
  • eigen commander/browser will not work
  • you’ll need to copy samples manually etc if you want to use these.
    (I do not really recommend installing eigend resources pkg which contains these)
4 Likes

Late to the party here, but that’s awesome work Mark!

1 Like

Pico and clean install is working - thanks to the intrepid @keymanpal for testing this :slight_smile:

this morning, Ive also tested the Tau (and Pico) and can confirm this is working too.

1 Like

– test for longer sessions, check stability
I find it quite stable, playing, interacting with instruments.

– drummer/sampler - need to load some samples.
WORKING great! (just drop files manually)
**Clarinet is also working **
NOT Cello (not sure why - convolution?

– headphone output (tau/alpha)
OK!

– mpe for midi
In a similar way to VST (tested with Arturia Pigments in Stand alone mode

– mpe for vst plugins
It was one of the first things I’ve tried using “SURGE” - pitch bend, matching 48 in both
EigenD and Surge, but needs more testing

M1 MacBook Air (16Gb ram version)

Loading times:Alpha - Factory Setup

  • 39s - 1 Split Standard Setup
  • 45s - 2 Split Standard Setup
  • 81s - 3 Split Standard Setup

STAGE on iPad 1 - StIlL works nicely :wink:

2 Likes

@keymanpal to confirm you think mpe is working ok, on plugins and via midi? (at least with basic testing?)

yes, cello and clarinet will not work, since they use convolution - which is currently disabled.

1 Like

phew - that took longer than expected BUT…

EigenD is now running on the latest version of Juce (6.0.8) - this seems to have got rid of all of the graphics glitches. yay!

only one minor issue when I shutdown Workbench I get an exception which I need to look into, but this does not affect functionality.

@keymanpal , sending you a copy now :slight_smile:

2 Likes