EigenLite 1.0.0 dev testing

Status: In Development

(api subject to change)

EigenLite Repo Branch : dev

Features:

  • device filtering
  • device notifications
  • new connection/re-connection logic
  • embedded firmware
  • no runtime requirements

Platforms:

  • MacOS - arm64/ x86_64 (now available in dev)

Linux I will add when there is a use-case/demand.

Description

The focus of EigenLite 1.0 is to make it much easier to integrate with applications and for users to have NO additional setup steps, just install the app and go.

also, as part of this, I wanted to support those with multiple Eigenharps, by allowing different apps/processes connect to specific eigenharps.
this was not possible previously as EigenLite would connect to ALL eigneharp’s that it found. with 1.0, you can specify a filter to limit connections to specific harps.

Breaking changes

there are a few breaking changes.

  • Firmware readers are now passed as pointers, not references as they are optional
  • types in eigenapi.h have been moved to base c++ types, so no include are necessary.
  • callback::device has been replaced, connected() is a close substitute.
  • callback breath - change val signed (to do)
  • tau mode keys (course 1) , id change

Testing etc.

the connection logic has changes quite substantially, so I’ll be keeping an eye on this whilst this is in the dev phase.

Stability of API

Im using and have been testing this version , but it is in development :wink:
this version is still in active development, so changes are still quite like.

note: I will not develop 0.6 further, so the USE_DYNAMIC will only exist for full release… so consider a 1.0 feature … that just sneaked into 0.6 for Mac/arm.

4 Likes

ok, Ive added x86_64 builds to EigenLite 1.0/dev

@Kai , you can give this a go with ECMapper… let me know if you run into any issues.
should only take a few mins to update to api to give existing functionality.

this would be very useful to get some more extended testing from users - in particular, so we know there are no issues (particularly with new connection logic).

from there…

adding device filter is easy to add, though likely not a priority, as EigenCore is (presumably) fine getting messages from all eigenharps and then distributing to multiple ECMappers!?
i.e. you don’t need to run multiple EigenCore for different harps, just one EigenCore → to N ECMappers.

however…
it could be handy for some ECMapper users with multiple Eigenharps, and they want to use alongside EigenD/other EigenLite apps on the same computer.
e.g. if I want to use my Pico with EigenCore/Mapper, but my Alpha with my new project.

its a simple add, basically, you just need to have a UI in EigenCore which says, do you want to file r… if so what (Pico/BaseStation) and which device… that’ll then limit EigenLite to attaching to that single Eigenharp.

1 Like

no another note… but related…

currently, Im supporting both dynamic/static libraries and embedded and multiple firmware readers.

but frankly, Im starting to wonder if we just stick to static and the embedded firmware?

theoretically, using file firmware reader and dynamic eigenlite/pico encoder means they can be updated without a new release of the client app.

however, we know:
a) there will never be a new Eigenharp firmware (ihx)
b) users will never update EigenLite without updating the client app

so its kind of functionality thats not going to be used, and for developers using the embedded firmware and static linking is just so much easier for distribution etc.

thoughts? have I missed something?

obviously we dont have to make this decision now… and of course, we should wait till we have a few releases under out belt, to know there are any oddities (e.g. different OS versions),
but Id like to consider it… KISS.

1 Like

a couple more breaking changes.

(done/fixed ) tau : mode keys
had incorrect id, now start correctly from 0

(todo) breath
will become a signed integer value, to better reflect usage.
Im also going to calibrate a zero point, and adjust scaling.

bigger change possible…

Im considering changing all the callback values to give float values
so, 0…1 (unipolar) and -1.0f / 1.0f (bipolar) , rather than send the raw integer values.
this would allow EigenLite to potentially find better scaling/calibration, without clients apps having to worry about it.

basically, like breath, I think theres some possibilities for improvements here, and it makes no sense for me to calibrate raw values in floating point, and then convert them back to 0 - 4096 to ‘pretend’ they are raw…
also I think generally floats are easier to work with (scaling etc)

1 Like

I’m positive. I can’t think of a reason why EigenLite users should need to know or care about the ihx files or picodecoder. So sounds like a simplification without any practical loss in flexibility to me. I struggled a lot with the dylibs/paths and how to properly bundle everything until I eventually learned about @executable_path, @loader_path and @rpath. That might just be me and my lack of Mac experience, but still, one less hurdle for the next newbie to tackle…

I like the idea of unipolar/bipolar floats. Easy to work with and to intuitively understand. Only thing I can think of, do you think there might be some value of EigenD/EigenLite “raw” data being exactly the same? E.g. EigenLite sending raw data into EigenD by whatever means and then have the rest of the EigenD setup behave just as if the device was internal? Or the other way around, where an app using EigenLite could receive that exact same data from EigenD instead? I don’t know how the data looks in EigenD, so wouldn’t know if this even makes theoretical sense.

Yeah, that was my current plan, at least. …but, I guess I could still use device filtering. Perhaps an advanced config (a.k.a. @keymanpal mode) with an editable list like this?
Enabled   Model   #   IP        Port
       Pico    0  127.0.0.1  1234
       Pico    1  127.0.0.1  1235
       Alpha   0  127.0.0.1  1234

4 Likes

yeah, rpaths are a bit tricky…
I do like using shared libraries generally, but as I said not sure adds much here.

bipolar/unipolar.
debate really is how much EigenLite is just a thin wrapper around eigend code vs how much it adds value and is easy to use.
but even then, its questionable… EigenD converts this data into bipolar floats anyway, so one could argue, it’s just where do you stop the abstraction.

overall, I think the benefits of a ease of use outweigh the theory of just give me raw data.
why?
basically because, Ive got a pretty good understanding of how EigenD massages this raw data, and I know the EigenD code well enough to check we get a similar view…
whereas other developers likely don’t want to go fishing around EigenD to find this information.
also… do we all want to reinvent the wheel in our client code?

I think the pitchbend/breath handling provides a good example where just taking these raw values and converting will not give you the same ‘feeling’ as EigenD unless you know a bit more about how EigenD handles these values.

so, overall I think its a move in the right direction :slight_smile:

device filtering … yeah, as I said, I think the main benefit of using the Device Filter with ECMapper EigenCore is if the users wants to use another application e.g. EigenD or my new App alongside ECMapper.
Currently this is not possible as EigenCore/EigenLite 0.5 will open ALL eigenharps and so make it impossible for another app to open and eigenharp. (as usb open is exclusive to one app)

so I do think is useful… and as I say, its very simplistic, all you need to do in EigenCore is two selectors:
Filter:
Device Type : All, Pico, Basestation
Device Number: 1…N

this maps onto api as

Device Type = ALL → setDeviceFilter (false, 0)
Device Type = Base, DevNum → setDeviceFilter (false, DevNum)
Device Type = Pico DevNum → setDeviceFilter (true, DevNum)

I could have made the filter more complex,
but honestly I see the main use case as linking an app to ONE specific device (advanced use)
OR just connecting to everything. (normal/default)

implementation note:
currently I only evaluate filter on device connection…
as I didn’t really want to start disconnecting devices when they fall outside the filter :wink:
but may review this later.

lets also remember, there are only a few of us with multiple eigenharps, and only a subset of those actively use multiple on the same computer at the same time :wink:
(which is why the current, no filtering, has not really been an issue for most)

1 Like

Sounds reasonable to me. Making things as easy as possible, but without crossing the line into making assumptions about what the user wants to do with the data. As an example, providing the Y axis data as a linear, bipolar float sounds good to me. But assuming that the Y axis will be used for pitch and apply a curve that suits that use is a decision better left to the user application.

So, I guess my preference would be for EigenLite to provide “linear” data that feels like a neutral starting point for further processing. Which probably means quite some massaging - at least for breath.

I also think sensitivity and behaviour close to the same, regardless of model would be useful. Iirc, at least breath sensitivity is different across models. EigenLite taking this into account and adjusting for it could potentially be of value. Someone with a only a Pico to test on could implement breath and have confidence in that the behaviour would be almost the same on an Alpha.

Hmm. Come to think of it, it doesn’t have to be either/or… Perhaps one consideration regarding simplicity and not having to reinvent the wheel, but still keeping EigenLite “raw” could be to add an example application with a set of utility functions for commonly needed stuff like scaling, velocity calculation, pitch bend curves, breath scaling and threshold, etc. Even conversion to floats, I guess… If you decide to keep EigenLite as light as possible, some of the calculations we are already doing in ECMapper and MEC could be copied to such an example project as suggested use. Sounds like something I’d be happy to assist with a first version of if that is the direction you want to take things in.

3 Likes

EigenLite is going to remain simple… its designed to be just an abstraction of the hardware…
its not going to be ‘opinionated’, rather just deal with the quirks of the hardware.

its focus is low level communication and prepping the data for delivery.
(preparation is limited to ‘calibration’ steps. really… the changes to the api are just to make this easier)

this was, and remains the goal of EigenLite…I do not want to extend the api beyond this goal.

so, as for how the data is used/interpreted that remains up to the client application…

could there be another layer/library that adds higher level abstractions , possibly…

however, Im busy working on a new project at the moment, this project has a (very) broad scope for Eigenharps.
it’s already doing a good job of highlighting what’s needed to get to EigenD levels of functionality.
this is why Im ‘poking’ at EigenLite a little bit as needed, however, my main focus is on this project.

once the project is ‘complete’ (in some form), I may take a look at which parts could be re-used in a more generic form, perhaps as a layer above EigenLite (and that layer WILL be opinionated ;))
… but its too early to commit yet, as its a big project… though progress is very fast so far!

3 Likes

bug fix: enable ‘tau mode’ on processing loop
main difference is led on mode buttons are now addressed correctly.

1 Like

BIG UPDATE (including API changes)

Ive done what I proposed above and now pushed to the dev branch.

this is currently being tested in Meta Morph, and looks very good !

I consider this a major improvement, that all should move to.

basic changes :

  • the main callbacks are now floats, bipolar and unipolar as appropriate.
  • a lot of improvements to values being sent by pico/tau and alpha.
    (there are a myriad of other changes, but hard to track them if Im honest!)

its relatively easy to migrate to the new api, as the the new api simplifies how applications use Eigenharps, so I found mostly I was removing code from the application.

Release Plan:
Meta Morph is being used as the primary ‘test’ for the new code, and will go into Early Access very soon, I will resolve any issues found during that testing.
Once, I’ve enough feedback that there are no major issues then EigenLite 1.0 will be released and moved to the main branch.

5 Likes

more changes to EigenLite (dev)

  • strips are now bipolar
  • pico’s strip is now less quantised
  • leds are cleared on stop()
  • Tau and Alpha now have percussion keys on separate course,

Pico
course 0 - main keys , 0-17
course 1 - mode keys, 0-3

Alpha
course 0 - main keys, 0-119
course 1 - percussion keys, 0-7

Tau
course 0 - main keys 0-71(iirc ?)
course 1 - percussion keys, 0-11
course 2 - mode keys, 0-7

2 Likes

Wow, so I am finally digging in to EigenLite and MEC and it has refreshed my desire to do some Eigenharp development. The original EigenD chewed me up and spit me out several times.

While learning about the library (dev branch) I noticed some more chunks of code from the pared-down eigend parts that are no longer used, so I opened a PR for those.

(BTW the dev branch does not build out of the box for me on Linux. I have some fixes but I’m not sure they’re correct, I need to look at the pico decoder situation more closely.)

Also, when running eigenapitest with a Tau plugged in to a pro basestation, I get swarms of pedal events (at full scale) peppered with usb errors (log:usbpipe_in_t::completed not completed packetLIBUSB_TRANSFER_ERROR (1) len = 512 actual= 0). Is that normal? Oddly, there are no usbpipe_in_t::completed unsuccessful messages.

1 Like

This is too cool… while I wait for Meta Morph, in very little time I whipped up a litte proggie to dump tab-delimited event data and pipe it to kst2. Bam, live oscilloscope of pitch/roll/yaw as I play the keys! Never got close to being able to do that with eigend.

Now I can see if I messed up the calibration of the percussion keys on my Tau when I opened it :smiley:

1 Like

sorry, this PR is not going to be merged.

from the outset, my aim was to use the EigenD code ‘as-is’, this means that in the event I find I need to fix it, I can just move the code freely between the two projects.
this is not just theory, as I’ve had to do this a few times in the past.

this would only change, if I decided that the EigenD code was ‘dead’, but we are not there yet.
as, frankly, whilst we have a few alternatives to EigenD, and for sure, we have some compatibility issue, for some users the alternatives are not a ‘complete’ replacement.

though, as I’ve said previously said, increasingly EigenD code base is become difficult to support, and my attempts to move to a more modern codebase (python3 in particularly) have proven to be very time consuming and difficult.
BUT, at some point, this will reach an ultimatum where I will have to decide either to push forward on EigenD and get it onto a more modern codebase, or call it a day.

BUT until then, this ‘shared’ EigenD code will remain the same between EigenLite and EigenD.


as for using dev branch…
as the name implies it not released, so may have some areas which are not updated as necessary.

if you have an issue with using the picodecoder (the library rather than the ‘fake’ one’), then mostly likely this is due to me not compiling it for the specific architecture you are using. esp. if you are using the new static library form.
(frankly, few use linux, so I tend to only put the effort in, when Im close to the actual release)

as mention earlier on this topic, this part is a work in progress.

I’d also point out that the api in the dev branch is still subject to change, partly because developments on MetaMorph are influencing some (much needed) changes - to make EigenLite responsibly for a bit more of the ‘data massaging’ thats needed.


no this is not normal. you should not seen usb transfer errors.
given the reference to libusb, Id assume this is under linux?

if its under linux, then yes, Ive seen this a few times, caused by particular kernel versions of the usb stack. I often encountered this with various SoCs where for a few kernel versions it’d work, then update it, and it stop… then again it’d start working again.
(that said, I think it always worked on desktop linux, at lease when I tried)

Ive done some pretty low-level debugging in the past, and my code and the libusb library appears to be doing exactly the right thing, but occasionally we just get back empty packets.
(as you can see in error, the packet is supposed to be 512 bytes, and its got zero data in it)
also, I found even the same/similar versions of the kernel, it’d work on some platforms but not others.

I did report to the devs of the kernel, but frankly, they were not that interested in it given they felt it was device specific (which in fairness, it does seem to be).
anyway, it seemed, to get fixed at some point, due to other updates/changes - but its not clear if the underlying cause has really been fixed, so could come back in later versions, or on different platforms.

anyways, this is one of the issues with Linux, there are so many variants - different hardware platforms, and different kernel versions. its difficult to support / test.

important note:
eigenapitest (and others) are just test programs, and actually make quite a few ‘simplifications’ to allow them to be readable, and so kind of ‘document’ the api.
in particular, its important when using EigenLite to consume messages at a guranteed sample rate of at least 3k, usually you would do this by tie-ing to you sound card or similar, which has an audio rate clock. this is not done in these test apps as it would increase their complexity considerably

to, although, Id say this is not the cause of the above error, id also not consider the test to be ‘reliable’.
… though in practice, it does usually work very well :wink:

1 Like