Joué Play and Other Stuff

Are there still users out there? I have a Play and I’ve enjoyed it quite a bit; so much so that I made a helpful MPE template for FL Studio. In addition to that I’ve taken it apart to see what’s under the hood: surprisingly solid wood and solid construction. The sensors look like nothing I’ve ever seen before.

At this point I’m considering doing some custom modules despite the lack of a SDK or spec. You can use RFID to harvest a little bit of energy (up to 3V at 10mA ) to charge a capacitor to run LEDs. The dream is something like the Linnstrument inside a module, having each touched point lit up. The big obstacle to this is MIDI information is not being transmitted and you need some sort of touch sensor, however crude, to actually trigger it; all the while it must be a very low power circuit. In any case it looks like the community around this is kinda dead so it would just be for myself.

Anyone else do some fun things with their board? I’m currently adding magnets to mine.

I think @keymanpal has one. No idea whether the existing modules are hardcoded in the firmware or more an (internal) configuration of existing building blocks that could be mixed and matched?
Perhaps if you ask the makers nicely they might provide you some material/information?

Hi there
I upgraded mine to a pro and use it a lot. Magnets would be a good addition. I don’t have the skill set to play about with it, but I’m interested to hear what you are doing so keep us informed :slight_smile:
I tend to use Equator 2 and Cherry audio MPE VSTs on Bitwig.
which are good fun.

I had the original one, long before they decided to offer separate Play and Pro versions, with a Pro firmware upgrade path for the Play.

I hardly used mine once I discovered that the pressure sensors were no good at all for polyphonic expression - the pressure applied by each finger interfered very badly with eachother, making a complete mockery of that side of things. I hoped it was just a firmware issue but when I asked on the official forum that they used to run years ago, they said it was a sensor limitation. I dont know if the ‘Play’ version has the same issue.

I have dug mine out again recently though, because its absolutely my favourite controller for MPE Y gestures. So I decided to accept that its useless for pressure, and to just use it in situations where I want to focus on pitch and Y gestures.

2 Likes

@NothanUmber I’ve perused the memory blocks on the modules and I’m seeing something resembling an XML structure or at least sets of nested parameters. While the patent and fcc documents note that the areas could be denoted within the memory, I think it’s hard coded in the firmware and the module is just storing parameters for the module type. There are settings for each element but I think the elements are hardcoded as areas in the firmware.

@matthewwood So far, I’m stuck on one of the blocks that code a feature set in the RFID memory. My hope is I can force the matrix pads into a glissando mode that we all so desire, and get a knock-off linnstrument. I can make the pads hold the note but they don’t want to pitch bend… yet. The magnets have worked pretty well but sadly I made my board a bit creaky since the sticky underside of the sensor isn’t… sticking as much.

@SteveElbows Yeah, I saw that being mentioned in old posts on the kickstarter. I’m not sure what the problem is because I looked at the aftertouch CC output when playing 8 notes and didn’t see any interference. The magnitude seemed a little off across the board… maybe. If the range was a little wider I might be able to know for sure, but I don’t have enough dexterity yet to be certain of my input. Maybe they fixed it in the play?

EDIT: Oh yeah, I got my Raspberry Pi Zero W working with the board so even if they don’t release the fabled wireless adapter, you can still play over bluetooth. Of course, if you get a Re-connect then you can plug in any old MIDI wireless host but I’m not brave enough to take mine apart. The only trick is, you need to convert it to USB A before going to the RPI. So Board → USB C to USB A → USB A to Micro USB → RPI. I tried other adapters and combinations but they didn’t work. I think if the CC/PD pins are connected the board shuts down. I’ve been running the board the the RPI on a 5000mah battery for the last 3 hours without the charge dropping so this seems like a great thing to DIY.

1 Like

The test to see if the pressure aftertouch issue I mentioned still exists is fairly easy:

Set device to send polyphonic aftertouch or MPE.

Use a midi monitor, or a synth patch that has really simple, intense, obvious aftertouch parameter controls, and then do the following:

Press one finger on the surface with maximum force, and observe aftertouch messages being sent with the highest possible value.

Add a second finger from your other hand, and gradually increase the pressure applied to that finger, while keeping full pressure on the original finger too.

On my device this cause a ‘tug of war’ - as pressure of the second finger increases, the aftertouch value from the first finger starts to fall. And when both fingers apply maximum pressure, the aftertouch values for each finger end up being about half the maximum aftertouch value. And it makes no good difference as to how far the fingers are apart - even if they are at opposite ends of the playing surface I still have this problem!

I would really quite like to know if their later product has this flaw, thanks!

Oh, thanks for that setup. Yes, I checked and there are effectively 4 horizontal strips and 12+ vertical strips that record aftertouch, based on the dimensions of the matrix pad. I was strangely able to fiddle with the exact placement of my fingers to get the full aftertouch values while in the same “general” horizontal plane so I think there are 8 horizontal strips, with even and odd strips being shared for the 4 horizontal “areas.” This is on the Play version.

You know, this honestly doesn’t sound like a sensor issue as much as a firmware issue on how it calculates pressure. That is to say, I believe Joué could fix it if they wanted to. Yes, there is interference along these dimensions, but there seems to be two colliding data streams rather than one “average”. I believe multi-touch screens have this issue but the firmware calculates the true values from interfering data streams. Plenty of libraries are out there that do this automatically

What’s more is we have the note information and this is repeatable. There might be a slight loss in the granularity of pressure, as bit-space is divided between the two inputs (i.e. 0-64, 65-127), but this can be compensated with an inertia function: providing infinite granularity based on a time-scale change. So basically, we have a script that is triggered when there are two or more notes and the aftertouch values suddenly drop, signaling a collision of data. This starts an inertia function which then calculates the true values by looking at the number of inputs, increasing it by some appropriate factor, and subtracting it or adding it based on the last inputs.

The really tricky thing about this is if there are two colliding notes, and one free note, you have to avoid applying that function to the free note somehow. I’m thinking the time-resolution will be appropriately larger so you “whitelist” an input from the function if it shows an appropriate magnitude or time-resolution. The last tricky thing is dealing with the notes slipping in and out of colliding, but the inertia function should have that in there.

Again, I am sure touchscreen libraries have functions to deal with this exact issue, but I don’t feel like learning an entirely different codebase to make a MIDI script. I think something in python should be able to handle it without too much trouble but who knows. Would you use the script if I wrote it? I can’t promise something bulletproof but it’s better than nothing, right?

Thanks very much for the thorough investigation!

I’d never investigated the sensor array myself, but when I first noticed the details of the problem my ‘programmers brain’ did assume it was a firmware issue, so I was absolutely gutted when I received the useless response from them about the issue on their old forum. I didnt want to get stressed about it so rather than bang my head against a brick wall, I just put the device back in its box and mostly forgot all about it for several years. And then at some point I think they deleted their old forum where this communication happened, so I dont even have reference to that bit of history now.

I hate it when companies dont try to sort their firmware, all that effort on product development and then they mess up in some other ways! More reasons to love the small world of open source firmware, eg the Linnstrument. With the original Joue Im not sure if they ever released even a single firmware update for it during its lifetime, I know the software editor got updated sometimes but not sure the firmware ever did.

I suppose another reason to wonder if it was just a firmware issue is if ‘initial note velocity’ calculations are still ok. But I dont use note velocity much myself and never bothered to look into that.

Anyway yes, I would use your script if you wrote one, thanks so very much for investigating this possibility! But dont write it just for me, only do it if you have a clear use for it yourself! I’m already using my own dodgy midi processing script with this device at the moment, because I wanted to convert the MPE Y CC74 messages into aftertouch messages for use with certain hardware synths. I have two versions, one which takes MPE mode data from Joue and keeps it as MPE, just converting CC74 into the appropriate midi channels channel aftertouch. And another one which I use with synths that dont support MPE but do support polyphonic aftertouch messages - this version still takes MPE from the Joue, but converts to single channel midi, monitors the active notes on each channel and turns CC74 from each channel into per-note polyphonic aftertouch, and uses ‘most recently pressed note’ to choose which pitch bend message to retain. But I just botched this stuff together using webmidi.js and my own small amount of javascript code, and then run it in a browser. Well year ago I may also have run it using node.js but I dont really remember. I dont mind whether your solution is python or javascript or something else, but I would probably like to run it on a rasperry pi eventually, as part of a permanent small solution that doesnt need to use my main computers.

1 Like

Ah, it’s great to hear someone else with a programmer brain working with this device. I’m not much of one, but this script is an exercise in my first open source project.

I’ve been testing cases for building out the logic and hoo boy there are many cases, some of which have no real solutions atm. Basically, your last held note blows out previous notes, whether reducing their magnitude or silencing them entirely. This means holding two notes will reduce the first pressed note greatly, while the second note will be roughly 1/3 its magnitude. Each additional note reduces the range of aftertouch values for previous note, and after 4 notes it’s very difficult to get any output. What is even more annoying is pressing hard will completely blow out the other notes aftertouch data, so with each note, you have to press lighter and lighter to not blow out their output. You might have already discovered this with your discussion with the manufacturer.

What this all means is the best strategy is to use values 0-30 for aftertouch data and throw out the rest. This reduces the dynamic range of aftertouch for single notes but allows each key to be sensitive, up to around 4 notes in the same horizontal zone (as the player adapts to this reduced range). Alternative strategy one is to detect colliding aftertouch data (this can be detected by a sudden drop in aftertouch output for a note, followed by another note on event) and use its output for all notes, until the collision stops. This means single notes will have a lot of dynamic range, and you can modify your playing to play across all 4 horizontal zones to avoid collisions. Alternative strategy two is to detect the collision and replay the last inputs. This might be interesting, especially coupled with a weight factor and inertia, to add a lot of dynamics to all the notes. You should also be able to adapt your playing to make ad hoc LFO effects.

Ultimately, we are still limited to 4 notes per horizontal zone, and 4 horizontal zones, but judging by the play-styles of many MPE players, that might not be a problem (also probably why Joué decided to set up their boards this way). My main critique of Joué is that simply making the limits transparent to players like us would have gone a long way in fully utilizing our boards, regardless of firmware limitations. On that front, I tried finding a whitepaper on the multiplexer chip that takes the board input, to maybe see if that was the source of the problem, but I came up with nada.

I think I might make a separate thread for ongoing discussion of development, if nothing else to keep me sane while I grok a new skillset. I think my first target is my idea for the “best” strategy, and apply some inertia function to help make up for the lost dynamic range. I’m kinda impressed by your js script so it’s inspiring to keep working on this project. I intend to keep the script extremely simple and performant so I can use it on my rpi 0 which takes care of the wireless connection currently. I chose python because I can use it natively in FL Studio as an instrument handler, making development a bit easier, otherwise I would love to do it in js.

Hey ya’ll, just had a cool idea. Technically something like the Joué could be made with an led array behind it (similar to the Erae) so wouldn’t it be cool if a Joué v2 had a DAW interface when you don’t have a pad on top? I’ve been wanting something like a groovebox but the Joué has enough lateral accuracy without a pad to have a decent interface. I don’t know if I’ll ever have the time, energy, or money to innovate, but I would love to see a product using the same idea behind the Joué, but usable in standalone. :slight_smile: Maybe something like a miniature projector would suffice, haha.

I’ve been experimenting with projection mapping onto the sensel morph. It works fairly well once you add the inventor’s overlay, and better still with a white sheet of paper.

I haven’t plugged my Joué in in a while, but last time I checked, there was no way to get any kind of sensor data out of it without one or more of the officially recognised overlays covering its surface. And none of them provide a big empty space to project into. So, my gut feeling is that this would require new hardware. (Or an open sourced firmware.)

Joué v2, as you say.

I’d give up on the miniature projector for this.

That’s sick. I’ve been working on the processing side of things, but it’s possible to cut out a usable projection surface, stick a NTAG213 RFID chip on the back, and copy the map data of one of the pads into it for an easy cludge. I was trying to reverse engineer the XML structure to make an arbitrary mapping, but it’s unclear to me yet if the physical map is stored within the Joué. When I’m done with the aftertouch script project, I’ll seek to answer that question. With any luck, I can tease apart the sysex messages and get the script to put out custom mappings.

1 Like

I don’t think the SysEx includes those definitions. If I remember correctly, it’s just “which overlay, what serial number, and which sensor was it detected on”.

I imagine a white neoprene rectangle would be ideal, once that part’s figured out.

Not sure if this helps, but I could maybe send you sysex dumps for placing an XY pad on each of the three slots.

You might have to massage the input to merge those into a single surface, or just add two black lines to the overlay.

And then, I guess you’d set those to MPE, each with a unique range of five channels so they don’t clobber each other? But I’m fairly sure you’d need the pro editor, to configure the base controller itself.

1 Like

I have the pro editor :slight_smile:

You’re right in that the sysex messages from placing a pad only contains that information. What I’m seeking to do is understand what is transmitted via sysex file dump message when assigning a configuration to a pad. I believe it corresponds to the XML configuration in the Joué editor but the hex data from the NTAG213 within the pad is a bit ambiguous. It’s structured similarly, but it also has extra data which unlikely could be physical mapping. My biggest barrier, besides time, is getting Windows to name the virtual ports, ugh.

I suppose the easiest thing to do is use the X-Y pads as a base configuration and then map the input within a script. If the rfid tags are partially shielded and offset from the center, it’s possible to override the the “base” with an instrument pad. So yeah, I guess it’s totally possible to implement that behavior. :smiley:

1 Like