I sort this under software, but given the hardware it runs on I suppose it’s a bit of a hardware topic as well for now.
Mod Note: Ive moved to hardware as a more general topic on the vision pro, as I think theres an interesting discussion on its usage in music, and galaxy is (hopefully?) just the first application in this area.
some more context:
Looks interesting, fingers crossed they also release this for Quest 3!
I’d suspect this is ‘unlikely’… the advantage for the Vision Pro, is that its basically another macOS/iOS variant, and has the same computing power.
the quest 3 is basically a mobile chip, also frankly its pass thru and hand tracking is no where near the Vision Pro.
as someone who has recently got into VR, I think the Vision Pro is a fantastic glimpse into the future.
however, there is no denying, its price will prevent it from become widespread, really its position in history (I suspect) is to pave the way for a future version, (undoubtedly) stripped down.
looking at iPhones, Id say in the $1000 region.
“but there are synths that cost more, its no more than a laptop”
indeed, in fairness, I don’t question its value for the tech at all… I think it’s reasonable value.
but its not got the application of a laptop or a high end synth, its a bit too leading edge.
of course, Im sure a couple of ‘performers’ will make it work, partly just because of the novelty and being different/experimental… but outside that?
for, sure, in a year or so, with more applications , the landscape may look different,
but by then , going on apple’s track record, we will see a new “Vision Pro” (or vision lite, vision se or whatever) announced.
anyway, its great to see Moog/Geert pushing this forward, seeing where XR can go with the best tech available - and it’ll make for great videos, showcase technology for sure.
whilst Apple has a very impressive history of bring new tech into the main stream, but Sony and Meta have spent millions attempting to push VR, with little success outside of its ‘niche’.
so this could be Apple’s next Newton 
As for Galaxy, looks very cool … and animoog is great !
best thing about it (for me) is the hybrid approach using a physical controller.
I think the bit of VR that doesn’t generally work well, is the lack of haptic feedback.
and whilst hand tracking for day to day usage makes sense… (one less thing to need), the lack of haptics is a major issue.
related, my favourite VR title by far is a VR (car) racing title, and thats because I have wheel/pedal setup, so I have the proper haptic input… VR just provides the visuals/soiund.
it’s ridiculously good…
and I think the hybrid music hardware setup with Vision Pro, will do the same.
give me a proper keyboard, and just ‘augment’ it with XR.
related, I think one of the most impressive AR apps Ive seen for music making is PianoVision (when used with a proper keyboard/piano)
its one of those apps that effortlessly demonstrates what AR is about, doesn’t feel ‘forced’.
(ok, I dont believe in this way of learning to play the piano BUT thats not really the point here ;))
I agree that the AR bit has way more potential compared to VR. Extra information laid over real physical controls is the only side of things I’m really interested in, not waving my hands in the air etc.
I’m interested in this for knobs and faders, not just keys.
I could also imagine it being possible to use AR finger tracking to give the likes of the Osmose and other keyboards the ‘slide your finger up and down the length of the key’ dimension of expression (that Roli etc have) but obviously accuracy and latency would matter, along with having the right hardware to ensure that the device can actually see your fingers even if you arent staring straight at the keyboard.
Personally having dabbled with VR over the years, Im going to try not to buy any more relatively heavy headsets that have to use cameras to pass through images of the real world. Which means I’ll be waiting god knows how many years till lighter devices that can use direct optical passthrough for vision of the real world are actually ripe, and hope that all the other pieces also fall into place by then.
I sort of expect thats Apples ultimate vision for this side of things, they just cant achieve that with current tech so they’ve had to make compromises. As long as they dont give up in the interim years, and manage to get just enough develop momentum sustained to move things forwards in all the other necessary areas, I dont mind. What I want to see next from them is more focus on really good spatial mapping and perpetual memory of the real world to the virtual world, and gradual improvements to real world physical object tracking with high accuracy, etc. eg in addition to all the stuff to do with tracking physical objects related to the making of music, I want to be able to place apps in specific places around my room and have all those choices stored beyond a single session. Not just highly functional app windows, one day I would like to decorate my room with strategically placed audio-visualiser elements. And obviously on the music side of things I want to permanently attach and map info widgets etc to particular real-world synths and control surfaces, and have the system recognise those again later, even if theyve moved location and lighting etc are different.
When it comes to AR I find one of the features that stand out the most with the Vision Pro is the eye tracking as a means of interacting with the surrounding environment, which I think may also be really interesting for musical expression.
My observation is that most AR applications until now has focused on the interaction of hands and fingers with the evironment. But the hands and arms in AR space has their usual limits, and you have to travel in physical space or perform some summoning gesture to extend it. Except the virtual part, it is quite the same as usual.
Eye tacking, on the other hand, has in general usually been applied as either data collection or as an alternative mouse cursor on regular screens.
I’d argue that in AR it makes complete sense to have the gaze as at the same prominence as hands/fingers, to complement each other and not just as a side note. It is a very natural and fast way of interacting and extending the regular range across f.i. a room, where eyes are not only input but also output.
For music, I think this can open up new ways of interacting with sound dependent not only on looking at virtual controlls to change paramters but also on what or how you look: distance, order, colours, blinking rythms, slow or fast eye movements, sizes…
I am travelling and have not had possibility to really read thuroughly about Animoog Galaxy, so I do not know if or how it incorporates tracking of eyes.
In any case, this is something I perceive as an interesting development with the device! 
(Edits to this post for clarity on AR focus.)
In 2017 I tried the Magic Leap device. It was quite brief and not very deep interaction (I remember a little astronaut, or something like that, walking on my palm and jumping down onto a table among other things) but it has stuck with me as a decisive before and after moment of having seen my living room, and by extension the world, in an entirely new way.
Boxing in screens close to my eyes has not really managed to make an appeal to me, especially since then. To be fair, I’ve not used much VR devices at all so my posistion may be a bit harsh or narrow minded but my point is I also find pass-through the most interesting approach to AR/VR/XR/MR (so many acronyms).
Perhaps Google Glass, Magic Leap, HoloLens skipped ahead too far. I’m not sure, but I perceive Vision Pro to be a step in that direction but from the position of VR-goggles. The Facebook/Meta Wayfarers are pass-through, aren’t they, though?
Regarding eye tracking, the Verges review (I watched the video version of it) of the new Apple Vision Pro brought up some interesting points about that side of things. For them the combination of it using your hands and where you are looking for input got tiresome really quickly, forcing you to think about where you were looking and what you were doing with your hands in a way that people arent used to having to think about in some of those contexts in real life.
Obviously theirs was a general review rather than being music related, and I dont have the opportunity to try this stuff for myself at the moment, but Im not sold on it being a wonderful way of doing things. Yes in a musical context we are used to paying attention to what we are doing with our hands, but Im still not convinced I will want the same burden placed upon exactly where I am looking too.
Yeah, referring again to the review by the Verge that I just mentioned, they think that Apple ultimately want to end up with real optical passthrough and lightweight glasses type device, but know that they cannot achieve their vision for all the other functionality and apps by using that method at the moment, so have had to go for the VR-type camera and screens for this generation, hoping that by making the camera and screens high enough resolution will be a suitable compromise for now. I would like to see what the lovely screens of this thing are like for myself, but I still know from other devices that in order to be worth it for me the device will have to be comfortable so I can use it for hours without wanting to escape the box strapped to my face. So I’m skipping this generation, and probably the next generation or two out. I am out of date with the devices that are already more like wearing glasses, because I just still tend to assume that they lack most of the other functionality I want, and this approach is just not ripe, these two different worlds cannot successfully merge the best of both worlds into one product yet.
Since I keep mentioning a particular review video, I suppose I should post it. A lot of it rang true to me, but without really trying this particular device I suppose cannot fully judge whether they are spot on with every observation.
Also after many years of getting carried away with my expectations, I’ve downgraded what I hope 3rd party developers will be able to offer in terms of apps too.
I should pay a bit more attention to how the APIs Apple offer evolve in future, to see if the stuff I want to do with mapping AR objects to particular physical objects and locations starts to improve and mature in the standard platform.
I’ll have a look in the morning, it’s nearing 01.00 in Ankara!
There will surely be quite alot of calibration when it comes to the ways we interact with a layered digital/physical environment. I do personally anticipate that gaze will have a place there, though, regardless of how intent is read from it.
Perhaps gaze+finger is a bit like the jarring experience of trying a mouse with a GUI for the first time in the early 80’s!
I’ll refrain from conclusions, though. I’ve not tried the Vision Pro either, not even watched the review in question yet!
From what little I know about those they are just glorified crappy cameras and speakers and voice-based smart assistants embedded in glasses, not anything nice involving an overlayed display layer.
This is partly why Im most interested in interacting with real hardware with real physical controls in an AR context. I assume that I will hate using gaze and hand combos for doing stuff for a long time, I bet the mouse experience is still better for that. I wont mind using it briefly for more casual stuff, for brief interactions, especially as some of these newer systems no longer require you to hold your hands out in front of you.
None of that diminishes my interest in having graphics overlays when touching traditional synth interfaces, but this stuff stubbornly refuses to arrive in a meaningful way. If this stuff is eventually done right, I wont complain if gaze detection is part of what makes it work, and any shortcomings in how thats implemented should show up pretty quickly.
I also know very. very little about those ones. My question was more a of question than a statement in the form a question!
I do know, though, that Meta has put alot of interest into eye tracking in regards to their AR/VR devices. I suspect this is just as much for collecting data as for anything else, if not more, if I may speculate.
Yeah.
For actual AR in a more glasses-like form, for now I suspect it comes down to stuff more like the following, and its various competitors, than the Meta Ray-Ban stuff. (I’m also leaving aside the companies that had to adapt and go for ‘industrial use’ markets after initially overpromising/bullshitting and then under-delivering in the wider consumer space, eg Magic Leap).
To be tempted by these sorts of devices, I expect what I’d need to see improve further will be:
The field of view, although I could probably live with the crappy field of view on offer if everything else was great.
Actually making the ‘spatial computing’ element good enough to be worthy of that name. Better tracking, less lag (the tracking and lag in the above demo appeared to be quite bad). Proper detection and perpetual memory of living spaces and objects.
Given time I expect Apple to do better with the real spatial stuff (in part because they dont skimp on computing power) so long as developer interest doesnt wane horribly and Apple dont pull the plug on the entire project. And so we are still left needing to wait for these two different worlds to merge back together, when things are ripe. I know I said I was going to ignore heavy boxes strapped to my face, but I expect that the wait for the best of both worlds will be a very long one indeed, so at some point I could still be tempted back to the heavier goggles side of things if they really nail the spatial & real world object interaction stuff. Either that, or the companies making the glasses will gradually get this side of things just about good enough to tempt me. I can live with certain compromises but the right mix of compromises isnt there yet from what I can tell. It always seems to be ‘at least a few years round the corner’, no matter how long I wait, but meanwhile some of the underlying tech is actually improving, just hasnt reached the promise land yet.
First reviews indicate that the Apple Pro is certainly better than the Quest 3 in pretty much every aspect. But in the end still a (high end) VR headset that improves more than it revolutionizes.
Sure, there will be applications that will fully rely on the additional performance capabilities of the Vision Pro hardware (not sure whether this is the case for the Animoog Galaxy app already honestly). But I think a lot should already be possible with the Quest 3 in the musical instrument direction. Will be interesting what people come up with!
PSVR2 from Sony has eye tracking, and has been used quite a few games, some quite effectively.
play Synapse on the PS5 if you want to feel how liberating eye tracking can be !
the main one, as has already been highlighted, is when we come proficient at something, we build muscle memory to the point where we do not look at what we are doing, so we can multi-task.
Im typing this, without looking at my keyboard, I play piano (poorly ;)) without looking at the keys.
unfortunately, really eye tracking, is like much of XR, its just one part of a solution - but its proven not to be the one ‘missing thing’.
in my experience, THE most important thing missing in XR is haptic feedback, when you have good haptics, it raises the immersion to a whole new level.
honestly, I think music making is (often) a physical experience… its part of the pleasure, connecting ourselves to sound. so for now, I don’t really think XR has much to offer musicians…
I think it will, at some point, but it’s still a bit too primitive (even the vision pro)
I know AR is the buzz at the moment, but for me it’s far from proven.
I think VR and AR are different, and both useful.
VR is more about experience and immersion, AR is more about being in the same space.
AR is really cool in that you are not disconnected from your environment, which can be very disorientating. but that lack of immersion is also misses something.
what I like about the Vision Pro is the way you can move ‘gradually’ between the AR and VR.
it makes AR->VR, less black n’ white, which I think is excellent.
this is something thats actually pretty new, and I think is an important step forward.
overall, I do feel the VR immersive experience could be exceptional for music.
but for now its a personal experience… as you cannot really share this with an audience.
ironically, high tech is not the issue here.
Id much prefer 8 people with Quest 3 sharing an experience over a singe person with a (fantastic) solo experience with Vision Pro.
(gaming also shows this… multiplayer / social experiences do very well, often over single player stuff)
on a personal note, Ive been considering doing some dev in the XR area, but really im not that tempted with Vision Pro. I’d be more tempted to play with the Quest 3.
why?
a) much broader user base
in this space, I think you need a lot of user feedback, users with different backgrounds etc.
you need to be able to iterate, find put what works.
also, see my comments above on user base and sharing experiences!
b) easier ecosystem to distribute
meta have a means (App Lab?) to distribute low volume apps without certification etc.
again, you can iterate faster.
frankly, mid term ( so when we see Vision Lite/SE from Apple) the best VR/AR apps will come not from ‘apple’ developers, but from XR developers, those that have evolved with the medium.
I don’t believe you need to be an apple early adopter to have (mid term) success in this space.
don’t believe me? I’ll lay a small wager here…
the best selling Vision Pro app (assuming its released) will be Beat Saber!
an application that was developed years ago, and is the top seller on every (?) VR platform.
understanding why this is the case, also demonstrates the ‘issue’ XR has, and how often its a solution looking for a problem 
Yeah, just to be clear i am referring to AR here (and interacting with distant digital objects in physical space with gaze), not the VR space of PSVR2.
I have edited the original post for clarity. There are so many acronyms in this nacent space, shifting into each other, I get dizzy!
You’re absolutely right reagarding delvelopment using gaze in VR-games but this, to me and at this present time, is closer to using a cursor or cross-hair on a regular screen (which of course is just fine in itself).
I’d also like to disclaim here that I am personally hestitant about some core conepts of VR and how it is applied. This is just me, though, and things also change.
I too, for realted reasons, also welcome the ability to gradually shift from one to the other. Great!