Using smartphone/watch as controllers (via gyro, accelerometer) for app based DAW

I’m thinking of a gamified music making app that could involve the use of moving phone and/or watch to modulate sounds. Is this something that has been done? Any feedback on concept? Thanks!

I think a few iOS music making apps have gone down this route, using the gyro data from phones.
also there are apps to send gyro data over osc ( probably also midi I suspect)

also, a lot went down the gamifying route - I think Endlesss is a good example of this, and actually the thing I least like about it :wink:

we’ve also seen quite a lot of dedicated motion sensor devices …
whilst a watch/phone is nice because it uses what you have already, the issue is your usually limited to one…
I think something like [instrumentsofthing’s 2.4 sink)(2.4SINK Kit – Instruments of Things) is really the most interesting (to me) as provides more option , its more expandable and although about 550 euros its not cheap, that includes connectivity (eurorack module) and sensors… so not that bad compared to a price of a phone/watch.

so yeah, definitely been “done”

is it useful? well some love it…
but it can be a bit gimmicky, usually its sold more for modulation than being played as an instrument (though thats been done too) , and then the issue is, if you are playing say a keyboard/guitar… your hands can’t just start ‘waving around’ … so its a bit limited.
(this is where IOT sink focus on modular is interesting, as modular is an area that we could do with more physical interaction - imho)

unfortunately, its a an area where we think of Imogen Heap’s Gloves concept,
but there, they are tied to hand gestures (bend sensors in gloves) and proximity data, and ideally also motion tracking (for position on stage)… so its much more than just the gyro data.

but hey, if you have an idea, the only true way to finding out if its good or not , is to implement / prototype it… even if others have ‘failed’, you may succeed :slight_smile:

1 Like


Might also be worth looking at:


There are some people who play ensemble music on iPhones:
The Michigan iPhone Orchestra was probably one of the first: U. Michigan iPhone Orchestra | Open Culture

This is also neat (well, no real reason to let an orchestra of professional musicians who are excelling at their instrument play an iPhone simulation of that - other than proof of concept, let’s try this :slight_smile: ): An orchestra with iPhones instead of instruments! - YouTube
P.S.: On second look the app that they are playing looks “gamified” - something along the lines of guitar hero :slight_smile:

TC-11 is great on an iPad. I haven’t used his iPhone app.