MUFF WIGGLER Forum Index
 FAQ & Terms of UseFAQ & Terms Of Use   Wiggler RadioMW Radio   Muff Wiggler TwitterTwitter   Support the site @ PatreonPatreon 
 SearchSearch   RegisterSign up   Log inLog in 
WIGGLING 'LITE' IN GUEST MODE

object tracking --> audio generation? Synesthesia Machine
MUFF WIGGLER Forum Index -> Video Synthesis  
Author object tracking --> audio generation? Synesthesia Machine
rayjohnson
Hello,

I'm thinking about starting a software project that takes video and generates music from the video.
one of my main ideas is tracking objects and using their detected parameters to relate to musical parameters.
the larger idea is a 'synesthesia machine', a framework of ideas and collection of software modules to patch between the audio and visual domains.

anyone see anything like this idea? i think it's different than most visualizers out there.

Thanks!
makers
David Rokeby's "Very Nervous System" immediately comes to mind. It's a different application of object tracking but certainly worth investigating. I haven't followed its development in years but there was a version of the code called SoftVNS for an old version of Max.

http://www.davidrokeby.com/vns.html
nathanxl
The IOS app called PhonoPaper, when used in freemode, converts the signal coming in from your phone camera into audio.
Check it out:
http://www.warmplace.ru/soft/phonopaper/
Annie Socoria
I was looking for something along these lines to generate audio from the results of my video feedback sessions. A visualizer in reverse of sorts. Haven't found anything that quite does what I had in mind and I lack the skills at present to make this. Given the complexities of synesthesia it would be interesting to see how a video image could be interpreted as sound.
makers
nathanxl wrote:
The IOS app called PhonoPaper, when used in freemode, converts the signal coming in from your phone camera into audio.
Check it out:
http://www.warmplace.ru/soft/phonopaper/


Right, and while your at Warmplace site check out VirtualANS. The original ANS synthesizer is certainly worth investigating in this context as well as it's namesake Scriabin.

https://en.m.wikipedia.org/wiki/Alexander_Scriabin
MUFF WIGGLER Forum Index -> Video Synthesis  
Page 1 of 1
Powered by phpBB © phpBB Group