→ Odile & Odette (Sensory Aggregate)
Visual Design / Generative Art / Audio-Visual Interactio / Machine Learning in Visual Design
Among human beings, very few people have the ability of "synaesthesia." They can connect different senses and present them realistically in their minds. I hope to use some engineering to capture the different sensory signals that people receive when watching Swan Lake, and I try to combine these signals. (stage, set, lighting, dancer movements, environment, symphony, etc. ).
Especially the 2006 version of Swan Lake by the Paris Opera in which the character of the black swan was added, which impressed me deeply. So I want to try to use this method to interpret the monologue between the white swan (Odile) and the black swan (Odette). Finally, it is presented in a 32*32 grid.
I utilize AI technology to analyze and integrate different sensory signals, simulating an experience similar to synesthesia. Also use OpenCV and MediaPipe to extract dance movement trajectories, capturing the dynamic features of the dancers. Additionally, I employ a CNN (Convolutional Neural Network) to analyze stage and lighting changes, extracting the colors, intensity, and dynamic variations of the stage design and lighting.
I leverage Google Magenta to analyze the emotional qualities of the music using AI, generating corresponding visual feedback and other sensory transformations. This AI-driven multimodal data fusion allows dance, music, lighting, and environmental elements to interact, creating a more immersive artistic experience.