Category: Science

Sounding the ocean

graph that depicts changes in the bottom pressure of the ocean over one month. The data was taken from sensors on the Juan de Fuca plate in the NE Pacific Ocean. The changes in bottom pressure in the graph depict daily fluctuations of the tidal cycle across the entire graph, but is marked by a large shift in bottom pressure in the middle of the data on April 24, 2015. The bottom pressure shifted as a result of the volcanic eruption — an indication that the seafloor dropped.In June 2024, Jon Bellona presented a paper at the 2024 International Conference on Auditory Display in Troy, NY on behalf of the NSF-funded Accessible Oceans project team. The paper, Iterative Design of Auditory Displays Involving Data Sonifications and Authentic Ocean Data, outlines their auditory display framework and their use of Kyma for all of their data sonification. Program with links to all papers are accessible online at https://icad2024.icad.org/program/

During the presentation, Jon played the full 2015 Axial Seamount Eruption. When an audience member asked about his use of earcon wrappers around each sonification, Jon shared a story about how his interviews with teachers at Perkins School for the Blind led him to include this feature.

In the coming year, Bellona plans to continue his work doing sonification with Kyma as part of a new project funded through NOAA: A Sanctuary in Sound: Increasing Accessibility to Gray’s Reef Data through Auditory Displays with Jessica Roberts, director of the Technology-Integrated Learning Environments (TILEs) Lab at Georgia Tech.

The Sounds of Data

In the July 11 2024 of Physics Magazine, host Julie Gould speaks with scientists who rely on senses other than sight, such as hearing and touch, to interpret data and communicate their research. They use sonification — the transformation of data into sound — to “listen” to hydrogen bonds, interpret gravitational-wave signals, and communicate a wide range of astrophysical data. Sonification also offers tools for visually impaired researchers and scientific outreach.

Included in the podcast are Martin Gruebele (University of Illinois at Urbana-Champaign) and Carla Scaletti (Symbolic Sound Corporation) describing how they could hear patterns of hydrogen-bond formation during protein folding that had been missed when relying solely on visual representations. (Their research is described in the 28 May 2024 issue of Proceedings of the National Academy of Sciences: “Hydrogen bonding heterogeneity correlates with protein folding transition state passage time as revealed by data sonification”).

Data sonification for scientific exploration & discovery

Data sonification, often used for outreach, education and accessibility, is also an effective tool for scientific exploration and discovery!

Working from the Lindorff-Larsen et al Science (2011) atomic-level molecular dynamics simulation of multiple folding and unfolding events in the WW domain, we heard (and analytically confirmed) correlations between hydrogen bond dynamics and the speed of a protein (un)folding transition.

The results were published this week in The Proceedings of the National Academy of Sciences (PNAS), vol. 121 no. 22, 28 May 2024: “Hydrogen bonding heterogeneity correlates with protein folding transition state passage time as revealed by data sonification”

Congratulations to everyone in the Biophysics Sonification Group:

Carla Scaletti (1), Premila P. Samuel Russell (2), Kurt J. Hebel (1), Meredith M. Rickard (2), Mayank Boob (2), Franz Danksagmüller (9), Stephen A. Taylor (7), Taras V. Pogorelov (2,3,4,5,6), and Martin Gruebele (2,3,5,8)

(1) Symbolic Sound Corporation, Champaign, IL 61820, United States;
(2) Department of Chemistry, University of Illinois Urbana-Champaign, IL 61801, United States;
(3) Center for Biophysics and Quantitative Biology, University of Illinois Urbana-Champaign, IL 61801, United States;
(4) School of Chemical Sciences, University of Illinois Urbana-Champaign, IL 61801, United States;
(5) Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign, IL 61801, United States;
(6) National Center for Supercomputer Applications, University of Illinois Urbana-Champaign, IL 61801, United States;
(7) School of Music, University of Illinois Urbana-Champaign, IL 61801, United States;
(8) Department of Physics, University of Illinois Urbana-Champaign, IL 61801, United States;
(9) Musikhochschule Lübeck, 23552 Lübeck, Germany

Science of Sound

Nate Butkus sending data from an iPad to control sound parameters in Kyma. Next step: substitute data from a scientific model for the iPad.

Nate and his team from the award-winning “The Show About Science” podcast had a question:

“Can sound help us understand the complex patterns in our universe?”

This question led Nate to Symbolic Sound in Champaign, Illinois on a journey where sound, music, and data intertwine in captivating and thought-provoking ways…

Listen to the podcast or read the transcript at the link below:

101: The Science of Sound with Kimberly Arcand, Martin Gruebele, Carla Scaletti, and Mark Temple

Martin Gruebele, Carla Scaletti, Nate Butkus, Eric Butkus (Photos by Jenny Butkus)

Displacement

Photo by Dr. Javier Alejandro González Ortega

After the 2010 El Mayor Cucapah 7.2 magnitude earthquake in northern Mexico, seismologist Alejandro González Ortega interviewed Don Chayo, a Cucapah native who witnessed the surface rupture. When Don Chayo drew parallels to the origin stories of the Cucapah people, González began to wonder if these stories may have recounted earlier seismic events that had been passed down over the generations.

Over the next several years, González and his colleague, choreographer/physicist Minerva Muñoz, created a performance piece based on 3D seismological data collected by 12 measurement stations during the event. Muñoz enlisted the help of composer Carla Scaletti to map the data to sound using Kyma and artist David Olivares to map the data to video using Unity.

As Muñez and González conducted further research and interviews with the Cucapah elders, a much more disturbing story began to emerge — that of a displaced people whose livelihood was being cut off and whose very language was being forgotten. What had originally been intended as a science/art collaboration about seismic activity began to morph into a deeper metaphor for displacement, disruption and loss.

The result — Wí Shpá, A journey in bare feet — is a poem in movement, images, sounds and words that explores pilgrimage, displacement, change, the relationship of humans with the environment, transformation and resilience.

The sound and visuals were created from seismological data and satellite geodesics of the El Cucapah Mw 7.2 earthquake that occurred on April 4, 2010. Consistent in many details with the cosmogony myths narrated by Don Chayo that had been passed down over generations, El Mayor-Cucapah Mw 7.2 was the most intense earthquake recorded in this region over the last century.

Wí Shpá, A journey in bare feet is an elegy to the ancestors and to the women and men of today; to the people of the river, of the earth, fire and wind. It is a glimpse into a universe in which animals are gods, and meaning is associated with each of the four cardinal directions, colors, the power of nature and of the land.

“Cosmogony of an Event, El Mayor Cucapah Mw 7.2” is an inter and trans disciplinary dialog of artistic creation and research combining the myths of Cucapah cosmogenesis and the scientific studies of El Mayor-Cucapah Mw 7.2, weaving a network of collaboration, tradition, scientific research, knowledge and experiences, but above all, creating a dialog between scientists, artists, native community, collaborators and the general public who participate in this live performance/ritual.

Credits:

Direction, stage creation and interpretation: Minerva Muñoz *
Production: Alejandro González, Minerva Muñoz / La Machina Productions
Scenic Advisor: Jorge Folgueira
Lighting: Minerva Muñoz
Composition and sound design: Carla Scaletti
Visual Art: David Olivares
Video: Marco Meza, Rommel Vázquez
Aerial Video (drone): Alejandro González
Photography: Alfredo Ruiz and Rommel Vázquez
Science: Javier González-García and Alejandro González
Audio Engineer: Rommel Vázquez
Scenography: Leoncio García
Makeup: Rosario Martínez
Lighting technician: Miguel Tamayo
Communication and networks: Stephanie Lozano
Support: Juan Sánchez

The rocket scientist of human hearing

In 1999, astrophysicist/musician David McClain spent an intense three-month period working on The Northern Sky Survey, mapping the sky in the near infrared while getting by on an hour of sleep per night. When he finished the survey, he was suddenly struck by a viral infection that nearly killed him; his doctors were never able to determine the cause and, after three months, the infection dissipated almost as quickly as it had appeared. But afterward David noticed that he could no longer understand his wife when she was speaking to him. He went to an audiologist and discovered he had a sensorineural hearing loss of 60-70 dB in the high frequency range. Hearing aids helped him understand speech, but he was devastated to discover that music never sounded right through the hearing aids. But as a physicist, he was determined to solve the problem.

Motivated by his love of music and informed by his scientific training, McClain has spent the last 16 years developing equations to describe the entire hearing experience – from the cochlea, to the afferent 8th nerve, to processing in the central nervous system, efferent 8th nerve interactions — and developing signal processing algorithms to adapt to and compensate for his hearing loss in a way that preserves the audio experience of music. The result is a collection of signal processing algorithms he calls Crescendo. Kyma is one of the tools David uses for testing out new ideas and prototyping them for Crescendo.

Now he’s blogging about his findings on his web site http://refined-audiometrics.com. In keeping with his motto “Keeping music enjoyable for all!” David hopes that his experiences, research findings, and extensive set of algorithms can benefit others.

 

Composer on a NASA mission

Composer Roland Kuit was recently interviewed on the prime time news program SBS 6 Hart van Nederland to discuss his Kyma sound explorations that will be launched into space on September 8, 2016 on NASA’s OSIRIS-REx mission to the near-earth asteroid Bennu.

While his music is being launched into space on September 8 2016, Kuit will be at the Kyma International Sound Symposium in Leicester, UK presenting his music and ideas along with filmmaker Karin Schomaker so you’ll have an opportunity to meet and talk with him at KISS2016.

Kyma creator on the cover of Computer Music Journal


The Spring 2016 issue of Computer Music Journal (Volume 40 Issue 1) includes a transcript of Carla Scaletti‘s keynote address for the 41st International Computer Music Conference.

In Looking Back, Looking Forward, Scaletti uses mythology, evolutionary anthropology, nostalgia research and a story about the origins of Kyma to illustrate the idea that software is “hardware with cognitive fluidity”.

 

The Listening Back, Listening Forward issue marks the beginning of Computer Music Journal‘s 40th year of publication and, citing recent research showing that nostalgia enhances creativity, CMJ editor Douglas Keislar invites readers to share their own computer music stories for possible publication as letters to the editor throughout this anniversary year.

Also in this issue: Silvia Matheus reviews The Seventh KYMA International Sound Symposium (KISS2015)!

QUANTUM in the desert: art, science, technology & collaboration

In April 2015, QUANTUM, the dance piece Jobin created inspired by his residency at CERN, will be touring northern Mexico including Culiacan, Hermosillo, Tijuana, Ensenada and Mexicali. Technical director Marie Predour will be running the live sound for the piece using a Kyma 7 Timeline.

Choreographer Gilles Jobin took a moment to talk a little about the piece, to explain his ideas on algorithmic choreography and to reflect on collaboration, art, science, and technology.

 

What’s interesting about technology is not so much the technology as a tool, but technology as a new way of thinking — as a different way to organize your work or to think about your work. The same is true when choreographers work with scientists or with musicians. There is a kind of exchange of practices that is enriching for everybody.

14 April Culiacan РFestival Danza Jos̩ Limon

16 April Hermosillo – Un Desierto Para La Danza

19 April Tijuana – Cuerpos in Transito

21 April Ensenada – Espuma Cuantica

23 April Mexicali – Entre Fronteras

27 April to 2 May Torreon – Gilles Jobin will be on the jury for Premio Nacional Guillermo Arriaga

What if the stars made music

Outdoor Culture presents Sounds of the Night Sky featuring Robert Jarvis‘ sound installation: aroundNorth, opening Thursday 20 February 2014 6.30-9.30 pm at the National Trust Stowe New Inn Farm Buckingham MK18 5EQ

This outdoor event will showcase the premier of Robert Jarvis’ new sound art installation, aroundNorth, a piece that was shortlisted for the 2010 PRS New Music Award.  A multi-speaker sound map of the stars driven by the turning of the Earth, aroundNorth uses Kyma to transform the night sky into a celestial music box rotating around Polaris, the North Star. As each star passes a virtual line in the sky, it triggers a musical note whose qualities are determined by the star’s spectrum, mass, brightness and distance from earth, creating a mesmerising sound map of the universe as viewed from our rotating planet.

Visitors will be accompanied down Bell Gate Drive on foot and into the gardens of Stowe after dark, before entering the semi-wilderness of the newly-opened Lamport Garden where the sounds of moving stars will be created like a giant celestial music box! Dress for the outdoors and bring a flashlight!  Click here for more information and tickets.