Week 1: Research on Word Analysis and Related Projects

I am curious about creating a communication platform that translates our words into audible animal calls for conversational play.

While I’m excited by this interest, I’m struggling to find an underlying reason or compelling argument other than it might be fun to see and hear what happens. So I started answering questions from last week and tried to make something to get closer to an answer.

What are some ways to analyze words?
One of my goals is to use material that is meaningful to people. For the moment, I’ve chosen to work with words and spent some time this week researching and brainstorming ways to analyze meaning from this form of input. Right now I’m imagining text input that is magically (ok, computationally) sonified.

So what’s to capture about a word or a phrase?

What it means?
How it sounds?
The speed at which it was typed?
Do I… Count letters, syllables, words, parts of speech*
Identify keywords, most common words, unique words
Determine level of subjectivity (fact <—> opinion)
Determine polarity (positive <—> negative)
Determine emotion (joy, surprise, anger, disgust, fear, sadness)
Compare words across a corpus of text (TF-IDF and word vectors)

*How do our word choices, specifically the use of function words (pronouns, position words, and auxiliary verbs) reflect “our personalities, emotional states, social connections and thinking styles?” (reference from Analyze Words, a Pennebaker project)

Resources:
Siraj Raval’s Sentiment Analysis – Data Lit #1
Shiffman’s Word Counting & Text Analysis Series
Shiffman’s N-Grams & Markov Chains Series
James Pennebaker’s The Secret Life of Pronouns and Ted Talk
Allison Parrish’s Introduction to Natural Language Processing and Word Vectors

Is there any work out there like this? What sound-related, participatory and preferably public installations can I find?
Blobchat by Yotam Mann and Sarah Rothberg
“Blobchat is an experimental platform for communicating – sort of – through sound. Initially debuted as a quadraphonic installation, Blobchat 1.0 invites you to try out conversation as performance. You can chat at other blobs on a state-of-the-art blobchat-enabled computer. As you chat, your words are sonified, and get garbled together with all the other chats into a collective stream of nonsense. But is it nonsense? We use *machine learning* to make new sense of the garbled blob. A new sense! Out of the collective gesture of chatting!”

Works by Daily tous les jours*:
Hello Trees!
“The installation takes the form of an illuminated arcade that extends from the ground to the seam of the tree canopies above. Passers­by are invited to input messages at stations located at each end of the promenade, and watch and listen as their voices are translated into light and sound patterns that travel along the arches as they walk beneath. The original input is gradually transformed from language to musical melodies. When two or more messages meet, a special light and sound effect is triggered.”

I Heard There Was a Secret Chord
“…a participatory humming channel that reveals an invisible vibration uniting people around the world currently listening to Leonard Cohen’s “Hallelujah”. The project comprises a room and a website. The room contains a simple numerical display of current online listeners, each represented by a humming voice in the space. Underfoot, these sounds are transformed into low frequency vibrations as visitors start humming along.”

21 Balançoires
“21 Balançoires (21 Swings) is a musical installation from which certain melodies emerge only through cooperation between players, thus stimulating a sense of community and ownership of space…. When in motion, each swing triggers different notes, and when used all together, the swings create a musical composition in which certain melodies emerge only through cooperation. This collaborative exercise stimulates intuitive play and experimentation amongst people of all ages and backgrounds, whether they know each other or not, and leads participants and spectators to become aware of each other, and their environment.”

Mesa Musical Shadows
“Musical Shadows is an interactive pavement that reacts to the shadows of passersby by playing sounds of singing voices. As visitors together explore different soundscapes with their shadows, they become part of a collective sound and body performance. This new scenario for public space engages strangers to bump up against one another and share a moment of magic igniting in them a sense of what is possible together.”

Works by Rafael Lozano-Hemmer:
Amodal Suspension
“One objective of the piece was to make a public spectacle by using the private medium of text messaging, slowing down communication and introducing the possibility of interception.”

Less Than Three
“…an interactive installation of light strips that form a network between two intercoms. As a participant speaks into an intercom, his or her voice is translated into corresponding flashes of light and this light pattern is transmitted visually along one of the several possible pathways through the network. When it reaches the other side, the viewer’s phrase is once again released as sound.”

Voz Alta
“…the megaphone amplifies the voice, a 10kW searchlight automatically ‘beams’ the voice as a sequence of flashes: if the voice is silent the light is off and as it gets louder so does the light’s brightness.”

Voice Array
“As a participant speaks into an intercom, his or her voice is automatically translated into flashes of light and then the unique blinking pattern is stored as a loop in the first light of the array. Each new recording pushes all previous recordings one position down and gradually one can hear the cumulative sound of the 288 previous recordings. The voice that was pushed out of the array can then be heard by itself.” (Similar Voice Tunnel in NYC)

Samba Surdo by Lucas Werthein
“Samba Surdo is an interactive eight-channel sound installation that allows viewers to experience Brazilian samba. Visitors are invited to walk into the center of a room where eight speakers are positioned around them in the shape of a circle. Each speaker represents a different instrument played in a samba orchestra. As the visitor moves throughout the circle, his/her position is tracked by a set of sensors that determine which speaker should turn on. In this way, Samba Surdo gives the visitor the role held by the Master of Percussion in a samba orchestra; by moving and experiencing the installation, the visitor determines which instruments are played.”
(reminds me of my spatial sound puzzle from last fall!)

*Thanks to Yeseul for pointing me in the direction of this Montreal-based design studio!

And just for fun, there’s no shortage of these:
Human voice changers
Humans making animal sounds (also compelling because of the humans’ expressions)
Animals making human sounds

Next Steps
Scour the internets for repositories of recorded animal sounds and learn more about how animal behaviorists decode animal communication.

Visit Sarah in office hours to learn more about Blobchat (scheduled).