Week 7: Color Sound Pen Iteration 2


Your midterm must include a project with strong concept, enclosure, presentation (5-7min as you will present in class), sketches (if you have them), schematic (in Fritzing), code on Github, and documentation as post on blog. Technically, you must include one tech from each section. SECTION I: P5.js visualization, Servo as Output, Serial Communication, Serial Handshake, and Tone as Output. SECTION II: Digital Input (at least 2), Analog Output (at least 2), DOM based input, Bluetooth, and Web Server.

Inspiration & Considerations
This is the second iteration of my Color Sound Pen to explore the translation of color into sounds, but this time using the Adafruit RGB Color Sensor with IR Filter. (The initial pen prototype and first iteration are described here and here.)  I'm excited to learn how this sensor works and to scale the project to an enclosure than fits comfortably in one hand. Ultimately, I hope to create a completely wireless device, which means using a smaller microcontroller, determining a battery solution, and figuring out the sound output. For the latter, I have three options: 1) incorporate a small speaker and utilize Arduino's tinny tone library, 2) incorporate a MP3 shield of some sort to ditch the tone library, and 3) play notes in a P5.js sketch. After office hours with Ayo, I was inspired to try P5 and apply our recent lessons in asynchronous serial communication. Choosing this option also means that I need learn how to connect to my computer via Bluetooth instead of the USB port if I want to go wireless.

With so many aspects to consider for this project, I divided up the work for first week into three parts: get the rgb color sensor working, create a P5.js sketch to play notes, and finally, connect the two. Anything else I noted above falls into the scope of next week's goals. 

Part 1: Get the RGB Color Sensor Working
This is my first time working with any sensor other than the ones included in the Arduino start kit, and I was so appreciative to follow Adafruit's Overview and Tutorial to set it up . Similar to my prototype, this sensor includes an onboard white LED to light objects in front of it and outputs red, green, blue, and clear light sensing values, blocking infrared light to increase accuracy. It connects to the Arduino's SDA and SCL pins for I2C communication, which is synchronous serial communication, meaning that both devices share the same clock over the SCL pin (SDA is for data transmission). Along with a library for the sensor, Adafruit provides several sample sketches: ColorView maps color sensing values to a RGB LED, tcs34725 prints color temperature and lux or illuminance, "the perceived brightness of visible light" according to the report below, and tcs34725autorange. My goal here was to not only get the sensor working but determine output values specific to each color reading so that I could map them to distinct sounds. I also aimed to prototype an idea for an smaller enclosure.

RGB Color Sensor Datasheet
ams TCS34725 Color Sensor
Lux and CCT Calculations using ams Color Sensors (DN40, calculations)
Ambient Light Sensor Auto-gain Algorithm (DN39, some thoughts on autogain)
Color Calculating Color Temperature (DN25, original Adafruit calculations)
Color Hex Color Codes

Materials & Tools
Arduino 101 (Bluetooth Low Energy capability)
Adafruit RGB Color Sensor with IR filter and White LED - TCS34725
Diffused RGB LED - Common Anode
1x 1K Ohm Resistor
2x 560 Ohm Resistor
8 ohm 0.5W Mini Metal Speaker with Wires (for prototyping)
Wires & Jumper Wires
USB Cable Standard A to B
Container from Foil Washi Tapes (from Michaels)
2x #2 56 x 3/8" Machine Screws
5/64" drill bit
Classic Crayola Markers

RGB Color Sensor:
3.3V to 3.3V (red wire)*
GND to GND (green wire)
SDA to SDA/Analog 4 (orange wire)
SCL to SCL/Analog 5 (yellow wire)

RGB Diffused LED:
LED Common Anode (long pin) to 3.3V (red wire)
LED Red Pin to 1K resistor to Arduino Pin 3 (white wire)
LED Green Pin to 560 ohm resistor to Arduino Pin 5 (gray wire)
LED Blue Pin to 560 ohm resistor to Arduino Pin 6 (brown wire)

Mini Speaker
Connect to Digital Pin 13 (white to red speaker wire)
GND to GND (green wire)

Arduino Sketches
My combined ColorView and tcs34725 sketch to see LED color changes and print color temp and lux values. 
ColorView sketch with speaker and specific color tone mappings.

*I hope to power a wireless version with 3.3V so best to start now instead of the recommended 5V in the tutorial. For the future, consider lowering the resistor values for the RGB LED since those recommended were based on 5V supply.

Part 2: Create a Sound Sketch in P5.js
Next I focused on a new set of notes to play via a P5.js sketch. First, I found and downloaded five free notes from freesound.org, each of which I slightly shortened using the free audio editor, Audacity. Then, I built out sketches in P5, beginning with one to just play these new notes with browser buttons. Next, I used asynchronous serial communication to control the playback with physical buttons on an Arduino. In the video below, notice the changing incoming values in the P5 console for each button press. Notice also that values are being sent multiple times for the duration of each press.

Materials & Tools
Arduino Uno
5 Buttons
5x 10K Ohm Resistors
USB Cable Standard A to B
Notes on Nylon Strings by Kyster

Step 1: Browser Buttons to Play Notes, P5.js sketch
Step 2: Arduino Buttons to Play Notes, P5.js sketch and Arduino sketch

Part 3: Connect the Sensor to the Sound Sketch
With everything working, I simply updated my color-tone mapped ColorView sketch (updated sketch here) to send bytes to serial port for my P5.js sketch (same sketch in Part 2/Step 2 above) to process and play notes. It works!

Wow, so much accomplished and still so much to do! It really helps me to build in small stages to move at a pace at which I can succeed and to keep the positive momentum going. Next up: figure out serial communication via Bluetooth using my Arduino 101. Order a smaller Bluetooth-enabled microcontroller and battery--will they all fit inside my Washi Tape container? Once those pieces arrive, I'll need to figure out how to configure them and how to move everything, including the RGB LED, into a smaller enclosure. Another suggestion from Ayo was to consider how to record sound sessions to archive all the many ways users might "play" paintings. Maybe include a button to start and stop the recording on the device itself.