Week 14: Painting Mirror

Move the pieces and assemble your reflections in this interactive Cubist mirror.
How many ways can you put yourself together?

Final documentation here

Project Description
In real life and in digital spaces we offer multiple versions of ourselves for numerous reasons. Painting Mirror plays with this idea by proposing continuous ways of seeing and presenting yourself. Using dual camera angles and the open source software, Processing, video reflections are fragmented, resized, duplicated, and scattered across the surface. The pieces blend together and moving them paints the screen anew. The impulse might be to reassemble your image into a whole, but can you? 

This is a continuation of a recent project, which I desired to push further to see what I might uncover. Though there's more to explore, I learned a lot along the way. Here’s a recap of the highlights. 

Related Work by Rafael Lozano-Hemmer (added January 2019)
Redundant Assembly
Bilateral Time Slicer

Part 1: Leaving the Land of Rects
My first instinct was to move beyond the land of rectangles and draw irregular quadrilaterals. I learned that I could use the beginShape() and endShape() functions, with vertices of X and Y coordinates in between, to draw more complicated polygons (see the PShape Shiffman tutorial). Vertices are drawn in a counterclockwise direction starting with the leftmost corner. 

Using this Danny Rozin example from class as a guide, I adapted my original sketch in a similar fashion. The polygon constructor still receives random values for an X and a Y, but instead of using these as the location for the leftmost corner of the shape, they become its centroid. Random numbers are then generated for the calculation of each X and each Y of every new vertex. Those values are stored in a two-dimensional array or matrix and each corner of the shape is created (again in counterclockwise fashion) by subtracting them from or adding them to the X and Y values of the centroid. Though we’ve been working with image matrices all semester, this exercise strengthened my understanding of how I might use them other contexts.

In the previous version of this project, I used the copy() function to grab and duplicate pixels from the canvas into my rects. Danny’s sketch and this tutorial showed me how to load and map parts of the camera feed into my irregular quadrilaterals using texture(). As in the earlier version, I’m still playing with the scale of the newly-mapped areas when doing so. 

In order to employ texture(), I changed the default rendering mode to P3D, which uses OpenGL and increases the overall speed of my sketch.

Part 2: Oh Yeah, Mouse Dragging
Once I converted the shapes, it was time to update the mouse dragging. It was no longer a simple process of checking whether the mouse was within bounds of a plain old rect. After falling down several rabbit holes for a solution, Shiffman recommended two options: drawing all the polygons offscreen as PGraphics objects each with their own unique color to check or try the library, Toxiclibs.js. And then I found a third: Karsten Schmidt, the author of Toxiclibs wrote an example on the Processing forum taking advantage of this point-in-polygon algorithm. It’s possible that I came across this before, but it was too early in the quest for me to understand what I was seeing and what I needed. In any event, I incorporated it into my sketch, using another array to store all of the vertices of each shape, and it worked!

(I remember adjusting the offSet calculations, too. In my original version, offSets for X and for Y where based on the center of each rect (width/2 and height/2), but now that I had a centroid, it was easy to just replace those with the X and Y centroid values.)

Part 3: Adding Another Camera
One camera is great, but with two webcams I can show the viewer dual angles of themselves simultaneously. So I created a new class in my Processing sketch to feed the second camera into a new batch of polygons. The code is nearly identical except the part of declaring a different camera in the texture() function. Gosh, I’m repeating so much code! Therefore, there MUST be a way to rewrite it more efficiently. I haven’t wrapped my head around how to do that just yet...it awaits as a very fun problem to solve. (I’m also still wondering if I might want to treat the camera feeds differently, and at least for right now, keeping them separated gives me the space to think about the possibilities.)

Of note, I took heed of Danny's suggestion to use different camera models to avoid conflicts and headaches: I'm using a Logitech C920 and C922

Part 4: Taking Advantage of ITP's Touch Screens
It's one thing to develop within the comfort of your own laptop environment, but I envisioned this to display on a touch-screen for users to play with their own images. First I messed with the screens along the hallway to the lounge, quickly realizing that any new piece of hardware in the mix requires it's own special attention with regards to the logistics of wire connections as well as finding the right resolution to agree with my Processing canvas size. But those screens are out of reach for kids, and I so turned my focus to the large vertical-hanging Planar monitor near the elevators in ITP's lobby. It took some wire wrangling, HDMI port-finding on the back, installing of drivers, and help from Marlon, but we got it to work on one of the ER's MSI laptops, which was waaay faster than my Mac.

Instead of changing the laptop display to portrait mode to fit the new screen as I began to do, Danny offered the BRILLIANT suggestion of simply rotating the actual cameras and attaching (with tape for now) to the sides of the monitor. For my final Pixel by Pixel iteration, I attached one camera to the side of the screen and mounted the other on a tripod behind the user. Both camera feeds streamed onto the display, but to be honest, the "back" version wasn't as noticeable. For the Spring Show version, I plan to mount both cameras on opposite sides of the screen; folks want to find their faces so give them more of those, I will!

Part 5: Finishing Up & Conclusions - Is it a Mirror, Painting, or Portrait?
Standing before the screen, one's image is elusive: the polygons act like jumbled lenses, and it becomes a puzzle to assemble yourself. Distorting the face and figure is endless entertainment for me, but it's possible shapes are too sharp. Their jaggedness, however, is softened by the fading painterly effect of the combined blend modes.

Shortly after I began this project, I started saving screenshots to document my process and not long after, considered it a type of photo booth. Without this feature, it's simply a funny mirror, albeit one with plenty more options to alter your appearance. Since it's migration to the big monitor, I incorporated a double-tap feature to save the screen sans keyboard using this old forum post as a reference. Processing tells me that the mouseEvent, getClickCount(), is deprecated, but hey, it still works.

Thank you so much, Marlon! I'm very grateful for your time with the Planar. Also to Danny Rozin for the inspiration and to Dan Shiffman for checking in on his way down the hall. And to my many wonderful and talented classmates who stopped by watch, play, chat, and give advice: I am in awe of you everyday. 

Code on GitHub
Portraits created at the ITP Spring Show 2018 @paintingmirror