Week 5: Color Image Processing

What fun experimenting with various color manipulation techniques in Processing the past two weeks to get a better understanding of how the red (R), green (G), blue (B), and alpha (A) values interact to produce different results. For this we learned a much faster programming technique, called bit shifting, to extract and set the color information of each pixel. Using bitwise operators, this technique works each pixel as a string of 32 bits--8 bits each for R, G, B, and A. The basic image processing sequence is as follows: first load the pixel array with nested loop, perform a right shift to extract the color information for each pixel (this shifts the bits to the right, masks them, and with & 0xFF pulls out the last 8 bits), manipulate color information as you see fit, and then perform a left shift to reassemble the color information (this packs the 8-bit numbers back into one 32-bit number). Right shifting is much faster for calling color information than the red(), green(), or blue() functions, and likewise, left shifting quicker for assembling color values than the color() function. This post helped me visualize the shifting and masking process.

I explored various algorithms while using my camera's live feed, and posted all the code here since it wasn't compatible with OpenProcessing. Here are the results of my permutations:

White Noise Texture
Riffing off of Rozin's examples, specifically this one, I really enjoyed the textures that emerged from different noise settings. For me the sweet spot for my "posterization" variable was around 2-3. Here the brightness of the pixels are evaluated and set to white if they are beyond the threshold.

import processing.video.*;
Capture camera;

int posterizeAmount;

void setup() {
size(1280, 720);
background (0);
frameRate(120);
camera = new Capture(this, width, height);
camera.start();
}

void draw() {
for (int x = 0; x<width; x++) {
for (int y = 0; y<height; y++) {
PxPGetPixel(x, y, camera.pixels, width);

//find a noise texture that I like
//int m = int(map(mouseX, 1, width, 1, 50));
//posterizeAmount = max(1, m);

posterizeAmount = 3;
if ((R+G+B) % posterizeAmount < posterizeAmount/2) {
R += 0;
G += 0;
B += 0;
} else {
R = 255;
G = 255;
B = 255;
}
PxPSetPixel(x, y, R, G, B, 255, pixels, width);
}
}
updatePixels();
//println(posterizeAmount);
}

int R, G, B, A;
void PxPGetPixel(int x, int y, int[] pixelArray, int pixelsWidth) {
int thisPixel=pixelArray[x+y*pixelsWidth];
A = (thisPixel >> 24) & 0xFF;
R = (thisPixel >> 16) & 0xFF;
G = (thisPixel >> 8) & 0xFF;
B = thisPixel & 0xFF;
}

void PxPSetPixel(int x, int y, int r, int g, int b, int a, int[] pixelArray, int pixelsWidth) {
a = a << 24;
r = r << 16;
g = g << 8;
color argb = a | r | g | b;
pixelArray[x+y*pixelsWidth]= argb;
}

White Noise in Black and White
For a completely monochromatic effect, simply replace the R and G values with B as the pixels are set back to the screen. (Or set R, G, and B to all R or all G.) For this example, I also set A to 150. What I most enjoy about this example is that it reminds me of an analog graphite drawing.

PxPSetPixel(x, y, B, B, B, 150, pixels, width);

Color Noise
In this example, I've change the color of the noise and also introduced the ability to adjust the R, G, B, and A values according the position of my mouse on the canvas before and as they are set to screen using the left shift function. For this particular image, my mouse was sitting in the upper middle portion of the window. For monochrome noise color, simply hardcode the R, G, and B values after "else" in the conditional statement and then remove the ability to set them dynamically with your mouse.

for (int x = 0; x<width; x++) {
for (int y = 0; y<height; y++) {
PxPGetPixel(x, y, ourVideo.pixels, width);

posterizeAmount = 3;
if ((R+G+B) % posterizeAmount < posterizeAmount/2) {
R += 0;
G += 0;
B += 0;
} else {
R= G+B;
G= R+B;
B= R+G;
}

R += mouseX;
G += mouseX;
B += mouseX;

PxPSetPixel(x, y, R, G, B, mouseY, pixels, width);
}
}
updatePixels();

Masking with Low Alpha
I discovered that in general a low alpha value upon setting pixel information back to the screen resulted in a blow-out and with the proper lighting, often created a mask around features in a certain tonal range. Here the alpha is set to 50:  PxPSetPixel(x, y, R, G, B, 50, pixels, width); This worked regardless of whether or not I added noise to the image. In the color noise example above, alpha is also set to low value due to the position of my mouse.

Inverting Colors without Noise
Without any noise applied and a constant alpha of 255, I played with inverting the colors for some very satisfying saturated results. Here I'm subtracting R for each channel.

PxPGetPixel(x, y, ourVideo.pixels, width);

R = 255-R;
G = 255-R;
B = 255-R;

PxPSetPixel(x, y, R, G, B, 255, pixels, width);

With Contrast Lighting
Different lighting sometimes produced dramatic results for all of the code examples. Again no noise applied here, but R, G, and B are each set to the sum of their partners.

PxPGetPixel(x, y, ourVideo.pixels, width);

R = G + B;
G = R + B;
B = R + G;

PxPSetPixel(x, y, R, G, B, 255, pixels, width);

All code adapted from these sketches by Danny Rozin.