My Writings. My Thoughts.

Foreground Subtraction (a visual mockup)

// November 18th, 2009 // No Comments » // Computational Media

I have revised the steps I have broken down for this project.  While I work through programming them I have put together a visual mockup of how I plan it to look.  First is a timelapse of a space to illustrate the motion involved.

Next is a video illustrating the stages of how the foreground will be subtracted and the background reconstructed.

The steps I outlined in my previous post had one point that needed some revisions. In determining a threshold to compare pixel values, it is easier to use a reference frame to establish a threshold from, as opposed to comparing each image to all of the others. So now, I plan on breaking the process up in to two stages. The first will average each pixel across all of the images and write the averaged value to the screen. The next will then determine the mean of each pixel below the threshold (with the threshold being determined in reference to the averaged pixels from the previous stage).

The Social Spotlight

// November 4th, 2009 // No Comments » // Physical Computing

social spotlight

The Social Spotlight was created in collaboration with Boris Klompus and Miriam Simun for our physical computing midterm. The project, consisting of a bench and three spotlights, sets the stage for social experimentation. Imagine how the setting would influence these experiments if it were to be installed in such places as subway platforms.

As a user approaches the bench the spotlight above the left seat is lit, enticing the user to sit there. This is sort of stage one of the experiment. Would people want to sit in the spotlight or feel they are supposed to? Would they be too shy and sit in one of other seats or avoid the bench all together. Once someone sits in the spotlight, that light turns off and the middle light turns on. As the etiquette of social norms in this type of situation may suggest, one would normally leave a seat in between empty before sitting down. With the middle spotlight turning on, the bench suggests otherwise. With the middle seat taken, the last spotlight lights up. Once the bench is fully occupied, the light above the first person to sit down turns back on again. This leads to new observations. Does the person under the spotlight feel awkward in this situation? Do they feel as though they are meant to get up? How do others sitting on the bench respond? The interest in how the answers to these questions change from user to user and location to location was one of the biggest factors in leading us to develop the project.

Here are a few more photos and a video below to demo how it functions.

Steps towards a larger project

// October 28th, 2009 // No Comments » // Computational Media

I have an idea for a project I don’t yet know how to make. To help me get there I figured I would break down each of the tasks I need to figure out and start working through them individually. I’ll get to that below, but first, I figured I would do a bit of explaining what the larger idea is all about.

Before getting to the technical side of things, I’ll describe some of the conceptual ideas. I would like to construct a system where I am able to edit out the motion from a scene (ie. people moving) and be able to create a static image void of any of it’s inhabitants. Think an image of Grand Central Station in New York completely empty on a saturday afternoon. Or, as another example, this clip from Vanilla Sky depicts Time Square completely empty.

The application of doing this I have envisioned in a number of ways. First one being that through a series of time lapse photographs a new photo could be constructed of a place that may never otherwise be empty. Also, the system could be integrated with live video so that the user’s can see themselves alone in these same spaces. Lastly I would like to input video footage into the system from movie scenes that have locked off shots. By creating a static image from this, the image can then be used for further background subtraction that would take away any characters from the scene and allow users in insert themselves into the scene.

As for the technical implementation, here is a brief explanation as to how I imagine it working out. The processing sketch will read and store all of the pixel data for each image. Once the sketch has cycled through the array of images, it will compare the values where it noticed a minimal change in pixel values and see if there is enough information to construct a new image with these values. If there are any gaps, it will continue reading images until it can construct an image without any. If not, the new image is displayed/saved.

Breaking this down further into smaller steps I can work through for now, I have listed a few key elements to familiarize myself with:

  1. Learn how to create an array of images.
  2. Learn how to store an image.
  3. Learn how to implement background subtraction.

Working my way through this list I first created an array of images.  I used just two to start. Alternating between a frame with an empty background and a frame with a subject in the foreground.


You can see the sketch in action here.

Next step was to figure out background subtraction. I learned a few things through this process. One of which was how to store an image off screen and into memory. The other thing which took a lot of trial an error to figure out, is that I may need to use an external library to handle the background subtraction as I couldn’t quite get the results I was hoping for.  I experimented with the process a bit to also do some foreground subtraction and some frame differencing. Here is a video of background subtraction (code), foreground subtraction (code) and frame differencing (code).

Working my way through this I put together a new list.  This one being some steps for the actual programming.  So the steps in that go:

  1. construct image array of 500 images
  2. load first 5 images of array into buffer
  3. analyze pixels across all 5 images above and below threshold
  4. write each pixel consistently below the threshold in all 5 images to a new image on the screen
  5. load next 5 images in array and write any pixels below threshold that haven’t already been written to the new image
  6. repeat step 5 until either the new image has all pixels in the image filled, or the end of the image array has been reached

Morphing and merging images

// October 22nd, 2009 // No Comments » // Representing Earth

Here is another example of the Bullet Time effect, but on a much more affordable scale than the Matrix budget. Using disposable digital cameras they managed to put together a rig to experiment with the effect. The focus on this project was to use the resulting images as a basis for temporal-spatial experimentation in relationship to the viewer. You can see videos on the project’s site.

In thinking about the idea of morphing and merging, the terminology leads me to think of a distorted representation of reality. In regards to distortion, Slit-Scan images come to mind as a good example of this relationship. Golan Levin has compiled “An Informal Catalogue of Slit-Scan Video Artworks and Research”. As well there are some ITP folks who have experimented with this technique and some basic processing sketches you can try out if you are interested in doing some tests of your own. To focus on an example that relates well to the nature of this course, Michael Awad created a work called “The Entire City Project”. The basic idea of the project was to take panoramic Slit-Scan images of a significant portion of the city of Toronto and then assemble them into an installation within the city’s major airport. In a related project, he teamed up with Evan Penny using portraiture to turn these types of images into 3D representations.

As another example of people creating misleading representations of reality, here are a few links to do with Polar Panoramas (aka little planets). There is a flickr group dedicated to these images if you would like to see more examples. If you would like to try creating any there are plenty of tutorials in the forum.

Observing interactions

// October 21st, 2009 // No Comments » // Physical Computing

This week in pcomp we were asked to observe people interacting with technology in a public space. I decided to observe people using subway turnstiles. As I did, I recorded a timelapse video in effort to document some of the patterns I noticed. The affordances that lead to the intended interaction are fairly straight forward in their design. The action associated with the turnstile itself is common enough to create accurate expectations. The signage helps to direct traffic and the device for swiping the card presents a familiar interface. The design of this interface didn’t seem to cause a problem for using the card, but instead the functionality of it. Most people didn’t have problems, but there was definitely some who did. You could often tell that in differentiating these two groups, those who would have troubles often seemed to be people not from the city (and in turn were not familiar with the system). Others you could tell had troubles with dexterity in general.

The most interesting observation through all of this I found was that when people had troubles swiping their card, they didn’t often try to do anything differently in their subsequent attempts. It seemed like they’re thinking might have been, “must be the machine, I didn’t do anything wrong”. The only feedback the machines give you is a message reading to swipe your card again. Improving upon this feedback would significantly help. If people don’t know they are making a mistake, or how to correct it, it can be difficult to correct. I only managed to catch one of these interactions in the video below (the last person through).

Serial communication

// October 21st, 2009 // No Comments » // Physical Computing

Here is my take on last week’s lab for physical computing.  I’ve been playing with spirals in ICM lately so I decided to rig one up to a potentiometer.  Next step is to bring in 4 or 5 more into the equation.  Here it is for now though …

Pages: Prev 1 2 3 4 5 6 Next