Our Live Experimental Interactive Television class is coming to a close. All four of our episodes have now aired on the Manhattan Neighborhood Network. It’s made for a busy but fun couple of weeks. Above are some of the highlights from our two episodes of UNITV. You can also check out some behind the scenes footage on my flickr page and check out the project in the ITP Spring Show.

For my Nature of Code final I’m thinking of creating a video that would have a particle system interacting with the native particles from the source footage. Using the image above as an example, the blurred out lights in the background would act as the primary set of particles. A second set of particles would then be introduced in processing (using textures stripped out from the lights). Through a variety of physics simulations, the new set of particles would interact with the lights as well as any objects that are focused in the screen. The chair above for instance, could be turned into a mask, where the particles would treat it as a boundary and react accordingly.

Using one shot to start with, I will try to implement as much of the code as I need to have everything up and running. Once I get to that stage I will start testing that code with new footage and try to introduce some new features. One of which that I am thinking of, is to add an audio reactive element to the particles’ behaviors. I still have lots to do before I get there though. I have started to use OpenCV to see how I can track the particles with blob detection. I am looking into the best way to handle the textures. I also need to decide how to handle the workflow of the video (GSVideo, an image sequence ?). While I do all of this, I will also be thinking of what type of footage will work best with the concept and what kind of foreground objects I can use to help build a stronger conceptual framework.

I have put together a first iteration of a project I’m working on called The Figure 8. As this is a project I will be continuing to work on, I will break down where I am so far and what I still plan to do with it. To start, here a few different descriptions of the project:

Short Description
The Figure 8 is a database film that explores the connections between stories and the people that tell them.

Long Description
The Figure 8 is a database film that explores the connections between stories and the people that tell them. Created using the software Korsakow, the film presents a collection of personal stories that allow for viewers to navigate the database while exploring the connections that create the paths between them. The structure of the database draws upon the idea that there are only eight stories in the world and all others are a derivative of those eight. The people in the film were asked to share any story they would like to tell, with the only constraints being time and that they tell a personal story of their own. These stories are then tagged with which of the eight plots they relate to and connections are then made with other stories in the database that share those same tags. Through these connections viewers are able to construct a narrative path more suited to their interests.

The eight basic plots:

• Sacrifice
• Rise and fall
• Transformation
• Revenge
• Chase
• Quest
• Romance
• Competition

Manifesto:

• A personal story
• No other narrative constraints or suggestions
• Told by by someone else
• 30 seconds to 3 minutes each
• One shot
• No cuts
• Source audio only

Problems encountered in this version:
The filming of this was all done using equipment I don’t have much experience with. As a result I had some audio issues to work through. I have since worked through all of these issues and will have to reshoot some of the material. I also encountered some hurdles with Korsakow. Mainly some compression issues that caused a few problems but I was able to work through. The one thing I couldn’t figure out though was how to create a lead in and have that move on to a starting clip. Ideally what I would like is for a title sequence to play (without the thumbnails at the side) and then randomly pull a clip with an associated start tag to follow. Not sure if this is possible yet.

Things to do:

• Film more stories
• Create promotional material to give to storytellers
• Reshoot the stories with bad audio
• Look into using conditionals and try to work text/tags into preview thumbnails
• Create new preview thumbnails with rollovers
• Create a new title sequence that includes a more in depth explanation of the project
• Create other pages for the site (about, manifesto, credits)
• Integrate links to other pages in the interface
• Integrate title in the interface

View the Project

The next thought that all of this led me to was how I can connect all of these stories in a meaningful way.  This brought me back to an idea I had heard before.  Various people have written how they believe there are only a few amount of stories in the world and all other stories are just derivatives of these.  This concept is something I’m still doing more research into, but I’ve already found one of these references that works really well with what I would like to do.  Nancy Kress wrote an article on how she believes there are only 8 stories.  She listed them as follows:

• Sacrifice
• Rise and fall
• Transformation
• Revenge
• Chase
• Quest
• Romance
• Competition

The other thing that I really like about this list is that it falls into 8.  I like how the figure 8 encompasses all of the basic plots, yet as you turn the figure on it’s side, it represents the infinite possibilities of all the stories these 8 basic plots can be turned into.

Taking these 8 plots, I plan on tagging all of the stories I enter into the database and using them to draw the connections between all of the clips.  Ideally, the more stories I can enter into the database the better.  Once I have good amount of videos to start with, I also plan on opening the site to encourage others to submit stories they’ve filmed themselves (whether it’s of themselves or of others).  In this sense I like that the piece becomes stronger as the database grows.

For Nature of Code I’m hoping to work on an audio reactive particle system.  I started in on this, but still have plenty to do.  I still need to spend more time with the beat detection to get a closer relationship between the two.  I also need to work out the details of rendering sketches and maintaining the synchronization of the audio and video (instead of using screen capture software).  As for the particles, I’ve just started off with one for now.  Next up, is to start to bring in more particles and apply more complex physics to them.  For now, you can click through the image above to watch a video of where I am at so far.

For last week’s class we were asked to build a web interface  to allow viewers to interactive with a 5 minute live program.  Working with Adam Harvey and Si Heun Cho, we created a networked version of twister.  The game streams live through ustream.tv and replaces the wheel that usually decides players moves with an interface that allows viewers to vote on the next move.  The counter resets after every move and voting opens back up.

One of the things that works well in this respect is the number of options for voting.  It creates a greater sense of impact for voters.  At the same time though, this impact diminishes as the scale of the viewership increases.  The game seems as though it would only work well on a smaller scale (say under 50 people) and even better with a viewership that has some sort of relationship with or prior knowledge of the players.  One of the other strengths of this type of idea is the sense of control it creates.  Although shared amongst the collective viewership, the concept of controlling something on the screen as an extension of  what a remote normally allows seems to bridge the gap between performer and audience.

If we were to try this again, I think it would be interesting to try a different game and see how the dynamic changes.  With this version, there is a tendency to vote for the most awkward move or for whatever move would help whoever you are hoping will win.  Instead, an interesting way of changing the dynamic could be choosing a game where the audience doesn’t control the competitors, but instead competes against them.

Last week in Live Experimental Interactive Television we were asked to create a site where users were able to interact with video by SMS. Teaming up with Adam Harvey, Edward Gordon and Mustafa Bagdatli we created the above video. Drawing upon the ridiculousness of the Daft Bodies meme, we thought it would be funny to give users the ability to control the characters with their phones. One of the things I thought worked well was the ability to send in a word via text and see that word acted out and sung with the typical Daft Punk vocoder sound. One of the drawbacks would be scale. It worked well with a class room full of people, but it might get out of control with thousands of viewers. A few ways to approach this problem may be to change the nature of the content, divide up the screen and then provide many more options. If this was the case, you could construct some sort of game based goal which people may be able to see the result of their actions clearer.

The interesting thing I found about this game was the social aspect of it. Another observation I had was that Twitter and Facebook were sort of loosely organized second screens in themselves. All of these options seemed to me more successful in the respect that people would rather read what their friends have to say about the game than strangers (or even respected bloggers in many instances).

For Live Experimental Interactive Television (LEIT) class this week we were asked to watch a live television broadcast and think of ways that it can be made interactive.  This worked well with my plans of getting to watch one of the few Pittsburgh Penguins hockey games that are televised here.  As far as adding interactivity in to this type of programming a few things came to mind.  One is that during intermission there is often a player or two that is interviewed.  Often that is dictated by their play (someone scores a goal that period).  An alternative to this system, could be for users to send in SMS messages voting for who they would like to hear from during intermission.  Away from the television and moving on to the computer screen, I thought it could be interesting if the camera selections or commentary could be influenced by keywords that occur in conversation amongst a chat room embedded into the site displaying the game.  Those are just a few thoughts so far, but it will be interesting to shift to this frame of mind in the future and continue to think of more.

I’ve continued on work from the first social instrument I created at the beginning of the semester.  With that one I wanted to make the interaction with the device as simple as possible and let the users focus on the interaction with each other instead.  With this second iteration I wanted to maintain that relationship but at the same time push it a little further.  This version now uses Ableton Live to generate the audio (using MIDI).  The users now have some additional controls.  Each have different controls available to them with the exception of a button to trigger notes being the common element between the two interfaces.  On one side there are controls for: note, tempo, reverb and playback.  On the other side there are controls for: instrument, volume, distortion and recording.  Also new in this version is a shared space.  It acts as both a form of feedback as well as introducing a new social interaction.  In addition to lights that indicate recording status and tempo, there will be a proximity sensor that controls pitch bend.  Seeing as this will be located in the middle, it becomes a different social interaction as it is the only control that the two users have to share.

With the addition of these added controls, the two users have to work harder and place an added focus on the way they communicate in order to create interesting melodies.  I have been prototyping what I have so far and am now moving on to building a more permanent structure for the instrument.  In doing so I am designing the interface in a flexible way so that I can easily make changes as I learn more from further user testing.  I now have the laser cutting done and all of the wood cut.  At this point I have a bit more sanding to do and a clear coat to apply to the enclosure.  After that I just need to assemble everything and reconnect all of the wiring.  Once the rest of the fabrication is done I’ll post some more documentation of the final result.  In the meantime, you can see some more photos and a video that demos it’s functionality here.