Data Horizon

An interactive installation about reimagining our digital existence within the space-time of virtual realms

Who are we really, in the densely packed world of binary data?

This installation piece is the exploration of  space-time manipulation of our digital memories by a digital black hole. Whenever I think of time, I also think about space, the vast cosmos, the great unknown. It is where our classical physics on Earth fractures and yields to the morphing, mysterious space-time continuum. This phenomenon is very cinematically theorized in the 2015 film Interstellar by Christopher Nolan. I wanted to take elements of this film and incorporate it into a journey across space-time that behaves slightly differently than our classical science that intuition has taught us.

Time to complete:

My Role: 

Tools used:

Installation space:

 

5 weeks,  Fall of 2023

Installation Designer and Coder in Max/MSP

MAX/MSP

NYU 370 Jay Street

Ideation - A Prompt about Time

This image is the visualization of a black hole in the movie. All that distinguishes the massive space occupied by the black hole is the light along its edges which is called the event horizon. This is the point near the black hole where even light isn’t fast enough to escape the pull of gravity. 

So how can I reflect this story in my installation?

“I pass by a lens that captures me walking by. But it shows me the world I left behind is different than I see with my eyes. The physics is not quite right. And the more I keep walking forward my form keeps changing, darker and there seems to be an event horizon like a black hole around me. I get the feeling I am walking into the past, not mine but someone else’s. 

 

Do I continue to exist any longer if the digital record of me shows me losing my 3D form and shape?”

Storyboarding

Starting point of Interaction : Squares on the ground to indicate the viewer is stepping into the event horizon. The screen at the front is the black hole. As the viewer moves forward, the edges of the human figure approaching spreads out. At the last step, the figure no longer resembles a human but is represented as white pixels, having completely been disintegrated into the black hole. 

End of Interaction: The frames to the side serve as a trail across time of the viewer’s journey. As the viewer reaches the “black hole” screen, the frames divide to show distorted camera captures of the viewer moving. The example from last class on detecting zones can help with tracking, taking screengrabs and necessary sound effects for the viewer as they move across the three squares. 

Technical Design

Inputs:

  • A camera starts capturing frames of the person walking across in 500 ms intervals.
  • OpenCV identifies humans coming from one side.
  • Also tracks when they exit the frame

 

Outputs:

  • Projects on a horizontal screen each of the frames slowly morphing the person into a black hole.
  • It also freezes the previous frames and adds distortion to the perspective.
  • As the person gets closer to the end of the camera view, the frames turn black, and the person’s outline becomes like the event horizon of a black hole.

Making the Event Horizon in Max

To achieve this effect I started working with the edge detection filters in OpenCV and Jitter. I really liked the bright neon effect of the Sobel filter and decided to use that as the center/source. Next, I tried to achieve the effect of the outlines spreading out from the source. I came up with the solution of zooming into an edge filter output and mixing it with the source.

The final effect the viewer will see as they stop in front of the black hole screen - total pixelation of self

Kinect or Camera?

The next challenge was to trigger changes to the digital output as the human walks to the front. For this part, I used blob detection, where the increasing height of the detected blob’s bounding box signaled the human coming closer to the camera. This solved my problem of having to work with a Kinect to set up depth tracking and find a more effective solution with the webcam.

The challenge later was that with multiple moving objects in the frame, there were too many blobs. So I switched to face detection instead which yielded a better result.

Approaching the Black Hole

The breakdown of physics

This is the sideview that has 3 reactive split screens as the viewer walks to the front screen showing the black hole interaction. The side view captures the journey of the viewer and their physical being breaking down, defying classical physics as they are nearing and eventually being absorbed into the digital void. The screens manipulate the real time video feed through various filters and output them sequentially based on the direction the human is walking. The screens will only activate when human motion is detected.

Scaling up in the Space

Once the projectors were set up in the space, I had to readjust the scale of the distance across which the human will move forward. I marked down the positions on the floor where the 3 different effects are triggered. 

When I set this up in the space initially, I used a projector. However, adjusting this to the screen and making sure all the effects were captured without blurring was an issue. So I set this up later on a TV monitor and the result was much better. I could also easily set up the camera to capture the space better where the viewer will walk past.

Audience Interactions

This installation was set up for one day of peer reviews. These were some of the observations I picked up from my classmates:

  • “I feel like I am inside a music video”
  • “There is something interesting happening with the 3 screens on the side”
  • “When I am watching from here (away from the interaction) I see everything. But when you walk there’s a lot going on”
  •  The audience experience seems different from the participants
  • The side view seems more exciting (probably because there was more exploration there, people trying to figure out what was happening since the responses were not as obvious)
  • The understanding of the 4D space was apparent. Someone even drew a connection to the tesseract of the bookshelves shown in the movie (even though I did not intentionally put anything there in reference)