Platform Project 1

enclosed ()

< All projects

In our daily lives, it is sometimes easy to forget how we connect with the world around us. We use our bodies and voices to express our thoughts and ourselves. It is easy to take for granted. But what if you were to lose all this, left only to converse with yourself? Your thoughts are trapped within your own body. You still understand the world, but the world can no longer understand you.

This is the narrative of patients with locked-in syndrome, a condition where individuals become completely paralyzed, making communication impossible. enclosed () seeks to tell this story. It’s about the feeling of isolation, of loneliness. Our work emulates this feeling of being misunderstood. Being present in physical space, but profoundly detached from reality. Showing the importance of communication through showing a world without it.

In our work, we are relating the waterfall to the barrier that those with locked-in-system experience. A barrier that, from one day to the next, suddenly appeared, closing you off from the outside world. The inability to move past the barrier - the waterfall - illustrates the frustration to be so close, yet far from the people around you. The water obstructs your view of reality, hindering interaction and disrupting our human connection.

The waterfall’s movement is driven by the brain activity of a recovered locked-in patient, reflecting on her isolated experience. Using EEG, a device which measures brain activity, we connected this data to specific emotions. Employing an in-house developed machine learning model, EEG data was classified into four different emotional states. The waterfall visually expresses her response in the only language she had at that time—her thoughts and emotions.

How humane is a world without human connection? What is this feeling of complete powerlessness? What is profound isolation when you lose all your ways of interacting with the world around you?

Emergence aims to participate and reflect on new technologies. Through this artwork, we explore the neuroscientific phenomenon of locked-in syndrome, reflecting on how our perception and cognition make us who we are. Additionally, the artwork pictures the development of neuroscientific technology. EEG technology has rapidly developed in recent years, appearing as a way to control games, create art and help people with motor impairments. We envision its potential application for locked-in patients as a way for them to express themselves and their emotions. Using brain-computer interfaces as an artistic medium is a great playground to experiment with specific use cases of this innovative technology.

Why.

enclosed () is an art installation that delves into the world and experiences of patients with lockedin syndrome, exploring themes like connection, isolation, and communication. It engages visitors, calling for reflection on humans’ fundamental need for interaction and expression.

The power of enclosed () lies in its ability to capture human vulnerability, reminding us of the fragility of our everyday abilities. It furthermore encourages appreciation for our capacity to engage with others using all forms of communication.

This installation serves as a statement, raising awareness for yesterday’s conversation, the connections formed today and tomorrow’s smile. Highlighting what is taken for granted by compelling visitors to reflect on often overlooked themes so central to human existence — themes that should be appreciated and understood, stimulating us to see the unseen.

Water curtain.

The water curtain represents an obstruction, when attempting to make human connection, by an encountered barricade. Falling water blurs the projection behind it. The more the projection is blurred, the more it adds to the sense of disconnection. When talking about her experience, Anne’s emotional responses were tied to the level of connection or disconnection she felt. By varying the droplet pattern of the water curtain, these states of emotion and disconnection can be made tangible.

The objective was to identify a range of emotions experienced by Anne during her interview. To achieve this, calibration data was required that indicated Anne’s brain activity patterns when she feels the following emotions: ‘happy’, ‘sad’, ‘stressed’, and ‘calm’. These specific emotions are selected based on the framework of Russell’s circumplex model of emotions. According to this model, each emotion can be mapped onto a two-dimensional plane, where the x-axis represents ‘valence’ (the degree of positive or negative emotion) and the y-axis represents ‘arousal’ (the level of stress or activation)2.

Because of this, it was chosen to classify 4 main emotions, each of one based on a quadrant.

High Arousal, High Valence indicates a ‘happy’ state.
High Arousal, Low Valence suggests a ‘stressed’ or ‘angry’ emotion.
Low Arousal, High Valence is associated with being ‘calm’.
Low Arousal, Low Valence is indicative of feeling ‘sad’.

When an interview answer is being played on the sound system, the watercurtain state will move to the accompanying classified quartile. After the audio snippet, the water curtain will decrease the flow and move back to its ‘rest state’. This rest state consists of randomised droplets, drizzling slowly out of the nozzles.

Imagine a place where the inside and outside world meet, like the doorstep of a house. In our art installation, flowing water is used to symbolise this kind of fluid boundary. According to the famous thinker Carl Jung, in dreams water is seen as a symbol of the edge between:

a. what we are aware of when we are awake, and;
b. the deeper thoughts and feelings that we are unaware of.

Now, think of people with the “locked-in syndrome”. These individuals are fully awake but can not move or communicate because their bodies are paralyzed. Their active minds are full of life. Comparable to a dream when you know you are dreaming, but cannot wake up. The artwork uses water as a metaphor. The water moves and changes, our way of representing brain waves inside these individuals that their still bodies cannot show. A wave oscillating between reality and a dream.

Visuals and audio.

We mimicked the experience of being in a Locked-in state by using visuals to enhance the feeling of disconnection. Showing the frustrating reality of observing the world around you vaguely but have no way to interact with it. In these visuals the silhouettes of people interacting will be visible asif you are looking from the hospital bed towards your surroundings. To make this experience more intimate, the video always portrays a small group of people. These people will interact with each other leaving the first person view in the dark. These videos will correspond to the audio being played consecutively by playing with the composition, amount and nature of the interaction of the silhouettes. The core emotions identified from the interview are happiness, calmness, sadness and stress. These emotions are centralised in the visuals.

The projections are accompanied by an audio component; re-recorded snippets of our interview with Anne, a recovered Locked-in patient, echoing around the room. At first, the auditory clips seem to describe more general feelings of frustration and isolation. But as guests approach the cascading waterfall barrier, the meaning becomes clearer – these are the unheard voices of those robbed of speech and interaction. The audio reveals increasing specificity on the lost ability to act upon one’s thoughts and emotions, though the condition is never explicitly stated. The snippets flow from a sense of vague constraint into an intimate revealing of what it means to be within an unmoving body. To make the experience more accessible, the interview recording will be translated into english and recorded in a studio.

img_7400

1/4

Fabrication.

The fabrication phase started with designing the watercurtain, which encompasses the frame, the watercurtain itself and the integration of the EEG data. The frame was chosen to be made of steel, but that was the easy part. In parallel with the watercurtain itself, we chose to use solenoid valves, since they can be easily controlled with arduino. The biggest hurdle was to make the whole watercurtain tube watertight, as we were working with a lot of electronics in close proximity to the water. After this was achieved, we could finally start testing the integrated EEG data, which our data engineers had worked on. To see all the hard work of the team finally come together was very rewarding and exciting!

- Camiel

Brainwaves and EEG

Integrating live EEG readings during the interviews scientifically grounds the work in real neural processes. Correlating and classifying emotional responses to interview questions demonstrate that these subjective experiences have quantifiable neurological foundations.

The link between neurology and portrayed inner life is created by visualising emotional responses extracted directly from EEG data, acquired as Anne recalls her memories.

The electrical activity recorded by EEG is complex, meaning it’s not just a single wave but a combination of many different waves, each oscillating at different frequencies. This complexity is like listening to a symphony with multiple instruments, each playing its own note. Fourier’s theorem states that you can take this complex signal and break it down into its individual components, which are simpler ‘pure’ waves. Each of these pure waves has its own specific frequency and amplitude.

Applying Fourier’s theorem to EEG data allows us to decompose the brain’s electrical activity into these pure waves. This process is crucial because it transforms a complicated signal into simpler parts that are easier to analyse and understand. By examining the frequencies and amplitudes of these component waves, we can gain insights into different brain states and activities.

In the actual model that was developed, a 40-dimensional (40D) space was used; as each data point contains 40 features. Consequently, there are four categories, corresponding to the four emotions. The SVM’s task is to partition this 40D feature space into four distinct regions, each representing one of the emotions. With a trained SVM model, when we receive new EEG data, we can process a 2-second segment, apply the Fast Fourier Transform to each segment, and obtain a 40-feature data point. We then determine the classification of this new data point based on its position in the 40D feature space.

Recovered locked-in patient

“After four hours, I was allowed to go to the room where my parents, sister, brother-in-law, and boyfriend were. But it was there that things went wrong. In the room, in the presence of everyone, I suffered a stroke. I was swiftly taken to get a CT scan, surrounded by stressful activity. Confusion and panic took over; I had no idea what was happening to me. They rushed me to the emergency room, where I tried to sit but realised my body wouldn’t respond to my commands. Then, I was given morphine and an intubation tube. From that moment, everything faded to black…” - Anne Cox (translated)

"'A few weeks ago I was approached by a Dream Team from the TU Delft, they were looking for someone who had experienced Locked in Syndrome. They wanted to measure my emotions with an EEG.

The entire dream team came to Someren. At home I was interviewed while wearing a cap that measured my brainwaves.

Today was the exhibition! They had created a waterfall with real water, the way this water fell was the speed of my brainwaves. Behind this waterfall, shadows could be seen and you could hear my answers I gave in the interview. The rest of the room was completely dark and filled with smoke. When you walked into the room you got a 'peek into my Locked-In syndrome.'"

"When you see water in your dreams, this is a transition from the conscious to the subconscious." - Anne