New Ways Forward: Immersive and Interactive Works at Tribeca 2021

Back to selection

Breonna Garden, at Wagner Park

Randy Astle

Festivals and events

Realization

Transmedia

June 30, 2021

augmented reality

Interactive movie

new media

podcasts

Tribeca Festival

Tribeca Film Festival

2021 Tribeca Film Festival

Virtual reality

As the first major festival to return in person as the pandemic recedes, Tribeca gave us one more sign of New York's return.

In the heights,

which opened the festival at United Palace on June 9, was a joyous celebration of community (even for those of us watching at home), and even at reduced capacity the festival was a wonderful opportunity to reconnect with the cinema. It also seemed that after the closure of the 2020 festival, this year's event was full of new types of content. Of course, shorts and feature films were always the main focus, but the festival has expanded with a new category of video games. , stand-up comedy, window art, lower Manhattan installations and other work.

In fact, it may have been largely due to the expansion of its immersive works that the Festival changed its name this year, ditching "Film" and becoming the more enigmatic, but presumably larger Tribeca festival. Moviegoers may lament the change — it still doesn't ring true to my ears — but it reflects the growing perception of Tribeca as it becomes something of a festival umbrella at SXSW.

The original film festival was eleven years old when it first presented Storyscapes in 2013, a collection of five interactive films, with no VR or AR in sight (which I

reviewed at the time

). Soon after, festival organizers began to differentiate this core program, which now consists mostly of high-end virtual reality pieces, from its broader immersive and interactive offerings. Everything has always been housed in the same space at Spring Studios on Varick Street, making the split nominal and arguably unnecessary. But because of the pandemic, this year it was crystal clear: the spacious room that normally accommodates everything was now limited to just five experiences – Storyscapes – while other rooms were divided between “Outdoor and Interactive Experiences” and “the Virtual Arcade”, which this year was truly virtual, mainly thanks to an agreement with the Museum of Other Realities to host films VR for remote users of Vive and Rift headsets.

The limited number of rooms in Storyscapes, while obviously occasioned by social distancing requirements, has also enabled other innovations. In a pandemic, virtual reality presents a potentially even bigger problem than a movie theatre, because over the past few years, Tribeca customers have crammed into tight lines to each take a ride in a helmet. The solution this year was not just to reduce the total number of attendees, but to admit in periodic waves of four, so they can move from room to room together. The order was organized to create a kind of narrative flow, and the extra space allowed the creators to create great works Visual arts essentially reflect the mood of their rooms, which in turn helps customers begin to mentally settle into a new project before they even reach for a headset.

Tribeca's divisions into three categories still allowed for different technologies in each program; for example, there were two AR plays in Storyscapes. Rather than proceeding through the plays the way the festival has organized them, I'd like to look at all of them the immersive pieces I've been able to experience in similar categories of two-dimensional films, audio works, augmented reality, and virtual reality. to see everything, especially compared to 2013, and there were several this year that I couldn't see. But here's what I was able to come up with.

TWO-DIMENSIONAL FILMS

Republic

is a surprisingly ambitious interactive film by the French writer

Olivier Demangel

and director

Simon Bouisson

, a team that has already collaborated on several films.

Republic

Demangel took the November 2015 terrorist attacks in Paris as a model, while moving much of the action underground to the underground, drawing parallels with the underground attacks in London, Madrid, Tokyo and elsewhere. starring a live-action young couple as they giddily explore the maze of abandoned pedestrian subway tunnels, but, rather than sticking to them exclusively, the film soon offers the opportunity to watch a second set of employees from office about to catch a train, then a third group of friends who are above ground near Republic Square, just north of the city center. As the attack unfolds , we can see how each of these three groups is affected, bringing together the wider action of the attack and the police response while learning the personal connections between those people who are caught up in the action - a young woman on the street worries about her missing husband, who viewers know is injured and picked up by two of the office workers, while at another two characters leave alone, only to arrive at another group.

We've seen this type of film before, but perhaps without the sprawling scale or seemingly large budget that Bouisson had to work with. What seems most innovative?

Republic

is the mechanism by which viewers select what they want to watch: where most interactive films have breakpoints where viewers can choose the next segment, here each of the three branches is shown simultaneously and without stopping , while viewers can switch between them by swiping up or down, like three stacked video tracks in any editing software (halfway through, I thought of them in terms of "the story of the top", "middle story", and "bottom story"). It's a smooth and intuitive way to move between stories at any time, but it also gets a lot of FOMO because viewers can wondering what they're missing in the other streams and constantly switching between them. I realized it was akin to watching three shows on TV during pre-pause/stream days, where we were clicking back and forth back and do our best to piece together the fragmented stories in our minds. The filmmakers are well aware of this, however, and so, at the end of the story, viewers are presented with a story map (three horizontal bars, essentially ) with the offer to go back and watch the parts they missed.

Creating three simultaneous/linked films is obviously an ambitious undertaking, increasing the total length of the film from 40 to 70 minutes, and it actually exceeded my expectations. Having accepted that I couldn't see everything, I enjoyed the chaos of jumping into each band's story in the media, without any context of what just happened ("Oh, they're on the street now," "Okay, now they're hiding in a closet," etc.). In the end, the three groups come together and the branches are cut, so to speak, but the mental chaos that comes from having only partial knowledge of what is going on remains - which is exactly what each of the characters in the film would have experienced.

Unfortunately, this also meant that the timing was sometimes off when the groups were supposed to cross paths. At one point the wife called her husband's cell and I switched to his video to watch the conversation from his side, but no phone rang. At another point, I caught a character in two places at the same time. The difficulty of planning and timing everything in production must have been immense, so these slight anachronisms weren't very troublesome. my opinion the conceit that each of the three videos was presented as being filmed by the characters themselves on their mobile phones, a 2020 version of

The Blair Witch Project

, which is here updated for the Facebook Live era. mute. The film's website presents the comments as another character who interacts with the actors on screen, and I thought my mute might reflect more about me and my age than the film. idea of the characters constantly filming themselves, especially in a life or death crisis, stretches the plausibility for me - wouldn't they want to turn off their phones and save their batteries while barracked in a darkened room ?—and I don't think the movie would have lost anything if it had just presented the action like a traditional movie, with no justification for the cameras that recorded it.Still, that's a quibble, and who would have made a rather diff erent from the one Demangel and Bouisson wanted, which even in this form is a towering achievement and an intuitive new way to move through interactive movies.

SOUND PARTS

Fluent

, which won the prize for the festival's best non-fiction creation competition, is an audio walking tour by a multidisciplinary artist

Annie Saunders

, with contributions from

André Schneider

and others. Beginning and ending at Zuccotti Park, ten years after the Occupy Wall Street movement, it was commissioned by Brookfield Properties' arts development arm, Arts Brookfield, to connect their One Liberty Plaza and One New York buildings Plaza.Saunders took this rather corporate commission and created a captivating meditation on the history of Lower Manhattan, reminding you that it is an island with a waterfront and a human history dating back centuries before the Dutch arrived. .It uses air and water imagery to bring to life the notion of Manhattan as a living, breathing entity, its shoreline stretching artificially beyond Pearl and Water streets, its buildings soaring and collapsing to make way for ever taller and newer constructions. The living island analogy reflects on listeners as it teaches how to take their own pulse and has them attend to their own breathing. tion and their heart rate; when my visit arrived at Zuccotti Park, my walking partner and I both automatically checked our pulse without even being asked. Sometimes the changes are cataclysmic, as in memories of Schneider from the neighborhood when Hurricane Sandy hit (and, though not mentioned, the shadow of 9/11 hangs over the entire room), but more often than not it reflects the gradual evolution of what is now the United States' most urbanized square mile. Colonial-era buildings like Fraunces Tavern and Trinity Church are now landmarks dwarfed by glass and steel skyscrapers, relics of centuries past that Saunders wants listeners stop and notice.

But it's not just about the scenery. Time and time again, she and Schneider repeat the theme that "it won't always be like this," and they're as much about our lives and relationships as they are about downtown buildings. city.Time is relentless and, for better or for worse, everything is constantly changing.The subtext of being present in the moment, understanding the past and moving forward into the future is reinforced by the buildings that surround you as you listen.The name

Fluent

, of course, is a play on words, obviously including the currents of water and air flowing in and around Lower Manhattan, but also involving the flow of time and the preciousness of the present moment.

The play is at its best when it involves outside voices: a man whom Saunders simply asks for directions but then explains the history of Pearl Street and the waterfront; a Native American who eloquently explains how to connect with the island and how its tunnels and streets are like airways that help him breathe, before adding that he also enjoys taking the subway to Mets games at Citi Field. The sound design is also superb, especially in the final moments. It lasts about an hour and covers nearly a mile and a half, which isn't very far, but sometimes the pace picks up really fast and listeners may find themselves running to catch up their delay. At one point early in my tour, we were interrupted by a man asking for money, so with no way to pause the audio, we missed the cue to turn off Wall Street and spent about ten mi utes in figuring out where we should be and catching on high.Of course, this unpredictability is what makes New York New York: vehicular and pedestrian traffic streams make a pre-recorded walking tour of midtown Manhattan very hard to time, but they reflect the lifeblood of the city today.

Fluent

is free and will be available until September (it starts every half hour from 5.30pm to 8.30pm, to encourage groups to group together); look for the large QR codes to scan in Zuccotti Park.

Knot: a trilogy

unfolds, as the name suggests, in three separate segments, lasting around 20 minutes each.

Fluent

, the audio is designed to be site-specific, but unlike

Fluent

the listener has complete control over precisely what those locations are. The first episode is designed to be listened to while seated on a park bench; the second, in the passenger seat of a car; and the third in a living room or home. The desired effect, of course, is to connect the audio to the physical environment of the listener, and it may be worth adding touch (e.g. wind), sight and sounds from the real world to the audio mix. It takes a bit of effort on the part of the listener, sure, but what film critic would complain about having to go to the cinema to watch a movie?

Tie

the creators of the audio-centric theatre/technology company

dark ground

simply demand the same attention from their audience. Of course, this also introduces an element of variability: in episode one I used a bench in Inwood Hill Park away from any road, so the moment a car stops breaks the verisimilitude slightly, and, more specifically, since I don't have access to cars (not wanting to, say, get a Lyft), I had to listen to episode two in my apartment and just imagine the situation. This limited the effect to audio, but it was still an engaging experience.

The title

Tie

could have several meanings, but most obviously it refers to the story itself, as it is. It's probably not a spoiler to say that the plot turns on itself, so the episode three ends where episode one begins, with recurring lines of dialogue and small events in different orders and contexts, or coming from the mouths of different characters. It's like peeling back layers of the Gordian knot only to find that you you're no closer to unraveling it, and this is accentuated by the stilted language and the formal, deliberately flat readings of the lines (think of Bresson "replaying" his actors emotions). Amnesia reigns: no character can remember where he went or what he did, and one of them can't even remember his name, defining himself by his only "character trait" of smoking. Another woman laments: "I don't know what I represent. At no time does anything seem naturalist e – we are in the realm of the absurd and the surreal.

The resulting tone gives the impression

Waiting for Godot

or, even more strongly, something David Lynch. Is this a dream sequence? Is this all real? Is this character - or am I, as a character in the drama - he's just hallucinating? This surreal feeling is strongest in episode two when a character recounts a dream in which a little boy terrifies a grown man before jumping inside him, likely destroying him from within .The boy is actually a motif throughout all three episodes, and if there's any logical explanation to the narrative, it revolves around the trauma caused when he's hit and killed by a car while chasing his ball. (he was looking for her in the park earlier).Episode three is largely a group session in which the boy dies again and again, the other participants claiming they are all fragments of the boy's own heartbreaking psyche. listener, and a mysterious box (which the listener carried in previous episodes) has the solution to break the knot and the time loop. This of course is straight out of

Mulholland Drive

, and offers just as much closure, because in this case the disappearance of the box from the apartment is what causes the need to return to the park and retrieve it (again).

Tie

not quite reaching Lynch level, it creates a wonderfully dark atmosphere in which to pass a pleasant hour.Through all its urgent and menacing sound and fury, in the end there is nothing left but

silence

In addition to Tribeca Immersive plays, the festival has now expanded to include 14 fiction and non-fiction podcasts (

listen to them here

).This was an addition that was planned for the aborted 2020 festival, which means that some pieces that could not be created at the time are now included.There is a wide range of news , ranging from a documentary about a police raid on a Tantric temple to

Maricopa's Mother

to a fictional Christmas teenage love affair in the projects of

Brooklyn Santa

to hallucinogenic comedy

Imperfection.

a podcast,

Unresolved,

is linked to an article in AR, on the unsolved murders of African-Americans, from the Immersive series.

AUGMENTED REALITY

After the trauma of 2020,

Breonna's garden

was probably the most emotionally satisfying play in Tribeca this year. Breonna, of course, is Breonna Taylor, a victim who quickly became a martyr and an icon as the nation's grief and rage boiled over the summer. latest.

Breonna's garden

is an augmented reality memorial to his memory, not just as an icon, although that is unavoidable, but as a human being.The piece was created by his sister Ju'Niyah Palmer in collaboration with the artist Lady PheOnix, and she bears the fingerprints of those who knew and loved Breonna long before her murder.

The piece, essentially a 3D sculpture, exudes tranquility from start to finish. When you fire up the app, it begins with a photo montage of family photographs of Breonna, which I hadn't seen before in the news, accompanied by mournful music. It cuts to the AR part, a full-fledged garden surrounding the animated figure of Breonna. Palmer appears beside her to pay homage to Breonna as she remembers her - their great -he misses mother, snuggling up on the couch, "not as the media has portrayed her"-then as she disappears, attention shifts to the garden itself.As the viewer moves their handheld device near each flower, it triggers an audio recording.In my viewing, the early ones were also from friends and family members who knew Breonna well, showing that this is a personal tribute from her loved ones, as they s 'speak directly to her to remind how how boring it could be, road trips and other experiences. Proximal audio works well to entice viewers to move around and explore, getting closer to each flower. Soon we move on to voices of people who didn't know Breonna but were moved by her death, and then there are others who take the opportunity to pay tribute to their own lost loved ones who influenced them. Viewers can, of course, record themselves- same one of these messages, thus planting their own flower and making

Breonna's garden

include everyone who cried. With the simple graphics of the flowers, this audio documentary of people simply talking about their grief and gratitude is extremely powerful, much more than a portable AR application could be.

During Tribeca, the piece was housed at the WarnerMedia Innovation Lab on West 21st Street, where it premiered with Palmer in attendance.Another space, however, has been set up and remains open in Wagner Park, which is around the corner northwest of the Battery, south of the Museum of Jewish Heritage and near where Little West Street joins Battery Place. Large QR codes invite passers-by to download the app and watch the experience there; took the photo above facing south towards Pier A, the first pier as you round the battery waterfront. But the app is also

available online

for anyone who wants to see the experience, making it a permanent, albeit digital, memorial to Breonna and all those remembered here.

While most of this year's AR parts were designed for wearable devices,

Critical

Distance

was developed with Microsoft for use on their HoloLens 2 headset. Co-created by Adam May and Amy Zimmerman, the article is about a group of Southern Resident orcas that reside off the coast of Washington. This pod is both endangered and studied extensively by biologists like

Ken Balcomb

, with each of the approximately 75 people named and recounted. And while the status is insufficient to determine the conservation status of killer whales globally, this group has been declining for years as it deals with issues such as canals busy boating areas off the Salish Sea and the dwindling supply of salmon which is the primary food source for resident killer whales ( i

interviewed director Josh Murphy

on his documentary

artificial

about salmon depletion in Tribeca two years ago).The main environmental problem of

Critical distance

is the underwater noise pollution caused by ships and how it interferes with echolocation and communication between whales. If they can't echolocate, they can't hunt the few remaining salmon. affects them in other ways: Zimmerman told me that in the calm of the COVID-19 shutdown, three calves were born, indicating that the vessels can also interfere with mating.

The room held at Storyscapes, and how it will appear when visiting science museums like the Smithsonian and London's natural history museums, consists of a large circular space with white walls that allow for video projection. This two-dimensional screen (which reminded me a little of the filming of Jon Favreau

The Mandalorian

) enables the communication of information, such as the family tree of the entire group and the lifespan of all known whales that lived and died there, and the transparent nature of the HoloLens enables both of these levels of display. The truly fascinating content, of course, occurs in the helmet, first a few fish, then a few individual killer whales, then the whole pod swims around you. This is first interrupted by a small fishing boat , then by a large ship, and audio and visual scrambling to convey to human viewers how disruptive this noise is to creatures that rely so heavily on their hearing. different places in the space, and this social aspect of the room is what made Zimmerman, a five-year Unity veteran, want to create it in AR rather than the more visually siled technology of reality. virtual. Although catalyzing action is always the most difficult task for environmental films and art, she hopes that if viewers have a social experience, it can better promote social change.

The animals in the piece are ethereal, ghostly, somewhat monochromatic translucent beings that glide past or through you and seem to respond to your touch. This is thematically appropriate, as the entire piece deals with the precarious nature of the very existence of these whales. But there's something about knowing that they're digital recreations of actual animals living in the Pacific right now - each one has a name and a story - that makes it even more haunting. These whale ghosts might soon to be all that's left, and, however close they feel in this room, the whales' physical absence from, say, a building in Manhattan, eerily echoes their potential future in the water .It may have something to do with the ontology of the photographic (or digital) image, or the work of art in the era of mechanical (or even digital) reproduction, but there is something thing to see as well the image of killer whales. it was different from, say, watching a traditional documentary about them. There's not as much information conveyed in

Critical distance

like in a book or a feature film, but there is a different emotion. You reach out to touch them but feel nothing. Distant so close! Perhaps it is an unintended sense of the title, that even when we are away from these animals, we can somehow feel – or be – close to them. And it is essential that we do so.

Jupiter Invincible

tells the story of a light-skinned slave in the antebellum South. Tribeca's most deceptively simple play, it's told through a comic book written by Pulitzer Prize-winning poet Yusef Komunyakaa, featuring the art of the accomplished artist

Ashley A.Woods

.In itself, as a printed work, it is a wonderful addition to graphic novels created by black people, and its opportunity to focus on the heart of the historical African-American experience, in the evils of slavery , couldn't be better. But Komunyakaa and Woods also hired a documentary filmmaker and an AR artist

Ram Devineni

To enhance the book: By downloading an app to a portable device, readers can hover their phones over each page to see Woods' works come to life. The variety of Devineni's work is impressive, with things like drawings springing from the three-dimensional page, character animation - a rearing horse, a man walking with his dogs - and live images - a stream in the woods, for example, bubbles to life. The audio matches all the visuals by Devineni, increasing the immersion of the work.

Nine years ago, I read (saw?) the book and the app

The Fantastic Flying Books of Mr. Morris Lessmore

from Moonbot Studios, in one of my first encounters with augmented reality (

Director

subscribers can review this article

here

).While their work was groundbreaking at the time, it's hugely gratifying to see how much the quality of book-AR combos has improved, both in the ease of triggering the app to view the AR and in the impressive depth, range and resolution of Devineni's work. This isn't his first foray into print AR, as he's previously used his non-fiction film background to include documentary interviews with rape victims in his comic book series.

Priya's Shatki

At Storyscapes, Devineni told me that the next step in his work will be to use green screen technology to create a full movie-comic book hybrid via AR. Like a movie where you literally turn the pages, it will be a new medium delightful as it gets perfected, especially when AR glasses become common enough to remove the wearable device between readers' eyes and the page.

With all of these exciting advances in technology, it can be easy to overlook the power of

Invincible Jupiter

Komunyakaa's story has created a realistic and magical tale that infuses superhero elements into a story of a runaway slave, as Jupiter gains the power to heal from any wound. This makes him a character passionnant lui-même, mais il y a plus à l'œuvre ici qu'un Black Wolverine ou un Luke Cage d'avant-guerre.Au centre de cela se trouve une vision que Jupiter a au milieu du livre, après avoir été battu presque à mort et frappé par la foudre, dans laquelle il rend visite à sa mère en Afrique et apprend la majesté de son peuple.Quand il se réveille, sa peau, initialement suffisamment claire à cause de l'albinisme, je n'ai pas réalisé au départ qu'il n'était pas caucasien, est devenue un brun riche, et il est maintenant un fier Africain.C'est la même fierté que le grand public a retrouvée avec

Black Panther

en 2018, et en effet Woods a dessiné Shuri pour une couverture de bande dessinée Marvel.Et c'est une manière appropriée

actionnable

fierté pour le moment présent, alors que Jupiter baisse alors la tête et se met au travail pour libérer son peuple.L'histoire reste inachevée à la fin, donc j'espère que d'autres problèmes, avec ou sans AR, suivront.

Director

Lauren Wissot s'est entretenue en profondeur avec Komunyakka et Devineni à Tribeca, que les lecteurs peuvent trouver

here

Procession

est une nouvelle œuvre de l'artiste

Dustin Yellen

qui s'appuie sur sa pratique sculpturale en ajoutant un composant AR pour les appareils portables.Le travail de Yellen, bien que multidisciplinaire, est largement basé sur la sculpture, utilisant des feuilles de verre feuilleté empilées pour intégrer des images bidimensionnelles et d'autres objets pour créer des tableaux figés ;thématiquement, son travail se concentre souvent sur l'Anthropocène et la façon dont les humains manipulent l'environnement, généralement à des fins catastrophiques.Dans le cas d

Procession

, une petite pièce verticale, de la taille d'un être humain, a été installée au sixième étage de Spring Studios, à un étage de la galerie Storyscapes.La sculpture elle-même présente un gratte-ciel chancelant construit essentiellement de colonnes et de sols ouverts, recouverts de végétation d'une manière qui évoque des scénarios post-apocalyptiques.Peupler ce bâtiment est une foule d'animaux anthropomorphisés miniatures - un lion de mer portant une veste verte et un jean bleu, un rat dans une chemise bleue portant un champignon géant, un kangourou dans une combinaison de travail avec des planches de bois sur le dos - tous construits à partir de photographies et collés entre les feuilles de verre.

Une application de réalité augmentée anime ces personnages, ne créant pas un récit linéaire mais donnant vie aux images figées de la sculpture.Ceci est disponible sur place à Tribeca mais, plus une pièce d'accompagnement qu'un élément lié à l'œuvre physique, ne nécessite pas du tout la sculpture.L'audio apporte le monde sonore de la nature, filtré à travers une musique tonale, à la pièce, tandis que le mouvement des créatures augmente leur intérêt visuel.Malheureusement, lorsque j'ai essayé d'exécuter l'application sur mon propre téléphone plus tard, elle n'a pas réussi à se charger à plusieurs reprises, un risque de pièces AR portables.La pièce est décrite comme impliquant les téléspectateurs dans les ravages causés par le changement climatique créé par l'homme. est chronométré à quinze minutes.Dans tous les scénarios, il sera intéressant de voir des artistes plus traditionnels comme Yellen intégrer la RA dans leur travail, ouvrant de nouvelles voies créatives pour eux et leurs collaborateurs centrés sur la RA.

Il aurait été facile de rater

Unresolved

dans son installation dans le coin nord-est de la batterie, derrière le monument des Pays-Bas.Là, une grande mais modeste sculpture en bois ressemblant à une pagode sans toit soutenait des feuilles de verre coloré, chacune portant des rangées de noms.C'était un mémorial aux Afro-Américains tués par les Blancs pendant les années des droits civiques, et il contenait tragiquement 151 noms.La sculpture, bien sûr, était contemplative en elle-même, mais la partie en réalité augmentée a donné vie à chacun de ces noms.Une œuvre créée par de multiples partenaires sous la direction de la directrice artistique Tamara Shogaolu,

Unresolved

dans ce format représente certaines des meilleures sculptures AR qui existent aujourd'hui.Pourtant, les téléspectateurs qui l'ont manqué là-bas peuvent regarder une version en ligne via un navigateur Web

here

Nouvelles voies à suivre : les œuvres immersives et interactives à Tribeca 2021

Dans la sculpture, chaque nom avait un petit code QR sur le côté, et le scanner ferait apparaître les données de cette personne.Cependant, cela ne s'est pas lancé directement dans leur histoire;d'abord, le spectateur devait prononcer le nom de la personne à haute voix trois fois.Cette « prononciation de leurs noms » ritualisée, suffisamment forte pour que les autres l'entendent, semblait être une manière respectueuse de les incarner à nouveau et de se rappeler qui ils étaient et comment ils sont morts ;comme il s'agit d'une partie cruciale de leur commémoration, la version en ligne contient également cet élément. It's similar to

Procession

in that it's a sculpture that triggers a handheld AR component, but the key difference of course is that where

Procession

shows just one large, intricate animation,

Un(re)solved

contains a legion of small ones. Perhaps the greater similarity is with

Kusunda

, as noted above, because these are people who are in danger of being forgotten, whose deaths may slip into the fog of time, and the act of saying their names brings them back and makes their memory that much stronger.

Once this was done, the app would present any information that was available about the person. I saw information presented in text, audio, and video, and, moving around the sculpture, I found names that were remarkably different in their stories, but all wrenchingly similar as well: a fifteen-year-old boy shot by a White police officer while waiting to go inside his New York City school; a girl killed in a drive-by shooting by three White men on the day she graduated from high school; a middle-aged couple, active in the civil rights movement, killed by a bomb in their home on Christmas night, which was also their anniversary. Fifty-eight-year-old Bessie McDowell was asleep in bed on June 14, 1956, when two White men, a father and son, came to collect on a loan they'd made to Bessie's nephew, who was staying with her. The young man told them he didn't have the money, and one of the men reportedly slapped him, causing him to run inside for safety. The father then shot a handgun through the window, striking Bessie in the face. He later claimed his shot was in self-defense—he thought the boy was going for a gun—and he didn't know anyone else was home, so he only served 12 months in prison and paid a $500 fine. Of course this story, which I chose randomly, bears uncanny resemblance to Breonna Taylor's, and it was fitting that this piece was so close to

Breonna's Garden

in the Battery as part of Tribeca's celebration of Juneteenth.

Un(re)solved

is deceptively simple, but I suspect that the more time you spend with it, the more similarities you'll see in the stories, causing you to not only honor these victims but wonder how far our society has actually come. In that way, it bears a striking resemblance to the VR piece

The Changing Same

, discussed below.

VIRTUAL REALITY

I wrote about

The

Changing Same

, which won Tribeca's Best Immersive Narrative Competition Award, when it

premiered at Sundance in February

. This time, however, I was able to view it on a proper headset that the film was designed for, rather than on my own Quest 2 at home, followed by a wonderful in-person conversation with the creative team (a small joy of the vaccine roll-out). Focused on the Black American experience, the title riffs on the fact that the more things change the more they stay the same; for all the country's progress in race relations and civil rights, many parallels with the past remain sadly ironclad. Realistically animated (seemingly from volumetrically captured performances) in six degrees of freedom (or 6DoF, meaning viewers can move around the space and change their perspective), the piece starts in what seems like a southern bayou, where the viewer meets the film's narrator, a young Black man, before flying through the air in a beautiful interlude and landing on a peaceful suburban street at night, where a police squad car pulls up, lights flashing, and the cops, one White and one Black, proceed to arrest you and your guide because of a match to a vague description. The situation escalates as the cops assault the young man, but as the scene shifts to the precinct the blame is falling on him, throwing him into a racist prison system through no fault of his own. A flashback to a slave auction house—where the same actor is protesting that he was born a free man and is not a runaway slave—drives the parallel between chattel slavery and modern policing home. It ends on an optimistic note, however, as viewers leave that reality and envision a future—or a present—of African-American equality, beauty, and pride.

For me, a white male, VR pieces like this and Roger Ross Williams'

Traveling While Black

(2019) are incredible tools for building empathy. As with

Critical Distance,

there is a definite educational component, but it's not the facts as much as the emotion of going through those situations that reveals new insight—insights that are sadly far too common for so many Americans today. Again, something about being embodied in the VR space, helplessly facing a police officer who could destroy or even end your life for no reason, feeling yourself getting sucked into the system, has a different impact than watching films, reading, or engaging in real-world activism—at least for me.

And that's the goal, the kind of empathy it's meant to catalyze. The creators Michèle Stephenson, Joe Brewster, and Yasmin Elayat want to lay bare the systemic roots of modern racism, and they want to affect how people feel about it so that they become involved in ending it. This is the first of three

Changing Same

films; future episodes will deal with topics like conditions in contemporary/1825 New York City and the horror of spectacle lynchings. And they plan on releasing the films not just online but through controlled venues and events like tent shows, museums, festivals, and community gatherings. They want viewers to see the films in public so that conversation can immediately ensue, community can be built, and societal progress made. Perhaps their most important outlet will be schools, where young people can be exposed to these issues with more emotional power than they would get from a textbook. At a time when many pundits and lawmakers are making hay out of critical race theory, works like this will provide educators a welcome counterweight to provide their students with accurate, impactful information. I can't help but think of

The 1619 Project

(and the recent political pushback over Nikole Hannah-Jones's tenure) in conjunction with

The Changing Same,

in terms of how both elucidate the connections between the present and the past, and I hope that as the films are finished they can help countervail those who would rather bury the United States' racist past. As many have said, it's only by understanding our history that we can alter our future.

Inside Goliath

, a British-French coproduction led by co-creators

Barry Gene Murphy

and

May Abdalla

, is an animated documentary about mental illness. An anonymous narrator recounts his story about living with schizophrenia and psychosis and his descent into alcohol and drug addiction in London's seedy streetlife. Meanwhile animation in 6DoF illustrates his morphing flat that grows increasingly claustrophobic, or his isolation at a club as his addictions surge. One fun sequence near the middle is an ironic video game, presented as an arcade machine, that the viewer plays; a side-scroller that approximates a 16-bit game at best, you navigate the narrator through the streets as he gathers booze and drugs and avoids the violent gangs that, in his real life, were far too common. This was a surprisingly engaging moment in the piece and could point the way for two-dimensional pieces that could convey a similar educational message.

The title

Inside Goliath

makes me think of a giant that not only stands in front of someone with severe mental illness, but that has fully enveloped them so that there's no way out. Another apt metaphor might be Kronos devouring his children, leaving them helplessly inert in the darkness. In fact, the narrator's despair, leading to indifference about the inevitability of his fate, is perhaps the most chilling part of his story; while there obviously is help for those battling mental illness,

Inside Goliath

does a wonderful job conveying how difficult it can be to reach out through the fog and try to reach it. Like

The Severance Theory: Welcome to Respite

, discussed below, this piece conveys less factual information about mental illness than it re-creates what it's like to live with it and the trauma that can result; in that way it resembles

The Changing Same

and some other of this year's pieces as well.

Kusunda

, which won the festival's Storyscapes Award, is a wonderfully timely document.ary about disappearing languages, a global phenomenon that's receiving far too little attention. Kusunda is a language so endangered that not only have most people never heard of it, fluent speakers probably barely reach into the double digits. The people all live in the mountains and jungles of Nepal, where Nepali has subsumed it as a dominant language. VR filmmakers Felix Gaedtke and Gayatri Parameswaran, co-founders of the Berlin studio

NowHere Media

, saw a chance to shine a light on the language, and perhaps help rescue it, through a woman named Gyani Maiya Kusunda who taught it to younger members of her community. Initial filming took place, but she died just as they were about to journey together to film the majority of the production in the backcountry; filming was then further hindered by the COVID pandemic. Parameswaran recounts all of this in voice over, and she told me that they decided to include the making of the film within its narrative to show how tenuous the language's position is. Her narration describes the difficulty of setting up a volumetric capture camera in a remote hillside village, for instance, and at one point their camera array overpowers the electrical supply and causes a blackout. This illustrates the remoteness of Kusunda speakers as much as any information conveyed in the film.

Despite all these difficulties, they succeeded in making an intriguing interactive film, with wonderful visuals and 6DoF; at Tribeca viewers were seated, but nevertheless they could physically lean and move around the space slightly to take in new perspectives, an effect that always increases a VR subject's verisimilitude. The Kusunda speaker who became their new subject, a shaman named Lil Bahadur Kusunda, has nearly forgotten it. This fills him with regret, and he seems delighted with the film crew and their interest in the language. He recounts some stories from his life, which are animated in TiltBrush, and is pleased with his granddaughter Hena Kusunda, a young woman who is determined to learn the language and keep it from extinction. In the end she performs a song for him in Kusunda, making it the only song that exists in the language, and it's a quietly moving moment that gives a dose of optimism in the face of such harsh odds. Capturing her singing with a 3D volumetric image enhances the presence of both her and her song.

Since

Kusunda

deals so much with a spoken language, it's appropriate that one of its most compelling features is in its audio. A language dies when it's no longer spoken, so at various moments in the narrative the viewer has to learn a Kusunda word and speak it out loud in order for the story to advance; in two cases this determines which animated story you see—choosing between a story about a bear or a tiger, for instance—which adds an element of interactivity to the film. But the emotional function is much like that in

Un(re)solved

: by speaking these words out loud, you become a partner in ensuring that this language and the people who spoke it are not forgotten, are still in a way present today. In this case it's also rather fun, and made me think more about how audio prompts can be used for interactivity in the highly visual medium of VR.

Parameswaran also told me some of the things their team is doing to preserve the language. The VR film itself will be available on Steam for the general public, and they're working to get it into museums as well for those who wouldn't encounter it online. Perhaps more importantly, they've made a traditional 2D film and are working in the community in Nepal to educate people and spread the use of Kusunda among those whose ancestors may have spoken it. This outreach, which is in the best tradition of Griersonian documentary, may be the most important element of the entire project, because if they succeed in exciting more young people like Hena, then Kusunda is in good hands for the future.

Lovebirds of the Twin Towers

is the latest work from filmmaker and VR documentarian Ari Palitz, who is known for work like producing

Clouds Over Sidra

(2015), about the Syrian refugee crisis, and co-directing

The Last

Good-bye

, about the Holocaust. This latter piece, which I

spoke with him about

when it premiered at Tribeca in 2017, used volumetric capture to record the testimony of survivor Pinchas Gutter at the Majdanek Concentration Camp, creating a physically palpable presence of the place as Gutter guides viewers through the various rooms (even though it was stitched together in postproduction).

The DNA of

The Last Goodbye

is present in

Lovebirds of the Twin Towers

, but the technology has only improved—and the tone is remarkably lighter. Created for this September's twentieth anniversary of the 2001 attack on the World Trade Center, it focuses on a couple, Carmen and Arturo Griffith, who both worked as elevator operators there. They met, fell in love, and had innumerable wonderful experiences in the towers and the surrounding plaza; they also lived through the 1993 car bombing—Carmen was right there when it happened—and the September 11th attack, when they were in different buildings and, both injured, didn't know the other's fate for several days. Their reunion made the news in 2001, and Palitz told me that he wanted to focus on an uplifting story like this for the anniversary because there was so much goodness and joy that came out of the place but that has been neglected in the face of the immense tragedy. While this is obviously appropriate, for this piece Palitz wanted to focus on the positive human stories of the WTC rather than just the attacks and their aftermath.

The piece at Storyscapes largely consisted of an interview with Carmen, who, like Lil Bahadur Kusunda, is seated but filmed volumetrically. She sits against a black background, allowing her words to carry the weight of her story, but while she talks the scene shifts to the outdoor plaza, the view from the roof, and other locations from her memory that no longer exist. The photorealism of these animations is remarkable, and, as someone who first came to New York in 2002, a year after the attack, it was a wonderful way for me to experience this part of the city's heritage, being fully immersed in a space that I had only ever seen in photographs and films. Visually, then, the accomplishment here arguably exceeds that of

The Last Goodbye,

and hearing Carmen describe her story is as emotionally fulfilling as anything else at Tribeca this year.

Arturo's interview remains to be completed by September, and the piece will be installed at other sites. As amazing as the finished VR piece should be, the piece also includes 2D interactive interviews with Carmen and Arturo, with innumerable questions that promise to be just as enlightening and educational; this was how I heard the details of her story about the 1993 bombing. Palitz is excited by the educational opportunities that this format presents, telling me that he's recorded his largest oral history yet with William Shatner, covering his entire life and career. If the topics are already ranging from 9/11 to

Star Trek

, the interactive video interview may be one of museums' prime tools in the next few years.

The Severance Theory: Welcome to Respite

is an immersive theater piece presented in VRChat. Designed as a social space where users can don animated avatars and then physically speak and interact with each other, VRChat has recently been adapted by theatrical innovators to create interactive theater pieces, where audiences and live performers alike inhabit the same world. This not only allows for geographically dispersed people to come together into a shared online space, but for actors to speak directly with viewers to create unique experiences with each performance; with titles like

The Under Presents: Tempest

and

Finding Pandora X,

this is a surging collaborative space between theater and VR professionals that filmmakers would do well to be aware of.

The Severance Theory

, writer and director Lyndsie Scoggin and her team address childhood trauma, specfically dissociative identity disorder. One participant takes the role of Alex (a name that fits multiple genders; during the setup the viewer has a chance to identity their pronouns), while others observe unseen. After the initial introduction Alex enters the narrative world by thinking back to their childhood, circa 1993, and an evening when they had just returned home to their parents after staying with their aunt for some time. Life at home varies wildly between loving and fraught, with both parents exerting obvious effort to make Alex feel welcome and cover over the cracks in their marriage: the mother involves Alex in making macaroni and cheese, and praises their drawing—in my performance the actress did a commendable job incorporating a picture of a unicorn I drew—while the father takes them up to the attic to experience a star projector and recount how he named a star after Alex the night they were born. But despite these efforts something more sinister than their marital struggles lurks in the shadows, literally, and revolves around an unspoken trauma that Alex experienced with their aunt. When the mother plays a music box that was a gift from the aunt, a dark mist oozes down the staircase and toward the viewers; in two subsequent episodes this cloud becomes more brazen, attacking Alex's self-worth through a disembodied voice and finally attempting, apparently, to become physically violent. These moments allow the rest of the audience to participate, as they come to Alex's aid as unseen voices, chanting that they must protect Alex and fight off the shadow. The visual metaphor is poignantly done, although it's broad enough that it could serve as a symbol of many types of mental illness and distress. After the piece ends—happily but ambiguously—a final salon includes information about dissociative identity disorder specifically for those who are interested.

There was enough locomotion between different rooms of Alex's house that by the end I was developing a good case of motion sickness, a common ailment in VR, but there was much more to praise about the piece, and I hope it has a post-Tribeca release as well. Scoggin uses her background in escape rooms to create a kind of metaphorical escape room here, with the feeling of claustrophobia and entrapment permeating even the piece's gentler moments, and some simple puzzle-like work to push the narrative forward. I saw the piece near the beginning of its Tribeca run, and even then the actors were tremendous in their adaptability and improvisation to my own actions, their clarity in helping shepherd a non-professional actor/VR performer through the process with them (including in the group onboarding process), and their vocal and physical performances—acting is no easy task when one's face is no longer visible. While in the role of Alex it was hard for me personally to not think like a screenwriter or actor performing for the rest of the audience, but with interactive VR “theater” set to grow exponentially it's something I think audiences will grow increasingly accustomed to in the next few years (it could be argued that we're seeing the emergence of an entirely new art form, not film, theater, or video game but something that uses parts of each). And hopefully in a few years improvements in photogrammetry, volumetric capture, and similar technology will allow us to see actors' actual bodies rather than animated avatars. That's the future of VR theater that I'm most looking forward to, and in the meantime pieces like this are developing the narratives that will get us there.

We Are At

Homepage

comes from the married team of

Michelle and Uri Kranot

, experimental animators and founders of the Danish animation studio

TinDrum

. An international co-production with

Floreal Films

in France,

Late Love Production

in Denmark, and the National Film Board of Canada, it's a VR adaptation of their 2D film

The Hangman at Home,

which in turn is based on Carl Sandburg's oft-adapted 1922

poem

. Sandburg muses about the home life of a man whose occupation is to kill, and whether it's a typically domestic scene, the horrors of his day buried or even blithely laughed off. The Kranots focus on a few key phrases—”play horse,” “bonfire”—and spin a series of tableaus, which, remarkably for a film based on a poem, are poignantly wordless. Presented in miniature within a proscenium arch, these moments are drawn mostly in inky, flowing black and white, and the hangman and other characters have a listless, uncomfortable feel to them, like a hybrid of Robert Crumb and Munch's

The Scream

—especially when they break the fourth wall and stare directly at the viewer. Topics go beyond the poem's literal content to evoke its tone of unease, with the hangman performing tasks like mourning a woman dying in bed, masturbating under his own covers, or arriving home to a bombed-out apartment with a view of a warzone through the missing wall.

In between these moments of miniature theater are full-scale 6DoF interactive scenes, where the viewer has to open and pass through a door (drawn in a chalky outline), or duck down to climb through a cabinet. At Storyscapes this meant that the four participants circled around a large open space, as they moved clockwise from one scene to another (and requiring a fair amount of wrangling by the staff to avoid collisions). Knowing this made these moments of walking forward somewhat uncanny, like a Kierkegaardian step into the dark, as I've rarely had license to move quite so far in a VR headset before. But it was also quite clever and entertaining as the four of us somewhat simultaneously figured out the tiny puzzle to move to the next scene.

The final scene breaks this pattern, as the proscenium is gone and the viewer is now within a full-scale drama in which a pregnant woman contemplates burning a stack of books and committing suicide by jumping out the window. This scene ends on a final moment of interactivity as the viewer now holds the matches and can decide whether to light up the books or not. The ending is enigmatic—it could literally go either way—but still encapsulates the tone of the entire piece, that just beyond these tranquil scenes everything is about to catch on fire. The Kranots, in fact, seem to have abandoned Sandburg's last line, “Anything is easy for a hangman, I guess,” in favor of its exact opposite: everything seems fraught for this character, on the brink of annihilation.

Leos Carax's Green Period: DP Caroline Champetier on

Annette

and Her Work with Jacques Rivette, Claude Lanzmann and Philippe Garrel

Nicolas Rapold

Cinematographers

Festivals & Events

interview

Aug 9, 2021

Sundance Festival Director Tabitha Jackson Announces Details of 2022 Event; Vaccinations Required for In-Person Utah Events

Scott Macaulay

Festivals & Events

Aug 3, 2021

“We Had to Combine Three Different Islands”: Antoneta Alamat Kusijanovic on Cannes Caméra d'Or-Winner

Murina

Nicolas Rapold

Directors

Festivals & Events

interview

Screenwriters

Jul 26, 2021