Comparative Media Studies/Writing

Subterms

    Latest story

    113 Shares119 Views

    Co-creating climate futures with real-time data and spatial storytelling

    by

    Virtual story worlds and game engines aren’t just for video games anymore. They are now tools for scientists and storytellers to digitally twin existing physical spaces and then turn them into vessels to dream up speculative climate stories and build collective designs of the future. That’s the theory and practice behind the MIT WORLDING initiative.

    Twice this year, WORLDING matched world-class climate story teams working in XR (extended reality) with relevant labs and researchers across MIT. One global group returned for a virtual gathering online in partnership with Unity for Humanity, while another met for one weekend in person, hosted at the MIT Media Lab.

    “We are witnessing the birth of an emergent field that fuses climate science, urban planning, real-time 3D engines, nonfiction storytelling, and speculative fiction, and it is all fueled by the urgency of the climate crises,” says Katerina Cizek, lead designer of the WORLDING initiative at the Co-Creation Studio of MIT Open Documentary Lab. “Interdisciplinary teams are forming and blossoming around the planet to collectively imagine and tell stories of healthy, livable worlds in virtual 3D spaces and then finding direct ways to translate that back to earth, literally.”

    At this year’s virtual version of WORLDING, five multidisciplinary teams were selected from an open call. In a week-long series of research and development gatherings, the teams met with MIT scientists, staff, fellows, students, and graduates, as well as other leading figures in the field. Guests ranged from curators at film festivals such as Sundance and Venice, climate policy specialists, and award-winning media creators to software engineers and renowned Earth and atmosphere scientists. The teams heard from MIT scholars in diverse domains, including geomorphology, urban planning as acts of democracy, and climate researchers at MIT Media Lab.

    Mapping climate data

    “We are measuring the Earth’s environment in increasingly data-driven ways. Hundreds of terabytes of data are taken every day about our planet in order to study the Earth as a holistic system, so we can address key questions about global climate change,” explains Rachel Connolly, an MIT Media Lab research scientist focused in the “Future Worlds” research theme, in a talk to the group. “Why is this important for your work and storytelling in general? Having the capacity to understand and leverage this data is critical for those who wish to design for and successfully operate in the dynamic Earth environment.”

    Making sense of billions of data points was a key theme during this year’s sessions. In another talk, Taylor Perron, an MIT professor of Earth, atmospheric and planetary sciences, shared how his team uses computational modeling combined with many other scientific processes to better understand how geology, climate, and life intertwine to shape the surfaces of Earth and other planets. His work resonated with one WORLDING team in particular, one aiming to digitally reconstruct the pre-Hispanic Lake Texcoco — where current day Mexico City is now situated — as a way to contrast and examine the region’s current water crisis.

    Democratizing the future

    While WORLDING approaches rely on rigorous science and the interrogation of large datasets, they are also founded on democratizing community-led approaches.

    MIT Department of Urban Studies and Planning graduate Lafayette Cruise MCP ’19 met with the teams to discuss how he moved his own practice as a trained urban planner to include a futurist component involving participatory methods. “I felt we were asking the same limited questions in regards to the future we were wanting to produce. We’re very limited, very constrained, as to whose values and comforts are being centered. There are so many possibilities for how the future could be.”

    Scaling to reach billions

    This work scales from the very local to massive global populations. Climate policymakers are concerned with reaching billions of people in the line of fire. “We have a goal to reach 1 billion people with climate resilience solutions,” says Nidhi Upadhyaya, deputy director at Atlantic Council’s Adrienne Arsht-Rockefeller Foundation Resilience Center. To get that reach, Upadhyaya is turning to games. “There are 3.3 billion-plus people playing video games across the world. Half of these players are women. This industry is worth $300 billion. Africa is currently among the fastest-growing gaming markets in the world, and 55 percent of the global players are in the Asia Pacific region.” She reminded the group that this conversation is about policy and how formats of mass communication can be used for policymaking, bringing about change, changing behavior, and creating empathy within audiences.

    Socially engaged game development is also connected to education at Unity Technologies, a game engine company. “We brought together our education and social impact work because we really see it as a critical flywheel for our business,” said Jessica Lindl, vice president and global head of social impact/education at Unity Technologies, in the opening talk of WORLDING. “We upscale about 900,000 students, in university and high school programs around the world, and about 800,000 adults who are actively learning and reskilling and upskilling in Unity. Ultimately resulting in our mission of the ‘world is a better place with more creators in it,’ millions of creators who reach billions of consumers — telling the world stories, and fostering a more inclusive, sustainable, and equitable world.”

    Access to these technologies is key, especially the hardware. “Accessibility has been missing in XR,” explains Reginé Gilbert, who studies and teaches accessibility and disability in user experience design at New York University. “XR is being used in artificial intelligence, assistive technology, business, retail, communications, education, empathy, entertainment, recreation, events, gaming, health, rehabilitation meetings, navigation, therapy, training, video programming, virtual assistance wayfinding, and so many other uses. This is a fun fact for folks: 97.8 percent of the world hasn’t tried VR [virtual reality] yet, actually.”

    Meanwhile, new hardware is on its way. The WORLDING group got early insights into the highly anticipated Apple Vision Pro headset, which promises to integrate many forms of XR and personal computing in one device. “They’re really pushing this kind of pass-through or mixed reality,” said Dan Miller, a Unity engineer on the poly spatial team, collaborating with Apple, who described the experience of the device as “You are viewing the real world. You’re pulling up windows, you’re interacting with content. It’s a kind of spatial computing device where you have multiple apps open, whether it’s your email client next to your messaging client with a 3D game in the middle. You’re interacting with all these things in the same space and at different times.”

    “WORLDING combines our passion for social-impact storytelling and incredible innovative storytelling,” said Paisley Smith of the Unity for Humanity Program at Unity Technologies. She added, “This is an opportunity for creators to incubate their game-changing projects and connect with experts across climate, story, and technology.”

    Meeting at MIT

    In a new in-person iteration of WORLDING this year, organizers collaborated closely with Connolly at the MIT Media Lab to co-design an in-person weekend conference Oct. 25 – Nov. 7 with 45 scholars and professionals who visualize climate data at NASA, the National Oceanic and Atmospheric Administration, planetariums, and museums across the United States.

    A participant said of the event, “An incredible workshop that had had a profound effect on my understanding of climate data storytelling and how to combine different components together for a more [holistic] solution.”

    “With this gathering under our new Future Worlds banner,” says Dava Newman, director of the MIT Media Lab and Apollo Program Professor of Astronautics chair, “the Media Lab seeks to affect human behavior and help societies everywhere to improve life here on Earth and in worlds beyond, so that all — the sentient, natural, and cosmic — worlds may flourish.” 

    “WORLDING’s virtual-only component has been our biggest strength because it has enabled a true, international cohort to gather, build, and create together. But this year, an in-person version showed broader opportunities that spatial interactivity generates — informal Q&As, physical worksheets, and larger-scale ideation, all leading to deeper trust-building,” says WORLDING producer Srushti Kamat SM ’23.

    The future and potential of WORLDING lies in the ongoing dialogue between the virtual and physical, both in the work itself and in the format of the workshops. More

    More stories

    • 138 Shares199 Views

      in Environment

      Using game engines and “twins” to co-create stories of climate futures

      by

      Imagine entering a 3D virtual story world that’s a digital twin of an existing physical space but also doubles as a vessel to dream up speculative climate stories and collective designs. Then, those imagined worlds are translated back into concrete plans for our physical spaces.

      Five multidisciplinary teams recently convened at MIT — virtually — for the inaugural WORLDING workshop. In a weeklong series of research and development gatherings, the teams met with MIT scientists, staff, fellows, students and graduates as well as other leading figures in the field. The theme of the gathering was “story, space, climate, and game engines.”

      “WORLDING illustrates the emergence of an entirely new field that fuses urban planning, climate science, real-time 3D engines, nonfiction storytelling, and speculative fiction,” says Katerina Cizek, lead designer of the workshop at Co-Creation Studio, MIT Open Documentary Lab. “And co-creation is at the core of this field that allows for collective, democratic, scientific and artistic processes.” The research workshop was organized by the studio in partnership with Unity Software.

      The WORLDING teams met with MIT scholars to discuss diverse domains, from the decolonization of board games, to urban planning as acts of democracy, to behind the scenes of a flagship MIT Climate Challenge project.

      “Climate is really a whole-world initiative,” said Noelle Selin, an MIT atmospheric chemistry professor, in a talk at WORLDING. Selin co-leads an MIT initiative that is digitally twinning the Earth to harness enormous volumes of data for improved climate projections and put these models into the hands of diverse communities and stakeholders.

      “Digital twinning” is a growth market for the game engine industry, in verticals such as manufacturing, architecture, finance, and medicine. “Digital twinning gives teams the power to ideate,” said Elizabeth Baron, a senior manager of enterprise solutions at Unity in her talk at WORLDING. “You can look at many things that maybe aren’t even possible to produce. But you’re the resource. Impact is very low, but the creativity aspect is very high.”

      That’s where the story and media experts come in. “Now, more than ever, we need to forge shared narratives about the world that we live in today and the world that we want to build for the future. Technology can help us visualize and communicate those worlds,” says Marina Psaros MCP ’06, head of sustainability at Unity, lead on WORLDING at Unity, and a graduate of the MIT Department of Urban Studies and Planning.

      In his talk on the short history of WORLDING, media scholar William Uricchio, MIT professor of comparative media studies and founder of the Open Documentary Lab, suggested that story and space come together in these projects that create new ways of knowing. “Story is always a representation,” he says. “It’s got a fixity and coherence to it, and play is — and, I would argue, worlds are —  all about simulation. Simulation in the case of digital twinning is capable of generating countless stories. It’s play as a story-generator, but in the service of envisioning a pluralistic and malleable future.”

      Fixed dominant narratives and game mechanics that underpin board games have been historically violent and unjust, says MIT Game Lab scholar Mikael Jakkobson, who shared findings for his upcoming book on the subject with the cohort. He argues that board games are built on underlying ideas of  “exploration, expansion, exploitation, and extermination. And, as it happens, those are also good ways of thinking about the mechanics of Western colonialism.”

      To counter these hegemonic mechanics and come up with new systems, community is vital, and urban planning is a discipline that plays a huge role in the translation of space, story, and democracy. Ceasar MacDowell, an MIT professor of the practice of civic design, told the WORLDING cohort that urban planning needs to expand its notion of authorship. He is working on systems (from his current position at the Media Lab) that not only engage the community in conversations but also prompt “the people who have been in conversations to actually make sense of them, do the meaning-making themselves, not to have external people interpret them.” These become dynamic layers of both representation and simulation that are not, as Uricchio suggests, fixed. 

      USAID Chief Climate Officer Gillian Calwell visited the group with both sharp warnings and warm enthusiasm: “When it comes to climate, this world isn’t working so well for us; we better start envisioning the new ones, and fast … We don’t have time to convince people that this is happening anymore. Nor do we need to. I think most of the world is having the hands-on, up-close-and-personal experience with the fact that these impacts are coming faster and more furiously than even the scientists had predicted. But one thing we do need help with on a more hopeful note is visualizing how the world could be different.”

      The WORLDING workshop is designed and inspired by the ideas and practices charted in the Co-Creation Studio’s new MIT Press book, “Collective Wisdom: Co-Creation Media for Equity and Justice,” which insists that “No one person, organization, or discipline can determine all the answers alone.”

      The five multidisciplinary teams in this first WORLDING cohort were diverse in approach, technology, and geography. For example, one is an Indigenous-led, land-based, site-specific digital installation that seeks to envision a future in which, once again, the great herds of buffalo walk freely. Another team is creating 3D-modeled biome kits of the water systems in the drought-stricken American West, animated by interviews and data from the communities living there. Yet another team is digitally twinning and then re-imagining a sustainable future in the year 2180 for a multi-player virtual reality game in a Yawanawà Shukuvena Village in the rainforests of Brazil.

      “While our workshop design was focused on developing and researching these incredible, interdisciplinary projects, we also hope that WORLDING can set an example for similar initiatives across global sectors where distances and varied expertise are not limitations but opportunities to learn from one another,” says Srushti Kamat, WORLDING producer and MIT creative media studies/writing grad.

      Most of the talks and presentations from the WORLDING workshop are available as archived videos at cocreationstudio.mit.edu/worlding-videos. More

    • 88 Shares159 Views

      in Environment

      Conversations at the front line of climate

      by

      The climate crisis is a novel and developing chapter in human and planetary history. As a species, humankind is still very much learning how to face this crisis, and the world’s frontline communities — those being most affected by climate change — are struggling to make their voices heard. How can communities imperiled by climate change convey the urgency of their situation to countries and organizations with the means to make a difference? And how can governments and other powerful groups provide resources to these vulnerable frontline communities?The MIT Civic Design Initiative (CDI), an interdisciplinary confluence of media studies and design expertise, emerged in 2020 to tackle just these kinds of questions. It brings together the MIT Design Lab, a program originally founded in the School of Architecture and Planning with its research practices in design, and the Comparative Media Studies program (CMS/W) with its focus on the fundamentals of human connection and communication. Drawing on these complementary sources of scholarly perspective and expertise, CDI is a suitably broad umbrella for the range of climate-related issues that humanistic research and design can potentially address. Based in the CMS/W program of the School of Humanities, Arts, and Social Sciences, the initiative is responding to the climate crises with a spirit of inquiry, listening, and solid data. Reflecting on the mission, James Paradis, the Robert M. Metcalfe Professor of CMS/W and CDI faculty director, says the core idea is to address global issues by combining new and emerging technologies with an equally keen focus on the social and cultural contexts — the human dimensions of the issue — with many of their nuances.  Working closely with Paradis on this vision are the two CDI co-directors: Yihyun Lim, an architect, urban designer, and MIT researcher; and Eric Gordon, a visiting professor of civic media in MIT CMS/W. Prior to CDI, when she was leading the MIT Design Lab research group, Lim says “At MIT Design Lab, I was working within the realm of applied research with industry partnerships, how we can apply user-centered design methods in creating connected experiences. Eric, Jim, and I wanted to shift the focus into a more civic realm, where we could bring all our collective expertise together to address tricky problems.”

      Deep listeningThe initiative’s flagship project, the Deep Listening Project, is currently working with an initial group of frontline communities in Nepal and Indigenous tribes in the United States and Canada. The work is a direct application of communication protocols: understanding how people are communicating with and often without technologies — and how technologies can be better used to help people get the help they need, when they need it, in the face of the climate crisis.

      The CDI team describes deep listening as “a form of institutional and community intake that considers diversity, tensions, and frictions, and that incorporates communities’ values in creating solutions.”

      Globally, the majority of climate response funding currently goes toward mitigation efforts — such as reducing emissions or using more eco-friendly materials. It is only in recent years that more substantial funding has been focused on climate adaptation: making adjustments that can help a community adapt to present changes and impacts and also prepare for future climate-related crises. For the millions of people in frontline communities, such adaptation can be crucial to protecting and sustaining their communities.Gordon describes the scope of the situation: “We know that over the next 10 years, climate change will drive over 100 million people to adapt where and how they live, regardless of the success of mitigation efforts. And in order for those adaptations to succeed, there must be a concerted collaborative effort between frontline communities and institutions with the resources to facilitate adaptation.“Communication between institutions and their constituents is a fundamental planning problem in any context,” Gordon continues. “In the case of climate adaptation, there will not be a surplus of time to get things right. Putting communication mechanisms in place to connect affected communities with institutional resources is already imperative.“This situation requires that we figure out, quickly, how to listen to the people who will rely on [those institutions] for their lives and livelihoods. We want to understand how institutions — from governments to universities to NGOs [nongovernmental organizations] — are adopting and adapting technologies, and how that is benefiting or hurting their constituencies.  People with direct frontline experience need to be supported in their speech and ideas, and institutions need to be able to take in the data from these communities, listen carefully to discern its significance, and then act upon it.” Sensemaking: infrastructure for connection

      One important aspect of meaningful, effective communication will be the ability of frontline and Indigenous communities to communicate likely or imagined futures, based on their own knowledge and desires. One potential tool is what the initiative calls “sensemaking:” producing and sharing data visualizations that can communicate to governments the experiences of frontline communities. The initiative also hopes to develop additional elements of the “deep listening infrastructure” — mechanisms to make sure important community voices carry and that important data isn’t lost to noise in the vast question of climate adaptability.“Oftentimes in academia, the paper gets published or the website gets developed, and everybody says, ‘OK, we’ve done our work,’” Paradis observes. “What we’re aiming to do in the CDI is the necessary work that happens after the publication of research — where research is applied to actually improve peoples’ lives.”The Deep Listening Project is also building a network of scholars and practitioners nationwide, including Henry Jenkins, co-founder and former faculty member at MIT CMS/W; Sangita Shresthova SM ’03 at the University of Southern California; and Darren Ranco at the University of Maine. Ranco, an anthropologist, Indigenous activist, and organizational leader, has been instrumental in connecting with Indigenous groups and tribal governments across North America. Meanwhile, Gordon has helped forge connections with groups like the International Red Cross/Red Crescent, the World Bank, and the UN Development. At the root of these connections is the impetus to communicate lived realities from the level of a small community to that of global relief organizations and governmental powers.

      Potential human futures

      Mona Vijaykumar, a second-year student in the SMArchS Architecture and Urbanism program in the Department of Architecture, and among the first student researcher assistants attached to the new initiative, is excited to have the chance to help build CDI from the ground up. “It’s been a great honor to be working with CDI’s amazing team for the last eight months,” she says. With her background in urban design and research interest in climate adaptation processes, Vijaykumar has been engaged in developing the Deep Listening Project’s white paper as part of MIT Climate Grand Challenges. She works alongside the initiative’s two other inaugural research assistants: Tomas Guarna, a master’s student in CMS, and Gabriela Degetau, a master’s student in the SMarchS Urbanism program, with Vijaykumar.“I was involved in analyzing the literature case study on community-based adaptation processes and co-writing the white paper,” Vijaykumar says, “and am currently working on conducting interviews with communities and institutions in India. Going forward, Gabriela and I will be presenting the white paper at gatherings such as the American Association of Geographers’ Conference in New York and the Climate and Social Impact Conference in Vancouver.”“The support and collaboration of the team have been incredibly empowering,” reflects Degetau, who will be co-presenting the white paper with Vijaykumar in New York and Vancouver, British Columbia. “Even when working from different countries and through Zoom, the experience has been unique and cohesive.”Both Degetau and Vijaykumar were selected as the first fellows of the Vuslat Foundation, organized by the MIT Transmedia Storytelling Initiative. In this one-year fellowship, they are seeking to co-design “climate imaginaries” through the Deep Listening Project. Vijaykumar’s work is also supported by the MIT Human Rights and Technology Fellowship for 2021-22, which guides her personal focus on what she refers to as the “dual sword” of technology and data colonialism in India.As the Deep Listening Project continues to develop a sustainable and balanced communication infrastructure, Lim reflects that a vital part of that is sharing how potential futures are envisioned. Both large institutions and individual communities imagine, separately — and hopefully soon together — how the human world will reshape itself to be viable in profoundly shifting climate conditions. “What are our possible futures?” asks Lim. “What are people dreaming?” 

      Story prepared by MIT SHASS CommunicationsEditorial and design director: Emily HiestandSenior communications associate: Alison Lanier More