Performance, Art, and Cyber-Interoceptive System

PACIS Pak (2019) and PACIS2 (2021) by M.D. Hosale and A. Macy

PACIS (Performance and Art Cyber-Interoceptive System) is a modular wearable wireless device solution for full spectrum biosensing* for use in the maker and arts communities. Data acquisition happens via the Biopac MP40 (https://www.biopac.com/biopac-science-lab-mp40-catalog-available/). The basis of the technology is a set of signal processing software tools for the analysis of real-time biophysical signals while transmitting the data it is collecting and analyzing wirelessly. The advantage is that the entire analysis system is encapsulated into a set of wearable components, making it much cheaper and less cumbersome to implement in interactive art and performance contexts than in similar systems. The system can be readily combined with additional hardware providing the capability of mapping biophysical signals to experiential, audio/visual media. The tools behind this technology are deployable on many platforms, including software tools that artists and others who are interested in the sonification, visualization, and translation of biophysical data to other media for deployment in performances, installations, and mixed reality environments.

* This includes: electrocardiogram (ECG), electromyogram (EMG), electroencephalogram (EEG), electrooculogram (EOG), electrogastrogram (EGG), ventilatory effort (RSP Effort), electrodermal activity (EDA), pulse plethysmogram (PPG), blood pressure (BP), and blood flow.


Projects Realized with PACIS technology:


HeartBeatDress (2021)

Anouk Wipprecht

PACIS Pak was used in making: HeartBeatDress - Anouk Wipprecht x Swarovski || 'Wearable Emotions' from Anouk Wipprecht on Vimeo.

    In collaboration with a leading crystal manufacturer, Dutch FashionTech Designer Anouk Wipprecht releases her new 3D printed 'HeartBeatDress' – a dress that records and broadcasts something very 'intimate' to you as a provocation to be true to your feelings.
    “At the dawn of this Modern Age, technology has given us the flexibility to investigate endless opportunities with electronics that have become smaller and smaller. Over the past 20 years, I've been connecting our bodies to electronics and integrate this through robotic fashion design. What does it mean when we can connect technological-expressive garments to our bodies, body signals, and even emotions?” the designer questions.
    Reaching farther than curiosity, Dutch designer Anouk Wipprecht's creations combine haute couture aesthetics with digital technology that examines the behavioral, neurological, and emotional states through visceral interaction. With a forward-thinking vision, she believes high-tech fashion is the key to a deeper understanding of how we process our emotions and their effects on our minds and bodies.
    One of her new deviously disruptive designs is a piece she's developed with crystal-maker Swarovski that uses built-in sensors, robotics, sound and light to respond in time with the wearer’s heartbeat. (http://www.anoukwipprecht.nl/)



Siren

#MTF Music Tech Fest, Aviero 2021

Siren is a ritualized performance inspired by the Aveiro para-hydromorphic Regosal - the constantly changing boundary between land and water. Breathing, voluntary and involuntary muscle movements and dual heartbeats compose the sound. The performance is accompanied by an interactive video-art piece edited from a film taken from a site-specific performance at the Aveiro wetlands.

Anya Yermakova (performance, improvisation and composition)
Alexandra Antopoulou (cinematography video art, costume and ritual, spoken word)
Scott Beibin (data collection assistance)
Konstantinos Damianikis (video art and design)
Gonçalo Guiomar (vocals)
Mark-David Hosale (sound design and live electronic composition)
Cindy Macy (breathing and spoken word)
Alan Macy (interactive bio-sensing and sound design)
Mónica Pedro (fish skin harvesting and processing and location scouting)
Diana Vieira (poetry and environmental science)
Débora Umbelino (spoken word)





the body in\verse

Alan Macy, Mark-David Hosale, Alysia Michelle James

Currents 2021 (Virtual) Livestream multimedia performance. June 19th, 2021

    The body in\verse is an online, interactive performance that combines biophysical sensing, emotive state sonification and visualization, and generative poetry to create the scene. The performance provides a deep dive from the world outside of ourselves, that is dissociated by mediated technology, into the interoceptive abyss of our emotive sea. A biophysical sensing system measures the emotional affect of the performer, and then uses that data to drive the sound, abstract imagery, and a generative poetry algorithm. Emotional affect of the performer is assessed through arousal and valence measures derived from correlation of the performer’s heart rate and heart rate variability. An algorithm generates poetry using conversations that take place with the audience as source material. The poetry source material is then algorithmically organized according to its sentiment (positive to negative), and mapped to the emotional affect of the performer driven by the emotional affect assessment from her biophysical measures.



Hand to Heart

Alan Macy

The base PACIS system was built around the Hand to Heart
(https://alanmacy.com/project/hand-to-heart/).

    By transporting a “mimic” of the heart, outside of a participant’s body, another person can touch this very central part of the participant’s nature. In the case of “Hand to Heart”, the embodiment principle is mediated by the senses of sight, sound and touch, so the recipient participant is gifted an intimate, physical connection to the source participant.
    The “Hand to Heart” project establishes a vulnerability for the participants. The two participants, source and recipient, establish a direct link between their sensory frameworks.
    The source participant develops a visceral awareness of the environment’s effect upon their own physiological state and then, by extension, their conscious state.
    The idea of the heart “mimic” is central to this art project because the heart has deep and abiding connections to the emotional and motivational foundational states that support judgement.
    The external heart “mimic” is solidly based in robust technology. The technology amplifies the electrocardiogram (ECG) of the source participant by a rough factor of 2000. This amplified data drives light sources, audio and tactile transducers inside the heart “mimic”. The ECG is representative of the contraction of cardiac muscle, so this same contraction signal drives the activation behavior of the heart “mimic”. The link between the real heart, and the heart “mimic”, is essentially instantaneous and fully analog.



Events

NeuroTechX Showcase Podcast

Applications (hardware and software) featuring Dr. Mark-David Hosale

Hosted by NeuroTechTO Leads Dr. Cris Micheli and Dr. John Griffiths. October 20th, 2021
Also on YouTube and CrowdCast.



About Biophysical Movement and Emotion as Computational Interfaces

Biophysical Movement and Emotion as Computational Interfaces (bioMECI) research explores how technology can help us create deeper connections with the world around us, each other, and ourselves by combining the latest advances in bioinformatic sensing technology with physiological awareness techniques. This research challenges notions of disembodiment and technology by engaging somatic awareness practice and computational media to explore questions of human and machine integration through technology-based artworks that critique our techno-scientific world. It is a point of convergence for a number of fields working in the larger domain of artistic and emotive research both internationally and within Canada. bioMECI research focuses on the development of new and novel interfaces that promise to be extremely concise methods to communicate difficult nuance based on the physiological measurement of emotional valence and arousal for the purpose of creating cyber-interoceptive systems in performance and computational art. These metrics are used to advance knowledge in the domain of performance-based computational art and somatic movement. This endeavour involves the development of novel interfaces and assessment techniques that promise to be extremely concise methods for the physiological measurement of emotional valence and arousal. We develop novel ways to use the real-time emotional state of an individual as an interactive interface in co-collaborative experiences. In addition, we explore how technology can facilitate transference, i.e. empathy, between audience and performer, audience with audience, and performer with performer for the purpose of creating cyber-interoceptive systems in performance and computational art.

Mark-David Hosale

Mark-David Hosale (www.mdhosale.com, www.ndstudiolab.com) is a computational artist and composer who has given lectures and taught internationally at institutions in Denmark, The Netherlands, Norway, Canada, and the United States. He is an Associate Professor in Computational Arts in the School of the Arts, Media, Performance, and Design, Toronto, Ontario, Canada. His solo and collaborative work has been exhibited internationally at such venues as the SIGGRAPH Art Gallery (2005), International Symposium on Electronic Art (ISEA2006), BlikOpener Festival, Delft, The Netherlands (2010), the Dutch Electronic Art Festival (DEAF2012), Biennale of Sidney (2012), Toronto’s Nuit Blanche (2012), Art Souterrain, Montréal (2013), and a Collateral event at the Venice Biennale (2015), Currents New Media (2017), among others. Mark-David’s work explores the boundaries between the virtual and the physical world. His practice is varied, spanning from performance (music and theatre) to public and gallery-based art. His interdisciplinary practice is often built on collaborations with architects, scientists, and other artists.

Alan Macy

Alan Macy (www.alanmacy.com) is currently the R&D Director and a cofounder of Biopac Systems, a biomedical company. Macy is also the founder of the Santa Barbara Center of Art, Science and Technology (www.sbcast.org), a live/work residency and arts laboratory. Macy designs data collection and analysis systems, used by life science researchers, that help identify the meaning of signals produced by life processes. He has 35+ years of product development experience in human physiological monitoring. His recent research efforts explore ideas of human nervous system extension and the associated influences upon perception. As an applied science artist, he specializes in the creation of cybernated art, interactive sculpture and environments.