Virtual Multi-Sensory Journey Through Art Collections

Project overview

Introduction

The project explores how digital art collections could become more accessible and engaging. We proposed a tool that enables artworks to be experienced through image, sound, movement, context and the personal interpretations of other users.

The concept was based on the idea that experiencing art should not be limited to describing its visible features. Emotions, associations and the diversity of individual responses are equally important.

Process

Understanding the user

Throughout all phases of the project, we had the opportunity to speak with people with different needs. We first conducted interviews and then consulted rapidly emerging concepts and prototypes with users, including people with disabilities, teachers, educators and technical professionals.

Key research insights:

— audio description should not be reduced to describing only the visual details of an artwork,

— users wanted to “hear the emotions of sighted people”,

— observing the diversity of how different people receive art can be valuable,

— engaging various senses and presenting different contexts are important.

Pain points:

— inability to experience art fully due to disabilities,

— difficulty sharing a personal experience with others,

— lack of independence during group museum visits.

Starting the design

We placed all ideas and insights on a chart that assessed how well they matched user needs and whether the available digital resources could support them.

Based on this, we defined two possible directions:

1. building a community for sharing experiences,

2. creating interactive, virtual and multi-sensory paths.

We collected feedback on both ideas from mentors and users. The final concept combined their strongest elements: a virtual multi-sensory journey through art collections in which users could share their unique experiences and enrich someone else’s encounter with art.

For the selected flow, I proposed the information architecture and low-fidelity mockups.

Refining the design

The next stage involved developing the visual identity and high-fidelity mockups.

An important part of the project was the Immerse experience, which allowed users to move around an artwork through head movement and to “hear” it through spatial sound.

View the interactive Immerse demo

High-fidelity prototype

The high-fidelity prototype was prepared after usability testing with users.

View the high-fidelity prototype

Accessibility considerations

The project included:

— appropriate contrast ratios, heading structure and correct HTML semantics,

— different navigation methods, including keyboard, voice commands and head movement,

— engaging different senses as a way of communicating experiences of art.

Hack(art)thon finale

The final task during the workshop was a developer presentation. Our project begins in the recording at approximately 1:43:00.

View the final presentation

The project received 2nd prize in the Hack(art)thon competition.

Outcome

The outcome was an advanced prototype combining digital art collections, multi-sensory interaction and the exchange of personal experiences of art.

Possible next steps included implementing VR presentation capabilities, creating a multimodal gallery experience using spatial binaural audio and developing AR and VR sharing experiences.

Takeaways:

— working in a new interdisciplinary team with diverse skills; we did not know one another beforehand, and the group included teachers as well as front-end and back-end developers,

— collaboration and user interviews at every stage of the work,

— working on a project directly related to accessibility, attending lectures by specialists and gaining access to materials on the subject.

SKAPPA Team: Agata Pałach-Bożek, Agata Tumidajewicz, Kamil Bożek, Paula Pniewska, Paweł Łuciuk, Stefan Paruch.

Resources

Official Hack(art)thon website

Final presentations — the project starts at approximately 1:43:00

High-fidelity Figma prototype

Interactive Immerse experience demo