What is the evolution of spatial experience through sound led by oto rea? (Part 1)

Jointly developed by GATARI Co., Ltd., an AR/MR startup that aims to create future infrastructure for the future city "super city", and NOMLAB, a laboratory of NOMURA Co., Ltd. with the theme of "digital innovation x place creation". , An article introducing a prototype for expanding the spatial experience with “sound” was posted on nomlog on September 23, 2020.

In this article, we would like to share with the members of both companies who are continuing to improve otorea, a device that expands the experience of space through "sound" during the protracted corona crisis, the appeal of otorea and the after-corona that they have seen through feedback and discussions from experience sessions. We talked about the prospects for the future.

From the left, Shunichi Takeshita (GATARI representative), Hitomi Nakamura (NOMLAB designer/engineer), Minori Hayashi (NOMLAB planner), Yohei Kuroki (GATARI production director). Yukiko Okamura of the Public Relations Department of NOMURA Co., Ltd. Co., Ltd. was in charge of the round-table discussion.

1. Looking back on the protracted corona era from the perspective of XR and real space

Okamura
Today, I would like to talk about otorea, a voice MR technology, with Mr. Shunichi Takeshita and Mr. Yohei Kuroki of GATARI Co., Ltd., who are jointly developing it, and Mr. Hitomi Nakamura and Minori Hayashi of NOMLAB, an open innovation laboratory of NOMURA Co., Ltd. Co., Ltd. I would like to hear from you. Thank you all.

all together
thank you.

Okamura
It's right away, but it's been more than a year since the epidemic of the new corona started. Today, I would like everyone to talk about the development of oto rea, which has continued even during the corona crisis. :Mixed Reality), and the industry of attracting customers, can you tell us what happened?

Takeshita
Speaking of commercial facilities and exhibition facilities, especially under the state of emergency declaration, it was difficult to provide an experience with a device like oto rea in a situation where people could not be invited to the real space. On the other hand, there is also data that at an exhibition facility, when the number of visitors was reduced to about 60% to prevent infection, overall sales decreased, but visitor satisfaction improved. In order to restore the sales of facilities after COVID-19, it is difficult to aim to return the number of customers to pre-corona, so we have increased the options that can raise the unit price in the form of improving customer satisfaction. I think I think oto rea will be one of the measures to increase that added value.

Okamura
I see. Are you currently in the phase of increasing both the sense of security and satisfaction in the real space?

Takeshita
When trying to provide a rich experience, for example, theme park attractions inevitably require a large initial investment, but oto rea can provide a new experience without major construction. It can be said that it is relatively easy to implement.

Okamura
Certainly, while it is difficult for facilities to attract customers and make new investments as before, in the sense that it can be introduced while ensuring safety, there is great potential for a voice experience that is highly satisfying and does not require large construction. I can feel it.

Nakamura
As people go out less frequently, the smartphone app club house has become popular, but the importance of sound content is increasing, and it can be said that the size of the market is expanding. As interest in sound increases, we believe that a rich sound experience will be one of the effective elements in XR, MR and AR.

Hayashi
In particular, oto rea can be said to be characterized by the presence of the viewing device itself. As long as we have devices, we can provide new experiences to many people without changing the existing space so much.

Nakamura
oto rea naturally reproduces sounds according to human actions. Since it is a voice experience using a smartphone, it is also an advantage that the experiencer does not need to wear a large device such as VR (Virtual Reality) or MR goggles.

Takeshita
Various companies are starting to work on AR, but as long as AR glasses are necessary (it is hard to say that they are still popular at the consumer level), it is difficult to proceed from the demonstration experiment. I can say We believe that “sound MR” like oto rea is a service that can be easily provided as an immediate solution.

Nakamura
I think that oto rea is a solution that makes good use of NOMURA Co., Ltd. 's spatial knowledge and the knowledge of Mr. GATARI, the "sound MR."

Okamura
Could it be possible to say that oto rea is “an AR that has evolved into an MR in response to the corona disaster”?

Takeshita
While working on the experience production of oto rea, I realized that it is a platform that creates "behavior of space". Up to now, spatial experiences have provided the same content to the people who are there, but by utilizing digital technology, it will be possible to interactively provide different experiences for each person. It is a new game-like experience that allows the user to interact with the space. I think it is more appropriate to call it MR rather than AR, because it is essential to work on space (not just voice information).

2. Looking back on the oto rea experience event at NOMURA Co., Ltd. RE/SP

Okamura
At the trial session held at NOMURA Co., Ltd. RE/SP in September 2020, although it was limited to the public due to the corona crisis, 150 people experienced oto rea during the period. What kind of feedback did you receive from those who experienced it, and how did you make use of it in development?

Hayashi
We had a lot of people come to the trial session, and everyone was as happy as a child, and they shouted, "Wow! I can hear the sound!" I realized that there was added value in the actions that the participants were doing, such as sitting on a chair and looking up, which they normally do, but for the participants, it turned into a new and surprising experience.​

Nakamura
AR on smartphones is experienced while looking at the screen, but oto rea is experienced without looking at the screen by hanging the smartphone from the neck. Because there is no visual information, it is easy to immerse yourself in the world view of the content, as the new experience is surprising.

Okamura
I believe that GATARI has also continued to work on space.Did you find any discoveries that strongly felt the connection between sound and space at this workshop?

Takeshita
Mr. Horii, a planner at NOMURA Co., Ltd., said during his experience of oto rea, "I feel like I've become the main character of an RPG," which left a deep impression on me. I thought that it would be a device that could be tackled as a game with a three-dimensional expansion of the real space.

Kuroki
I strongly feel that oto rea can add a "playing experience" to the real world. Digital information that does not exist physically can be brought in (in a form that can be recognized on the spot). I believe that this series of actions, such as playing back audio through the movements of my body, led to a highly original and powerful experience.

Takeshita
​ ​I haven't experienced the tension of not feeling the device, in other words, "facing the space directly without intervening an interface." I was happy when I heard the impression that "there was fun not found in the smartphone AR experience" at the trial session.

Nakamura
Since sound can be placed here (in a specific place) in the space, I thought that, for example, "putting sound in an exhibit" would enhance the appeal of various spaces. (Continued in the second half)

The photos in this article were taken by Yuichi Tsumoto, who promotes the oto rea project, and this article was written by Jiro Takano, former editor-in-chief of nomlog and director of NOMLAB.

In the second part of the article, we will start with the possibilities of otorea, which we saw at the trial session during the corona crisis, and expand to the new normal of people, society, and digital technology. looking forward to!

About the oto rea experience
You can experience it at NOMURA Co., Ltd. in Odaiba or GATARI Akihabara Studio. For facility operators who are interested in this article, here Please contact Sorry to trouble you, but please specify "I want to experience oto rea" at the beginning of the "Inquiry content" input field. We may not be able to meet your expectations depending on the guidance of the government and the Tokyo Metropolitan Government regarding measures against the new coronavirus infection. Please note.