IEEE Technology and Society Magazine - March 2022 - 34

Feature
communication is key and the desire for (perceived)
nonmediation of technology is high.
In the remainder of this article, we describe the
three main project components: 1) virtual wharenui;
2) voxelvideo storytelling; and 3) 3-D tele-co-presence
in more detail. We will conclude the article
with lessons learned and future directions.
Virtual wharenui
Normally, most marae have a communal gathering
place, the wharenui, wharepuni, or here wharerau
(Te Rau Aroha), which takes a central and prominent
position on their marae. To emphasize tı¯rangawaewae
or a " place to stand " / " sense of belonging "
for dispersed Ma¯ori, the marae takes center stage with
the wharerau providing the focal point. The artistry
evident at Te Rau Aroha, created by renowned artist
and master carver Cliff Whiting, has resulted in one
of the most contemporary and artistically stunning
marae in Aotearoa. The wharerau is uniquely octagonal,
modeled on the temporary dwellings that were
used for extended food gathering excursions. Each
of the eight walls focuses on a particular cultural
theme and aspect, highlighted by intricate carvings
and weavings. In addition, the wharerau is home to
twelve 9-foot-high wooden female tı¯puna (ancestors)
representing the primary genealogical ancestors for
descendants connected to this marae (see Figure 2).
To create a sense of " being there, " a virtual representation
of the actual marae should depict the
characteristic elements of the building in sufficient
detail and with appropriate cultural respect. While
the physical wharenui (wharerau) is very rich in fine,
artistic detail, an exploration with virtual reality techniques
require simplification to enable an interactive
real-time experience.i There is an apparent conflict
to be resolved: Which parts of the whare (building)
should be represented in what detail for a meaningful
and suitable experience? In addition, the creation process
should be affordable and time-efficient to be easily
handed over to Ma¯ori sovereignty. In partnership
with the marae, we decided on a hybrid model of fine
detail reconstruction and overall sense of presence.
According to Schubert et al. [11], the sense of
presence can be decomposed into spatial presence,
involvement, and realism. Spatial presence focuses on
the embodied relationship between the user and the
iOnly a certain fidelity of detail, as measured with number of polygons and
resolution of textures, can be rendered in interactive real-time (>20 updates per
second).
Figure 1. Te Rau Aroha marae and its
wharerau at Motupo¯hue.
Figure 2. Photograph of wharerau interior
depicting one of the eight artistically
decorated walls with tı¯puna figures.
(to scale) virtual environment. Involvement mainly
stems from the way people interact with and self-navigate
within the virtual environment. The weakest
factor for the presence is the perceived realism which
allows us to neglect this aspect for some parts of the
building. Hence, we produced models with varying
degrees of fidelity and shared them with the domain
experts to decide on required levels of detail.
For the 3-D reconstruction of the inside of the
wharerau building, we use a hybrid modeling and
photogrammetric approach. In a first iterative step,
the (simplified) architectural shape is modeled and
the eight walls are textured with photographs taken.
This already allows for an immersive exploration of
the wharerau and provides a sense of being there.
However, all structures are flat (2-D), including the
artistically shaped tı¯puna figures. In a second iterative
step, hundreds of photographs of one of the eight
walls including the accompanying two tı¯puna figures
to the left and right are taken. The programs COLMAPii
iihttps://github.com/colmap/colmap
34
IEEE Technology and Society Magazine
https://github.com/colmap/colmap
IEEE Technology and Society Magazine - March 2022

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - March 2022
