Metaverse Meets Consumer Electronics

The first year of the Metaverse Era is considered 2021 mainly because the gaming company Roblox was listed on March 10, 2021. Different from the traditional games, in the Roblox metaverse, the game players can construct their own games to let others play together.

& THE FIRST YEAR of the Metaverse Era is considered 2021 mainly because the gaming company Roblox was listed on March 10, 2021. Different from the traditional games, in the Roblox metaverse, the game players can construct their own games to let others play together.
In October 2021, Facebook's parent company was rebranded Meta Platforms, which further boosted metaverse. Mark Zuckerberg announced Meta's vision of metaverse called Horizon in which users can throw parties, buy/sell houses and cars, open concerts, customize the clothes of users' avatars, and design different environments for home and working offices. He believes the aforementioned immersive interaction would completely change human life in the future.
Following Meta, Microsoft announced its hybrid collaboration platform Mesh. Nvidia announced Omniverse, which aims to realize highly productive remote work through metaverse. Currently, the most immediate emerging products in metaverse are considered to be gaming and cooperative working.
The dominant companies in metaverse have different advantages in jumping into the metaverse area: Epic's Unreal Engine renders a high-fidelity 3D virtual environment; Meta's Quest VR (virtual reality) headsets provide users with a fully immersive environment to interact; Microsoft's HoloLens MR (mixed reality) headsets enable the users to overlay the virtual objects onto the real environment and interact with real-world through gesture vocabulary, speech recognition, etc.

EVOLUTION OF THE METAVERSE
Nowadays, different technical giants are separately promoting their own metaverses resulting in "islands" of decentralized, multicentered metaverse systems. Although users join different metaverses, respectively, their virtual assets and customer data are not shared between the metaverses. In order to build a unified fully connected metaverse ecosystem, the standards of the metaverse as well as corresponding communication protocols are highly required as shown in Figure 1. In this regard, NVIDIA Omniverse is one of the pioneers, which integrates digital models across multiple metaverse systems through its opensource file format and framework Universal Scene Description. Furthermore, it provides the capability to build and render different 3D scenes and assets for different metaverse systems, and deploy them to a variety of eXtended reality devices remotely.
Additionally, Khronos Group builds an open standard named OpenXR for developing applications running on any major eXtended reality devices and platforms through a unifying application programming interface. According to Deloitte's report, 1 it is expected that the distributed metaverse systems will gradually share data and establish unified standards across platforms, thus achieving full integration in the mature stage.

METAVERSE-CE TAXONOMY
The metaverse requires consumer electronics (CE) to measure users' interaction information and meanwhile create an immersive environment with multimodality. Additionally, the consumer electronics take the users in the metaverse to experience a new virtual life with other peers as shown in Figure 2.
There are a few key technology pillars that support the metaverse in the virtual world 2 as the following: Sensing, Actuating: To create a fully immersive metaverse environment, various kinds of sensory measurements are made, such as binocular stereo scenes computed by camera transformations. Furthermore, different kinds of actuator actions are generated, such as interaction forces calculated by a physics engine.
Representation, Assets: High-fidelity virtual scenes and customized avatars are rendered and displayed in 3D. In addition, digital assets authenticated by NFT (non-fungible token), a digital collectible as an example, are also included.
Artificial Intelligence: It enables the metaverse to be self-cognitive, such as interactively talking with users by natural language processing, effectively engaging with users by making inferences from users' behaviors, etc.
Blockchain, Cybersecurity: The virtual metaverse requires privacy protection, encryption, and authentication to protect users from cyberattacks. In addition, blockchain as a decentralized ledger would keep all critical transactions safely recorded.
To enable the users immersively interact in the metaverse, consumer electronics use many different ways to strengthen the sense of reality, including: 1) Vision, Sound, Haptics, Smell, Taste: The users look at binocular 3D high-resolution displays by an eXtended reality headset or a smartphone. 3 Broader field of view (130 horizontally by 100 vertically or even 200 by 130 ), higher resolution (4K or even 8K), and higher refresh rate (90 fps or even 120 fps) provide a natural view to the users. Spatial audio enriches the 3D surround sound experience of sonic changes with the movement of the head. Haptic vest and haptic gloves enable the users to feel the stiffness and even texture of a virtual object in the metaverse. The electrical generator or simulator of smell and taste could further enhance the ultimate immersion experience. 2) Voice, Gesture: Multichannel microphone array makes the users' voice input clear enough, followed by speech recognition. Users' gesture vocabularies are recognized by a set of RGB-D cameras.
Between the metaverse and users, the Web 3.0 network 4 transmits the datagrams between the virtual and real world with ultra-low end-toend latency (ideally less than 7 ms), including: 1) Downlink Traffic: Mainly the rendered virtual environment frames from the metaverse cloud to the consumer electronic end host.
As the rendered frame is too big to fit in one packet for transmission, it is divided into a number of small packets resulting in the traffic pattern as a microburst of packets. 5 2) Uplink Traffic: Typically including the pose measurement of the user's head, signals from the controllers, and synchronization from the consumer electronic end host to the metaverse cloud, etc. The traffic pattern is a much more frequent packet transmission with much smaller packet sizes.

OUTLOOK: WHEN METAVERSE COMES TO LIFE
Metaverse brings a future where the virtual and real worlds are intertwined. With an immersive user experience, it digitizes the real experience in the metaverse. Users leverage metaverse to not only make their work more efficient and intuitive (e.g., by using Meta Horizon workrooms), but also enrich their social life and entertainment. All these immersive experiences greatly rely on the collaborative virtual environment provided by consumer electronics, realizing the high quality of experience for users.
On the other side, the metaverse also emphasizes the concept of realizing the digital experience. Virtual objects would have a great impact on user's lives in the real world. For instance, through augmented reality glasses, the metaverse could overlay vivid navigation guidance on the real road by taking measurements via consumer electronics (such as LiDAR sensors or cameras). More interestingly, virtual assets such as cryptocurrency and NFT create real economic value, namely metaeconomy.