Three Truths 关于 Virtual Reality Success From a VR Novice

I just started a consulting project relating to 360° VR video, 并得出一些介绍性结论. I am only an egg, as Robert Heinlein might say, but I thought I would share them.

第一个, 做得正确, VR是一种非常强大的媒介, capable of a level of immersion that can’t be matched in the world of “flat video,” the pejorative designation for the 2D video we watch on phones, 电视, 还有电脑屏幕. 关键是“做得对”.”

Second, VR is a challenging medium with little margin for error. Where producers discuss quality of experience for flat video with terms like abandonment rate, 在虚拟现实世界中，它是晕机或恶心, which presumably generates a much stronger antipathy toward your company or brand than does a pre-roll ad.

Third, resolution is critical for effective VR video, and the numbers work against you. YouTube提供4K高质量的VR视频, which sounds great until you realize that this 4K represents the entire 360° view. Devices like the Oculus Rift or 宏达电 Vive have field of views of 110 percent, 或者30%的视频(110/360), 也就是说在任何时候, 你只能看到30%的水平像素, 或者比原来的4K低1200像素. The Rift and Vive have a display resolution of 1080x1200 per eye, 所以这是一个很好的匹配, 但如果你提供2K视频, you have to scale the video to double the resolution. 这会导致像素化和柔软.

These are the numbers for mono video, where both eyes see the same image. 如果你拍摄立体VR, 或者每只眼睛一个视频, 你必须把两个视频都打包成4K流, 将各自的分辨率减半.

While field of view is your enemy from a resolution perspective, it can also be your friend. 这是, since the viewer only sees what’s in the field of view at any one time, 为什么不直接发送那个视野呢, with a lower-quality buffer around the edges to make sure there’s something to watch if the viewer quickly turns his or her head.

开始, understand that the default frame for VR video is called an equirectangular layout, which is what you get when you map a sphere to a rectangle. As an example, a typical world map is an equirectangular image. There’s lots of distortion at the poles since you have to stretch them horizontally to achieve the same width as the equator. The problem with the equirectangular image is that it shows the same quality for all parts of image, 即使观众只能看到, 最多, 每次只看30%的图像.

在一篇名为 “Next-Generation Video Encoding Techniques for 360° Video and VR,” 两名Facebook员工讨论了金字塔编码, which divides each uploaded video into 30 viewports. Each viewport contains a full-resolution version of the current viewport, plus much lower resolution detail of the rest of the original frame. 使用1秒的片段, the player monitors the field of view and changes quickly with head movement. 如果观众转动她的头, 她会暂时看到一个低分辨率的版本, 但质量应该会迅速提高. The authors claim that this approach reduced delivered file size by 80 percent.

A different approach was described in a paper called “Viewport-Aware Adaptive 360° Video Streaming Using Tiles for Virtual Reality.” 在这里, the authors divided the equirectangular frame into multiple tiles that were encoded separately at declining quality, 就像传统的编码阶梯. The player retrieves the highest quality rungs from the field of view, 随着其他领域的质量下降. So the tiles directly behind the current field of view, 还有极点, 可能是质量最低的, with improving quality delivered closer to the current viewport. 这些都是通过DASH清单管理的.

I’m working out what all of this means from a compression perspective, but it’s refreshing to see such creative solutions to bandwidth-related problems.

[This article appears in the January/February 2018 issue of 流媒体杂志 “关于VR的三个真相”."]