Disney R&D Create Real-Time Tool to Improve 3D Perception

submit to reddit Dec 06, 2011 2 Comments by

The Disney Research Centre, ETH Zurich, Black Rock Studios and Disney Interactive Studios have published a report describing OSCAM – a solution to  provide high quality S3D regardless of screen size, reducing production costs. 

review dividing line Disney R&D Create Real Time Tool to Improve 3D Perception

While the business model for 3D television and gaming is still being established, research in lab centres around the globe are tackling the issues inherent in fooling the brain into ‘seeing’ a 3D image. Many problems can be fixed in a post process but researchers at the Disney Research Centre, ETH Zurich, Black Rock Studios and Disney Interactive Studios have been very active in analysing how various factors such as screen size, viewing distance and individual physiological states can affect stereoscopic perception in real-time interactive applications like S3D gaming.

The team have just released a paper describing their OSCAM solution which attempts to solve the problems with stereoscopic creation processed in real-time. For example, in a first-person perspective game, where the player is in control of the view, a simple collision with another object will result in excessive disparities that cause visual fatigue or destroy stereopsis (see Figure 1). In order to guarantee correct S3D, a controller is required that adjusts the range of disparities to the viewer’s preferences such as camera convergence and interaxial width without the need for expensive computational solutions.

figure one Disney R&D Create Real Time Tool to Improve 3D Perception

In the report, researchers acknowledge that perceived depth is not just influenced by the disparity between the left and right images (other factors include vertical size and focus pulling) but the ‘geometrical approach’ (distance between the two images) is the most important influence on perceived depth and this is where their research has been focussed.

In it, they establish there is a comfort zone for perceived depth for each individual viewer. This ‘range’ (also called a ‘depth budget’) consists of the sum of the negative and positive disparities. Disney researchers are proposing a “stereoscopic control mechanism for disparity that is able to guarantee an exact mapping of arbitrary (personally chosen) content to a comfortable target depth range."

Their controller allows any scene content to be mapped into a specific depth range (‘depth budget’ or range between positive – screen elements appearing ‘behind’ the screen, and negative – screen elements appearing 'out' of the screen) when the viewing geometry values are known (screen size and viewing distance).

What does this mean exactly? Well screen size plays a major role in perceived depth for stereoscopic 3D content. For example, 3D content designed for the silver screen would look very flat if ported to a device such as the LG Optimus 3D or Nintendo 3DS without any post processing. Small screen 3D viewing platforms require a considerably higher depth range – a greater parallax. Also, current post processing technology is limited – the amount the parallax can be artificially increased is quite small before artefacts occur.

The Disney/Black Rock/ETH Zurich solution means content created by a producer is guaranteed to create the desired stereoscopic depth effect independent of the actual display device, be it for example a large polarised display or a small Nintendo 3DS. Moreover, an application can be easily adapted to the depth or disparity constraints of a particular device, which helps to reduce ghosting or crosstalk artefacts. The research team claim this will considerably simplify production and reduce associated costs. The consumer may even adjust the target depth range intuitively to accommodate to personal preferences, such that proper stereopsis without excessive disparities.

In the OSCAM report, they write production and consumption of S3D in broadcasting, cinema and medical applications has been researched for many years but studies into S3D real-time interactive applications like gaming have been rare (perhaps not surprising due to the newness of this sector).

The basic principle of stereoscopic perception is simple – a left camera and right camera replicate the human eyes. When each eye sees the corresponding image, the brain processes the images together to create depth. However, this principle is too simple as it does not consider the hidden processes involved within the brain. Artefacts can include keystoning or depth plane curvature – when the extreme edges of a scene are distorted (although tools such as the Foundry’s Ocula suite can fix this). Cardboarding is the artefact when images appear as 2D planes in a 3D scene (can often occur when a 3D camera has zoomed into a scene). Other factors such as when a viewer moves his or her head in the yaw, pitch and roll motions and viewing distance from the screen can introduce distortions too.

OSCAM focusses on real-time stereoscopic rendering, with control over perceived depth for dynamic 3D environments, minimisation of nonlinear depth distortions, and high efficiency for demanding real-time applications running times less than 0.2ms per frame even at full HD resolution.

OSCAM considers …

  • Perceived depth not only depends on the amount of disparities seen by the viewer, but also on monocular cues such as vertical size, focus, or perspective and viewing distance.
  • Results from perceptual experiments and research on stereoscopy clearly indicate a large variation in the physiological capabilities and preferences of different people.
  • Research shows that different people have significantly varying stereoscopic comfort zones, indicating that individual control over stereoscopic depth is desirable.
  • These indications motivate the need for tools that allow for a content, display, and user-adaptive control of stereoscopic disparity.

Applications for OSCAM

Adaptive stereoscopy and automatic fail-safe
When moving the camera through a scene, the render-depth is constantly changing. Improper camera convergence and interaxial separation can cause large disparity values when an obstacle suddenly comes close to the camera. An example is shown in Figure 4, where a player is moving fast through the environment. The OSCAM solution is able to adapt to vast depth changes which is useful as uncontrolled stereoscopy causes excessive disparities and inducing eye-strain to the viewer.

figure four Disney R&D Create Real Time Tool to Improve 3D Perception

Another typical situation encountered in interactive, dynamic environments is the sudden change of scene depth. An example is where a camera is horizontally tracked across a street. If it passes very closely in front of a couple, a sudden discontinuous decrease in the minimum scene depth occurs. If this is not handled properly, the depth perception is immediately destroyed. The OSCAM method adapts the stereoscopic parameters to prevent exceeding disparities to appear for too long.

Changing target screen size and viewing conditions
Stereoscopic imagery that is produced for a certain target screen and viewing distance is optimised to create a particular depth impression. If, however, the content is shown under different viewing conditions, the viewer may experience a completely different depth sensation. OSCAM is able to adapt content automatically for screen size. It also provides intuitive and exact stereoscopic control by allowing the viewer to adapt the perceived borders of the depth range to the respective personal comfort zone.

Post Production
OSCAM also provides an interesting tool for content creators and artists in production. The controller can be used to intuitively script perceived depth for different scene parts and hence create artistic effects such as emphasised depth for certain objects, stereoscopic flatlands, etc. Moreover, since the method can map any scene content into a pre-defined target depth range without changing any camera parameters except interaxial separation and convergence, it effectively de-couples classical ‘2D’ camera work from stereoscopic camera work. This allows for a stream-lined production pipeline, where both stereoscopic and classical camera artists can work independently.

In order to further evaluate the utility of the method, a user study with 31 subjects was conducted. The study was aimed at comparing the OSCAM to standard stereoscopic rendering with fixed camera convergence and interaxial separation. When asked which one is more comfortable to watch, OSCAM was preferred in 61.7% of the examples, while the fixed stereo was preferred in 38.3% of the cases. When asked which one looked more realistic, the OSCAM controller was preferred in 60.7%  of the scenes compared to 39.3% for the static stereo settings. The Disney headed team concluded that their experimental evaluation showed that our controller is preferred over naïve stereoscopic rendering.

As producers and platform owners become increasingly keen to push 3D content on a wide variety of devices from IMAX to mobile phones, Disney’s solution could be a compelling proposition, especially to drive down costs. For example, when Atlantic Productions produced Flying Monsters 3D, they had to re-shoot different versions for the 3D cinema presentation and 3D television presentation. Also, such a rendering engine could allow 3D games intended for small handheld devices be ported to large screens.