Image from: The Polar Express by Castle Rock Entertainment (A Time Warner Company), Shangri-La Entertainment and ImageMovers

We’ve all encountered it: when we watch a movie, or play a video game, sometimes we would notice something that is slightly off: scenes and especially characters that are supposed to look realistic, but sort of miss the mark. Only by a tiny margin, but enough to make the character - or the scene as a whole - to appear somehow wrong, unsettling, even scary sometimes.

Maybe the skin doesn’t look natural enough, or maybe the lighting does not reflect as it would have in a real-life scene. Maybe mimics and facial expressions are wrong, or maybe all of the above is fine, but the character simply doesn’t move quite like a living being would.

Oftentimes we can’t even clearly tell what exactly is wrong - but we always feel if something is indeed wrong. Moreover, this only happens with scenes aimed at realism, not with cartoons or otherwise stylized imagery. And the more realistic these scenes are supposed to be, the more pronounced this feeling gets.
Amazingly, for decades now there has been a name for this effect: this is the so-called “uncanny valley” effect. But what exactly is it? Why does it happen? And what can we do to mitigate it?

A Bit of History

The concept was coined by the robotics professor Masahiro Mori in 1970. Mori’s hypothesis was that as the appearance of a robot is made more human, the observers’ emotional response becomes increasingly positive - until it reaches a certain point beyond which the response quickly becomes strong revulsion. However, as the robot’s appearance continues to become more and more human-like, the response becomes positive once again.

Image by Smurrayinchester / Wikipedia .

This area between “somewhat human” and “fully human” came to be known as the uncanny valley.
Originally, this concept was only applied to robotics, but as visual effects and especially computer graphics continued to evolve, it became apparent that the same effect can happen with any form of synthesized imagery. This “uncanny valley” effect was very common in the works from the 90s and 2000s. But even today it is by no means extinct. It is often associated with anatomy, lighting and shading, but it can also apply to animation:

Image from “Heavy Rain / The Casting” 2006 Tech Demo by Quantic Dream

This is not always a bad thing. Sometimes it can be used for artistic purposes: to show, for example, that something is wrong with the character:

Image from: “Kara” demo by Quantic Dream

But these are specific cases. More often than not you’d like animation to be as convincing as possible.
The effect can be tricky to overcome because more often than not we can’t consciously tell what exactly is wrong and what changes have to be made.
And while recently significant progress has been made in the field of rendering, animation technology did not develop to the same degree. As a result, we get photo-real environments populated by almost lifelike characters that… don’t exactly move in a very convincing manner?

Ways to Bypass

 It seems reasonable to assume that as visuals get more and more advanced and lifelike, so should the animation. This, however, is not always the case for a couple of reasons. Currently, realistic animations are often made using motion capture. This is the most obvious way to get a fully realistic animation: as it records motions that happen in the real world, so the outcome is by default as real as it can be. This approach is used widely, and it truly does allow for creating highly expressive and convincing animations. However, it presents its own set of limitations.

The process of capturing real-life movements is not an easy one. It requires participation of many specialists from multiple fields of work. It can get very expensive, especially if we’re talking about scenes with multiple participants. And mocapping animals is its own set of problems that only grow bigger if we try to capture something that does not have a direct counterpart in our world - some fantasy creature, for example.

Of course, all these issues are by no means fatal. With enough dedication and resources they can all be overcome. But once they do, a different challenge arises. An animation recorded with motion capture is rarely used “as is”. Usually it has to be cleaned, or retimed, or otherwise adjusted before it can be put into action. All these are necessary preparations, but they can lead to a pitfall:
...when you edit an animation to suit your needs, you rarely do take into account its realism and physical accuracy. So if you are not careful, you can edit realism out of the animation - and in fact, that is exactly what happens too often.

What To Do About It

So what can we make of it? It would  seem that oftentimes the “uncanny valley” effect in animation is caused by the loss of physical accuracy. And if this is indeed the case, this means that at least for some of these cases the effect can be surmounted by applying physics to the animation - which is something that physics-based animation software like Cascadeur excels at.

Because of this, we decided to use Cascadeur to demonstrate how exactly an “uncanny” animation can be improved. As it turns out, in many cases applying physics tools that Cascadeur provides does indeed improve the looks of an “uncanny” animation:

But unfortunately, there always will be cases where this is not enough.

However: even if the problem cannot be fixed right away, it can be circumvented. For example, you can try to obscure the motion, only partially showing the character on the screen. Alternatively, you can use the environment to conceal the parts that don’t look convincing enough. The limitation of this approach is obvious: it cannot be applied to motions that are supposed to be clear and easy to recognize. But for cases like these, there is another solution: the one that might seem paradoxical at first.

In the end, it would seem that overcoming the uncanny valley effect is not as much a technical task as it is an artistic one. So instead of making the animation more realistic, you can try making it more stylized. After all, the “uncanny valley” effect only really applies to realistic images and motions, and even then, only to those which are not realistic enough. And if realism is something you can afford to sacrifice for a better overall impression, maybe sometimes this sacrifice has to be made.

Because stylized animation does not necessarily equal stylized visuals, nor does it mean completely abandoning realism. In fact, physics can very much be applied to stylized animation, and stylized animation can just as well benefit from physical accuracy. Combining these two approaches can produce impressive results - you won’t be getting a completely realistic animation, but you might get one that looks convincing enough. This solution is obviously not for everyone, but as a last resort, it can be used.

Thanks for taking the time to read our essay on the Uncanny Valey Effect. We hope this topic is as interesting for you as it was for us!
If you have anything to add, feel free to share your thoughts in our forum!