Category: Act Of Observation

The Act of Observation Postmortem

Initial Goals

The Act of Observation (AO) was intended to be an exploration into the societal space of virtual reality. This experience was my first time developing for VR, and I wanted to play with the lines between player and player character, the power of voyeurism in virtual space, and environmental character development.

When a player puts on the headset and enters the virtual space, they perceive it as real, or at least more real than a traditional format video game. I recently played a horror VR experience at the CMU ETC. My friend played the game first and I watched her and the experience on unfold on the television. The experience used a predictable set of horror tropes to establish the mood, but my friend screamed and jumped. When it was my turn, I thought I already knew the experience well enough to get out unscathed, but I too was terrified. The immediacy of the space, both as a novelty and as a spatial setting, really do make the experience. This experience is also an example of how individualized the current VR scene is. One player, one headset, one experience. The lines for VR experiences are long for this reason, and the experiences are generally taylored for the singular player (though this is not all VR experiences, of course, just the bulk.)

In AO, I wanted to address this odd space and the powers that come with the VR headset. As a player in the VR experience, you are you, not a player character. There is no blaming Mario when you fail to do something. It is your fault, always. And if you are you in the experience, then should your actions have real consequences in the virtual environment? Do the inhabitants of the world see you pop in an out of existence when you put on the headset? What does your presence in their space mean? And what does it mean to look into their world through this ocular device of the headset? A device with, in my case of the Google Cardboard, very limited tangible input but the incredible power to view a different existence?

Process

I built the experience for the Google Cardboard and used a droid turbo phone to host the experience. I chose the GC because of its accessibility ( I am a huge fan of the cheap/free democratization of game development) and its inputs. The GC has a one button input system (and not a haptic system) and is held up to the head instead of strapped on. This gives the player the power to immediately chose to enter and exit the virtual system whenever they please.

I modeled the game in Maya and animated the blendshapes there. The rest of the animation happened in Unity’s animation system.

I chose a young woman’s bedroom because of the “classic” voyeurism scenario of the peeping tom. As a player stuck sitting on the edge of her bed, you see her most private actions through the animations. The animated objects have a separate material (translucent gold) to differentiate them from the room. I did this to give mystery to the objects and the dweller of the space: is this a playback of her or is this how you imagine her in her space?

I separated the animations into escalating reaction “scenes”, beginning with her unsuspecting actions and ending with her realization that you are in her space. I transitioned these scenes with messages left on her answering machine by a friend, and eventually her, which echo her thought process of what is happening in her space. I did all the voice acting for the game.

A big part of setting the tone of the experience was lighting and sound. I went through several iterations of each to fine tune the late night/early morning feel and to get the core of her interactions across in the animation. Originally I did not plan to put sounds to her animation, but tried it at the suggestion of playtesters and found that it worked.

Acquired Knowledge and Skills

Over the course of making this game I became very familiar with Unity’s animation system. I learned how to edit animations via curves, something I had never touched before. I also learned how to create and export blendshape animations from Maya to Unity. I learned how to use normal maps in Unity materials and the importance of uniform geometry on models. I also became more confident in my 3D modeling, and learned the beautiful Sculpt Geometry Tool in Maya, which made all my modeling dreams come true. This was also my first attempt at a VR/360 experience.

Setbacks and Solutions

My first big issue with the project was importing animations from Maya to Unity. My initial plan was to animate everything in Maya and export it into Unity where I would cut the clips up and trigger them from scripts. No matter what format I used from FBX to Maya binary I could not get animation to export. The only thing I could get to export was blendshapes in Maya binary format. So I decided to animate in Unity. This led to my second big problem and I think the biggest issue: automatic curving in Unity’s animation system. Since I didn’t have cut clips I chose not to use Mechanim’s graphical editor, but animated all the objects from scratch in the scene. Unity automatically curved between my keyframes, creating wildly over-exaggerated movements. I solved this by exploring the different curve types in Unity: some situations called for linear or constant breaks between keyframes. In some cases the added motion ending up working for the motion, so I kept it. I also had issues with relational animation. This was especially bad with rotation. If an object had multiple parts that really didn’t need to be animated through a blendshape (for example: the lock tumbler and the lock latch) I parented them to an empty gameobject and put animation on that game object. This moves the center of rotation to the empty game object and rotations occur in essentially random relational space to the parent. I some cases I could fix this with a grandparent object with animation that could animate both the grouping of objects and the individual pieces, and in some cases I had to adjust the scaling to get it to look right.

I also had issues with trigger zones for animations. The player triggers all the animations by looking at things (there is no traditional input). I did this via a long raycast from the center of the player’s camera viewport that collides with a trigger zone around each animatable object. I had to fine tune each zone so that animations could be triggered one at a time from the central vantage point. I also had to space out trigger zones for the 360 viewing. I really wanted to take advantage of the 360 viewing, so I tried to put interesting triggerable things in all four corners of the room to give the player a reason to turn around.

Conclusion

AO has been an exploration into voyuerism, 3D animation, and VR design. I hope to continue to work in this space (both technically and conceptually) with later iterations of this work.

 

Lighting Iterations

I played around with lighting to better communicate the time of day and the mood of the game. I wanted something that was in the early morning, slightly creepy, dark but colorful, and I needed to do it in only a few lights. Going in I had no idea how the lights would effect the android build. Luckily through baked lighting I managed to get the same look in the VR build. There are only 3 realtime lights: one of the lavalamp lights, the yellow light behind the blinds, and the flashing answering machine light.

 

 

Getting Weird with Normal Maps and Final Agency

I’ve finished all the major animations except the locks. This gun-eyeball thing came to me in a sleepless haze and I couldn’t stop laughing about it. Later I realized the eye motif actually related to the voyeurism topic I’m exploring with the work. So I’ve made it part of the final scene in which the player has to make a final choice. (Playtesters have requested more explicit agency in the experience.) The player will be presented with the gun on the bed or the door. All the other objects in the room will disappear and the lighting will intensify (with more red.) A message will play from the dweller explaining that she know you are there. She will present you with the final choice: leave by looking at the door or stay and she will confront you (with the eyeball gun.)

I also added normal maps to most of the objects in the room, which made them less toony.

normals1normals2normals3

Scene Two Initial Animations

Today I played around with lighting and the shadows, trying to make them less pixelly and crunchy. Still need to work on that. I also finished some of the animations for the second scene: the blinds, the sheets, and started the meds. A colleague introduced me to the geometry sculpting tool in Maya which made the sheets not look terrible. I’m planning on revisiting some of the models with this tool to make them less terrible too.

First Scene Rough Animations + Triggers

I had some trouble organizing animator controllers in Unity for each object which caused me to lose a lot of work. Each animatable object has an idle “hiding” animation as a default that transitions via a trigger to their actual animation which. Each animation plays once. I ended up having to make an empty parent object for every animating object that contains the animator controller and accesses the moving object as its child. This fixed a “missing!”/yellow text  error that caused animations not to play.

Animation Textures and Script Progression

This experience is most like an interactive depiction of the invasion of voyeurism, character development through environment/animation, and mood through lighting. Each scene has a set of triggerable animations the player activates by looking at objects. Each scene has a message on the answering machine explaining the emotional state of The Dweller. I’m still unsure of how to end it.

*Needs blend shapes

^Needs new base state

%New Item

  1. The Dweller’s Initial Room: her default state.
    1. Animations:
      1. Cigarette boxes: timidly deciding against smoking
      2. Drawers: struggling to open crammed/stuck drawers; accidentally pulling one all the way out (top 3 visible drawers have unique animations)
      3. Guitar case: spins and thuds against the wall heavily (she puts it away after a successful gig)
      4. ^*Pillow: fluffs up and squishes down (trying to get comfortable)^needs head dent
      5. ^*Sheets: legs kick underneath the sheets/the sheet crinkle^needs to be crumpled by default
      6. ^Magazine: flips through pages bored^needs to sit as if thrown to the side/pages spread open in teepee shape
      7. Record: flinging it like a frisbee as she walks in the door
    2. Sounds: City sounds from outside, air conditioning
    3. Message: None
  2. Initial Suspicions: A ghost?
    1. Animations:
      1. %Meds + pill: nervously checking the label(rotating the container) and taking a pill
      2. Sheets: pulled up around her close as she cowers underneath
      3. ^Magazine: intently reading^texture becomes a magazine about paranormal activity
      4. Lightbulb: screw in and out of socket, checking to make sure it’s fine
    2. Sounds:Answering Machine beep, city sounds, air conditioning
    3. Message: “Hey, sorry I missed you! I was booking our next show at the Square Bar, so forgive me? Uh, so what’s this about a ghost in your apartment? You’re kidding, right? Call me back when you’re ready to make the setlist, weirdo.”
  3. Scared: An invader
    1. Animations:
      1. Blinds: opening halfway through as though she is looking through them to the outside
      2. %Pepper Spray: reading directions (rotating canister), opening, and testing the spray
      3. ^Magazine: intently reading^new texture “Self Defense Techniques”
      4. %Guitar Case: putting lock on guitar case
      5. %Drawers: locking each drawer
    2. Sounds:Answering Machine beep, city sounds, air conditioning
    3. Message: “Hey, pick up the phone right now and talk to me! Are you sure? Just a feeling of being watched? Do you have any evidence of a break in? Call the police right now and get them to check! Right now, damnit, I’m serious! Pick up the phone!”
  4. Confrontation: Addressing the player, not the player character
    1. Animations: %Items arranged on the bed in a message: “Stop watching me”?
      1. %Gun: (on the bed in front of the player) raising to be aimed at the player
    2. Sounds: Nothing
    3. Message: From the dweller: “I know that you are there. This is my place. You are invading my life, but I won’t let you control it. So, Watcher, here’s your option.”

Google Cardboard VR

actobvr1actobvr2

After going through several crits of the assets for Act of Observation and explaining the mode of interaction, it was suggested that I throw it in VR. Since the player is stationary and only only rotates, it seemed a god fit. I chose Google Cardboard because I could borrow one, and I am a big supporter of the cheap/DIY/democratization of the medium. Also I felt that fit the self motivated/desire driven voyeurism of the interactions: with a headset that the player holds up to their face, they have the option of immediately leaving or engaging the experience. The player isn’t strapped in to the game. Like invading a personal space, they direct their gaze, when they gaze, and how they gaze. This control over another (the dweller character in the game) makes the headset a tool of control and integrates it into the experience.

Lighting Tests

Some lighting tests for lights to send to Unity. All of this will be baked since the animation happens in a transparent psychscape. I wanted the lighting to be early morning dramatic.

I also made some of the more aggressive objects: pepper spray, lock, and gun. I think I’ll also make the magazine change topics too.