Level Designer @ Question Games. These opinions and projects are mine unless noted.
Category: Merge Conflict
I was a programmer and puzzle designer (and made/animated the main game UI) for this game that I worked on with Temi Olujobi, Isabella Djurovic, Avery Holmes, and Kris Atanasoski.
Play the game on itch.io: https://rachelpmoeller.itch.io/otto-the-combo-butler
I inserted analytics into Otto The Robot Butler (ORB). I implemented the analytics hooks, implemented the changes between builds A and B, and distributed build A. ORB is a 10 to 15 minute 3D puzzle game in which the player controls a small Alexa type robot who can navigate an apartment complex to find and bring back objects for the tenants. Puzzles in the game consist of a tenant asking for an object that has two qualities. The player must find an object in the world matching one of the qualities and another matching the other quality, combine them, and return them to the requestor. There are six of these puzzles in the game, each with one solution. We wanted to measure how long it took players to solve each puzzle (how long it took them to find and combine the correct items, not including the time it took them to deliver it to the requestor, which had the potential to skew the data) and how many incorrect combination attempts they made per puzzle. Since we were in the fine tuning stages of the level design, we specifically wanted to test how the placement of objects in the level affected playtime. We thought this could correspond to the difficulty of finding hidden objects in the space. Measuring the incorrect combination attempts was an attempt to understand strategy and player interpretation. Since the ‘correct’ adjective of an object in the game was revealed to the player if they combined the object incorrectly with a non-requested quality, we wanted to see if in general players used this to learn about the world quickly (and if so at what point in the game.) We also thought this might correspond to players not understanding the quality of an object they were supposed to find. For example, do they interpret a broom as “clean” or “dirty”?
Our AB test measured time to solve each puzzle and wrong combination attempts per puzzle against differences in item placement. Build A was our ‘control’ using the item placements the team had designed as part of our gatekeeping strategy to get the player to explore the entire apartment complex. Build B had more carefully considered item placement in which sightlines, color, framing, and the composition of other objects in the room drew players toward items. The changes in Build B included: changing the positions of all objects except the scythe and lime, the hot sauce and TNT were moved closer to the people who would ask for those objects’ qualities, the beige colored cheese and plug items were moved to sit on a stack of black tires and away from the beige floor, moved the soiled bag item from behind a door to be in the frame of the door, placed noodle item in a box to frame it, placed the candle item in a chair facing the door the player entered the room from, and placed the corn item at the end of a hallway so it was framed by a door.
Our major finding about the time to solve was that puzzles 3 through 6, and particularly 3 and 4, took longer to solve in Build B. This was surprising to us, since Build B was changed to help the player find the objects faster. In puzzle 3, the hot sauce item was moved to be closer its requestor in Build B, however, this placed the object in a closet out of a direct sightline from the door. In Build A the hot sauce is close by the door, if not directly visible. We think this may have been a time sink for players who explored the space quickly and might not have noticed the closet. To address this in the next iteration of builds, we chose to keep the hot sauce item in its Build A location in order to shorten the average solve time for puzzle 3.
Our major finding from the number of wrong combination attempts was that Build A had a significant increase in the amount of wrong combination attempts. We think this is linked to the short puzzle solve times in Build A and points to a particular strategy. Build A players may have used the adjective reveal feature often to learn an object’s adjective when they came across it. Therefore, they knew what object corresponded to what qualities when a new puzzle request started. This allowed them to remember where the correct object is rather than always needing to find and interpret the correct object. High wrong combination attempts and short solve times point to this strategy. Whether or not this is indicative of the item layout is not directly measurable, but this efficiency behavior might be linked to the long traversal times and search times it takes to find the objects that are not intentionally highlighted by the level architecture. When applying this data, we chose to allow this strategy to continue, so we kept many of the item layouts from Build A.
In conclusion, we measured the time to solve each puzzle and the number of wrong combination attempts per puzzle. Our data showed that item placement has an effect on how long it takes players to decipher and remember the game space, as well as the formation of traversal and puzzle solving strategies.
I am a programmer/designer/animator on this game that I’m working on with Temitope Olujobi, Avery Holmes, Isabella Djurovic, and Kristina Atanasoski.
I am a programmer/designer/animator on this game that I’m working on with Temitope Olujobi, Avery Holmes, Isabella Djurovic, and Kristina Atanasoski.
We ran an A/B test at NYU Game Center’s Playtest Thursday. I wanted to test for general errors and bugs (which we found) and how item placement affects puzzle solve time and the number of wrong combination attempts. So, I made two builds with the puzzle items placed in slightly different locations and wrote some analytic hooks to silently capture those data points.
We found that in both versions there was a significant spike in time for the second puzzle (find a power strip and a scythe). Players found the scythe quickly but took forever to find the power strip. Since this is the second puzzle and the third puzzle took way less time, I looked into why this might be. I had a hunch that people had trouble finding the power strip because:
Another version we are trying:
We also found that players had more incorrect puzzle combinations in the build with more “hidden” objects. I think this is because players who learned the tag reveal tried it immediately to get a payoff for their long search for the objects – the level of effort wasn’t being rewarded appropriately. -> To fix this in subsequent builds I’ve been more intentional about where puzzle items are placed – keeping them within sightlines (though not always immediate ones – some are clearly visible when the player turns to leave, for example). Using analytics has been very smooth and I’ve set up another set of A/B test builds to measure another round of item placement.
I am a designer, programmer, and animator on this project that I am working on with Temitope Olujobi, Avery Homes, Isabella Djurovic, and Kristina Atanasoski.
We named the project Merge Conflict because comedy arises from tragedy.
We decided to change the puzzle order (and add new ones) to better make sense with our gatekeeping structures. Now:
We also structured where puzzle objects will go in these rooms. For example, there must be at least one puzzle item that’s part of the solution to the following puzzle in a newly unlocked room. This is to reinforce the player’s reward of new place to explore with a new way to progress.
I also animated and implemented some UI elements.
This project is being made in collaboration with game designers Temitope Olujobi, Kristina Atanasoski, Avery Holmes, and Isabella Djurovic. I am a programmer and puzzle designer on the project.
After Temi took an art pass on the halls and apartments, we fleshed out the puzzles into this proof of concept that demonstrates the kinds of combination puzzles the player completes during the game. There is also an example of a gatekeeping puzzle. We iterated over gatekeeping strategies for a while before deciding on having NPCs unlock rooms for the player. This made the most sense as opposed to have players combine items to make a key and then use the key on a door (use being a verb that’s only used in that context.) This solution removed the verb use, streamlining player understanding.
To cut down on asset creation and scope, we decided to limit combined objects to appear as presents. All presents are the same, with the text on the tag changing to describe the contents depending on the unique combination. Each puzzle has one unique combo solution that always combines only two items. Presents and people cannot be combined.
Rachel Moeller's Game Development Blog 