Exploratory Study
First, I sought to learn about Associates’ experience working this job and working near the Proteus robots. To do so, I visited a facility that had Proteus robots, and I interviewed 12 Associates in that facility. I walked around the facility with them and asked them to show me what they did. I asked what they liked and disliked about the robots. I also spoke with 3 General Managers about what they would need to introduce Proteus into their own facilities. Constraints during this time, were that only a few robots were active on the floors in testing mode because it was early in Proteus development.
Process
I performed the following:
- I received training to work as an Amazon Associate moving carts, to better understand their experience.
- I conducted semi structured interviews. At times, I also asked Associates to use the think aloud method and tell me what they were thinking as they performed parts of their jobs near the robots.
- I audio-recorded interviews and later transcribed them.
- I used thematic analysis to determine themes from interviews.
- I synthesized findings and wrote of a report on the results. I described what was working well and poorly with the robots, which pain points were most critical and needed to be addressed earliest, and which elements were important to study further to best determine how the robots’ behavior should change.
- I ran a meeting discussing my findings with the rest of the team. I sought their feedback about my recommended plan of action based on their understanding of the robot engineering and our project resource allocation.
Outcomes
- Learnings on what about Proteus was working best and what most needed improvement or further study.
- My next immediate goals on the Proteus project.
- My findings and report affected the next year’s planning and prioritization for the Proteus project.
Next Steps
One main finding from my exploratory study was that some elements of the robots’ motion (e.g., the driving behaviors) confused Associates. These behaviors are deeply coded into the robots, and a myriad of engineering and technical constraints prevent the team from changing those behaviors in the short term. This provided two main next steps, which I worked on in parallel:
- Develop training to help Associates understand Proteus' motion in the short-term
- Identify how Proteus should act (e.g., its behaviors) to better communicate with Associates in the mid- to long-term
1. Training to Understand Proteus Motion
I explored what types of training would best help Associates develop a better mental model of Proteus’ motion so they could work alongside the robots with greater ease.
Process
- I led meetings with the team to understand what types of training options were feasible for associates at this facility (e.g., video, demonstration).
- I selected three main training methods that differed the most from each other to test.
- I developed the procedure for each training method, including writing video scripts myself.
- I drove creation of training materials through collaboration with designers and contractors.
- I conducted AB testing of different trainings at the facility.
- I measured success of trainings using surveys and interviews. I also collected information on what Associates need from training and how to make trainings most successful.
- I synthesized findings and wrote a report of the results. I described which elements of each training were most useful to Associates, which trainings might work best for people who work daily near the robots versus who pass through the robot areas from time to time. I wrote recommendations on how to create the trainings.
- I ran a meeting to discuss the results with the rest of the team. I integrated their feedback about my recommended plan of action.
- Based on the feedback, I continued driving development of the training with the Learning team and other collaborators.
- From time to time, the product launch team asked for trainings on new engineering features, which continued to roll out every few weeks.
- I proposed a way to pivot the training we created to be more modular and scrappy to account for frequent changes. In a meeting, the team agreed on my proposal.
- I drove development of the modular training materials.
- I used learnings from my study to provide information for posters to help educate Associates about Proteus.
Outcomes
- Posters providing information about Proteus to anyone in the facility
- Short-term plan for modular and scrappy training Associates while the Proteus robot has frequent behavioral updates
- Materials for short-term training
- Long-term plan for training Associates once Proteus is in a stable state and ready to launch in further locations
- Rough draft of long-term training materials (updated as Proteus continues to update)
Next steps
- Continue developing the training as new Proteus behaviors are released
- Continue gathering feedback on training materials to improve them for the final product
- Finalize the long-term training materials for Proteus before it is released in additional sites
2. How should Proteus Act?
A second next step from my exploratory study was to determine behaviors that Proteus could engage in that would improve its working relationship with Associates. I ran a bodystorming study to gather this information. In bodystorming, people act out the part of the humans and the part of the robots to more fully demonstrate what the robots might do in the given environment. It also helps people double check that their intuition on what the robots should do feels good when physically played out. For example, someone might say that they want the robot to pass close them quickly, but if someone passes them at that speed, they may feel uncomfortable.
Process
- Based on my above-described exploratory study, I determined the three main contexts in which Proteus' behaviors were most confusing or startling.
- I designed a bodystorming study and gathered feedback from my team to improve the feasibility in the facility.
- I visited the facility and ran bodystorming with associates. We used materials to to help improve our communication in the scenes, including wearing inner tubes to pretend to be the robots (see Figure), using a bicycle bell to represent Proteus' sounds, and using flags to represent Proteus' lights. I video and audio recorded the study.
- During and after each bodystorming session, participants and the researchers took notes on Post-it’s indicating what types of behaviors they preferred and in what contexts.
- I used thematic analysis and synthesized findings to create a report on main recommendations.
- I shared the report with the team, which was happy about the findings and asked for indication of what engineering requirements I suggested to achieve these recommendations
- I worked with an engineer to develop a document showing which behaviors required which existing vs new robot capabilities.
- Together we presented the table to a diverse audience including team leads on various engineering efforts. The team leads asked questions and prioritized efforts to develop these capabilities.
Outcomes
- A list of main pain points with Proteus, indication of outcome if these pain points are not resolved, suggested behaviors to resolve these pain points, robot/engineering capabilities we need to develop to create these behaviors
- Buy-in across teams and prioritization to create robot capabilities and implement recommended behaviors
- Developed the bodystorming method for improving robots in industry, and have an academic paper under review to help other researchers implement this method.
Next Steps
- As features develop, I will test how they can best be deployed to alleviate pain points.