In a recent Journal of the American Medical Association paper, Ross Koppel reported on a field study of a hospital's order-entry system, which physicians use to specify patient medications. The study identified twenty-two ways in which the system caused patients to get the wrong medicine. Most of these issues are usability problems.
MY ROLE
I served in many different roles for DaVita's UX Research team. On multiple occasions I acted as a note taker, observer, and moderator. I also created the data reporting forms, analyzed the data and presented the study findings and design recommendations to DaVita's many stakeholders.
CRAFTING THE STUDY
I started crafting the study by having an initial meeting with the UX Design Team, Business Stakeholders, and Project Managers. The initial meeting gave me the core goals and questions in which the study needed gather information.
THE PROBLEMS
As the UX Researcher for DaVita, I was challenged to research and design best practices for their CWOW and HWOW software. The technology needed to solve multiple design problems used by 70,800 medical professionals. The software they were using was out of date, often had poor visibility, and an overly complicated workflow that dramatically slowed their pace of treating patients.
INTERVIEWING AND USABILITY TESTING
According to Erika Hall and the Nielsen Norman Group, ethnographic interviews are still the best form of getting inside a participants head and learning about their habits. I conducted each interview based on the needs of the designers and business stakeholders while also keeping my methods flexible enough to accommodate a medical environment.
MODERATORS GUIDE
In each study, I was tasked with uncovered potential usability issues and learning how each prototype worked in multiple situations. For the interview and usability studies I created, I always include a moderators guide with an introduction, body, and wrap up section (See the sample moderators guide above).
For each moderators guide, I paired it with a clickable InVision prototype. In some cases, the medical concepts that needed to be tested were so complex that multiple design solutions had to be presented to see what outcome would work best for a participant. For these scenarios, I created A/B tests.
USABILITY TESTING with PROTOTYPES
THE TESTING ENVIRONMENT
In each interview session, I also included a checklist of items to help create a smooth and comfortable environment for my test participants. DaVita was a unique client in that I consistently was able to visit their Nurses, Dietitians, and Facility Administrators in their place of work. This meant that my test had to be as non invasive as possible just in case the test needed to place on the medical floor. In order to make sure that the best feedback was captured, I included these items in my research practice.
- A research plan
- A working InVision Prototype
- A Moderator
- Note Taker(s)
- One or more methods of documentation, such as a camera or Lookback. (*If not on the medical floor)
- A timer
EVALUATIVE AND QUALITATIVE RESEARCH
In my studies, the note-taker recorded both evaluative, qualitative measures during each testing session. My evaluative data included the participant’s frequency and severity of errors as well as the rate of completion for each task. The qualitative data included the participant's comments, expectations of actions, concerns, and my observations of their behavior.
FINDINGS
My many studies at DaVita uncovered several opportunities for software improvement. In the case of the Diet Order wireframe shown above, 5 out of 5 participants understood how to enter 75Kg and understood what "standard" body weight meant as a medical term. But 4 out of 5 of those participants also wanted to be able to enter their own notes, a capability that had been removed on this prototype.
FINAL ANALYSIS
The process of final analysis should always include everyone that was involved with the research. In my practice I took care to follow these steps.
- Closely review the notes.
- Interesting behaviors, emotions, actions, and verbatim quotes.
- Observations on a sticky note.
- Ability to group the notes on a whiteboard.
- Rearrange the notes to assess the patterns.
Each studies analysis and results were input into a readable chart that was put on Confluence for multiple departments to reference.
FINAL PRESENTATION
In order to present a course of action based on the research that I conducted, I always included two research presentations. The first was a quick analysis that designers could quickly implement into their designs. The second was a high fidelity account of what took place in the field visit. It was important for me to iterate each concept that was tested and the objectives that I used in each section of the test.
For certain concepts I found it helpful to create visuals of the wireframes and participant feedback in order to create a frame of reference for where my participants were especially frustrated. I also created in depth video analysis so stakeholders at the DaVita Headquarters building could observe the reactions and daily life in the clinics.