PRODUCT DESIGN — USER EXPERIENCE — ACCESSIBILITY — USABILITY TESTING

AI-Enabled Interview Experience

Gathering feedback at scale was important for customers, so branching the existing survey features into an adaptable, AI-enabled experience called “Flex” to gather richer feedback became necessary.

  • Create a new feature to gather direct voice-of-the-customer feedback.

  • Build upon and update an existing, yet outdated framework for a modern feedback delivery experience.

  • I was the primary designer for this feature, working closely with product management and engineers over the iterative project.

IDENTIFYING THE PROBLEM

Surveys Weren’t Enough

Our customers were hesitant to adopt our survey feature, due to the static and less engaging nature of the feedback. They much preferred our live interviews, which gave rich feedback on account of a human interviewer being able to pivot. Additionally, they were able to hear directly from the voice-of-the-customer what their thoughts were, their tone, etc. Live interviews were more expensive, inhibiting customers from scaling how much feedback they could analyze.

Primary issues identified:

  • Static and often short answers from survey feedback

  • Expensive human-to-human interview processes

  • Limited data causing customers to not be able to pull trends out easily

PRODUCT DESIGN

A Scalable System

A key challenge was launching the MVP quickly. We adapted the existing survey feature for asynchronous interviews by recording video questions and letting participants answer with audio or video. To reduce scope and optimize time to customer value, I identified which components and conventions that could be reused vs. net new:

  • Adding an “Open Ended” question to the builder so video questions can mix with static “Custom Question” types and reuse branding settings.

  • Building a new video-recording feature that supports recording questions in the builder and participant answers.

73%

of organizations with at least 1 active Workflow

after 1 year post-launch

USER EXPERIENCE

Adaptable AI Follow-ups

To bolster the Flex interview experience, our next iteration was to enable AI follow-up questions. Once a participant answers, AI would determine 1. If a follow-up question is needed, and 2. What the follow-up question should be.

How to generate AI questions quickly, for participants who are already investing more than they get back?

We immediate ran into technical limitations, causing the question generation to be 7-9 seconds. In this situation, it was a non-starter to reduce the time according to our Response Time Standard documentation (see right). Advocating for the user’s experience was important, so our team worked on lowering the time.

We were able to get down to 3-5 second wait time, which was better, but still needed some clear feedback.

Decision: Modal with clear determinate loader

I built a modal overlay for answered questions. We branded the AI with Clozd’s gradient mesh and used generic copy: “…while we consider the best follow-up question” if none is found. A determinate loader with a progress bar and exact percentage was used to reassure users during the up-to-5-second wait.

OUTCOME

Iterative Testing

From the launch of the MVP, customers mentioned a quality increase in insights compared to the survey experience. Being able to hear the customer’s voice from the video increased the level of detailed answers.

Additionally, we looked at improving the completion rates. This launched a couple quick investments that lead to this completion rate increase in only a month.

26 -> 34%

completion rate over 1 month alone, by improving and iterating on the user's experience

from direct customer feedback

Iteration 1: “No audio detected” error states

We saw a handful of submission with no audio. These were contributing to the completion rate positively, but couldn’t contribute to our customer’s insights and had to be manually excluded from the completion rate. To prevent these empty submissions, we implemented an alert if no audio was detected after 3 seconds. We saw a stark decrease in empty submissions weeks after launch.

Iteration 2: Fewer button clicks

A common theme in user feedback was how many clicks it took to record a question. This is rather vague feedback, so after following up with users, we clarified these user experience improvements:

  • Once the user has selected “Video” or “Audio”, automatically show their choice to prevent the mental friction of selecting an option.

  • Upon clicking their answer, users had to previously then “Start recording”. This was replaced with a countdown.

These solutions didn’t eliminate clicks (as the user feedback phrased it was), but reduced the perceived clicks and burden users felt they had to do to answer a question. Users reported immediate improvement in the experience.

Iteration 3: Navigation “Back” and “Next” buttons

Originally, we launched Flex interviews as a very linear process for simplification. We found a group of users weren’t able to complete the interview due to reasons like poor internet timing them out, being pulled away for other responsibilities, etc. Once the page reloaded, users were frustrated that their answers weren’t saved and had to restart the whole interview. To fix this:

  • We started temporarily saving partial responses so participants didn’t have to restart.

  • We added “Back” and “Next” buttons, so participants could revise answers freely. This helped if they were booted to a question unexpectedly, they could return to their desired progression point.