As of 06/06/19, we are happy to announce that all Advanced and Enterprise Users on Botmock have access to our latest major feature, Usability Testing, on all projects. 

At its core, usability testing is our take on taking your prototypes to your users and giving them an opportunity to play around with a chatbot or voice skill that you have created. This can allow for testing before a bot/skill is deployed, or even as a method to do closed testing on new versions of a bot, all without a single line of code.

For Usability Testing, all teams with the feature can have an unlimited number of testers for every project.

Getting Started

To start using Usability Testing, you start by opening the "Test" tab on the left side of your Botmock Editor. As of right now, Usability Testing is only available for Flow Projects.

When opening up the tab, you'll be presented with a screen that allows you to activate Usability Testing for your project.

Simply press the "Start Usability Testing" Button to begin the configuration process for testing.

Setting Up

Once you activate Usability Testing, you will be prompted to setup an initial set of options that are required to move forward with a successful Usability Testing experience.

Here is an overview of all of the Settings: 

Intro Image - This is an option to upload an image that a tester will be above the title on your Usability Test starting or "Intro" screen.

Intro Title - The title that will appear in the starting or "Intro" screen for the tester when starting a Usability Test.

Introduction Message - A message that will appear under the Intro title in the Starting screen in a Usability Test for testers.  

End Message - A message that will appear to testers after they either end, or hit a goal block in a Usability Testing session.

Password Protection - An option to enable for a Usability Test to be password protected when a tester goes to testing link. Users will only be able to access a test if they know the password added in this setting.

Free Form Test Type - The "Free Form" test type gives the option to an Editor to set the test mode that allows testers to simply go through a Usability Test session without having to reach any content/goal blocks to complete the session.

Goal-Based Test Type - The "Goal-Based" test type gives the option an Editor to set the test mode that allows testers to have to reach a certain point in the conversational interface to end/complete the session. A "Give Up" button is also given to testers when this option is enabled.

Session Length - A configuration that allows a Project Owner/Editor to set the maximum length of time that a tester can try and be in a Usability Test session before the session is automatically completed for the tester.

Deploying Usability Testing

Once all the options are setup and configured by a Project's Owner or Editor team, the next step is activating usability testing and sending the link to a target test group. By default, a Usability Testing link is activated once the initial configuration is complete, but can be deactivated for any period of time to work on changing a flow or configuration settings.

Measuring Usability Testing

With Usability Testing, a few ket metrics can be measured using Botmock.

  • Effectiveness: Did the user achieve their goals?
  • Efficiency: How much effort did it take user to reach their goal?
  • Satisfaction: How satisfied was the user with the solution presented?

The way to see the proper results in Botmock can be found under the "Sessions" section of the Usability Testing tab. Once you have given a link and let testers go through tests, your user sessions will appear in this section. 

When opening up an individual session overview, you can see a few different pieces of info: 

User Info - An Email and name is collected when the user firsts enters a Usability Test, and is shown in the editor as well.

Completion Time - The amount of time (in minutes) that a tester was in a session, or how long it took them to complete a Goal-based session.

Feedback - Feedback that is collected on the completion screen when a tester reaches the end of a usability testing session.

Session Replay - A video version of the Usability Testing session that shows the entire session and can be paused/unpaused for easier analysis.

Visualize Journey - A version of the Usability Testing archive that shows in a Journey-style format to see the exact path that the tester took, all within the actual Project Editor.

There is also an option to delete any individual sessions if needed, which can be done by simply clicking the trash icon under the User Info.

Did this answer your question?