Galileo Cloud

Proving the case for bringing user testing to GM

Galileo is the cloud services platform and process automation hub for General Motors Information Technology teams. It provides self-service tools for developers to create and manage environments, project management teams with streamlined online work flows, and automated coordination between IT-to-IT services teams.

This project was a case study to prove the value of user testing, and to establish a user testing lab within GM IT. We used the Galileo Cloud internal tool for our example, conducting user tests on a prototype and comparing the results.

The Challenge

As of June 2017, the features and design of GM's digital products (web-based applications, marketing web sites, native mobile apps, etc) are being informed by biased, opinion-based decisions, not from direct user insights and user-verified need. Consequently, these products often fail to meet user objectives and expectations. This results in low adoption, general user dissatisfaction, low-quality brand perception, and expensive re-development.

The Scope

This project was a case study to prove the value of user testing, and to establish a user testing lab within GM IT. We used the Galileo Cloud internal tool for our example, conducting user tests on a prototype and comparing the results.

The Goal

By integrating direct, continual user insight into the product development process, we can significantly reduce wasted development and operational support costs, maximizing return on investment. By using the methods performed in a typical usability lab, a reduction in development costs of at least 20% can be expected.

What is Galileo?

And why are we working on it?

As of September, 2017, the Galileo Project Team is seeing lower-than-desired adoption, reducing ROI.

We hypothesize that users are rejecting Galileo due to confusion caused by

  • having three separate web sites

  • unintuitive navigation

  • and a lack of clear, explanatory content.

By obtaining direct user insight through the User-Centered Design process and usability testing,

We believe these sites can be successfully consolidated; improving product understanding and speed of task accomplishment, resulting in improved adoption. Additionally, the consolidation should reduce development costs associated with the maintenance of three sites instead of one.

User Testing

Methodology

Through existing user feedback analysis, additional interviews and observations, we will learn where/why users attempting to navigate the 3 separate sites of Galileo are becoming confused. User insight will help us write usability test cases that will enable us to compare the time it takes a user to accomplish a task with the current 3-site approach, with same test cases performed on an informed prototype.  Additionally, after completion of test cases, we will survey users on their likelihood to use Galileo over their previous method. Finally, we will measure the development cost difference between the currently implemented solution, and the user-accepted solution to establish cost savings.

Metrics will be measured by

  • Time to accomplish a task

  • Future usage survey

  • Uninformed vs. Informed Development costs

User Test Script

We asked them the following questions and analyzed the results:

  • Can you tell me what Galileo does?

  • Based on what you’ve seen so far, can you tell me what Galileo offers?

  • How would you find out more?

  • Do you know what Pivotal Cloud Foundry is?

  • How would you find information about Pivotal Cloud Foundry?

  • Based off this page, can you tell me what Pivotal Cloud Foundry is?

  • Can you tell me what you think Pivotal can be used for?

  • Do you find this page to be not at all useful, somewhat useful, or very useful?

The Prototype Test Results

Below are some highlights from what we found during the user tests.

Overall the new prototype tested positively but still had areas for improvement.

The Current Design Test Results

 

The Takeaways & Costs

Our findings indicate that we should move forward with:

  • The Service Overview Pages

  • Search Functionality

  • Consistent Navigation

More work needs to be done on:

  • The Clarity of What is Galileo

  • The Categorization of Services

  • We could also incorporate more help/tutorial videos.

 

The End Result

As a result of putting this case together showing the value of user testing on Galileo, the GM usability lab was approved and funded.

With these findings we would make adjustments to the prototype, conduct another user test session and see if our changes improve the site experience.