Alfie Martin

Billing & Payments

Usability Study

Study Objective

Evaluate the usability of the updated 'Bill is Ready' email, the discoverability of the 'My Bill Insights' page from the email, and the clarity and usefulness of the information on the 'My Bill Insights' page.

Study Details

Users were given the following scenario then asked to complete seven related Tasks :

"
Imagine you've just received your monthly utility bill in your inbox. The bill was higher than expected, so you want to check the details to understand what's driving your bill. You're going to open the email and search for more information about your usage and charges.

METHODOLOGY

  • Unmoderated Usability Study via UserZoom
  • Eight Participants
  • Opco Mix
  • Average 12 minutes per session
  • Results include screen recordings, transcript, task success and questionnaire responses
The study concluded with a five-part final reflection questionnaire designed to assess users' affinity for the designs presented.

Executive Summary

The usability study validated the appeal, clarity, and discoverability of the authenticated 'My Bill Insights' page and enhanced 'Bill is Ready' email, while also surfacing future opportunities to elevate the post-MVP experience.

  • Participants mentioned how easy, smooth and straightforward the experience was overall.
  • The amount of information or specificity of information (such as the energy use breakdown) was a pleasant surprise to participants.
  • Many participants first clicked 'View Bill' from the 'Bill is Ready' email when asked to find more information on their bill, indicating the 'My Bill Insights' page could be even more discoverable if linked from the bill itself.
  • Six of eight participants were 'extremely likely' to use this webpage in the future to understand their bill.

FUTURE OPPORTUNITIES

  • Position link to 'My Bill Insights' page on bill itself
  • Include more support options in addition to FAQs
  • Explore feasibility of more granular usage chart for customers to see peak usage times (daily or hourly)
  • Separate personalized tips from Home Energy Analysis breakdown, or make more discoverable (QPower)
  • Make the Home Energy Analysis breakdown more interactive (ie. Clicking into usage)

Open Email

TASK

Please review the email as if you had just received it in your inbox.

  • 1

    Two of eight participants found the 'My Bill Insights' page without prompting

  • 2

    Participants commented on how clean and organized the email appeared

QUESTIONNAIRE

What is your first impression of this email?
Please describe what you see.

"Appreciate how clean it is, not an overwhelming amount of information"

"...feels organized and not overwhelming"

"Explore your bill insights probably covers the changes [to my bill]"

Open Webpage

TASK

If you haven't yet, click the link under "Explore Your Bill Insights" that reads: "View Details"

What did you expect to see after clicking 'View Details'?

"A comparison of the prior month to the current month showing the difference,"

"I expected to be shown the reasons for the change"

"I expected to see a comparison for usage or a change in cost"

"I thought I'd see like a general, this is what people use"

First Impressions

TASK

You've arrived at the Bill Insights page. Take a moment to look around and try to understand your charges.

"

This feels like everything I would want to know if I was going to call [my utility company]

"
Total Amount
Due
Taxes & Fees
Delivery
Supply

Whats Included in My Monthly Bill?

Your total bill consists primarily of:

Supply

The cost of generating electricity.

Distribution and Delivery

The expense of transporting electricity across long distances.

Taxes & Fees

State and local taxes, fees, and surcharges.

"I love an infographic, I love how easy it is to absorb."

Understanding

What do you understand about your bill from this page?

"I understand that my bill is higher than this month, mainly due to increased energy costs, a longer billing period and more usage likely caused by the weather conditions, and also the 10% increase."

"I can see what part of my home I'm using the most electricity like the electronics"

Is anything confusing or unclear?

"No, everything seems very clear. The layout is easy to follow, and the explanation is simple."

"there was nothing confusing or unclear at all in my opinion"

Investigating Increases

TASK

Based on the information available, what do you think could explain the increase in your bill this month?

6/8

Participants cited the rate increase mentioned in the editable module

8/8

Used the monthly comparison widget to understand higher cost

Likely reasons your electricity charges are about $28 more

Weather Conditions or Daylight Hours

You used more electricity due to changes in weather.

+ $9.73

Bill Period ÷ $6.85

You used more electricity because this bill period was 1 day(s) longer.

+ $11.42

Energy Cost / Energy Use

Your bill may have changed due to differences in your energy consumption or adjustments in energy prices. To get a breakdown of your energy use, update the information about your home.

QUESTIONNAIRE

How easy or difficult was it to find an explanation for the increase in your bill?

Six of eight answered "very easy"

• One participant thought a granular breakdown of usage over time (minutes, hours) would be most helpful:

"Perhaps a chart that shows you the time breakdown of your usage, so you could narrow down what is causing the increase, whether it's your air conditioning or the usage of electronics."

Where did you look first?

"Well, right at the top... it answered the question immediately."

"where I looked first: I, I loved the breakdown... The first point is the energy costs rose, and then I looked at the breakdown of the weather and the extra day....."

First click distribution

Electronics Module

27%

Top Breakdown

73%
See a Personalized Tip

More Information

TASK

You are still looking for support on your high bill and want to find more answers to bill-related questions. Where would you go to find more information?

Seven of eight participants pointed to the FAQs to find more information on bill-related questions.

• One participant thought FAQs were generally unhelpful because they didn't address his uncommon questions.

• Opportunity to explore pathways to more support options in this real estate.

"I hate [FAQs] because I must not have frequently asked questions... I must have isolated individual questions."

QUESTIONNAIRE Results

Were you able to find information on ways to reduce your energy bill?

6/8
Answered 'Yes'

"The whole lower half of this page has things that can help me"

How easy or difficult was it to find ways to reduce your energy bill?

7/8
Answered 'Very Easy'

Follow Up Questionnaire

Users were asked how they would describe the overall experience, if anything surprised or confused them and if there is anything they would improve.

Participants mentioned how easy, smooth and straightforward the experience was overall.

The amount of information or specificity of information (such as the energy use breakdown) was surprising to some participants.

Ideas for improvement included:

Phone number or chat to contact support

More granular usage chart (daily or hourly)

Make the breakdown more interactive to explain categories

Opportunities Summary

1

Because users first clicked 'View Bill' when prompted to find more information on their charges, consider positioning link to 'My Bill Insights' page on bill itself.

2

Consider including more support pathways on the bottom of the page (ie. Chat or other unassisted support options).

3

Explore feasibility of more granular usage chart for customers interested in seeing peak usage times on a daily or hourly basis.

4

Because so few customers discovered the personalized tips from Home Energy Analysis breakdown, make them more discoverable either in UI design or break them out into a separate widget (OPower).

5

Consider how to make the Home Energy Analysis breakdown more interactive to allow users to click into categories for more information.