Water from a stone: User research in government without users

The Code for Canada fellowship program is no longer active. To learn more, visit our fellowship yearbook.

Users are part and parcel of user-centred product design. They are the customer and the lifeblood of a business. For any product to be successful, it must centre user needs early in the product ideation stage. This user research ideally starts early, is iterated on regularly, and builds the foundation for the product.

Bureaucratic Barriers

In the case of the government, our user profile is a little different. Within the Government of Canada, our customer is often everybody; people from all ages and life stages. They use our products for essential services like applying for employment insurance. They rely on us during key life events like immigrating to Canada or registering the birth of a child. These services have multiple moving parts — including legislation, policy, strategic direction, financial considerations, political climate, operational requirements, human resources, electoral priorities, content design, communications, bilingual translation, and more.

Given the complexity of government services, there are checks and balances to ensure that important things aren’t left out. Policy legislation ensures that government employees stick with rigid guidelines and approval processes. These are not limited to; but include:

• privacy guidelines to prevent the sharing of personal identifying information
• interview techniques to ensure all participants are able to provide informed consent
• inclusivity criteria so all user research methods are inclusive of people with disabilities and bilingualism

All these (necessary) formalities mean that it can take a lot longer to deploy a solution than it would take in the private sector. User research in civic tech is tricky to perform because of the scale of users, the number of stakeholders and implicated organizational structures, and mistrust of the government. Unfortunately, these processes are slow to adapt — techniques and processes don’t always come with up-to-date regulatory requirements. The pace at which things change across government can render an environment that can sometimes feel hostile to user-centred, agile product design. This pace, however, is an opportunity to innovate in non-traditional ways to access key user insights. This is the user research that should inform the delivery of digital products that our population uses.

Changing the Status Quo

I then paraphrase the folder of images into a spreadsheet with tags for further analysis. I tried to tag whether the query seemed to be on the behalf of the author or on behalf of someone else. I also add a region if available (provincial or national), the source of the post, what EI program the query pertains to, and what keywords might be applicable to their situation (i.e. pregnancy, retirement, or low income).

Finally, I aggregate my research into some basic metrics to better understand how to potentially prioritize product requirements. I used an informal word cloud on the 50 top referenced keywords to better understand the scale at which some subjects may be of more concern than others.

Within this particular sample set, for example, we discovered that 1 in 5 of our data points was looking for information on behalf of someone else (e.g. an elderly parent), which gives us insight into how we should be designing service access to serve not just individuals directly, but the people they might be supporting in their life journeys.

4.3. Authority Bias

The case for performing passive research within government lies within authority bias. Research shows that people generally have a “[deep-seated] duty to authority” (Milgram, S. (1963).). They are less likely to complain about the government TO the government. There is a fear that it may impact their own benefits somehow. There are guidelines around compensation and recruitment as well which can inhibit participation. Passive research allows us to better understand the true sentiment that we might not always capture with a phone call or in-person interview.

The opposite can also be true in some cases — a lack of trust, particularly within communities experiencing historical (and in many cases, ongoing) marginalization within Canada — can have an impact on formal user research participation rates and diversity.

5. User research is not user testing

Governments can sometimes treat user research, testing and usability as the same thing when they are different, and serve dramatically different purposes.

User research should take center stage prior to the production of the product. It’s an opportunity to better understand the basic bounds within which you should be designing. Some good questions to answer here are:

• What do we know about the people who use the service?
• What have we created in the past for them? Do they like using it?
• Who else has created something like this in the past? Can we do a knowledge share?

Don’t jump into user testing early — or perish the thought — use it to replace user research. Create a product that’s worthy of user-testing first. A considerable practice is our user research and testing methods to take a page out of pharmacology. When vaccines are being developed globally, they have to meet minimum quality and safety standards before they can be given to human research participants — let alone distributed widely.

As designers, our preliminary research should inform the design of an optimal product based on our informed predictions so that we can have something worthy of testing. User testing can be time and resource-intensive, so optimizing the time we have with our users lies in ensuring we’re asking the right questions — the questions we haven’t been able to answer yet.

Ensure interactions match what the user might expect — time with the external user is valuable and should be based on actionable tasks that can be measured in a quantifiable way. Here’s an example:

1. Scenario: Can you please show us how you would find the application criteria for a specific benefit?
2. Testing outcome: It took the user 35 seconds and 8 clicks to find the application criteria for a particular benefit. This same application criteria can be found within 3 seconds and 2 clicks on a search engine.
3. Requirements building: How might we design an experience that not only considers how a user might intuitively navigate a page.
4. Actionable: We should design and test a predictive search that filters benefits below based on plain language keyword tags.

I encourage you as a designer to speak up when you sense that user research might be conflated with user testing within your organization.

Summary

In summary, user-driven design within government can be a complex beast to navigate because of bureaucratic and regulatory guidelines. The stakes are high given that the services we design impact the vast majority of the population. Many government agencies are currently in the early stages of digital transformation, which means that it’s an optimal time to start introducing user-driven and agile design methods to product delivery. As designers, it is in our best interest to get creative and resourceful when we run up against barriers. We can explore non-traditional channels for user research to subsequently create digital government services people use because they want to, and not because they have to.

Want Code for Canada's support conducting user research in government? Fill out a form on our inclusive user research page.