In the last year, Yukoners may have noticed an uptick in government public engagement surveys calling for input on a wide range of subjects from cannabis legalization to school calendar dates to a carbon tax rebate.

The Yukon Liberal government has iterated on multiple occasions the priority it’s making public engagement.

On Oct. 30, it announced the launch of a new website dedicated to sharing government consultations and their results. Twenty-two engagement initiatives have been completed this year, and many included surveys.

While public consultation has value, market researcher Donna Larsen acknowledges, it’s the blurring of lines between Yukon government engagement surveys and professional public opinion and market research that has prompted her to share a number of concerns with the Star about the way these survey results are being gathered, presented and interpreted.

Larsen is a partner with DataPath Systems, a market research company out of Marsh Lake.

After the Yukon government published its hugely popular cannabis legalization survey results Nov. 9, at least three Yukon media outlets – including the Star – reported these results in a way that made them seem representative of all Yukoners.

A significant number of Yukoners responded to the cannabis survey – more than 3,000.

However, the survey’s finding that 81 per cent of respondents support the federal government’s plan to legalize cannabis, for example, cannot be extended to mean 81 per cent of Yukoners support the federal government’s plan.

Public opinion and market researchers use a variety of methods and controls to obtain survey results that are applicable to a wider population.

A number of factors prevent the Yukon government’s cannabis survey and other engagement surveys from being representative.

“When they invite people to do a survey on a particular topic, they’re inviting people who have a vested interest in the outcome of that survey. So that right away disqualifies it from being a scientific, representative study,” Larsen explained.

Another red flag is that the demographic breakdown of survey respondents does not match the demographic breakdown of the Yukon’s population.

Just over five per cent of respondents, for example, were aged 65 and older.

According to the 2016 Statistics Canada Census, almost 12 per cent of the Yukon’s population is 65 and older.

“It does not represent or reflect the opinions of Yukoners. It only reflects the people that took the time to respond to that survey,” Larsen told the Star.

“The Yukon government seems to be totally now embracing this; they’re doing it on every topic.”

Kara Johancsik is a public engagement advisor with the Executive Council Office.

She told the Star Thursday that the Liberal government has indeed embraced these public surveys on government policy since taking office in December 2016.

“The reason we’re engaging a lot right now is because we’re also taking action on a lot of items as we enter the second year of this mandate,” Johancsik said.

“In order to make those decisions in the best possible way, we want to hear views of Yukoners and have those views reflected in decisions.”

So why is the government making these decisions based on survey results that do not represent the views of the Yukon’s population as a whole?

And why are media outlets reporting on these survey results as if they do?

To the former question, Johancsik said the government isn’t striving to do market research nor create a statistically representative picture of Yukon public opinion with these engagement surveys.

That’s because not everyone cares about or will be affected by every government policy decision, she said.

“We know that depending on the issue that we’re asking people about, some people are going to be very impacted by it, care a lot about it, have a lot to say. Some people may not care at all,” she said.

Johancsik pointed to the Legal Profession Act as one example. An engagement survey on the legislation saw 45 respondents.

“If we were doing a random sample every time we wanted to engage on a policy issue, it might be quite exhausting for people,” Johancsik said.

She was referencing one of the methods market researchers use to obtain representative survey results.

It would also be expensive and time-consuming, she added.

“We want to give people the option to not participate if they don’t have anything to say about it, but also give them the option to participate if they want to contribute.”

And the government does so in a variety of ways, she pointed out.

Surveys are made available online and often in hard copy as well, and face-to-face engagement sessions are also hosted.

“Public engagement in terms of the online surveys we do are designed to be inclusive and accessible rather than representative, and I don’t see that changing,” Johancsik said.

But Larsen argues that distinction needs to be made clearer.

“Listening to Yukoners, getting input from Yukoners, sure, great,” Larsen countered. “But they need to not step over the line between consultation and research.”

The government has an obligation to make it clear that they are participating in the former and not the latter, she explained.

Using graphs and percentages in presenting survey results – as the Yukon government does – blurs the line between professional research and the public engagement the government says it’s facilitating, Larsen said.

The public sees these symbols and thinks they represent the results of a scientific study.

“It’s not research, it’s not scientific, it’s not representative. Don’t pretend it is,” Larsen said. A disclaimer could help, she noted.

Asked about this possibility, Johancsik said she thought it was obvious that the surveys the government advertises are not representative.

The government invites people to participate, so it’s clear it’s not a controlled sample, she said. In presenting survey results, the number of respondents and their demographic breakdown are listed.

“To me, that’s pretty clear that what we’re talking about here is the people who responded. The respondents, not Yukoners as a whole.”

But, Johancsik went on to say, if this in not in fact evident to those reading the survey results, “we can look at tweaking wording.”

Yukon media outlets are not the only ones to misinterpret government survey results.

In Saskatchewan, the provincial government also put out a survey about the legalization of cannabis.

CBC Saskatoon noted in a story on the results that “because it is an online survey, no margin of error is available and it is considered non-representative.”

However, the Regina Leader-Post headlined its story with “Majority of Saskatchewan residents believe legal age for marijuana should be 19: Survey,” implying representativeness.

So is it the media that have to scrutinize survey results more carefully before reporting on them, or the government that has to make it clear that engagement surveys aren’t representative?

“I think it goes both ways,” Johancsik said. “If we can be clearer about it, then we should be.”