Focus Groups

 

I’ve written often about the concerns I have with research techniques.  This column was inspired by my attendance at a focus group.

 

I was just asked to attend some focus groups and I was reminded of the dozen or so reasons I don’t like them.

 

First, rich and busy people usually won’t attend, skewing data to a narrow demographic.  This would taint research in a decent undergraduate social science class.  Worse, focus group companies know where they can get repeat attendees on short notice.  These are often students, the unemployed or at least people who live near the focus group facilities.  All of this taints the data, too.

 

In this particular focus group I feel a twinge as soon as the leader walks in the room.  She is too talkative, as if trying to entertain a client, not conduct research.  Conducting professional scientific research is a learned skill.  Notice I use the term scientific and do not make a distinction with social science.

 

About fifty years ago Heisenberg said the instrument used in a laboratory experiment affects the results of the experiment—the Heisenberg uncertainty principle.  A thermometer used to measure temperature changes the temperature of the thing it’s measuring.  In 1962, Kuhn pointed out that scientists have a tendency to collect data that supports their views and ignore that which conflicts.  He used the term paradigms to describe this phenomenon.  Merton and Feyerabend discussed how science is a value laden pursuit, driven by those performing the science.

 

So the focus group leader is in a difficult position.  Waltzing in like the star of the show, trying to impress, performing for the client behind the two-way mirror or any other false behaviour can make that position untenable and skew the data.

 

A focus group leader should dress and talk one step up from the subjects in the room.  S/he is there for scientific purposes to gather data, not make friends or entertain.

 

Most focus groups begin way too quickly.  The leader should allow discussion to evolve slowly with open ended questions to see what the respondents want to discuss.  Closed, specific questions, especially ones that can be answered with “yes” or “no” or a short sentence won’t reveal as much as an open ended questions encouraging discussion.

 

Closed questions yield what researchers call an aided response.  The questions tell respondents what to talk about.  Open questions yield an unaided response where the respondents can tell researchers what to think about—the way it should be!

 

In the focus group I was in recently, we were testing confidential matters.  But let’s say it was reactions to Candidate X.  One of the first questions was, “What kind of car do you think Candidate X drives?”

 

This sparks a discussion of myriad types of cars.  I’m reminded that some people follow car models more than others.  Some may name one car thinking it’s the sportiest on the market, while another person may name the same model because it represents fuel efficiency, economy or some other attribute.

 

In the end, without knowing what respondents mean by the car models they name, you end up with a mish-mash of information that could mean anything.  “Cadillac” means luxury, high-price, high fuel consumption and perhaps the ability of American industry to compete with anyone on the planet.  Who knows?

 

But if there really were value in the car question, it should have begun in an open ended fashion such as, ”How does Candidate X get around the district or campaign trail?”

 

Respondents might name trains, boats, planes and cars.  If we find out people think the candidate flies around in a private jet, that could be a problem.  If cars are really relevant, after an open discussion of other types of travel, the closed ended question could be asked about what kind of car the candidate uses.

 

In the focus group I watched, one respondent got up on his hind legs and said he didn’t think the candidate used a car.  It takes guts to challenge an authority figure and say the question is wrong.  If a respondent does that, it’s a very powerful message that you could be on the wrong track.

 

Next the leader held up pages from the candidate’s web page, passed them around and asked for a review.  That’s not how people use web pages, so it’s hard to tell what was being tested.

 

Next, the leader passed around literature that the candidate regularly sends out.  The question was, “What do you like about the brochures, householders and other communication?”

 

I have two problems with this line of questioning.  First, it misses the opportunity to find out if anybody remembers getting anything in the mail from the candidate.  People may feel boxed into saying they received and remembered the material, when the most valuable information might be that they didn’t.  Next, what if people hate the literature?  Asking what they like about it, cuts off the discussion about what they don’t like.

 

People like to be cooperative when being paid, so you have to be careful they aren’t so cooperative that you don’t find out what they’re really thinking.

 

Next came the Barbara Walters question.  She was famous for asking an interviewee, “If you were a tree, what kind of tree would you be?”

 

In this session, respondents were shown pictures of all kinds of people—young, old, various races, both genders and so on.  As they are looking at the pictures, the leader asks, “If the candidate’s campaign were a person, what person would it be?”

 

Not only is the question a bit odd, it’s hard to tell what people mean when they pick a picture of a fit, muscular looking man.  Does mean the campaign is intimidating, full of thugs or has staying power for the long run?  Next, respondents start answering questions that weren’t asked.  One picks the construction worker because the candidate comes from a part of the district where there’s lots of construction.  Another picks the young Asian woman because the district is becoming more multi-cultural. Some seem to be picking people they find attractive or would like to be with, not who embody the campaign.  This ends in another mish-mash of unusable information.

 

I innocently asked the client where the pictures came from?  It turns out employees of the research company picked them.  They may use them every session, for all I know.  Regardless, it does not make a lot of sense to me to have respondents judging pictures picked either at random or purposefully by a research company to make their sessions go more smoothly.

 

Next, the leader asks what does the person in the picture do for a living, and what does the person do on weekends?  (I assume the construction worker works in construction).

 

Ironically, after going to all this trouble to stimulate fairly irrelevant discussion, the leader cuts off dialogue by asking if anyone has any final thoughts?  When the leader leaves the room to get more instructions from the client, I listen in to the continuing discussion in the focus group room.  It’s actually a better discussion, unaided by the leader.  One older woman began waxing nostalgic about the district, the party and the candidate.

 

So, what’s the right approach?  There are lots of them.  Credible scientists use several different methods and compare them to obtain more reliable data.  Polls, questionnaires, elite interviews with opinion leaders and lay-elite dialogues to see where there are gaps between those in the know and regular folks can all help.  So can old-fashioned research.  Online databases now make is really easy to search everything from newspapers to academic journals.  For all a campaign knows five distinguished academics and journalists have written 10,000 excellent words on the topic over the years.

 

Focus groups can be helpful, if run properly.  But they also need to be augmented with sound, old-fashioned research, preferably in libraries, surrounded by musty books.

 

 

 

For Safety’s Sake…Think Again

We often assume the police, fire, EMS, mayor, and other first responders are fulfilling their duties to keep us safe.

 

But according to my studies of plans from the top 100 English-speaking cities in the world, this isn’t always the case.

 

Many of our cities are in danger from their own emergency plans which sometimes feature pages of jargon, acronyms, missing appendices, irrelevant lists of contributors, and wishful thinking.

 

Researchers in the field call them “fantasy plans,” relying on public transit, private cars, and even expertise and help that often doesn’t really exist.  Up to 56% of urbanites don’t have cars, and often the remedies listed in crisis plans turn out not to exist. In fact, some evacuation orders have killed more people than the emergency did.  Contra-flow for evacuations, with all roads leading out, has been called potentially life-threatening by researchers.  

 

Where does San Francisco fit in?

 

San Francisco has two main emergency plans: the All-Hazards Strategic Plan and the Hazard Mitigation Plan

 

The biggest strength of San Francisco’s All-Hazards Strategic Plan is the emphases on a comprehensive training program for city workers, the use of volunteers, and social media.

 

But the All-Hazards plan was last updated six years before I studied it, and the plan has not followed the maintenance schedule set for it. Moreover, when I studied it, I found myself wondering whether anyone is really safer as a result of reading San Francisco’s statements describing how the plan was developed or what the visions, missions, and guiding principles of it are. It’s hard to see how these help anyone prepare for an emergency, but they take up about a quarter of the plan.

 

The Hazard Mitigation Plan is different. It’s newer that the All-Hazards plan. But it has some of the same problems.

 

The Hazard Mitigation Plan is full of very general statements and is often overly preoccupied with semantics definitions. The section on “seismic hazards,” for example is mostly about distinguishing earthquakes from landslides and tsunamis. A brief history of the effect of those events on San Francisco follows. But I could find no procedures describing specifically what to do in the even of those emergencies.  How does it help  victims to be absolutely sure they are being swept away by a landslide versus a tsunami or earthquake?

 

It looks as though San Francisco’s plans were not intended for the general public. They offer hardly any information on individualized, micro-level measures that citizens can take to prepare themselves to deal with emergencies.

 

No city’s crisis plan is perfect. But a good plan can mean the difference between a well-handled crisis and disaster which can cost lives.

 

Some other cities’ emergency plans do have something serious to say — Boston’s climate change study, Kansas City’s dealing with pets, for whom residents will risk their lives, and Richmond, B.C.’s links to great information on personal preparedness. These are just some of the elements that stand out and which could be duplicated by other cities.

 

If San Francisco or any other city lacks the money or time to write a better plan, the best advice from publicly available plans could be cut and paste it into a better document than is on most websites in North America.

 

Some of the plans, including San Francisco’s may have been updated in the months since I read them. No plan will ever be able to foresee every form disaster can take, so it’s important to be flexible and learn from the experiences of other cities.  But the main focus should be on useful, clear information for the average citizen.  

Stirred, not Shaken

 

Every now and then the topic of earthquakes comes up in Vancouver.  The most recent event was the 4.8 quake on 29 December, 2015.  I have worked on the topic of earthquakes for federal authorities for some years and continue to follow the issue.

Vancouver’s emergency planning documents put the risk of an earthquake in perspective.  They note that 60% of Vancouver’s building stock was built before seismic building codes.   There’s been no damaging earthquake in modern times and so all these buildings are vulnerable. 

Interestingly, my father was partly responsible for construction of one of the first earthquake resistant buildings in Vancouver — 200 Granville Square.  I remember first hearing about earthquakes when I lived in Burnaby in the late 1960s.  I was a young teenager and my father was General Manager of “Project 200”, an early attempt to revamp the waterfront.  Nobody played nicely, not even consortium members Woodwords, Canadian Pacific, Grosvenor Lang, or Sears.  City planners didn’t react quickly or lead the discussion, and neither the federal nor provincial government seemed interested.  Project 200 is another story—a missed opportunity.  But the earthquake resistant building is highly relevant today.

 

There was a lot made about Granville Square.  It was among the first density transfers of its kind in North America with my father having to help obtain a mortgage for the air above the CP railway tracks.  Usually mortgages are for land and buildings, but this building was built on stilts above the CP railway tracks. 

While this deal was being done and the building designed, my father would regale us at the dinner table about his day at work.  One day it was the promotional copy that told of the “giant pedestrian mall” that would abut Granville Square.  “And where will you find the giant pedestrians?” I asked.  Another night my father toyed with some promotional copy that would tell the story of Granville Square being the only building left standing after a major earthquake.  He thought it would be fun to tell potential tenants that they’d be able to work in peace, quiet, and safety, if only they could climb over the rubble of all the other buildings in the city.

 

In those days, earthquake-proofing a building meant constructing sockets, lined with neoprene on which concrete pillars stood, which in turn held up the building.  My father said he had no idea what neoprene ended up like after he’d put a building weighing many hundreds of tonnes on top of it.  He certainly had no idea how to perform maintenance on the assembly after construction.

 

Now, Vancouver’s plan goes well beyond building materials. Memoranda of Understanding with other Canadian cities are now part of the plan in order to ensure rapid deployment of resources after a disaster.  Similarly, the Vancouverites are recommended to reduce dependency on electricity and natural gas grids and develop back-up power sources. Even if buildings are still standing after an earthquake, supply lines and energy infrastructure might be severely damaged or destroyed.

 

Even the best-designed earthquake-proof building might need to be evacuated in case of fire or other emergency. This is why Vancouver’s plan requires buildings’ structural drawings and fire plans to be copied and stored centrally in order to speed-up assessment of complex and high occupancy dwellings.

There’s more to be done in Vancouver and neighbouring cities.  Some have designated disaster routes (DDRs) and others don’t.  Some link to useful provincial government preparedness documents, and others don’t. We’ve never sure how many individual citizens are well prepared and would rather not find out after an earthquake.   

Today in History: July 12th

1995:  A heat advisory is issued in Chicago, warning of a record-breaking heat wave. By the time the heat breaks a week later, nearly 1,000 people are killed. Record high use of air conditioning caused some power failures. People opened so many hydrants to cool themselves off that water pressure was lost. The heat warped train rails, causing delays for commuters.

For more on crisis management: Buy ‘An Ounce of Prevention’

1990: Just two days after Mikhail Gorbachev is re-elected head of the Soviet Communist Party, Boris Yeltsin, president of the Republic of Russia, announces his resignation from the Party. Yeltsin’s action was a serious blow to Gorbachev’s efforts to keep the struggling Soviet Union together.

1960: Louis Robichaud is sworn in as Premier of New Brunswick, replacing Hugh John Flemming.

For more on politics: Buy ‘Political Conventions’

Today in History: July 10th

1971: Death of Samuel Bronfman, prominent Canadian businessman.

Tough Love at the Table – Pipe Cleaner

1958: Prime Minister John Diefenbaker and President Dwight Eisenhower sign an agreement to have Canada and the United States set up a Joint Committee to guide North American defences in the event of enemy attack.

For more on politics: Buy ‘Political Conventions’.

1887: A dam breaks in Switzerland, killing 70 people in their homes. The water pressure on the dam slowly eroded the concrete. Rescue boats launched to assist people caught up in the sudden flood were ineffective, as some of those on the boats drowned when they capsized in the roiling waters. For more on Crisis Management: Buy ‘An Ounce of Prevention’.

Today in History: July 9th

2010: Prime Minister Stephen Harper appoints legal scholar David Johnston the next governor general.

1991: In St. Lazare, Manitoba, 400 residents flee their homes when a train carrying highly corrosive acetic anhydride derails. The emergency evacuation ends after six days. For more on Crisis Management:Click Here

1960: President Dwight Eisenhower and and Soviet Premier Nikita Khrushchev trade verbal threats over the future of Cuba. The relationship between the United States and Cuba deteriorated rapidly after this exchange. The Castro regime accelerated its program of expropriating American-owned property. In response, the Eisenhower administration severed diplomatic relations with Cuba.

Today in History: July 8th

2000: The Canadian Alliance choses former Alberta Treasurer Stockwell Day as leader, replacing founder Preston Manning.

1991: A study by the C.D. Howe Institute points out dangers of a post-separation economic alliance between Quebec and Canada.

1991: A Gallup Poll reports that 69% of Canadians want Prime Minister Brian Mulroney to resign, including 80% in Ontario, but only 54% in Quebec.

Today in History: July 7th

1996: Canadian Space Agency astronaut Dr. Bob Thirsk lands with his Shuttle mission crew mates at the Kennedy Space Center, after Columbia completed 272 revolutions of the earth, and a record 16-day, 21-hour, 48-minute and 30-second flight.

1975: Ed Broadbent is chosen leader of New Democratic Party on fourth ballot in Winnipeg, replacing David Lewis. Finishing in second place was Rosemary Brown.

1887: Blyth built a cloth-sailed wind turbine (or “windmill”) in the garden of his holiday cottage in Marykirk and used the electricity it produced to charge accumulators; the stored electricity was used to power the lights in his cottage, which thus became the first house in the world to be powered by wind-generated electricity.

Today in History: July 6th

1998: Two Canadians are among the 167 crew members killed as a gas leak leads to an explosion and fire on the Occidental Petroleum drill rig in the North Sea off the coast of Scotland. For more on Crisis Management: Click Here

1946: On this day in 1946, George Walker Bush, is born in New Haven, Connecticut. When he was two years old, Bush’s parents moved to Texas. George W. Bush was elected president in 1999, and served until 2008. Learn more: Click Here

1944: In Hartford, Connecticut, a fire breaks out under the big top of the Ringling Bros. and Barnum & Bailey Circus, killing 167 people and injuring 682. An investigation revealed that the tent had been treated with flammable paraffin thinned with gasoline to make it waterproof. Ringling Bros. and Barnum & Bailey Circus eventually agreed to pay $5 million in compensation. For more on Crisis Management: Click Here

Today in History: July 5th

1998: Japan launches a probe to Mars, joining the U.S. and Russia as a space exploring nation.

1971: President Richard Nixon formally certifies the 26th Amendment, which lowers the voting age in the U.S. from 21 to 18 years. To learn more: Click Here

1970: An Air Canada DC-8 going from Montreal to Los Angeles makes a heavy landing at Malton Airport in Toronto, bounces and loses one engine. As the pilot tries to take off and land again, another engine falls off and the airplane crashes, killing all 109 people on board.