Focus Groups

 

I’ve written often about the concerns I have with research techniques.  This column was inspired by my attendance at a focus group.

 

I was just asked to attend some focus groups and I was reminded of the dozen or so reasons I don’t like them.

 

First, rich and busy people usually won’t attend, skewing data to a narrow demographic.  This would taint research in a decent undergraduate social science class.  Worse, focus group companies know where they can get repeat attendees on short notice.  These are often students, the unemployed or at least people who live near the focus group facilities.  All of this taints the data, too.

 

In this particular focus group I feel a twinge as soon as the leader walks in the room.  She is too talkative, as if trying to entertain a client, not conduct research.  Conducting professional scientific research is a learned skill.  Notice I use the term scientific and do not make a distinction with social science.

 

About fifty years ago Heisenberg said the instrument used in a laboratory experiment affects the results of the experiment—the Heisenberg uncertainty principle.  A thermometer used to measure temperature changes the temperature of the thing it’s measuring.  In 1962, Kuhn pointed out that scientists have a tendency to collect data that supports their views and ignore that which conflicts.  He used the term paradigms to describe this phenomenon.  Merton and Feyerabend discussed how science is a value laden pursuit, driven by those performing the science.

 

So the focus group leader is in a difficult position.  Waltzing in like the star of the show, trying to impress, performing for the client behind the two-way mirror or any other false behaviour can make that position untenable and skew the data.

 

A focus group leader should dress and talk one step up from the subjects in the room.  S/he is there for scientific purposes to gather data, not make friends or entertain.

 

Most focus groups begin way too quickly.  The leader should allow discussion to evolve slowly with open ended questions to see what the respondents want to discuss.  Closed, specific questions, especially ones that can be answered with “yes” or “no” or a short sentence won’t reveal as much as an open ended questions encouraging discussion.

 

Closed questions yield what researchers call an aided response.  The questions tell respondents what to talk about.  Open questions yield an unaided response where the respondents can tell researchers what to think about—the way it should be!

 

In the focus group I was in recently, we were testing confidential matters.  But let’s say it was reactions to Candidate X.  One of the first questions was, “What kind of car do you think Candidate X drives?”

 

This sparks a discussion of myriad types of cars.  I’m reminded that some people follow car models more than others.  Some may name one car thinking it’s the sportiest on the market, while another person may name the same model because it represents fuel efficiency, economy or some other attribute.

 

In the end, without knowing what respondents mean by the car models they name, you end up with a mish-mash of information that could mean anything.  “Cadillac” means luxury, high-price, high fuel consumption and perhaps the ability of American industry to compete with anyone on the planet.  Who knows?

 

But if there really were value in the car question, it should have begun in an open ended fashion such as, ”How does Candidate X get around the district or campaign trail?”

 

Respondents might name trains, boats, planes and cars.  If we find out people think the candidate flies around in a private jet, that could be a problem.  If cars are really relevant, after an open discussion of other types of travel, the closed ended question could be asked about what kind of car the candidate uses.

 

In the focus group I watched, one respondent got up on his hind legs and said he didn’t think the candidate used a car.  It takes guts to challenge an authority figure and say the question is wrong.  If a respondent does that, it’s a very powerful message that you could be on the wrong track.

 

Next the leader held up pages from the candidate’s web page, passed them around and asked for a review.  That’s not how people use web pages, so it’s hard to tell what was being tested.

 

Next, the leader passed around literature that the candidate regularly sends out.  The question was, “What do you like about the brochures, householders and other communication?”

 

I have two problems with this line of questioning.  First, it misses the opportunity to find out if anybody remembers getting anything in the mail from the candidate.  People may feel boxed into saying they received and remembered the material, when the most valuable information might be that they didn’t.  Next, what if people hate the literature?  Asking what they like about it, cuts off the discussion about what they don’t like.

 

People like to be cooperative when being paid, so you have to be careful they aren’t so cooperative that you don’t find out what they’re really thinking.

 

Next came the Barbara Walters question.  She was famous for asking an interviewee, “If you were a tree, what kind of tree would you be?”

 

In this session, respondents were shown pictures of all kinds of people—young, old, various races, both genders and so on.  As they are looking at the pictures, the leader asks, “If the candidate’s campaign were a person, what person would it be?”

 

Not only is the question a bit odd, it’s hard to tell what people mean when they pick a picture of a fit, muscular looking man.  Does mean the campaign is intimidating, full of thugs or has staying power for the long run?  Next, respondents start answering questions that weren’t asked.  One picks the construction worker because the candidate comes from a part of the district where there’s lots of construction.  Another picks the young Asian woman because the district is becoming more multi-cultural. Some seem to be picking people they find attractive or would like to be with, not who embody the campaign.  This ends in another mish-mash of unusable information.

 

I innocently asked the client where the pictures came from?  It turns out employees of the research company picked them.  They may use them every session, for all I know.  Regardless, it does not make a lot of sense to me to have respondents judging pictures picked either at random or purposefully by a research company to make their sessions go more smoothly.

 

Next, the leader asks what does the person in the picture do for a living, and what does the person do on weekends?  (I assume the construction worker works in construction).

 

Ironically, after going to all this trouble to stimulate fairly irrelevant discussion, the leader cuts off dialogue by asking if anyone has any final thoughts?  When the leader leaves the room to get more instructions from the client, I listen in to the continuing discussion in the focus group room.  It’s actually a better discussion, unaided by the leader.  One older woman began waxing nostalgic about the district, the party and the candidate.

 

So, what’s the right approach?  There are lots of them.  Credible scientists use several different methods and compare them to obtain more reliable data.  Polls, questionnaires, elite interviews with opinion leaders and lay-elite dialogues to see where there are gaps between those in the know and regular folks can all help.  So can old-fashioned research.  Online databases now make is really easy to search everything from newspapers to academic journals.  For all a campaign knows five distinguished academics and journalists have written 10,000 excellent words on the topic over the years.

 

Focus groups can be helpful, if run properly.  But they also need to be augmented with sound, old-fashioned research, preferably in libraries, surrounded by musty books.

 

 

 

For Safety’s Sake…Think Again

We often assume the police, fire, EMS, mayor, and other first responders are fulfilling their duties to keep us safe.

 

But according to my studies of plans from the top 100 English-speaking cities in the world, this isn’t always the case.

 

Many of our cities are in danger from their own emergency plans which sometimes feature pages of jargon, acronyms, missing appendices, irrelevant lists of contributors, and wishful thinking.

 

Researchers in the field call them “fantasy plans,” relying on public transit, private cars, and even expertise and help that often doesn’t really exist.  Up to 56% of urbanites don’t have cars, and often the remedies listed in crisis plans turn out not to exist. In fact, some evacuation orders have killed more people than the emergency did.  Contra-flow for evacuations, with all roads leading out, has been called potentially life-threatening by researchers.  

 

Where does San Francisco fit in?

 

San Francisco has two main emergency plans: the All-Hazards Strategic Plan and the Hazard Mitigation Plan

 

The biggest strength of San Francisco’s All-Hazards Strategic Plan is the emphases on a comprehensive training program for city workers, the use of volunteers, and social media.

 

But the All-Hazards plan was last updated six years before I studied it, and the plan has not followed the maintenance schedule set for it. Moreover, when I studied it, I found myself wondering whether anyone is really safer as a result of reading San Francisco’s statements describing how the plan was developed or what the visions, missions, and guiding principles of it are. It’s hard to see how these help anyone prepare for an emergency, but they take up about a quarter of the plan.

 

The Hazard Mitigation Plan is different. It’s newer that the All-Hazards plan. But it has some of the same problems.

 

The Hazard Mitigation Plan is full of very general statements and is often overly preoccupied with semantics definitions. The section on “seismic hazards,” for example is mostly about distinguishing earthquakes from landslides and tsunamis. A brief history of the effect of those events on San Francisco follows. But I could find no procedures describing specifically what to do in the even of those emergencies.  How does it help  victims to be absolutely sure they are being swept away by a landslide versus a tsunami or earthquake?

 

It looks as though San Francisco’s plans were not intended for the general public. They offer hardly any information on individualized, micro-level measures that citizens can take to prepare themselves to deal with emergencies.

 

No city’s crisis plan is perfect. But a good plan can mean the difference between a well-handled crisis and disaster which can cost lives.

 

Some other cities’ emergency plans do have something serious to say — Boston’s climate change study, Kansas City’s dealing with pets, for whom residents will risk their lives, and Richmond, B.C.’s links to great information on personal preparedness. These are just some of the elements that stand out and which could be duplicated by other cities.

 

If San Francisco or any other city lacks the money or time to write a better plan, the best advice from publicly available plans could be cut and paste it into a better document than is on most websites in North America.

 

Some of the plans, including San Francisco’s may have been updated in the months since I read them. No plan will ever be able to foresee every form disaster can take, so it’s important to be flexible and learn from the experiences of other cities.  But the main focus should be on useful, clear information for the average citizen.  

The City of the Future

 

I’ve always thought of Seattle as a symbol of modernity.  Growing up in Vancouver, I viewed the Space Needle as an example of American prosperity and progress.  It reached toward outer space while the monorail sleeked along the ground toward the future.  I now live in Toronto, but visit Seattle when I can, to keep in touch with the future. 

  

The world almost missed getting these two lasting symbols from the 1962 World’s Fair.  The Fair was originally going to be called The Festival of the West, as hard as it is to mix cowboys and technology—Broncos and Boeing.  More futuristic heads prevailed and the fair became known as the Century 21 Exposition with the motto “Living in the Space Age.”  The Fair turned a profit and left a legacy of civic amenities and revitalization. 

 

Edward E. Carlson, one of the great civic boosters of any time and any city, extracted a victory within a victory.  He famously drew the Space Needle on a napkin, obtained funding for the project, started it a little late, but finished on time.  The Space Needle retired it’s debt in 18 months.  It now has a new pod for more weddings and parties.

 

My book Safer Cities of the Future is a study of better urban experiences through better design.  I began the book with a picture of the Space Needle and the Seattle skyline.  I recounted the story of Mr Carlson and the world’s fair as an example that we get things right occasionally.

 

It’s a lifetime later and Mr. Carlson and Seattle should be remembered for success and foresight.  But now we have new challenges — terrorism, a sluggish economy, severe weather events and little faith in space, the future, or in anything.  This is where Seattle can lead the world again.

 

Seattle’s All Hazards Mitigation plan is one of the best I’ve studied. It offers a very realistic view of the risk of floods, earthquakes, and terrorism. The plan notes, for example, that 32.75% of Seattle’s housing stock was built before 1939, an astonishingly high percentage considering that the seismic building code dates from 1992. If there were a serious earthquake, the damage could be enormous and could come at a very high cost. Seattle’s plan notes that property damage claims against the city have totalled as much as $12 million since 2006. In the event of a serious storm or earthquake the cost would be exponentially higher.

 

Seattle’s plan also includes a detailed summary of activities to mitigate the impact of earthquakes. These include upgrades to buildings to correct structural deficiencies, disaster recovery needs for all IT systems, and replacement of older underground cables. And the plan notes that city departments, the police, and public utilities have all received briefings and training in order to prepare them for seismic events.

 

These are all steps in the right direction. But Seattle’s plan could be improved by including specific instructions for residents on what to do or where to go in the event of earthquakes, floods, or other disasters; what to take with them; and even how to ensure the safety of their pets.

 

Don’t get me wrong: no plan is perfect, and there are some very good plans out there that Seattle could borrow from. Auckland, New Zealand, for instance is the only plan I’ve seen that precisely quantifies the economic cost of a volcanic eruption. Kansas City has a unique plan for rescuing citizens’ pets. And Boston’s plan for dealing with climate change is an excellent example for all other coastal cities.
Most plans leave much to be desired.  But Seattle’s is a template for progress.  

Stirred, not Shaken

 

Every now and then the topic of earthquakes comes up in Vancouver.  The most recent event was the 4.8 quake on 29 December, 2015.  I have worked on the topic of earthquakes for federal authorities for some years and continue to follow the issue.

Vancouver’s emergency planning documents put the risk of an earthquake in perspective.  They note that 60% of Vancouver’s building stock was built before seismic building codes.   There’s been no damaging earthquake in modern times and so all these buildings are vulnerable. 

Interestingly, my father was partly responsible for construction of one of the first earthquake resistant buildings in Vancouver — 200 Granville Square.  I remember first hearing about earthquakes when I lived in Burnaby in the late 1960s.  I was a young teenager and my father was General Manager of “Project 200”, an early attempt to revamp the waterfront.  Nobody played nicely, not even consortium members Woodwords, Canadian Pacific, Grosvenor Lang, or Sears.  City planners didn’t react quickly or lead the discussion, and neither the federal nor provincial government seemed interested.  Project 200 is another story—a missed opportunity.  But the earthquake resistant building is highly relevant today.

 

There was a lot made about Granville Square.  It was among the first density transfers of its kind in North America with my father having to help obtain a mortgage for the air above the CP railway tracks.  Usually mortgages are for land and buildings, but this building was built on stilts above the CP railway tracks. 

While this deal was being done and the building designed, my father would regale us at the dinner table about his day at work.  One day it was the promotional copy that told of the “giant pedestrian mall” that would abut Granville Square.  “And where will you find the giant pedestrians?” I asked.  Another night my father toyed with some promotional copy that would tell the story of Granville Square being the only building left standing after a major earthquake.  He thought it would be fun to tell potential tenants that they’d be able to work in peace, quiet, and safety, if only they could climb over the rubble of all the other buildings in the city.

 

In those days, earthquake-proofing a building meant constructing sockets, lined with neoprene on which concrete pillars stood, which in turn held up the building.  My father said he had no idea what neoprene ended up like after he’d put a building weighing many hundreds of tonnes on top of it.  He certainly had no idea how to perform maintenance on the assembly after construction.

 

Now, Vancouver’s plan goes well beyond building materials. Memoranda of Understanding with other Canadian cities are now part of the plan in order to ensure rapid deployment of resources after a disaster.  Similarly, the Vancouverites are recommended to reduce dependency on electricity and natural gas grids and develop back-up power sources. Even if buildings are still standing after an earthquake, supply lines and energy infrastructure might be severely damaged or destroyed.

 

Even the best-designed earthquake-proof building might need to be evacuated in case of fire or other emergency. This is why Vancouver’s plan requires buildings’ structural drawings and fire plans to be copied and stored centrally in order to speed-up assessment of complex and high occupancy dwellings.

There’s more to be done in Vancouver and neighbouring cities.  Some have designated disaster routes (DDRs) and others don’t.  Some link to useful provincial government preparedness documents, and others don’t. We’ve never sure how many individual citizens are well prepared and would rather not find out after an earthquake.   

Today in History: July 13th

2007: Conrad Black is convicted in U.S. District Court in Chicago of mail and wire fraud and obstruction of justice. The charges related to diverting funds due to Hollinger International for personal benefit, and to Black and his chauffeur removing documents from Hollinger offices in violation of a court order prohibiting removal.

Political Conventions – Lordly Body Language

1991: Gwich’in people of Mackenzie Delta settle a land claim, getting 15,000 square kilometres of land and $75 million. This is the first regional settlement with northern native groups.

For more on negotiations: Buy ‘Tough Love – Power, Culture and diversity in Negotiations, Mediation & Conflict Resolution’

1968: the Hong Kong Flu pandemic begins. It would go on to kill 1 million people worldwide.

An Ounce of Prevention – On Pandemics

An Ounce of Prevention – Black Death

Today in History: July 11th

2005: Deh Cho First Nations agree to a deal with the Canadian government to get participation in the environmental assessment and regulatory review of the $5.7 billion Mackenzie Valley Pipeline gas project.

For more on negotiations: Buy ‘Tough Love at the Table: Power, Culture and diversity in Negotiations, Mediation & Conflict Resolution’

1991: Carla Hills, American trade representative, says the North American free-trade deal NAFTA will not endanger auto pact or harm Canadian culture.

For more on politics: Buy ‘Political Conventions’

1955: Seven American teenagers die on Mount Temple, near the Valley of the Ten Peaks near Lake Louise, Alberta. It is Canada’s worst single mountaineering accident.

For more on crisis management: An Ounce of Prevention

Today in History: July 1st

2002: A Russian passenger plane with 69 passengers and crew collides in the air over Germany with cargo plane with a crew of two, killing all 71 people on both planes. As they approached each other, an automated system told one pilot to go up, and the other to go down. However, a Swiss air-traffic controller ordered both pilots to descend.

1958:  CBC starts nationwide TV broadcasting as new Trans-Canada microwave relay system goes into operation.

1867: The British North America Act creates the Dominion of Canada, uniting the British colonies of the Province of Canada (now Ontario and Quebec), the Province of New Brunswick, and Nova Scotia.

Political Conventions – Happy 43rd Birthday, America

Today in History: June 30th

1984: Pierre Trudeau steps down as Prime Minister of Canada and retires from politics.

1997: The United Kingdom hands over control of Hong Kong to the People’s Republic of China.

Political Columns – Pugnacious Pierre — Trudeau

1912: The deadliest tornado in Canadian history roars through downtown Regina at 4:50 pm, killing 28 people. In all, 2,500 people are left homeless. Mayor Peter McAra cancels Dominion Day celebrations.

Today in History: June 27th

1984: Pierre Elliot Trudeau is named the winner of the Albert Einstein Peace Prize for his global campaign to ease East-West tensions.

1969:  U.S. President Richard Nixon begins a starts visit to Canada. He speaks in Montreal, and marks the tenth anniversary of the St. Lawrence Seaway, which he said is a lesson “ how two nations can work together; how they can dream together, and make those dreams come true.”

1949: Louis St. Laurent leads the Liberal Party to re-election  in 21st federal general election, winning 190 of 262 seats, an increase of 65 seats. It’s the fourth consecutive majority for the Liberals.

Today in History: June 6th

1981: More than 500 passengers are killed when a train plunges into a river in India. Heavy rains meant the tracks were slick. The engineer was Hindu and believed that cows are sacred. As the train was going over a bridge, a cow appeared on the track. The engineer, wanting to avoid hitting the cow, braked too hard, and the train derailed.

1971: A mid-air crash between two airplanes near Los Angeles kills 50 people. A DC-9 with five crew and 44 passengers collided with a U.S. Navy F-4 Fighter. One of the two F-4 crew members was also killed.

1944:  The Battle of Normandy begins. D-Day begins with the landing of 155,000 Allied soldiers on the beaches of Normandy in France, in the largest amphibious military operation in history.