Tuesday, December 4, 2012

The Numbers Don't Lie

So let's pretend, and I stress pretend, Metro's customer satisfaction survey is 100 percent accurate and 80 percent of Metrorail riders are indeed satisfied with the service, and 84 percent are happy with bus service.

After seemingly constant fare hikes, a new board, a new GM, years of weekend and nighttime track work, escalator replacements, straight As on all their "vital signs" reports, tons of new hires, new buses, Metroforward, Momentum, Rush+, pats on the back from every "stakeholder,"  and hundreds of millions spent "rebuilding the system" to "improve safety and reliability," it's about time the hard work and "investment" started paying off with higher numbers of satisfied riders, right?

Well...

Back in 2008,  less than a year before the fatal Red Line crash, 85 percent of Metrorail riders and 78 percent of Metrobus riders were satisfied with Metro service.

Back in 2007, 87 percent of Metrorail riders and 81 percent of Metrobus riders were satisfied with Metro service.

I could not find releases about earlier surveys, but in the two above, Metro said the numbers "remain consistent" from 2004.

Weird thing is that in 2003, when I'm pretty sure everyone thought Metro was at least little better than it is now (it certainly was cheaper), the satisfaction with Metrorail was 72 percent and 76 percent for Metrobus. Metro used what appears to be a different system of surveying in 2003.

I suppose some of this year's lower numbers could be explained by the constant track work over the weekends, but perhaps, if Metro were honest and forthcoming with riders and did a better job at managing expectations instead of rolling out self aggrandizing PR campaign after PR campaign, its relationship with its riders might be a little better. Instead, Metro continuously brags and boasts about how everything is just fine.

If you live on PR instead of real accomplishment,  eventually, even the cooked numbers won't lie.

If you're interested in some readers' comments on the methodology of the Metro survey, check out this and this. Also you can vote in the Unsuck rider satisfaction survey here

Other items:
Metro busts OT budget (Examiner)
Metro backs off cutting SE bus service (WaPo)

Comments (23)

Loading... Logging you in...
  • Logged in as
Great job exposing their BS. All that work, and they haven't moved the needle. FAILURE.
I love that they spouted this BS for years and years and no one ever called them on it!

Metro's respect for it's customers is astonishingly low.
Well, comparing Unsuck's numbers to Metro's numbers is akin to comparing the President's approval ratings on one Democratic and one Republican news site. Obviously the numbers aren't going to match up!

I have no idea if Metro's numbers are biased or not, but the fact that their satisfaction numbers are higher than before the 2009 crash is indeed fishy. Yet the Unsuck approval rating (5% as of December 4, 2012) seems just as inaccurate. People who like Metro don't go on this website, the vast majority of people here are unsatisfied with Metro- as is reflected in the site's name- "Unsuck DC Metro!"

I don't think the Unsuck survey can reflect how the general public views Metro, for the reason I just stated above. It's like polling for the public's approval rating of the President, and only asking Republicans
7 replies · active 640 weeks ago
H8 rider's avatar

H8 rider · 640 weeks ago

Um, I think you're missing a major point here. Unsuck is not a taxpayer funded organization and is not trying to tout its poll as scientific.

Regardless, your points have nothing to do with this post.
"Um, I think you're missing a major point here. Unsuck is not a taxpayer funded organization and is not trying to tout its poll as scientific. "

Well, in all fairness, I don't think Metro is touting it's poll as scientific either. However I could be wrong about that, I have no idea what makes a poll be classified as "scientific."

"Regardless, your points have nothing to do with this post."

The post concerns Metro's survey. I know the author didn't directly tie it to the Unsuck survey, but there is a link to it at the bottom of the post. It wouldn't be there if the author didn't want to provide contrast to the Metro survey.
Jeff, in some respects it does not matter if Metro's poll was scientific or not, what matters is that it is driving Metro's decisions regarding a whole host of important issues.

It is concerning, however, that Metro it putting such stock into this survey, yet is not releasing basic information like crosstabs and the like.
I spent $250,000 and hired an army of consultants to create that poll. Don't diss.
including a consulting company based in Omaha named SoulShaker
They made an interesting case, but I went with SoulSucker instead. It was more expensive and offered less.
"I don't think the Unsuck survey can reflect how the general public views Metro, for the reason I just stated above"

That's part of the point... the numbers can be skewed based on the target audience of the survey.
Here's my analysis of their methodology.

Dan Stessel, Dr. Gridlock and Ben Schumin got together, ordered a few pizzas and filled out 650 surveys. The other 150 were sent out to metro customers...
3 replies · active 640 weeks ago
Who is Ben Schumin? I've heard of the other two....
He's just a douche on twitter who's over the top sympathetic to everything metro. his handle is @SchuminWeb
WMATApologist
hrh king friday 13's avatar

hrh king friday 13 · 640 weeks ago

New Poll: 98% of peopple think Metro's poll is bullcrap.
There is one statistic which shows very clearly that the poll was less than useful.

"Trip planning information and service scored 100 percent among customers who used the website"

I call BS... not to downplay the website, but because nothing would be 100% on a survey about WMATA with more than 700 participants....
OT: having just gotten off the phone with the smarttrip people to get *another* refund for a situation where a faregate/parking gate charged my card twice for a single swipe, please everybody check your smarttrip usage online every few weeks to make sure WMATA isn't robbing you blinder than they already are. sometimes i'll catch the gate double billing, but sometimes i don't.

happy commuting!! :-)
russell.j.coller.jr's avatar

russell.j.coller.jr · 640 weeks ago

This whole controversy is racist. Put me down as a critic of EVERYTHING Metro... except the scratchy announcements on the train PA systems. It drowns out the racist thoughts in my head --- almost.
1 reply · active 640 weeks ago
you probably also believe in slavery
I wanted to fill out a WMATA survey at one time, but the form was worded in such a way as to leave no room for comment, questions, interpretation. Questions such as: At what station did you receive this survey? Where did you exit? What was the time of day? Innocuous, yes. .... But then: Where do you live? What is your salary? What is your race? .... Really WMATA? No wonder you report a relatively high 80% customer satisfaction. Those surveys were a joke!
more OT: wmata employees are tired. awww.
http://wmata.com/about_metro/news/PressReleaseDet...
This "survey" is a joke. If you want to know what a real transit survey looks like, check out this one from Seattle: http://metro.kingcounty.gov/am/reports/2010/2010_...

Notice how much emphasis the King County study puts on "Regular Riders" vs. "Infrequent Riders". For the Metro survey, they called random numbers. Of those who picked up, they asked if the respondent had used Metro in the last 30 days. My guess is that they picked up a great deal of infrequent riders in their survey. Infrequent riders, often don't find problems that they experience to be problematic whereas a regular user would find the situation to be chronic and unacceptable.
1 reply · active 640 weeks ago
There is also some nice current dynamic charts on their website regarding performance, etc. (opposed to WMATA's incorrect old static image of a graph)

Both a well-implemented survey and a readable maintained system of infographics takes foresight, attention to detail, and man-hours. WMATA does one of these well-unfortunately, all three are required.

Post a new comment

Comments by

 
Creative Commons License
This work is licensed under a
Creative Commons Attribution-Noncommercial 3.0 Unported License.
Site Meter