Statistics are altogether too bourgeois for the public sector elite

I've been thinking for a while about Graeme Archer's article on how 'getting' statistics is as important as 'getting' Shakespeare yet people still, in that giggling way, exclaim how they were never any good at maths.

There is no “either/or” between words and numbers; at the same time, of course, there is. CP Snow’s two cultures exist, most clearly in the words/numbers dichotomy, but only because we structure them into existence through conscious decisions about how to educate our young. It’s a human choice, not a law of the universe. Good with words or good with numbers is a story we tell about ourselves, and like all stories it has its heroes, and villains.

Now I'm a social scientist (if I'm anything properly academic) who once gloried in the title of Research and Planning Director and I recall a number - see what I did there - of stories about folk dumbfounded by maths and stats. We once prepared an analysis of customer geography for a knitting wool company (sexy, eh!) based on a simple index and had to spend the first twenty minutes of the presentation explaining what indices were. And when I did my 'Research Methods' module for my MSc, we were told to avoid quantitative research because (I summarise) maths is hard and you don't need to do it.

Such is the reality of a world where my personal experience of buying a railway ticket is a better guide to policy than statistics explaining why I am an outlier in buying such an expensive ticket. Or where - as I put it in a debate at last year's Battle of Ideas:

We’d reckon on uplift in response of around 2X or maybe 3X compared to a random selection. Great until you realise that the response to random was around 0.2% - all that clever technology means that, instead of getting ignored by 998 out of 1000 people, you only get ignored by 994.

Only the 2X or 3X gets reported not the actual numbers even though they tell us we needn't fret so much.

Statistics matter yet the extent to which we understand them or appreciate them is troubling. I recall being told - in very definite terms - that my comments on the relative poverty line were wrong because I was using the median household income not the mean and "median isn't an average". This is despite the fact that median is the right measure where there's a lower bound (i.e. you can't have a negative household income - which might come as a surprise to bankrupts).

The thing with Graeme's article, however, is not that people don't know enough maths and statistics but that they are proud of this fact. And, you know, I might have just found a little glimpse at the reasons why this is the case - numbers, counting, accounting are terribly bourgeois things. Here's a chunk from a chapter in Deirdre McCloskey's Bourgeois Equality (the chapter is entitled 'Aristocrats Scorned Measurement'):

In England before its bourgeois time Roman numerals prevailed. Shakespeare's opening chorus in Henry V...apologises for showing battles without Cecil B. DeMillean numbers of extras: yet "a crooked figure may/ Attest in little place a million/ And let us, ciphers to this great accompt, / On your imaginary forces work." The crooked figure he has in mind is not Arabic "1,000,000" but merely a scrawled Roman M with a bar over it to signify "multiplied by 1,000": 1,000 times 1,000 is a million.

McCloskey - like Graeme Archer - quotes Samuel Johnson (albeit a different reference):

"To count is a modern practice, the ancient method was to guess; and when numbers are guessed they are always magnified."

And that modernity was, in Johnson's time, the advent of Bourgeois England - Capitalist England (although McCloskey would be cross at the sloppiness of this latter designation). Before this time, accounting was a grubby thing of merchants, pursers and bursers not something the great and good should be bothered with. Indeed, McCloskey goes on to note that, prior to 1803, students arriving at Yale were expected to be fluent in Latin and Greek but not to be qualified in arithmetic. Such mundane things as statistics were beneath such grand folk.

We appear to have come full circle with quoting poetry and qualitative (for which, usually, read anecdotal) analysis given greater credence than any sort of proper measurement, analysis and appraisal based on statistics, probabilities and other scary number-crunching things. As Simon Jenkins disdainfully put it (when it seemed his basic statistical knowledge was found wanting):

I seriously doubt this poll, since it implies that two-thirds did know the answers. All on whom I tested it failed, including myself. Nor could they see the point. With one voice they replied: “That’s the sort of thing you learned at school.” So what is the point?

What were the sorts of question? Calculating the mean, median and mode - terms used every day in the Guardian where Jenkins writes. The area of a circle. And even long division and multiplication. Not even GCSE maths, not much advanced on what we'd like an eleven year old to know. We don't need to know this stuff, say the bookish elite, it's smelly maths.

Yet how can you know whether government policies - reducing poverty, improving health and so forth - are effective if you've no idea how the things we're reducing or improving are measured? I'll tell you how - we use anecdote, stories. So, rather than look at the statistics we relate the tale of someone's grandma or how a single mum is struggling on benefits. We sometimes gather together several of such stories and embellish them with commentary from 'experts' relating their own understanding based on further anecdote - or 'qualitative research' as they like to call it.

Statistics, maths, measurement, accounting, calculation - these are the tools of the capitalist, the merchant, the city trader. Thing to be disdained and dismissed in favour of our, often pre-judged, opinion founded on at best a reasoned assessment but more commonly on a collection of tales. We might then hold our noses and wander into the world of maths seeking out some numbers that seem to match what we've found from our tales. People compare apples with pears, use percentages when the numbers are too small for this to make sense, and calculate means when the bounds make the mean meaningless. Journalists - often with the title Science Editor or Health Correspondent - present headlines that bear no relationship to the research they report on. Not because they want to mislead but because they've no idea what 'statistical significance' means let alone what a p value is.

Hardly a day passes without us witnessing people in powerful and influential positions dismissing the evidence of statistics in favour of anecdote and conjecture. And, so long as people can respond by quoting Disraeli - lies, damned lies and statistics - without being laughed at, there is little chance that we'll see the proper use of measurement in the formulation of public policy. Meanwhile, out in the world of markets, capitalism, trade and investment, people have no choice but to use measurement and statistics, without them they would fail. For the public sector elite, however, such concerns are just too bourgeois and instead they make policy inspired by stories in the Guardian or Daily Mail.

.....

Journalists and maths - things that don't really mix. The case of the boozy MPs.

It's from the Sun so all the gory details (doubtless given the source more of the former than the latter) are behind Rupert's paywall. However the point is this:

BOOZY MPs have sparked a fierce backlash from campaigners by splashing out £11,000 in just one week in Parliament’s bars.

Terrible. All these MPs clogging up bars, sloshing back copious quantities of booze while they should be running the country (or something like that). The problem here is shown by some basic maths. There are 650 MPs which means that, to spend £11,000 in a week, the average MP spent £16.92 - just £2.42 per day. A bear in mind that there are a load of other folk who can buy beer in the House of Commons. So MPs are drinking less than a pint of beer on average (the price list is here - the cheapest beer is £2.70 for a pint) and people are having a go at them?

The 'campaigners' in question appear to be folk working for Alcohol Concern - or so it seems. Quoted in the Daily Express, Jackie Ballard (a former Liberal Democrat MP and professional nannying fussbucket) the boss of Alcohol Concern showcased another lie to make a lame point:

Jackie Ballard, head of Alcohol Concern, said: "At a time when alcohol is causing grief to individuals and costing our society £21billion a year, Parliament should be leading by example."

Seems to me that Parliament is absolutely setting an example - the level of consumption is well within the guidelines of the health fanatics and reveals that, far from being a bunch of drunks, MPs are avoiding boozing on the job. The idea that spending an average of £6.62 on the first day back represents partying "hard into night following election gains" seems to be stretching the point given that this won't even buy half a bottle of the House's house sauvignon.

This is just another example of prohibitionist campaigners taking advantage of the seeming inability of journalists to grasp simple maths. With the result that we get shock horror headlines over takings that would represent a pretty lousy night for a typical city centre boozer.

....

Maths and social science...

****

The research methods lecture for my MSc sticks in my mind. Not just because I have a curiosity about different research methods and their rationale but also because the lecturer told us not to worry about maths, indeed that she wasn't any good at maths.

Sadly, the results of such wilful ignorance look like this:

"Not many psychologists are very good at maths," says Brown. "Not many psychologists are even good at the maths and statistics you have to do as a psychologist. Typically you'll have a couple of people in the department who understand it. Most psychologists are not capable of organising a quantitative study. A lot of people can get a PhD in psychology without having those things at their fingertips. And that's the stuff you're meant to know. Losada's maths were of the kind you're not meant to encounter in psychology. The maths you need to understand the Losada system is hard but the maths you need to understand that this cannot possibly be true is relatively straightforward."

In the social sciences (and this is psychology among the most maths rich of these disciplines) the use of maths appears almost discouraged - we're told about qualitative research, how it gives greater insight and understand than mere number crunching. And, when someone comes up with a complicated quantitative explanation of everything ("The Spirit Level" springs to mind as a good example here) the legions of non-mathematicians leap upon the research with glee and excitement. Sadly, what they can't do is explain the maths.

It is a deceptive idea that we can call something 'science' - even with the qualifier 'social' - and then pretend that it can be studied without a reasonable degree of competence in maths and with research methods based on experiment, empirical study and data analysis. This isn't to dismiss qualitative studies - I used to be Planning Director in an ad agency, I love a nice focus group - but to say that, for all their value, such methods simply aren't science.

....

The idea of differential risk is beyond much of the UK's male population

****

The EU, in a fit of lunatic equality-mongering, plans on making insurance companies charge women more for their motor insurance despite them not being as big a risk on the roads as us men.

If this wasn't stupid enough, the results of a poll conducted by Uswitch reveals that this idea of differential risk is beyond much of the UK's male population:

The survey found that despite the hardship facing female drivers, two-thirds (66%) of men said they believed the new legislation was needed. A fifth (18%) of men felt they had been unfairly paying the lion’s share of premiums, and just under half (45%) believed they have been unfairly discriminated against.

Presumably the next step will be to prevent discrimination on the basis of age - meaning that we'll all be paying insurance premiums made more costly by the risks associated with teenaged boy racers. Who I'm sure feel 'discriminated against'.

The answer to those blokes who feel discriminated against is simple - learn how to drive as safely as women and you'll pay less!

....

A glimpse of higher education's future?

****

Time will no doubt tell but might this be a glimpse of higher education's future?

There are no professors in Virginia Tech’s largest classroom, only a sea of computers and red plastic cups.
In the Math Emporium, the computer is king, and instructors are reduced to roving guides. Lessons are self-paced, and help is delivered “on demand” in a vast, windowless lab that is open 24 hours a day because computers never tire. A student in need of human aid plants a red cup atop a monitor.
 
The Emporium is the Wal-Mart of higher education, a triumph in economy of scale and a glimpse at a possible future of computer-led learning. Eight thousand students a year take introductory math in a space that once housed a discount department store. Four math instructors, none of them professors, lead seven courses with enrollments of 200 to 2,000. Students walk to class through a shopping mall, past a health club and a tanning salon, as ambient Muzak plays.

But does it work? First indications are good:

"...Virginia Tech students pass introductory math courses at a higher rate now than 15 years ago, when the Emporium was built. And research has found the teaching model trims per-student expense by more than one-third, vital savings for public institutions with dwindling state support."

Interesting!

H/T Tyler Cowan

....

Checkout queues - not quite what you thought...

Queuing is important. I know you all hate it but it really is important. I’ve blogged before about this matter – in a little moan about the value and importance of differential pricing. So now we’re going to consider the checkout.

Now one of the things about queues (other than the fact that the other queue always moves faster) is that we make an assumption that the shorter queue will take less time. Now this is – on the basis of the information available to us and our common sense – a rational decision. However, it could also be wrong.

Here’s a mathematician studying the issue:

There are easily a dozen variables affecting the line speed that have nothing to do with the number of customers in each line or the number of items in their baskets.

And the other variables tell us that:

Cheque is slower than credit which is slower than cash – not really a surprise and perhaps explains (the rapidly vanishing) cash-only lines

The y-intercept is non-zero – in other words:

It should take you zero seconds to purchase zero items but you can't ignore the fixed time cost of the pleasantries ("Hi. How are you doing? Do you need any help out?") and the transaction itself.

The express lane isn't faster.

You attract more people holding fewer total items…when you add one person to the line, you're adding 48 extra seconds to the line length … without even considering the items in her cart.

So this answers the question about “express” queues – they are not a strange privilege for people who aren’t buying very much but a means by which these people are removed from the main queues – thereby speeding up your passage through with a trolley while making no difference to the customer with but a few items. In truth the rational decision is to separate different queues (i.e. different customer behaviours) in the knowledge that the asymmetry of understanding will improve the rate of throughput. Not only does this make your customers happy but it also reduces the supermarket’s costs (which – in a benign cycle of benefit – further increases the customers’ happiness by helping reduce prices).

Today, there is a further complication with the introduction of self-service checkouts. Again these will remove some customers from the checkout queues (and leave them standing frustrated while the machinery doesn’t work). And the jury is out on this wondrous technology:

It’s easy to see why retailers are turning to more and more self-service kiosks. They put the customer in control, they can reduce operational costs and they save on staff costs. When they are designed well they can improve customer experience because customers grab their goods and leave the store with minimal fuss. When they are designed poorly, they can be time consuming and frustrating having a negative impact on customer experience.

And they might not have cut queues either (although this might not have been the point):

Figures compiled by The Grocer magazine show that average queuing times for staffed tills at Tesco and Sainsbury's, the retailers with the most self-service checkouts, have increased over the past two years. At Tesco, which has 6,000 self-service checkouts in its 1,200 stores, the average wait for a staffed till lengthened from 5min 15sec in 2008 to 5min 42sec this year. Sainsbury’s saw a smaller rise, from 5min 30sec to 5min 35sec.

Although the supermarkets say these figures are nonsense (and they do seem to originate in some way from USDAW, the shopworkers’ union who aren’t going to be in favour of automation). My guess is that there's a befefit to be gained but both the technology and also the customer's familiarity have yet to reach the point at which that benefit is realised.

...