## Posts Tagged ‘statistics’

### How can we make statistics lovely? Interactive graphics?

Posted on: May 17, 2017

How can we make love statistics? Interactive graphs?

Think you’re good at guessing stats? Guess again. Whether we consider ourselves math people or not, our ability to understand and work with numbers is terribly limited, says data visualization expert Alan Smith. Smith explores the mismatch between what we know and what we think we know.

Alan Smith. Data visualisation editor
Alan Smith uses interactive graphics and statistics to breathe new life into how data is presented. Full bio
Filmed April 2016

Back in 2003, the UK government carried out a survey. And it was a survey that measured levels of numeracy in the population.

And they were shocked to find out that for every 100 working age adults in the country, 47 of them lacked Level 1 numeracy skills. Now, Level 1 numeracy skills — that’s low-end GCSE score. It’s the ability to deal with fractions, percentages and decimals.

this figure prompted a lot of hand-wringing in Whitehall. Policies were changed, investments were made, and then they ran the survey again in 2011. So can you guess what happened to this number? It went up to 49.

0:57 And in fact, when I reported this figure in the FT, one of our readers joked and said, This figure is only shocking to 51 percent of the population.”

But I preferred the reaction of a schoolchild when I presented at a school this information, who raised their hand and said, “How do we know that the person who made that number isn’t one of the 49 percent either?”

1:20 (Laughter)

So clearly, there’s a numeracy issue, because these are important skills for life, and a lot of the changes that we want to introduce in this century involve us becoming more comfortable with numbers. (Can’t learn numeracy without using a pen and pencil?)

it’s not just an English problem. OECD this year released some figures looking at numeracy in young people, and leading the way, the USA — nearly 40 percent of young people in the US have low numeracy. Now, England is there too, but there are seven OECD countries with figures above 20 percent. That is a problem, because it doesn’t have to be that way. If you look at the far end of this graph, you can see the Netherlands and Korea are in single figures. So there’s definitely a numeracy problem that we want to address. (It is the method used to learning numeracy)

as useful as studies like these are, I think we risk herding people inadvertently into one of two categories; that there are two kinds of people: those people that are comfortable with numbers, that can do numbers, and the people who can’t.

And what I’m trying to talk about here today is to say that I believe that is a false dichotomy. It’s not an immutable pairing. I think you don’t have to have tremendously high levels of numeracy to be inspired by numbers, and that should be the starting point to the journey ahead.

one of the ways in which we can begin that journey, for me, is looking at statistics. Now, I am the first to acknowledge that statistics has got somewhat of an image problem.

2:52 (Laughter)

It’s the part of mathematics that even mathematicians don’t particularly like, because whereas the rest of maths is all about precision and certainty, statistics is almost the reverse of that.

But actually, I was a late convert to the world of statistics myself. If you’d asked my undergraduate professors what two subjects would I be least likely to excel in after university, they’d have told you statistics and computer programming, and yet here I am, about to show you some statistical graphics that I programmed. (You think you comprehended probability and statistics, but you forget them if Not practiced)

what inspired that change in me? What made me think that statistics was actually an interesting thing? It’s really because statistics are about us.

If you look at the etymology of the word statistics, it’s the science of dealing with data about the state or the community that we live in. So statistics are about us as a group, not us as individuals. And I think as social animals, we share this fascination about how we, as individuals, relate to our groups, to our peers. And statistics in this way are at their most powerful when they surprise us.

there’s been some really wonderful surveys carried out recently by Ipsos MORI in the last few years. They did a survey of over 1,000 adults in the UK, and said, for every 100 people in England and Wales, how many of them are Muslim? Now the average answer from this survey, which was supposed to be representative of the total population, was 24. That’s what people thought. British people think 24 out of every 100 people in the country are Muslim. Now, official figures reveal that figure to be about five. So there’s this big variation between what we think, our perception, and the reality as given by statistics. And I think that’s interesting. What could possibly be causing that misperception?

I was so thrilled with this study, I started to take questions out in presentations. I was referring to it. Now, I did a presentation at St. Paul’s School for Girls in Hammersmith, and I had an audience rather like this, except it was comprised entirely of sixth-form girls.

And I said, “Girls, how many teenage girls do you think the British public think get pregnant every year?” And the girls were apoplectic when I said the British public think that 15 out of every 100 teenage girls get pregnant in the year. And they had every right to be angry, because in fact, I’d have to have closer to 200 dots before I could color one in, in terms of what the official figures tell us.

And rather like numeracy, this is not just an English problem. Ipsos MORI expanded the survey in recent years to go across the world. And so, they asked Saudi Arabians, for every 100 adults in your country, how many of them are overweight or obese? And the average answer from the Saudis was just over a quarter. That’s what they thought. Just over a quarter of adults are overweight or obese. The official figures show, actually, it’s nearer to three-quarters.

5:56 (Laughter)

5:57 So again, a big variation.

I love this one: they asked the Japanese, for every 100 Japanese people, how many of them live in rural areas? The average was about a 50-50 split, just over halfway. They thought 56 out of every 100 Japanese people lived in rural areas. The official figure is seven.

So extraordinary variations, and surprising to some, but not surprising to people who have read the work of Daniel Kahneman, for example, the Nobel-winning economist. He and his colleague, Amos Tversky, spent years researching this disjoint between what people perceive and the reality, the fact that people are actually pretty poor intuitive statisticians. (I read many of their research papers in the late 80’s)

And there are many reasons for this. Individual experiences, certainly, can influence our perceptions, but so, too, can things like the media reporting things by exception, rather than what’s normal. Kahneman had a nice way of referring to that. He said, “We can be blind to the obvious” — so we’ve got the numbers wrong — “but we can be blind to our blindness about it.” And that has enormous repercussions for decision making.

at the statistics office while this was all going on, I thought this was really interesting. I said, this is clearly a global problem, but maybe geography is the issue here.

These were questions that were all about, how well do you know your country? So in this case, it’s how well do you know 64 million people? Not very well, it turns out. I can’t do that. So I had an idea, which was to think about this same sort of approach but to think about it in a very local sense. Is this a local? If we reframe the questions and say, how well do you know your local area, would your answers be any more accurate?

I devised a quiz: How well do you know your area? It’s a simple Web app. You put in a post code and then it will ask you questions based on census data for your local area. And I was very conscious in designing this. I wanted to make it open to the widest possible range of people, not just the 49 percent who can get the numbers.

I wanted everyone to engage with it. So for the design of the quiz, I was inspired by the isotypes of Otto Neurath from the 1920s and ’30s. Now, these are methods for representing numbers using repeating icons. And the numbers are there, but they sit in the background. So it’s a great way of representing quantity without resorting to using terms like “percentage,” “fractions” and “ratios.”

So here’s the quiz. The layout of the quiz is, you have your repeating icons on the left-hand side there, and a map showing you the area we’re asking you questions about on the right-hand side. There are 7 questions. Each question, there’s a possible answer between zero and a hundred, and at the end of the quiz, you get an overall score between zero and a hundred.

And so because this is TEDxExeter, I thought we would have a quick look at the quiz for the first few questions of Exeter. And so the first question is: For every 100 people, how many are aged under 16? Now, I don’t know Exeter very well at all, so I had a guess at this, but it gives you an idea of how this quiz works. You drag the slider to highlight your icons, and then just click “Submit” to answer, and we animate away the difference between your answer and reality. And it turns out, I was a pretty terrible guess: five.

How about the next question? This is asking about what the average age is, so the age at which half the population are younger and half the population are older. (This is the definition of the median) And I thought 35 — that sounds middle-aged to me.

9:35 (Laughter)

9:39 Actually, in Exeter, it’s incredibly young, and I had underestimated the impact of the university in this area. The questions get harder as you go through. So this one’s now asking about homeownership: For every 100 households, how many are owned with a mortgage or loan? And I hedged my bets here, because I didn’t want to be more than 50 out on the answer.

these get harder, these questions, because when you’re in an area, when you’re in a community, things like age — there are clues to whether a population is old or young. Just by looking around the area, you can see it. Something like homeownership is much more difficult to see, so we revert to our own heuristics, our own biases about how many people we think own their own homes.

the truth is, when we published this quiz, the census data that it’s based on was already a few years old. We’ve had online applications that allow you to put in a post code and get statistics back for years. So in some senses, this was all a little bit old and not necessarily new. But I was interested to see what reaction we might get by gamifying the data in the way that we have, by using animation and playing on the fact that people have their own preconceptions.

It turns out, the reaction was more than I could have hoped for. It was a long-held ambition of mine to bring down a statistics website due to public demand.

11:06 (Laughter)

This URL contains the words “statistics,” “gov” and “UK,” which are three of people’s least favorite words in a URL. And the amazing thing about this was that the website came down at quarter to 10 at night, because people were actually engaging with this data of their own free will, using their own personal time.

I was very interested to see that we got something like a quarter of a million people playing the quiz within the space of 48 hours of launching it. And it sparked an enormous discussion online, on social media, which was largely dominated by people having fun with their misconceptions, which is something that I couldn’t have hoped for any better, in some respects. I also liked the fact that people started sending it to politicians. How well do you know the area you claim to represent? (All candidates to public office must go through such quizzes in their locality and the nation)

then just to finish, going back to the two kinds of people, I thought it would be really interesting to see how people who are good with numbers would do on this quiz. The national statistician of England and Wales, John Pullinger, you would expect he would be pretty good. He got 44 for his own area.

12:16 (Laughter)

Jeremy Paxman — admittedly, after a glass of wine — 36. Even worse. It just shows you that the numbers can inspire us all. They can surprise us all.

12:31 So very often, we talk about statistics as being the science of uncertainty. My parting thought for today is: actually, statistics is the science of us. And that’s why we should be fascinated by numbers.

##### Patsy Z shared this link · 7 hrs

“Whether we consider ourselves math people or not, our ability to understand and work with numbers is terribly limited.”

Alan Smith explores the mismatch between what we know and what we th…
ted.com

### Damned lies and statistics?

Posted on: April 14, 2017

Damned lies and statistics?

Sebastian Wernicke turns the tools of statistical analysis on TEDTalks, to come up with a metric for creating “the optimum TEDTalk” based on user ratings. How do you rate it? “Jaw-dropping”? “Unconvincing”? Or just plain “Funny”?

Sebastian Wernicke. Data scientist

After making a splash in the field of bioinformatics, Sebastian Wernicke moved on to the corporate sphere, where he motivates and manages multidimensional projects. Full bio

Filmed in Feb 2010

If you go on the TED website, you can currently find there over a full week of TEDTalk videos, over 1.3 million words of transcripts and millions of user ratings.

And that’s a huge amount of data. And it got me wondering: If you took all this data and put it through statistical analysis, could you reverse engineer a TEDTalk?

Could you create the ultimate TEDTalk? (Laughter) (Applause) And also, could you create the worst possible TEDTalk that they would still let you get away with?

0:49 To find this out, I looked at three things:

I looked at the topic that you should choose,

I looked at how you should deliver it and the visuals onstage.

Now, with the topic: There’s a whole range of topics you can choose, but you should choose wisely, because your topic strongly correlates with how users will react to your talk.

to make this more concrete, let’s look at the list of top 10 words that statistically stick out in the most favorite TEDTalks and in the least favorite TEDTalks. So if you came here to talk about how French coffee will spread happiness in our brains, that’s a go. (Laughter) (Applause) Whereas, if you wanted to talk about your project involving oxygen, girls, aircraft — actually, I would like to hear that talk, (Laughter) but statistics say it’s not so good.

If you generalize this, the most favorite TEDTalks are those that feature topics we can connect with, both easily and deeply, such as happiness, our own body, food, emotions. And the more technical topics, such as architecture, materials and, strangely enough, men, those are not good topics to talk about.

How should you deliver your talk?

TED is famous for keeping a very sharp eye on the clock, so they’re going to hate me for revealing this, because, actually, you should talk as long as they will let you. (Laughter) Because the most favorite TEDTalks are, on average, over 50% longer than the least favorite ones.

And this holds true for all ranking lists on TED.com except if you want to have a talk that’s beautiful, inspiring or funny. Then, you should be brief. (Laughter) But other than that, talk until they drag you off the stage.

While you’re pushing the clock, there’s a few rules to obey. I found these rules out by comparing the statistics of four-word phrases that appear more often in the most favorite TEDTalks as opposed to the least favorite TEDTalks.

I’ll give you 3 examples. First of all, I must, as a speaker, provide a service to the audience and talk about what I will give you, instead of saying what I can’t have.

Secondly, it’s imperative that you do not cite The New York Times. (Laughter) And

finally, it’s okay for the speaker — that’s the good news — to fake intellectual capacity. If I don’t understand something, I can just say, “etc., etc.” You’ll all stay with me. It’s perfectly fine. (Applause)

let’s go to the visuals. The most obvious visual thing on stage is the speaker.

And analysis shows if you want to be among the most favorite TED speakers, you should let your hair grow a little bit longer than average, make sure you wear your glasses and be slightly more dressed-up than the average TED speaker.

Slides are okay, though you might consider going for props. And now the most important thing, that is the mood onstage. Color plays a very important role.

Color closely correlates with the ratings that talks get on the website. (Applause) For example, fascinating talks contain a statistically high amount of exactly this blue color, (Laughter) much more than the average TEDTalk.

Ingenious TEDTalks, much more this green color, etc., et. (Laughter) (Applause) Now, personally, I think I’m not the first one who has done this analysis, but I’ll leave this to your good judgment.

it’s time to put it all together and design the ultimate TEDTalk.

Now, since this is TEDActive, and I learned from my analysis that I should actually give you something, I will not impose the ultimate or worst TEDTalk on you, but rather give you a tool to create your own. And I call this tool the TEDPad. (Laughter)

And the TEDPad is a matrix of 100 specifically selected, highly curated sentences that you can easily piece together to get your own TEDTalk.

You only have to make one decision, and that is: Are you going to use the white version for very good TEDTalks, about creativity, human genius? Or are you going to go with a black version, which will allow you to create really bad TEDTalks, mostly about blogs, politics and stuff? So, download it and have fun with it.

I hope you enjoy the session. I hope you enjoy designing your own ultimate and worst possible TEDTalks. And I hope some of you will be inspired for next year to create this, which I really want to see.

##### Patsy Z and TEDxSKE shared a link.

Posted on: May 6, 2012

Toby Ng Design turned into graphics population spread in “The Word of 100”

If the world were a village of 100 people, how would the composition be? This set of 20 posters is built on statistics about the spread of population around the world under various classifications. The numbers are turned into graphics to give another sense a touch.  Look, this is the world we are living in.

I urge you to forward the statistics that you have strong doubt about. Occasionally, a little good common sense of people who have good background knowledge can question statistics that might be biased.
Note: red dot award: communication design 2009. GDC(Graphic Design in China) Awards 2009,
International Design Awards 2009
HOW International Design Awards 2010
HKDA Global Design Awards 2011

### Blog Stats

• 1,516,375 hits