How to mine 5 million books already scanned by Google
Mind you that this speech was delivered in 2011. At the time, Google was targeting to scan 15 million books, Not counting magazines and other artistic productions.
Erez Lieberman Aiden: Everyone knows that a picture is worth a thousand words. But we at Harvard were wondering if this was really true. (Laughter) So we assembled a team of experts, spanning Harvard, MIT, The American Heritage Dictionary, The Encyclopedia Britannica and even our proud sponsors, the Google. And we cogitated about this for about four years. And we came to a startling conclusion. Ladies and gentlemen, a picture is not worth a thousand words. In fact, we found some pictures that are worth 500 billion words.
Jean-Baptiste Michel: So how did we get to this conclusion? So Erez and I were thinking about ways to get a big picture of human culture and human history: change over time. So many books actually have been written over the years. So we were thinking, well the best way to learn from them is to read all of these millions of books. Now of course, if there’s a scale for how awesome that is, that has to rank extremely, extremely high. Now the problem is there’s an X-axis for that, which is the practical axis. This is very, very low.
01:32 Now people tend to use an alternative approach, which is to take a few sources and read them very carefully. This is extremely practical, but not so awesome. What you really want to do is to get to the awesome yet practical part of this space. So it turns out there was a company across the river called Google who had started a digitization project a few years back that might just enable this approach. They have digitized millions of books. So what that means is, one could use computational methods to read all of the books in a click of a button. That’s very practical and extremely awesome.
02:03 ELA: Let me tell you a little bit about where books come from. Since time immemorial, there have been authors. These authors have been striving to write books. And this became considerably easier with the development of the printing press some centuries ago. Since then, the authors have won on 129 million distinct occasions, publishing books. Now if those books are not lost to history, then they are somewhere in a library, and many of those books have been getting retrieved from the libraries and digitized by Google, which has scanned 15 million books to date.
02:33 Now when Google digitizes a book, they put it into a really nice format. Now we’ve got the data, plus we have metadata. We have information about things like where was it published, who was the author, when was it published. And what we do is go through all of those records and exclude everything that’s not the highest quality data. What we’re left with is a collection of five million books, 500 billion words, a string of characters a thousand times longer than the human genome — a text which, when written out, would stretch from here to the Moon and back 10 times over — a veritable shard of our cultural genome. Of course what we did when faced with such outrageous hyperbole … (Laughter) was what any self-respecting researchers would have done. We took a page out of XKCD, and we said, “Stand back. We’re going to try science.”
03:34 JM: Now of course, we were thinking, well let’s just first put the data out there for people to do science to it. Now we’re thinking, what data can we release? Well of course, you want to take the books and release the full text of these five million books.
Now Google, and Jon Orwant in particular, told us a little equation that we should learn. So you have five million, that is, five million authors and five million plaintiffs is a massive lawsuit. So, although that would be really, really awesome, again, that’s extremely, extremely impractical. (Laughter)
04:03 Now again, we kind of caved in, and we did the very practical approach, which was a bit less awesome. We said, well instead of releasing the full text, we’re going to release statistics about the books. So take for instance “A gleam of happiness.” It’s four words; we call that a four-gram. We’re going to tell you how many times a particular four-gram appeared in books in 1801, 1802, 1803, all the way up to 2008. That gives us a time series of how frequently this particular sentence was used over time. We do that for all the words and phrases that appear in those books, and that gives us a big table of two billion lines that tell us about the way culture has been changing.
04:34 ELA: So those two billion lines, we call them two billion n-grams. What do they tell us? Well the individual n-grams measure cultural trends. Let me give you an example. Let’s suppose that I am thriving, then tomorrow I want to tell you about how well I did. And so I might say, “Yesterday, I throve.” Alternatively, I could say, “Yesterday, I thrived.” Well which one should I use? How to know?
04:59 As of about six months ago, the state of the art in this field is that you would, for instance, go up to the following psychologist with fabulous hair, and you’d say, “Steve, you’re an expert on the irregular verbs. What should I do?” And he’d tell you, “Well most people say thrived, but some people say throve.” And you also knew, more or less, that if you were to go back in time 200 years and ask the following statesman with equally fabulous hair, (Laughter) “Tom, what should I say?” He’d say, “Well, in my day, most people throve, but some thrived.” So now what I’m just going to show you is raw data. Two rows from this table of two billion entries. What you’re seeing is year by year frequency of “thrived” and “throve” over time. Now this is just two out of two billion rows. So the entire data set is a billion times more awesome than this slide.
06:05 JM: Now there are many other pictures that are worth 500 billion words. For instance, this one. If you just take influenza, you will see peaks at the time where you knew big flu epidemics were killing people around the globe.
06:16 ELA: If you were not yet convinced, sea levels are rising, so is atmospheric CO2 and global temperature.
06:24 JM: You might also want to have a look at this particular n-gram, and that’s to tell Nietzsche that God is not dead, although you might agree that he might need a better publicist.
06:35 ELA: You can get at some pretty abstract concepts with this sort of thing. For instance, let me tell you the history of the year 1950. Pretty much for the vast majority of history, no one gave a damn about 1950. In 1700, in 1800, in 1900, no one cared. Through the 30s and 40s, no one cared. Suddenly, in the mid-40s, there started to be a buzz.
People realized that 1950 was going to happen, and it could be big. (Laughter) But nothing got people interested in 1950 like the year 1950. (Laughter) People were walking around obsessed. They couldn’t stop talking about all the things they did in 1950, all the things they were planning to do in 1950, all the dreams of what they wanted to accomplish in 1950.
In fact, 1950 was so fascinating that for years thereafter, people just kept talking about all the amazing things that happened, in ’51, ’52, ’53. Finally in 1954, someone woke up and realized that 1950 had gotten somewhat passé. (Laughter) And just like that, the bubble burst.
And the story of 1950 is the story of every year that we have on record, with a little twist, because now we’ve got these nice charts. And because we have these nice charts, we can measure things. We can say, “Well how fast does the bubble burst?” And it turns out that we can measure that very precisely. Equations were derived, graphs were produced, and the net result is that we find that the bubble bursts faster and faster with each passing year. We are losing interest in the past more rapidly.
08:24 JM: Now a little piece of career advice. So for those of you who seek to be famous, we can learn from the 25 most famous political figures, authors, actors and so on. So if you want to become famous early on, you should be an actor, because then fame starts rising by the end of your 20s — you’re still young, it’s really great. Now if you can wait a little bit, you should be an author, because then you rise to very great heights, like Mark Twain, for instance: extremely famous.
But if you want to reach the very top, you should delay gratification and, of course, become a politician. So here you will become famous by the end of your 50s, and become very, very famous afterward. So scientists also tend to get famous when they’re much older. Like for instance, biologists and physics tend to be almost as famous as actors. One mistake you should not do is become a mathematician. (Laughter) If you do that, you might think, “Oh great. I’m going to do my best work when I’m in my 20s.” But guess what, nobody will really care.
09:17 ELA: There are more sobering notes among the n-grams. For instance, here’s the trajectory of Marc Chagall, an artist born in 1887. And this looks like the normal trajectory of a famous person. He gets more and more and more famous, except if you look in German.
If you look in German, you see something completely bizarre, something you pretty much never see, which is he becomes extremely famous and then all of a sudden plummets, going through a nadir between 1933 and 1945, before rebounding afterward. And of course, what we’re seeing is the fact Marc Chagall was a Jewish artist in Nazi Germany.
09:55 Now these signals are actually so strong that we don’t need to know that someone was censored. We can actually figure it out using really basic signal processing. Here’s a simple way to do it. Well, a reasonable expectation is that somebody’s fame in a given period of time should be roughly the average of their fame before and their fame after. So that’s sort of what we expect. And we compare that to the fame that we observe. And we just divide one by the other to produce something we call a suppression index. If the suppression index is very, very, very small, then you very well might be being suppressed. If it’s very large, maybe you’re benefiting from propaganda.
10:34 JM: Now you can actually look at the distribution of suppression indexes over whole populations. So for instance, here — this suppression index is for 5,000 people picked in English books where there’s no known suppression — it would be like this, basically tightly centered on one. What you expect is basically what you observe. This is distribution as seen in Germany — very different, it’s shifted to the left. People talked about it twice less as it should have been. But much more importantly, the distribution is much wider. There are many people who end up on the far left on this distribution who are talked about 10 times fewer than they should have been. But then also many people on the far right who seem to benefit from propaganda. This picture is the hallmark of censorship in the book record.
11:11 ELA: So culturomics is what we call this method. It’s kind of like genomics. Except genomics is a lens on biology through the window of the sequence of bases in the human genome. Culturomics is similar. It’s the application of massive-scale data collection analysis to the study of human culture. Here, instead of through the lens of a genome, through the lens of digitized pieces of the historical record. The great thing about culturomics is that everyone can do it. Why can everyone do it?
Everyone can do it because three guys, Jon Orwant, Matt Gray and Will Brockman over at Google, saw the prototype of the Ngram Viewer, and they said, “This is so fun. We have to make this available for people.” So in two weeks flat — the two weeks before our paper came out — they coded up a version of the Ngram Viewer for the general public. And so you too can type in any word or phrase that you’re interested in and see its n-gram immediately — also browse examples of all the various books in which your n-gram appears.
12:06 JM: Now this was used over a million times on the first day, and this is really the best of all the queries. So people want to be their best, put their best foot forward. But it turns out in the 18th century, people didn’t really care about that at all. They didn’t want to be their best, they wanted to be their beft. So what happened is, of course, this is just a mistake. It’s not that strove for mediocrity, it’s just that the S used to be written differently, kind of like an F. Now of course, Google didn’t pick this up at the time, so we reported this in the science article that we wrote. But it turns out this is just a reminder that, although this is a lot of fun, when you interpret these graphs, you have to be very careful, and you have to adopt the base standards in the sciences.
12:42 ELA: People have been using this for all kinds of fun purposes. (Laughter) Actually, we’re not going to have to talk, we’re just going to show you all the slides and remain silent. This person was interested in the history of frustration. There’s various types of frustration. If you stub your toe, that’s a one A “argh.” If the planet Earth is annihilated by the Vogons to make room for an interstellar bypass, that’s an eight A “aaaaaaaargh.” This person studies all the “arghs,” from one through eight A’s. And it turns out that the less-frequent “arghs” are, of course, the ones that correspond to things that are more frustrating — except, oddly, in the early 80s. We think that might have something to do with Reagan.
13:30 JM: There are many usages of this data, but the bottom line is that the historical record is being digitized. Google has started to digitize 15 million books. That’s 12 percent of all the books that have ever been published. It’s a sizable chunk of human culture. There’s much more in culture: there’s manuscripts, there newspapers, there’s things that are not text, like art and paintings. These all happen to be on our computers, on computers across the world. And when that happens, that will transform the way we have to understand our past, our present and human culture.
Let me tell you a story about a little girl named Naghma.
Naghma lived in a refugee camp with her parents and her eight brothers and sisters. Every morning, her father would wake up in the hopes he’d be picked for construction work, and on a good month he would earn 50 dollars.
The winter was very harsh, and unfortunately, Naghma’s brother died and her mother became very ill. In desperation, her father went to a neighbor to borrow 2,500 dollars.
After several months of waiting, the neighbor became very impatient, and he demanded that he be paid back. Unfortunately, Naghma’s father didn’t have the money, and so the two men agreed to a jirga.
So simply put, a jirga is a form of mediation that’s used in Afghanistan’s informal justice system. It’s usually presided over by religious leaders and village elders, and jirgas are often used in rural countries like Afghanistan, where there’s deep-seated resentment against the formal system. At the jirga, the men sat together and they decided that the best way to satisfy the debt would be if Naghma married the neighbor’s 21-year-old son. She was six.
01:23 Now, stories like Naghma’s unfortunately are all too common, and from the comforts of our home, we may look at these stories as another crushing blow to women’s rights. And if you watched Afghanistan on the news, you may have this view that it’s a failed state.
However, Afghanistan does have a legal system, and while jirgas are built on long-standing tribal customs, even in jirgas, laws are supposed to be followed, and it goes without saying that giving a child to satisfy a debt is not only grossly immoral, it’s illegal.
In 2008, I went to Afghanistan for a justice funded program, and I went there originally on this 9-month program to train Afghan lawyers.
In that nine months, I went around the country and I talked to hundreds of people that were locked up, and I talked to many businesses that were also operating in Afghanistan. And within these conversations, I started hearing the connections between the businesses and the people, and how laws that were meant to protect them were being underused, while gross and illegal punitive measures were overused.
And so this put me on a quest for justness, and what justness means to me is using laws for their intended purpose, which is to protect.
The role of laws is to protect. So as a result, I decided to open up a private practice, and I became the first foreigner to litigate in Afghan courts. Throughout this time, I also studied many laws, I talked to many people, I read up on many cases, and I found that the lack of justness is not just a problem in Afghanistan, but it’s a global problem.
And while I originally shied away from representing human rights cases because I was really concerned about how it would affect me both professionally and personally, I decided that the need for justness was so great that I couldn’t continue to ignore it. And so I started representing people like Naghma pro bono also.
Since I’ve been in Afghanistan and since I’ve been an attorney for over 10 years, I’ve represented from CEOs of Fortune 500 companies to ambassadors to little girls like Naghma, and with much success. And the reason for my success is very simple: I work the system from the inside out and use the laws in the ways that they’re intended to be used.
I find that achieving justness in places like Afghanistan is difficult, and there’s three reasons.
The first reason is that simply put, people are very uneducated as to what their legal rights were, and I find that this is a global problem.
The second issue is that even with laws on the books, it’s often superseded or ignored by tribal customs, like in the first jirga that sold Naghma off.
And the third problem with achieving justness is that even with good, existing laws on the books, there aren’t people or lawyers that are willing to fight for those laws.
And that’s what I do: I use existing laws, often unused laws, and I work those to the benefits of my clients. We all need to create a global culture of human rights and be investors in a global human rights economy, and by working in this mindset, we can significantly improve justice globally.
Now let’s get back to Naghma. Several people heard about this story, and so they contacted me because they wanted to pay the $2,500 debt. And it’s not just that simple; you can’t just throw money at this problem and think that it’s going to disappear. That’s not how it works in Afghanistan.
So I told them I’d get involved, but in order to get involved, what needed to happen is a second jirga needed to be called, a jirga of appeals. And so in order for that to happen, we needed to get the village elders together, we needed to get the tribal leaders together, the religious leaders.
Naghma’s father needed to agree, the neighbor needed to agree, and also his son needed to agree. And I thought, if I’m going to get involved in this thing, then they also need to agree that I preside over it.
05:29 So, after hours of talking and tracking them down, and about 30 cups of tea, they finally agreed that we could sit down for a second jirga, and we did. And what was different about the second jirga is: this time, we put the law at the center of it, and it was very important for me that they all understood that Naghma had a right to be protected.
And at the end of this jirga, it was ordered by the judge that the first decision was erased, and that the $2,500 debt was satisfied, and we all signed a written order where all the men acknowledged that what they did was illegal, and if they did it again, that they would go to prison.
And most importantly, the engagement was terminated and Naghma was free. Protecting Naghma and her right to be free protects us.
06:29 Now, with my job, there’s above-average amount of risks that are involved. I’ve been temporarily detained. I’ve been accused of running a brothel, accused of being a spy. I’ve had a grenade thrown at my office. It didn’t go off, though.
But I find that with my job, that the rewards far outweigh the risks, and as many risks as I take, my clients take far greater risks, because they have a lot more to lose if their cases go unheard, or worse, if they’re penalized for having me as their lawyer. With every case that I take, I realize that as much as I’m standing behind my clients, that they’re also standing behind me, and that’s what keeps me going.
07:13 Law as a point of leverage is crucial in protecting all of us. Journalists are very vital in making sure that that information is given to the public. Too often, we receive information from journalists but we forget how that information was given. This picture is a picture of the British press corps in Afghanistan. It was taken a couple of years ago by my friend David Gill.
According to the Committee to Protect Journalists, since 2010, there have been thousands of journalists who have been threatened, injured, killed, detained.
Too often, when we get this information, we forget who it affects or how that information is given to us. What many journalists do, both foreign and domestic, is very remarkable, especially in places like Afghanistan, and it’s important that we never forget that, because what they’re protecting is not only our right to receive that information but also the freedom of the press, which is vital to a democratic society.
08:10 Matt Rosenberg is a journalist in Afghanistan. He works for The New York Times, and unfortunately, a few months ago he wrote an article that displeased people in the government. As a result, he was temporarily detained and he was illegally exiled out of the country.
I represent Matt, and after dealing with the government, I was able to get legal acknowledgment that in fact he was illegally exiled, and that freedom of the press does exist in Afghanistan, and there’s consequences if that’s not followed. And I’m happy to say that as of a few days ago, the Afghan government formally invited him back into the country and they reversed their exile order of him.
08:59 If you censor one journalist, then it intimidates others, and soon nations are silenced. It’s important that we protect our journalists and freedom of the press, because that makes governments more accountable to us and more transparent. Protecting journalists and our right to receive information protects us.
Our world is changing.
We live in a different world now, and what were once individual problems are really now global problems for all of us. Two weeks ago, Afghanistan had its first democratic transfer of power and elected president Ashraf Ghani, which is huge, and I’m very optimistic about him, and I’m hopeful that he’ll give Afghanistan the changes that it needs, especially within the legal sector.
We live in a different world. We live in a world where my eight-year-old daughter only knows a black president. There’s a great possibility that our next president will be a woman, and as she gets older, she may question, can a white guy be president? (Laughter) (Applause)
Our world is changing, and we need to change with it, and what were once individual problems are problems for all of us. According to UNICEF, there are currently over 280 million boys and girls who are married under the age of 15.
Two hundred and eighty million. Child marriages prolong the vicious cycle of poverty, poor health, lack of education.
10:33 At the age of 12, Sahar was married. She was forced into this marriage and sold by her brother. When she went to her in-laws’ house, they forced her into prostitution. Because she refused, she was tortured. She was severely beaten with metal rods. They burned her body. They tied her up in a basement and starved her. They used pliers to take out her fingernails. At one point, she managed to escape from this torture chamber to a neighbor’s house, and when she went there, instead of protecting her, they dragged her back to her husband’s house, and she was tortured even worse.
11:24 When I met first Sahar, thankfully, Women for Afghan Women gave her a safe haven to go to. As a lawyer, I try to be very strong for all my clients, because that’s very important to me, but seeing her, how broken and very weak as she was, was very difficult. It took weeks for us to really get to what happened to her when she was in that house, but finally she started opening up to me, and when she opened up, what I heard was she didn’t know what her rights were, but she did know she had a certain level of protection by her government that failed her, and so we were able to talk about what her legal options were.
12:18 And so we decided to take this case to the Supreme Court. Now, this is extremely significant, because this is the first time that a victim of domestic violence in Afghanistan was being represented by a lawyer, a law that’s been on the books for years and years, but until Sahar, had never been used. In addition to this, we also decided to sue for civil damages, again using a law that’s never been used, but we used it for her case. So there we were at the Supreme Court arguing in front of 12 Afghan justices, me as an American female lawyer, and Sahar, a young woman who when I met her couldn’t speak above a whisper. She stood up, she found her voice, and my girl told them that she wanted justice, and she got it. At the end of it all, the court unanimously agreed that her in-laws should be arrested for what they did to her, her fucking brother should also be arrested for selling her — (Applause) — and they agreed that she did have a right to civil compensation. What Sahar has shown us is that we can attack existing bad practices by using the laws in the ways that they’re intended to be used, and by protecting Sahar, we are protecting ourselves.
13:48 After having worked in Afghanistan for over six years now, a lot of my family and friends think that what I do looks like this. (Laughter) But in all actuality, what I do looks like this. Now, we can all do something. I’m not saying we should all buy a plane ticket and go to Afghanistan, but we can all be contributors to a global human rights economy. We can create a culture of transparency and accountability to the laws, and make governments more accountable to us, as we are to them.
14:22 A few months ago, a South African lawyer visited me in my office and he said, “I wanted to meet you. I wanted to see what a crazy person looked like.” The laws are ours, and no matter what your ethnicity, nationality, gender, race, they belong to us, and fighting for justice is not an act of insanity. Businesses also need to get with the program. A corporate investment in human rights is a capital gain on your businesses, and whether you’re a business, an NGO, or a private citizen, rule of law benefits all of us. And by working together with a concerted mindset, through the people, public and private sector, we can create a global human rights economy and all become global investors in human rights. And by doing this, we can achieve justness together.
In Oxford in the 1950s, there was a fantastic doctor, who was very unusual, named Alice Stewart. And Alice was unusual partly because, of course, she was a woman, which was pretty rare in the 1950s.
And she was brilliant, she was one of the, at the time, the youngest Fellow to be elected to the Royal College of Physicians. She was unusual too because she continued to work after she got married, after she had kids, and even after she got divorced and was a single parent, she continued her medical work.
00:43 And she was unusual because she was really interested in a new science, the emerging field of epidemiology, the study of patterns in disease. But like every scientist, she appreciated that to make her mark, what she needed to do was find a hard problem and solve it.
The hard problem that Alice chose was the rising incidence of childhood cancers. Most disease is correlated with poverty, but in the case of childhood cancers, the children who were dying seemed mostly to come from affluent families. So, what, she wanted to know, could explain this anomaly?
01:23 Now, Alice had trouble getting funding for her research. In the end, she got just 1,000 pounds from the Lady Tata Memorial prize. And that meant she knew she only had one shot at collecting her data. Now, she had no idea what to look for.
This really was a needle in a haystack sort of search, so she asked everything she could think of. Had the children eaten boiled sweets? Had they consumed colored drinks? Did they eat fish and chips? Did they have indoor or outdoor plumbing? What time of life had they started school?
01:54 And when her carbon copied questionnaire started to come back, one thing and one thing only jumped out with the statistical clarity of a kind that most scientists can only dream of. By a rate of two to one, the children who had died had had mothers who had been X-rayed when pregnant. Now that finding flew in the face of conventional wisdom.
Conventional wisdom held that everything was safe up to a point, a threshold. It flew in the face of conventional wisdom, which was huge enthusiasm for the cool new technology of that age, which was the X-ray machine. And it flew in the face of doctors’ idea of themselves, which was as people who helped patients, they didn’t harm them.
02:46 Nevertheless, Alice Stewart rushed to publish her preliminary findings in The Lancet in 1956. People got very excited, there was talk of the Nobel Prize, and Alice really was in a big hurry to try to study all the cases of childhood cancer she could find before they disappeared. In fact, she need not have hurried.
It was fully 25 years before the British and medical — British and American medical establishments abandoned the practice of X-raying pregnant women. The data was out there, it was open, it was freely available, but nobody wanted to know. A child a week was dying, but nothing changed. Openness alone can’t drive change.
So for 25 years Alice Stewart had a very big fight on her hands.
How did she know that she was right? Well, she had a fantastic model for thinking.
She worked with a statistician named George Kneale, and George was pretty much everything that Alice wasn’t. So, Alice was very outgoing and sociable, and George was a recluse. Alice was very warm, very empathetic with her patients. George frankly preferred numbers to people. But he said this fantastic thing about their working relationship. He said, “My job is to prove Dr. Stewart wrong.”
He actively sought disconfirmation. Different ways of looking at her models, at her statistics, different ways of crunching the data in order to disprove her. He saw his job as creating conflict around her theories. Because it was only by not being able to prove that she was wrong, that George could give Alice the confidence she needed to know that she was right.
04:55 It’s a fantastic model of collaboration — thinking partners who aren’t echo chambers. I wonder how many of us have, or dare to have, such collaborators. Alice and George were very good at conflict. They saw it as thinking.
05:21 So what does that kind of constructive conflict require? Well, first of all, it requires that we find people who are very different from ourselves. That means we have to resist the neurobiological drive, which means that we really prefer people mostly like ourselves, and it means we have to seek out people with different backgrounds, different disciplines, different ways of thinking and different experience, and find ways to engage with them. That requires a lot of patience and a lot of energy.
05:57 And the more I’ve thought about this, the more I think, really, that that’s a kind of love. Because you simply won’t commit that kind of energy and time if you don’t really care.
And it also means that we have to be prepared to change our minds. Alice’s daughter told me that every time Alice went head-to-head with a fellow scientist, they made her think and think and think again. “My mother,” she said, “My mother didn’t enjoy a fight, but she was really good at them.”
06:35 So it’s one thing to do that in a one-to-one relationship. But it strikes me that the biggest problems we face, many of the biggest disasters that we’ve experienced, mostly haven’t come from individuals, they’ve come from organizations, some of them bigger than countries, many of them capable of affecting hundreds, thousands, even millions of lives.
So how do organizations think?
Well, for the most part, they don’t. And that isn’t because they don’t want to, it’s really because they can’t.
And they can’t because the people inside of them are too afraid of conflict.
In surveys of European and American executives, fully 85 percent of them acknowledged that they had issues or concerns at work that they were afraid to raise. Afraid of the conflict that that would provoke, afraid to get embroiled in arguments that they did not know how to manage, and felt that they were bound to lose.
Eighty-five percent is a really big number. It means that organizations mostly can’t do what George and Alice so triumphantly did. They can’t think together. And it means that people like many of us, who have run organizations, and gone out of our way to try to find the very best people we can, mostly fail to get the best out of them.
08:14 So how do we develop the skills that we need? Because it does take skill and practice, too.
If we aren’t going to be afraid of conflict, we have to see it as thinking, and then we have to get really good at it. So, recently, I worked with an executive named Joe, and Joe worked for a medical device company.
And Joe was very worried about the device that he was working on. He thought that it was too complicated and he thought that its complexity created margins of error that could really hurt people. He was afraid of doing damage to the patients he was trying to help.
But when he looked around his organization, nobody else seemed to be at all worried. So, he didn’t really want to say anything. After all, maybe they knew something he didn’t. Maybe he’d look stupid. But he kept worrying about it, and he worried about it so much that he got to the point where he thought the only thing he could do was leave a job he loved.
09:21 In the end, Joe and I found a way for him to raise his concerns.
And what happened then is what almost always happens in this situation. It turned out everybody had exactly the same questions and doubts. So now Joe had allies. They could think together. And yes, there was a lot of conflict and debate and argument, but that allowed everyone around the table to be creative, to solve the problem, and to change the device.
Joe was what a lot of people might think of as a whistle-blower, except that like almost all whistle-blowers, he wasn’t a crank at all, he was passionately devoted to the organization and the higher purposes that that organization served. But he had been so afraid of conflict, until finally he became more afraid of the silence.
And when he dared to speak, he discovered much more inside himself and much more give in the system than he had ever imagined. And his colleagues don’t think of him as a crank. They think of him as a leader.
10:42 So, how do we have these conversations more easily and more often? Well, the University of Delft requires that its PhD students have to submit five statements that they’re prepared to defend. It doesn’t really matter what the statements are about, what matters is that the candidates are willing and able to stand up to authority. I think it’s a fantastic system, but I think leaving it to PhD candidates is far too few people, and way too late in life. I think we need to be teaching these skills to kids and adults at every stage of their development, if we want to have thinking organizations and a thinking society.
11:29 The fact is that most of the biggest catastrophes that we’ve witnessed rarely come from information that is secret or hidden. It comes from information that is freely available and out there, but that we are willfully blind to, because we can’t handle, don’t want to handle, the conflict that it provokes. But when we dare to break that silence, or when we dare to see, and we create conflict, we enable ourselves and the people around us to do our very best thinking.
12:10 Open information is fantastic, open networks are essential. But the truth won’t set us free until we develop the skills and the habit and the talent and the moral courage to use it. Openness isn’t the end. It’s the beginning.
Lebanon holds breath for deal to export trash abroad
Florence Massena. Posted February 1, 2016
It has been six months since Lebanon’s garbage crisis began in July, after the closure of the Naameh landfill and the end of the Lebanese state’s contract with Sukleen, the company in charge of transporting the garbage. Still no garbage treatment and storage solution has been found by the Chehayeb Commission, led by Agriculture Minister Akram Chehayeb since September to address the garbage piling up in the streets.
Instead, the Cabinet announced on Dec. 21 that Lebanon would export the garbage as a temporary solution. But since then, waste still lies in dumps and public fields everywhere in Lebanon, rotting on the ground and being swept about by the winter rains.
Although the Council for Development and Reconstruction announced Jan. 11 that Lebanon would export its waste over the next 18 months, nothing certain can be said right now about the details of the deal.
Nabil Abou Ghanem, adviser to the Agriculture Minister, told Al-Monitor, “The export of garbage is an alternative.” He added, “We found the solution of sending it away by boat because the municipalities and civil society refused the opening of new public landfills.” This measure doesn’t please everyone, and concerns remain regarding the nature of the deal, the cost of the entire process and the condition of the garbage that would be exported.
“The whole deal is quite blurry right now,” Ziad Abi Chaker, engineer and CEO of Cedar Environmental, told Al-Monitor. “I know for sure that maritime transport costs $125 per ton. And that is not the only cost; you have to add collection, transportation, sorting and wrapping of waste to a local facility, the transportation from this facility to the port, the loading on the ship, port fees and maritime transport fees. Also think of the unloading at the port of arrival and local port fees, as well as the transport and the gate fees of the local facility that would accept them,” Abi Chaker added.
According to his calculation, the whole process would cost between $250 and $300 per ton, whereas the Agriculture Ministry’s office communicated to Al-Monitor an estimate of $193 per ton. Abou Ghanem said, “Port fees are not going to be applied to waste; only transportation fees from the trucks to the ships, and it is the company exporting the garbage that will take care of it.”
He added, “The rest of the cost will be supported by the municipalities at the same cost they were paying for Sukleen’s services. In case of a general problem to finance this process, the government will find an alternative.”
Another issue remains at the heart of the waste being exported: will it be the new trash, still easy to sort, or the old piles still lingering from last summer and already decomposing? “The whole idea was to get rid of the old, to sort the new locally,” Abi Chaker pointed out. He added, “If the old [garbage] stays, it is an expensive measure that will be useless.
Plus, it depends of the destination. If a country takes the old, it means it is a place where the Basel Convention on the Control of Transboundary Movements of Hazardous Wastes and Their Disposal is not applied.” It means that Lebanon might export toxic and dangerous garbage that could bring disease to another country.
Although the Agriculture Ministry declared that it was acting “according to the Basel Convention,” nothing is certain concerning the waste that would be exported, especially its destination. As-Safir reported Jan. 9 that it would be exported to Sierra Leone, but the African country denied this information soon after, creating a diplomatic incident with Lebanon, according to Abou Ghanem.
“We still don’t know where the garbage is going,” he said, adding that “45-day-old and less waste will be sorted out locally. The French company Coral will do it in Beirut, but we still don’t know what to do about the older waste, including the waste that has been burnt. A French company and the British company Chinook will try to solve this issue.”
Chinook is the final company chosen by the Lebanese government after six international companies bid on the project, including the Dutch company Howa. Al-Monitor called both companies, but they refused to comment about the deal. Howa said it is not in the running.
On that matter, Abou Ghanem said, “We asked the companies to give a deposit of $2.5 million each, which the government would [keep] if they did not fulfill their part of the contract. Howa did not agree that the government would touch their deposit.” He went on, “Chinook agreed to pay $5 million and that this money would be [kept] by the government in case of violation of the contract.”
The contract between Chinook and the Lebanese government is still not signed after the agreement was delayed twice in January. According to a report published by Magazine Jan. 22, a falsification by Howa is the cause of this current blockage. The company, the report alleges, provided a fake document stating that Sierra Leone had agreed to accept Lebanese waste.
The Agriculture Ministry has not confirmed this aspect of the prospective deal so far, but the head of the Council for Development and Reconstruction, Nabil al-Jisr, confirmed Jan. 29 that Chinook had obtained Russia’s agreement to accept Lebanon’s trash, although no official comment was made from Russia.
But this 18-month solution would be a way for the government to push the municipalities to find their own solutions to treat their waste and reduce transportation costs. “Our goal is to have zero waste through better management of the issue,” Abou Ghanem said, adding, “We have to go through this crisis in order to start recycling … and use resources that could be obtained from the sorting of our garbage,” such as energy.
He declared that the Agriculture Ministry was already using organic waste to make compost, but that it had been sorted after being collected, leaving it contaminated and useless for this purpose. He is confident that within a few months, pure compost will be available for local farmers for free, if the deal is applied and sorting solutions are found.
Sebastian Castelier contributed to this article.
Almost All West Bank Land Deals for Illegal Settlements Forged
From straw men to cash-stuffed suitcases, new investigation reveals that 14 out 15 acquisitions by right wing firm of West Bank lands from Palestinians were forged.
In recent years, whenever the state sought to evacuate illegal outposts in the Binyamin region, Al-Watan officials would announce that they had bought the local lands from their Palestinian owners. The documents often turned out to be forged.
A police investigation that was opened in the case ended, and the case was transferred to the state prosecutor. The police did not interrogate Hever, but by questioning a chain of straw men it emerged that 14 of the 15 supposed deals were forged.
An investigative report by Raviv Drucker documents two Palestinians who acted as straw men for Al-Watan.
The straw men would buy the lands from people using forged documents purpoting to be those of the real landowners (many of whom are no longer alive) and then transfer them to Amana in a fake deal.
One of them, Akram, said to the camera, “I signed on many plots and lands.” Referring to the land upon which the outpost of Amona was established he said: “I signed in Silwad,” he said, referring to a West Bank Palestinian town northeast of Ramallah. “They told me there is land with such and such a number, and another plot with number… sign. I signed so many times for many lands. I made five or six deals.”
He added: “I told them I am signing for all the Palestinians, but let me live in Israel because of my children. My children are in Israel, and I don’t see them, and I am always entering Israel and going to jail because I have no permit, and I tell them in the investigation that I am threatened. I have a problem in the Palestinian Authority, and I am threatened unless you let me live in Israel. I sold [land] because they would help me, and no one is helping me. I am suffering from complexes now.”
Straw men, cash suitcases
The two straw men attest that at the signing, an Israeli lawyer from Jerusalem, who was investigated in the case and who name is barred from publication under a gag order, handed them a suitcase with half a million shekels ($126,400).
When they go outside, the lawyer takes the suitcase back. The goal is to fake a deal. Mohammed, one of the straw men who was interviewed, said that he sat with the lawyer in a restaurant, and afterwards went up to his office.
“He told me that this man’s friend cannot put the land under his name because of the Income Tax Authority,” he explained. “I signed on the papers and afterward he gave me a suitcase full of money and told me, ‘Take this.’ When I went downstairs, he took the money back and told me, ‘Take, this is what you get.’ That’s all.”
The investigative report documents attorney Eytan Lehman meeting with Akram at a gas station after he learned that Akram was speaking with Drucker, and instructs him to call him and tell him that he does not remember anything, without realizing that he was being recorded.
“The question is if you can have the conversation with him,” he said in the recording. “Tell him [Druker] perhaps I no longer remember he story blah blah blah, and then he will tell you: ‘but don’t worry, I know the whole story.’ Tell him what does he mean ‘know the whole story,’ for you don’t remember what I said to the police or something like that … Try to get it out of him without him understanding.”
Hever responded by saying he does not have knowledge of the report’s content.
Al Watan responded to the report, saying all of their land deals were done legally.
How To Be a Polymath
by Steven Mazie
Thinking back on the college recommendations I’ve written over the past few weeks, a pattern leaps up: the most successful students, the ones who are the most lively and engaged in class, the most interesting and most dedicated, are never merely great students.
They are also utterly devoted to six other pursuits.
This used to puzzle me. How can a kid write such detailed and analytically involved nightly reading journals on Augustine and Dante, schedule meetings with me about multiple drafts of her essays, excel in a Dostoevsky seminar, third-semester Calculus and painting and find the time to edit the school newspaper, run the debate club, take photography classes, volunteer at her city councilman’s office, sing in a band and write prize-winning poetry on the side?
I exaggerate, but only slightly. As humbling as it is to write letters for students like these, it’s also enlightening, and it’s not just about the elite few humans who can handle doing more than one thing well. “Our age reveres the specialist,” writes Robert Twigger, “but humans are natural polymaths, at our best when we turn our minds to many things.” It’s not just the youngsters who can join the polymath party:
The pessimistic assumption that learning somehow ‘stops’ when you leave school or university or hit thirty is at odds with the evidence. It appears that a great deal depends on the nucleus basalis, located in the basal forebrain.
Among other things, this bit of the brain produces significant amounts of acetylcholine, a neurotransmitter that regulates the rate at which new connections are made between brain cells. This in turn dictates how readily we form memories of various kinds, and how strongly we retain them.
So what’s the trick to letting the acetylcholine flow more abundantly? Twigger again:
People as old as 90 who actively acquire new interests that involve learning retain their ability to learn. But if we stop taxing the nucleus basalis, it begins to dry up.
In some older people it has been shown to contain no acetylcholine — they have been ‘switched off’ for so long the organ no longer functions.
In extreme cases this is considered to be one factor in Alzheimers and other forms of dementia — treated, effectively at first, by artificially raising acetylcholine levels. But simply attempting new things seems to offer health benefits to people who aren’t suffering from Alzheimers. After only short periods of trying, the ability to make new connections develops. And it isn’t just about doing puzzles and crosswords; you really have to try and learn something new.
Trying something new. Hmmmm. What kind of thing?
There’s evidence that something as trivial as changing the path you use when you walk home from the subway can rewire your brain for the better. But beyond tweaking your habit trail, there are more meaningful pursuits you might try, or adopt.
Two years ago, while on a fellowship that cut my teaching load in half and brought me from New York City to a bucolic liberal arts campus a couple of hours away, I had enough newfound headspace to write a piece for the New York Times and soon thereafter accepted an offer to launch Praxis here at Big Think.
I had no idea if I’d be able to keep up the writing while being a dad and a teacher and a runner, but I thought I’d give it a try. The experience has been busy, yes, but manageable, and a few months later I started blogging for The Economist as well.
Adding new activities to my plate—not just any activities, but stuff I really enjoyed doing and had some affinity for—seems to have given me a new source of energy, and sometimes when I’m exhausted I’m also, strangely, exhilarated.
Modern capitalist society bears part of the blame for generating generations of “monomaths.” A monomath, in Trigger’s words, is “a person with a narrow mind, a one-track brain, a bore, a super-specialist, an expert with no other interests.”
You can’t have a modern economy without some degree of specialization, but taken too far the division of labor turns individuals, in Marx’s words, into automatons, “appendage[s] of the machine.” It’s the price we pay for our species’ relentless progress and ever-increasing gains in productivity:
For as soon as the distribution of labour comes into being, each man has a particular, exclusive sphere of activity, which is forced upon him and from which he cannot escape. He is a hunter, a fisherman, a herdsman, or a critical critic, and must remain so if he does not want to lose his means of livelihood. (from Marx’s German Ideology)
Does this conundrum sound familiar? You can raise your skeptical eyebrows, all my critical critics, about the plausibility or desirability of Marx’s alternative—my students certainly do—but close your eyes and imagine this for a second:
In communist society, where nobody has one exclusive sphere of activity but each can become accomplished in any branch he wishes, society regulates the general production and thus makes it possible for me to do one thing today and another tomorrow, to hunt in the morning, fish in the afternoon, rear cattle in the evening, criticise after dinner, just as I have a mind, without ever becoming hunter, fisherman, herdsman or critic.
This fixation of social activity, this consolidation of what we ourselves produce into an objective power above us, growing out of our control, thwarting our expectations, bringing to naught our calculations, is one of the chief factors in historical development up till now.
Few of us can dream of becoming such radical polymaths. (And some of us may consider this extreme de-specialization to be nightmarish.) But it undervalues our lives to willingly enter into mindless ruts. If you’re in a rut, at least be aware of the fact, and let it spur you to take some action. Take that sabbatical, if you are lucky enough to get one. Make stuff. Pursue a new interest. Learn a new language. Stop this, start that. Consider career changes, even if you don’t actually make one. Do something new. Come on, it’s good for you.
Photo courtesy of Shutterstock
Why does life exist?
Popular hypotheses credit a primordial soup, a bolt of lightning, and a colossal stroke of luck.
But if a provocative new theory is correct, luck may have little to do with it. Instead, according to the physicist proposing the idea, the origin and subsequent evolution of life follow from the fundamental laws of nature and “should be as unsurprising as rocks rolling downhill.”
From the standpoint of physics, there is one essential difference between living things and inanimate clumps of carbon atoms: The former tend to be much better at capturing energy from their environment and dissipating that energy as heat.
Jeremy England, an assistant professor at the Massachusetts Institute of Technology, has derived a mathematical formula that he believes explains this capacity.
The formula, based on established physics, indicates that when a group of atoms is driven by an external source of energy (like the sun or chemical fuel) and surrounded by a heat bath (like the ocean or atmosphere), it will often gradually restructure itself in order to dissipate increasingly more energy. This could mean that, under certain conditions, matter inexorably acquires the key physical attribute associated with life.
From the standpoint of physics, there is one essential difference between living things and inanimate clumps of carbon atoms: The former tend to be much better at capturing energy from their environment and dissipating that energy as heat.
“You start with a random clump of atoms, and if you shine light on it for long enough, it should not be so surprising that you get a plant,” England said.
England’s theory is meant to underlie, rather than replace, Darwin’s theory of evolution by natural selection, which provides a powerful description of life at the level of genes and populations. “I am certainly not saying that Darwinian ideas are wrong,” he explained. “On the contrary, I am just saying that from the perspective of the physics, you might call Darwinian evolution a special case of a more general phenomenon.”
His idea, detailed in a paper and further elaborated in a talk he delivered at universities around the world, has sparked controversy among his colleagues, who see it as either tenuous or a potential breakthrough, or both.
England has taken “a very brave and very important step,” said Alexander Grosberg, a professor of physics at New York University who has followed England’s work since its early stages. The “big hope” is that he has identified the underlying physical principle driving the origin and evolution of life, Grosberg said.
“Jeremy is just about the brightest young scientist I ever came across,” said Attila Szabo, a biophysicist in the Laboratory of Chemical Physics at the National Institutes of Health who corresponded with England about his theory after meeting him at a conference. “I was struck by the originality of the ideas.”
Others, such as Eugene Shakhnovich, a professor of chemistry, chemical biology and biophysics at Harvard University, are not convinced. “Jeremy’s ideas are interesting and potentially promising, but at this point are extremely speculative, especially as applied to life phenomena,” Shakhnovich said.
England’s theoretical results are generally considered valid. It is his interpretation — that his formula represents the driving force behind a class of phenomena in nature that includes life — that remains unproven. But already, there are ideas about how to test that interpretation in the lab.
“He’s trying something radically different,” said Mara Prentiss, a professor of physics at Harvard who is contemplating such an experiment after learning about England’s work. “As an organizing lens, I think he has a fabulous idea. Right or wrong, it’s going to be very much worth the investigation.”
At the heart of England’s idea is the second law of thermodynamics, also known as the law of increasing entropy or the “arrow of time.”
Hot things cool down, gas diffuses through air, eggs scramble but never spontaneously unscramble; in short, energy tends to disperse or spread out as time progresses.
Entropy is a measure of this tendency, quantifying how dispersed the energy is among the particles in a system, and how diffuse those particles are throughout space. It increases as a simple matter of probability: There are more ways for energy to be spread out than for it to be concentrated.
Thus, as particles in a system move around and interact, they will, through sheer chance, tend to adopt configurations in which the energy is spread out.
Eventually, the system arrives at a state of maximum entropy called “thermodynamic equilibrium,” in which energy is uniformly distributed. A cup of coffee and the room it sits in become the same temperature, for example.
As long as the cup and the room are left alone, this process is irreversible. The coffee never spontaneously heats up again because the odds are overwhelmingly stacked against so much of the room’s energy randomly concentrating in its atoms.
Although entropy must increase over time in an isolated or “closed” system, an “open” system can keep its entropy low — that is, divide energy unevenly among its atoms — by greatly increasing the entropy of its surroundings. In his influential 1944 monograph “What Is Life?” the eminent quantum physicist Erwin Schrödinger argued that this is what living things must do. A plant, for example, absorbs extremely energetic sunlight, uses it to build sugars, and ejects infrared light, a much less concentrated form of energy. The overall entropy of the universe increases during photosynthesis as the sunlight dissipates, even as the plant prevents itself from decaying by maintaining an orderly internal structure.
Life does not violate the second law of thermodynamics, but until recently, physicists were unable to use thermodynamics to explain why it should arise in the first place. In Schrödinger’s day, they could solve the equations of thermodynamics only for closed systems in equilibrium. In the 1960s, the Belgian physicist Ilya Prigogine made progress on predicting the behavior of open systems weakly driven by external energy sources (for which he won the 1977 Nobel Prize in chemistry). But the behavior of systems that are far from equilibrium, which are connected to the outside environment and strongly driven by external sources of energy, could not be predicted.
This situation changed in the late 1990s, due primarily to the work of Chris Jarzynski, now at the University of Maryland, and Gavin Crooks, now at Lawrence Berkeley National Laboratory.
Jarzynski and Crooks showed that the entropy produced by a thermodynamic process, such as the cooling of a cup of coffee, corresponds to a simple ratio: the probability that the atoms will undergo that process divided by their probability of undergoing the reverse process (that is, spontaneously interacting in such a way that the coffee warms up). As entropy production increases, so does this ratio: A system’s behavior becomes more and more “irreversible.” The simple yet rigorous formula could in principle be applied to any thermodynamic process, no matter how fast or far from equilibrium. “Our understanding of far-from-equilibrium statistical mechanics greatly improved,” Grosberg said. England, who is trained in both biochemistry and physics, started his own lab at MIT two years ago and decided to apply the new knowledge of statistical physics to biology.
Using Jarzynski and Crooks’ formulation, he derived a generalization of the second law of thermodynamics that holds for systems of particles with certain characteristics: The systems are strongly driven by an external energy source such as an electromagnetic wave, and they can dump heat into a surrounding bath. This class of systems includes all living things. England then determined how such systems tend to evolve over time as they increase their irreversibility. “We can show very simply from the formula that the more likely evolutionary outcomes are going to be the ones that absorbed and dissipated more energy from the environment’s external drives on the way to getting there,” he said. The finding makes intuitive sense: Particles tend to dissipate more energy when they resonate with a driving force, or move in the direction it is pushing them, and they are more likely to move in that direction than any other at any given moment.
“This means clumps of atoms surrounded by a bath at some temperature, like the atmosphere or the ocean, should tend over time to arrange themselves to resonate better and better
Note: this article is lengthier, but you got the gist: You efficiently dissipate heat and energy and you are more likely to be a living entity and grow.