Your Movie Rating Is Lying To You

Data & Perception

Your Movie Rating Is Lying To You

Why the difference between a mountain of data and a rumor of art is often hidden behind three decimal points.

John Graunt sat in a small room in London in . He was a man who sold buttons and cloth but he liked numbers. He collected the Bills of Mortality from the local churches.

These papers listed the names of the dead and the reasons they died. Some died of the plague and some died of old age. Graunt looked at the lists and he saw patterns. He noticed that the small parishes had numbers that changed every week.

One death in a small village could make the plague look like a monster. One death in a large city was just a mark on a page. He understood that a number needs a crowd to be true. He wrote a book about these things and people began to understand the weight of a sample. He was the first man to see that a small count is a nervous thing.

The Illusion of Equality

Saga lives in Sundsvall and she is . The snow is outside her window and the kitchen is warm. She holds her phone and she wants to find a movie.

She looks at a thriller from America. The thriller has a score of 6.3. It has a famous actor and the poster is bright. She looks at a drama from Sweden. It is a story about the rural north and it also has a score of 6.3.

She thinks these movies are the same quality. She chooses the thriller because she knows the actor. She does not look at the number of votes. She does not see that the thriller has nine hundred thousand votes. She does not see that the Swedish drama has nine hundred votes. She makes a choice based on a number but the number is a lie.

The American Thriller

900,000 Votes

The Swedish Drama

900 Votes

The statistical “weight” of these two identical 6.3 ratings is separated by a factor of 1,000.

The 6.3 on the thriller is a solid thing. It is built from the opinions of a million people and it will not move. If ten people hate the movie the score stays at 6.3. If ten people love the movie the score stays at 6.3. It is a mountain of data.

The 6.3 on the Swedish film is a thin thing. It is built from a few hundred people. Many of these people worked on the film or they live in the town where it was made. If ten people watch it tonight and they hate it the score will drop to 5.8. If ten people love it the score will rise to 6.7.

The number is a snapshot of a small group and it carries the noise of their lives. Saga does not know this and she treats the numbers as equal.

A rating is a measurement but people treat it like a property. They think a film possesses a 6.3 in the same way it possesses a director or a run time. This is a mistake. The rating is a report of what happened when people watched it.

When few people watch it the report is not finished. It is a rumor. Small national productions suffer from this rumor. They are filmed in places like Dalarna or Kiruna and they are seen by local audiences first.

These audiences have their own biases and their own reasons for voting. A Swedish film on SVT Play or Viaplay might have a mediocre score because it has not yet found its crowd. It is penalized for being small and it stays small because it is penalized.

I was writing these thoughts and my boss walked into the office. I moved my fingers on the keyboard and I stared at the lines of text. I tried to look busy and I hoped he would not stop.

He walked past my desk and his shoes made a sound on the carpet. I felt the tension in my shoulders and then it was gone.

I returned to the numbers. I thought about how we look for certainty in a world that is full of noise. We want a simple answer to the question of what to watch. We want the number to tell us the truth so we do not have to think.

The Arithmetic of Anxiety

People use these scores as a shield against a bad evening. They have of free time and they do not want to waste them. They look for a 7.0 or an 8.0 because they think these numbers are a guarantee.

They see a 6.3 and they feel a sense of risk. This risk is real for a big movie with a million votes. A 6.3 on a blockbuster usually means the movie is flawed. It means the script was weak or the ending was poor.

But a 6.3 on a small film from a small country means something else. It means the sample is small and the error is large. It means the movie might be a masterpiece that has only been seen by a few people who were in a bad mood.

“The machine tells you a number but the air in the room tells you if the number matters.”

– Natasha R.J., Clean Room Technician

Natasha R.J. is a clean room technician and she works with precision instruments. She measures particles that the human eye cannot see. She knows that a sensor can be perfect but the context can be wrong.

If a single dust mote hits the lens the reading is ruined. This is what happens to small films. A single group of critics or a single week of bad luck can ruin the reading for a . The score becomes a barrier and the barrier becomes a wall.

We are asked to make decisions like this every day. We look at restaurant reviews and we look at star ratings for schools. We see a 4.2 and we compare it to a 4.2. We do not ask how many people stand behind the number.

We are training our brains to ignore the uncertainty. We want the world to be a list of sorted items but the world is a messy place. Local culture is hidden in the mess. Small voices are drowned out by the volume of the large ones.

When we ignore the sample size we ignore the minority. We give the power to the biggest crowd and we let the small parishes disappear.

Tools for the Smarter Viewer

The Swedish film industry produces many works that are quiet and deep. These films appear on services like

Movie Owl

where viewers can see what is streaming on Netflix or Max or Cineasterna.

The service shows the real IMDb rating as a shortcut. This is helpful for the viewer who is in a hurry. But the viewer must be smarter than the shortcut. They must see the 6.3 and then they must look for the count.

They must ask if they are looking at a mountain or a rumor. If they see a small count they should be brave. They should realize that the number is not a fence. It is a suggestion.

The thriller with the famous actor started on Saga’s television. She sat on her sofa and she watched the first . The explosions were loud and the music was fast. She felt nothing.

The movie was a 6.3 and it felt like a 6.3. It was a product that had been tested and polished until it was average. It did not take a risk and it did not fail but it did not succeed. Saga felt bored and she turned it off.

She thought about the Swedish drama. She thought about the film from the north with the quiet poster. She wondered what those nine hundred people saw. She wondered if their 6.3 was different from the thriller’s 6.3.

The Soul in the 5.9

There is a cost to our obsession with precision. We lose the ability to be surprised. We stay in the safe zones where the numbers are high and the samples are large. We watch the same ten shows that everyone else is watching because the 8.4 is a comfort.

We do not want to be the person who watches the 5.9 and finds a soul. But the 5.9 is often just a 8.0 that hasn’t been discovered yet. It is a film that was watched by ten people who wanted something else. It is a victim of the math of the mean.

🏢

The 8.4 Consensus

The comfort of the crowd.

✨

The 5.9 Discovery

The soul in the outliers.

The clean room where Natasha works is quiet and the air is filtered. She wears a white suit and she moves with care. She knows that her presence changes the environment. Every person who votes on a film changes the environment of that film.

In a big movie your vote is a drop of water in the ocean. In a small movie your vote is a stone in a pond. The ripples go to the edge and they change the shape of the water.

We should value the stones more than the drops. We should look for the ponds where our presence matters.

Sundsvall is dark now and the snow is still falling. Saga goes back to her phone. She finds the Swedish drama again. She sees the nine hundred votes and she sees the 6.3.

She thinks about John Graunt and his parishes. She thinks about the weight of a single life. She presses play.

The film starts with a shot of a forest. There is no music and there are no stars. There is only the sound of the wind and the sight of the trees. It is beautiful and it is strange. It is not an average movie. It is a specific movie. It was made for a specific moment and it found a specific viewer.

🌲

A film with ten votes is a rumor and a film with ten thousand is a fact.

We need to learn how to read the rumors. We need to understand that the gap between 900 and 900,000 is not just three zeros. It is a different kind of reality. One is a conversation and the other is a broadcast.

If we only listen to the broadcast we will never hear the secrets of the conversation. We will live in a world of 6.3s that feel like nothing. We will miss the 6.3s that feel like everything.

Saga watches the forest on her screen and she is glad she chose the rumor. She is glad she looked past the number. She knows that the truth is often hiding in the small counts where the noise is the loudest.

Exploring the Weight of the Sample