Fun with the movies

“The game is afoot!”  [Sherlock Holmes in “The Adventure of Abbey Grange”.]

Recently, Stacker published their list of the 100 “best” mystery movies, which they came up with by combining scores from the Internet Movie Database (IMDB) and Metacritic. It’s definitely fun to look through and we’ll probably try watching some of the ones we haven’t seen. While I don’t really understand their scores (see the notes at the end of this post if you’re curious), it’s meant to be fun, so let’s have some!

One thing we can do is compare IMDB and Metacritic scores. (I’ll multiply IMDB scores by 10 so that they are comparable — they are on a scale of 0-10 while Metacritic is on a scale of 0-100.)

IMDB vs Metacritic scores

What we can see is that Metacritic is a lot more positive than IMDB: the dots are mostly above the gray line, which indicates the same score for both IMDB and Metacritic. The movie “The Souvenir” is particularly striking with its relatively low score on IMDB but relatively high score on Metacritic.

Those are the scores, but we’re also interested in the ranks. It could be that while the scores are different between IMDB and Metacritic, the ranks are roughly the same. However, that it is not the case. There is, in fact, not a lot of agreement about the relative merits of mystery movies across the two sites. In this chart, as above, the further a dot is from the diagonal gray line, the more different the rankings.

IMDB vs. Metacritic ranks

Since Stacker is combining these two disparate sources, what happens with those differences? Here are the top 10 movies according to the three sites, so you can see how they compare with the rankings on the others. There’s clearly something odd going on with Stacker, since it lists “Rear Window” as number 1, which neither IMDB nor Metacritic do. Hmmm… In any case, you can decide which site is closest to your own preferences.

Top 10 Mystery Movies on Stacker

Stacker IMDB Metacritic Title Director(s)
1 10 4 Rear Window Alfred Hitchcock
2 15 3 Vertigo Alfred Hitchcock
3 14 2 Citizen Kane Orson Welles
4 4 8 Psycho Alfred Hitchcock
5 2 11 Spirited Away Hayao Miyazaki
6 28 1 Three Colors: Red Krzysztof Kieslowski
7 13 5 North by Northwest Alfred Hitchcock
8 20 6 Rashomon Akira Kurosawa
9 27 7 The Third Man Carol Reed
10 12 13 Double Indemnity Billy Wilder

 

Top 10 Mystery Movies on IMDB

Stacker IMDB Metacritic Title Director(s)
18 1 59 Harakiri Masaki Kobayashi
5 2 11 Spirited Away Hayao Miyazaki
56 3 98 The Usual Suspects Bryan Singer
4 4 8 Psycho Alfred Hitchcock
62 5 99 Oldboy Park Chan-wook
40 6 92 Memento Christopher Nolan
37 7 89 Coco Lee Unkrich, Adrian Molina
16 8 29 The Lives of Others Florian Henckel von Donnersmarck
11 9 14 Apocalypse Now Francis Ford Coppola
1 10 4 Rear Window Alfred Hitchcock

 

Top 10 Mystery Movies on Metacritic

Stacker IMDB Metacritic Title Director(s)
6 28 1 Three Colors: Red Krzysztof Kieslowski
3 14 2 Citizen Kane Orson Welles
2 15 3 Vertigo Alfred Hitchcock
1 10 4 Rear Window Alfred Hitchcock
7 13 5 North by Northwest Alfred Hitchcock
8 20 6 Rashomon Akira Kurosawa
9 27 7 The Third Man Carol Reed
4 4 8 Psycho Alfred Hitchcock
21 96 9 Don’t Look Now Nicolas Roeg
12 38 10 The Maltese Falcon John Huston

 

Sticking just with Stacker, let’s take a look at the rank versus year — are older movies better than newer ones?

Stacker Rank vs Year

I’ve divided the chart into 4 quadrants. The patterns are pretty clear: the “better” movies are split pretty evenly between the top two halves, while the “lesser” movies are predominantly after 1979 (bottom right). On the other hand, the earlier period (left half) skews towards “better” movies, while the later period (right half) skews towards “lesser” movies. For those of you who like numbers, here are those same quadrants with the number of movies in each, along with the average ranks and average years for each quadrant.

 

Ranks\Years 1935-1977 1978-2020
1-50 N: 23
Ave Year: 1956
Ave Rank: 18
N: 27
Ave Year: 2003
Ave Rank: 32.2
51-100 N: 6
Ave Year: 1961
Ave Rank: 72.7
N: 44
Ave Year: 2008
Ave Rank: 75.9

 

Turning quickly to the movies themselves, we can compare the duration in minutes (runtime) to the year the movie was released. At least in these top 100, the later movies tend to be longer. The one extreme example is “Mysteries of Lisbon” which was originally a TV mini-series rather than a theater released movie.

MM Runtime vs Year

Finally, we can also see which directors show up the most. The number one mystery director is no mystery… Alfred Hitchcock.

Director Number of movies
Alfred Hitchcock 8
Denis Villeneuve 4
Asghar Farhadi 3
Krzysztof Kieslowski 3
Bi Gan 2
Carol Reed 2
David Fincher 2
Francis Ford Coppola 2
Michael Haneke 2
Robert Altman 2

 


Technical notes

I accessed the Stacker site, and a republished version of the story on SFGate on October 5, 2020. I collected the data (in the browser using javascript) from the latter version, filling in a missing entry (#7, “North by Northwest”) as well as interpolating a missing Stacker score of 93 for (#17, “LA Confidential”). All of the analyses and visualizations were done in R, using ggplot2.

Although Stacker says that they use “a weighted index split evenly between IMDb and Metacritic scores” I was not able to recreate their scores. In particular, their statement suggests that the Stacker Score is a linear combination of the IMDB and Metacritic scores:

Stacker = a * IMDB + b * Metacritic + c

However, I could not find any suitable values for (a, b, c) — the Stacker scores seem to be inconsistent with the linear combination model, so I don’t really know how they computed their scores. Another oddity is that even though the maximum Stacker score is 100 (as on Metacritic), over 3/4 (78) of the Stacker scores are higher than both of the other scores.

I recomputed a simple average of the IMDB (multiplied by 10 for comparability) and the Metacritic scores (i.e. a=5, b=0.5, c=0), and I found that those scores led to different rankings for about 1/3 of the movies (32). However, those differences were usually just 1 rank, and occasionally 2, so not a huge difference, and certainly not one that interferes with the fun here.