Criterion Recommendation System: Looking for Testers

[cdnchris]

While rebuilding the site last year I decided to implement a few other features since the new tech stack would allow me to do them, and one of those was a recommendation system for Criterion releases. It's been a lot of work, but it's finally getting very close to what I had originally envisioned, and I'm now at a point where I need others to help test it.

I'm looking for a handful of users who would be interested in trying it out.

If you're interested, please continue reading.

The current plan is to open it up within the next 2 to 3 weeks. It won't be complete and a number of features will still be missing, but I want to start testing the core recommendation/scoring system as soon as possible.

I'll get more into what it does and how it works once it's ready to be tested, but the basic idea is that it's designed around exploration or "expanding one's horizons," whether you're a seasoned cinephile or someone just getting into film. A user enters their current interests (directors they like, genres, movements, etc.) and from there the system makes recommendations based on how far outside your comfort zone you want to go.

It's not a "you like Wes Anderson so here's every Wes Anderson movie in the collection, conveniently available in one big box set" type system, and the results can differ quite a bit depending on the parameters provided. Results are not based on surface-level similarities, but draws from things like cinematic movements and eras, influences and similarities between filmmakers, style, themes, and other areas within film history. The hardest part about building this was building the datasets (and the math, there's a lot of freaking math).

What I'm mainly looking to test is:

• General functionality
• Whether the interface makes sense (though the current interface is mostly placeholder at the moment)
• Impact on server resources with multiple users running searches at the same time
• Whether the results mostly make sense

Resources are a big concern. Initial runs during the early stages last year could take upwards of 2 minutes, but I've managed to get that down to anywhere from about 1.5 to 20 seconds depending on the complexity of the request.

If you do volunteer I'd ask that you test it periodically, and there will be a dedicated thread where you can leave comments, suggestions, or notes. Every result will also be stored in a database for me to review afterward. There will additionally be a scoring system around whether a recommendation "makes sense," though I'm still deciding whether that should be a scale-based system or just a simple thumbs up/down setup. In either case, I'd ask that you score each item in the results when possible.

If you're interested, or have any questions, please respond in this thread.

I'm not sure yet how many people I'm looking for, so for now I'll just base it on the amount of interest.

[LastMinit]

Hey, so these days I'm heavily interested in a movie's backdrop, I'm always on the lookout for a film that strongly depicts a city or a setting or an era, and the more exotic and off-the-beaten path the better. I'm sure there are plenty of good examples in the film world I've yet to discover, but it's a hard thing to search for, and asking for recommendations is tricky because at the same time I'm very particular about a director's touch. So it's a bit of a niche within a niche. I recently caught Bi Gan's Kaili Blues on the channel and LOVED it, it's exactly the kind of thing I'm after. Not sure if this set of parameters would fall within your equations, but if it might, then I'd be glad to take part.

[cdnchris]

Actually, that is exactly what I'm trying to test for, at least in this initial phase. One of the parameters (and the only required parameter as of now, with everything else being optional) are directors you are interested in/fond of. So yes, you would enter Bi Gan and then adjust various knobs and get recommendations for Criterion releases (haven't expanded to their Premiers or Eclipse releases yet).

So Bi Gan with all the basic settings (meaning "safer") leads to:

Badlands
Chungking Express
A Man Escaped
In the Mood for Love
Mysterious Object at Noon
Japon
Yi Yi
Flowers of Shanghai
Mulholland Drive
Werckmeister Harmonies

Cranking things a bit more then gives

Color of Pomegranates
Eraserhead
Andrei Rublev
Japon
Flowers of Shanghai
Mysterious Object at Noon
Werckmeister Harmonies
Stalker
This Is Not a Burial, It's a Resurrection
La Cienega

So the idea is not necessarily "you like Kaili Blues or Resurrection so here are films exactly like that" but more "you like the work of these directors so here's some you may find of interest in your comfort zone."

For another example, Spielberg (with full "Safe" settings) would be:

Pee-wee's Big Adventure
WALL-E
I Wanna Hold Your Hand
The Big Chill
Thelma and Louise
The Others
The Three/Four Musketeers
Following
The Princess Bride
The Manchurian Candidate

More adventurous:

Seconds
Ashes and Diamonds
Robinson Crusoe on Mars
His Girl Friday
Foreign Correspondent
The Most Dangerous Game
It Happened One Night
Thief of Bagdad
My Darling Clementine
The Others

I've been much happier with the results it's throwing at me the last few months, but there's still some off items and I think it comes down to how it's weighting things in certain circumstances, so I need to tune it and having people throw whatever at it and ranking results will help.

[onedimension]

I don't have time to contribute to testing right now, but this sounds like a great project!

[godshammgodard]

Hi - I'm definitely interested in assisting with this. I'm a long time user of Criticker, which I use primarily for its recommendation algorithm. I also have my own personal recommendation spreadsheet. Anyway, this is right up my alley. Please let me know when and how I can assist.

[cdnchris]

I don't have time to contribute to testing right now, but this sounds like a great project!

Just to clarify, I appreciate if people are able to spend a bit of time on it, but even if it's something someone hits up once in a while, that's fine, too.

Hi - I'm definitely interested in assisting with this. I'm a long time user of Criticker, which I use primarily for its recommendation algorithm. I also have my own personal recommendation spreadsheet. Anyway, this is right up my alley. Please let me know when and how I can assist.

Great! It'll be a few weeks out but I'll send out more info once it's ready. I'm ironing out some issues, still, including one where I broke one of the most important parts of it and didn't realize it until this morning. So there's a chance it's going to be rough