Human/AI partnerships, and laptop stickers!

Dear Clog Loss competitors,

As I mentioned in the THANK YOU post, we are grateful for your contributions and I am impressed by the high MCC scores achieved so far. I especially like to see how each team is also in competition with itself, always seeking to improve its own score regardless of its position on the leaderboard.

What the leaderboard does not reveal is that the diversity of approaches will ultimately be most valuable in the ensemble context - sometimes having three completely different models exhibiting average behavior can be equivalent or even better than one supermodel.

Also we are currently experimenting with AI assistance - that is, what happens when you have a model suggest an answer to a skilled human, and then the human can decide how to incorporate that information into a final answer. So far, our research results are quite tantalizing - it looks like under certain conditions, the human + model performs better than human alone or model alone. As you might guess, having a model that complements human performance is actually more effective in this context than a model that approximates human performance.

In any case, as a small token of our thanks, we’d like to send you a nice quality Stall Catchers laptop sticker.

So if you have made at least one submission to the Clog Loss competition, please feel free to send your name and mailing address to with the subject line “sticker”. We will do our best to accommodate international addresses.

If there’s one thing we are good at, it is extracting value from each information processing agent in our collective intelligence system. Please keep up the good work and remember your model, regardless of where it falls on the leaderboard, is potentially useful for accelerating Alzheimer’s research.

Best wishes,
Pietro Michelucci (and the Stall Catchers team)
Director, Human Computation Institute