Last night I found myself printing a pile of various WBC scoresheets with some scribbles on them. Doesn’t sound very interesting first, does it? Nonetheless, reading these WBC finals scoresheets from amazing baristas including Colin Harmon (he deserves all the respect for releasing his sheets first), Michael Phillips, James Hoffman and Gwilym Davies has already made me to think again my approach to the whole competition and its scoring.
Just watching the routines, live or not, is actually quite bizarre. Watching delicious, some world’s best coffee being made in front of your eyes and you don’t get to taste any of it. You can’t say too much about judges impressions either.1 But examining the scoresheets from the competitors kind of gives you a glimpe of how that coffee might have tasted and more importantly, why did that competitor get x amount of points for his performance. By studying the routines and scoresheets not just the competitors themselves but also other people could do better in their next competition.
There was an excellent comment by Alex Bernson in James Hoffman’s post. He points out that the judge has most likely assumed that the scoresheet would be shared with other judges as well as with the competitor and his training team, not the whole Internet barista community. A lot of writing on the scoresheets I’ve been reading is judge’s own notes, some is obviously meant for the competitor as feedback. Most of it doens’t make any sense for someone who hasn’t really tasted the coffee. Alex suggests that the problem might be solved if the judges writing the scoresheets knew they might and hopefully would go into the public. Hopefully this would improve the feedback given to the barista, increase scoresheets’ value to the coffee community as well as decrease the amount of incompetent judges.2
As the secretary of the Finnish chapter of SCAE, I am involved in the forthcoming barista competition. Today I got an idea of making all the scoresheets of the six finalists public and posting them on the national body’s website after competition. Not only would it improve the level of judging, hopefully making it fair and transparent for everyone, but it also might attract more baristas for next year’s competition. Showing the new kids what they are expected of, where do they lose or get points and make them realise that the top baristas don’t get all “fives” or “sixes” (something I didn’t before cheking out these sheets. This, of course, depends heavily on the Head Judge and the criteria and varies from competition to another). Getting new baristas to compete just might be the best way to improve the quality and presence of speciality coffee both in Finland and worldwide.
I really would like to hear any feedback, comments, views regarding this subject. There has been some rumour going on about last year’s competition and its transparency. There’s no need for speculations anymore, I would just like to make sure there’s no room for such rumours this year.
10 Comments
I hope you know that every possible score sheet in Finland is confidential and publishing them is either a) illegal b) highly unethical or c) both.
Of course it would require the permission of each and every competitor. There’s no point forcing anyone to reveal their scoresheets but giving them the possibility and maybe encouraging them a little without creating any pressure might be the way to go. Or would that be illegal too? The biggest reason why I think this would be good idea is that just the fact that the scoresheets might be posted to the public could – and probably would – make the judging process more fair and transparent to everyone.
At wbc they were really strict about not showing the scorecards to anyone else but the competitor. The competitors can already publish their scorecards if they want, so the result would stay the same. Disguising curiosity in names of progression or education is, in my opinion, quite shady.
There are different kind of scoresheets around. At the other end is the strict confidentiality (which in Finland is supervised by at least VALVIRA I think) and will lead to sanctions, losing practitioner’s license etc. No matter how you look at it, every possible score sheet is at face value, confidential. I won’t start to list all the possible philosophical bases I could justify myself. I’d bore you to death.
If you somehow still need to take the possibility from the competitors to use their own consideration, you could use wikileaks.
I think it is a great idea Kalle. It could benefit aspiring baristas and de-mystify some elements that make barista competions difficult to understand for those not competeing or judging themselves. To publish all sheets in a uniform standard, in a timelly manner and at a SCAE/SCAA hosted location would be a great start to make the competitions more meaningful and accessible. If there are SCAE rules stating you can’t publish this, try to get an excemption for the Finish comps and/or push to change the rules?
If indeed this is illegal, how is this illegal? Under what law and paragraph?
I think as a general idea it’s a really good thing to release these. What might get lost in translation though is that not too many people understand Finnish
The two main benefits as I see it in your post is 1. the achievement of getting judges to know the transparency of their judging and making them a bit more clear and 2. that globally other nations can look at each others score sheets for self improvement.
Thought I posted this comment yesterday, guess I forgot to hit submit…
I’m glad you liked my comment. I don’t have any experience with the Finnish legal system, but I’d find it hard to believe that publicizing scoresheets would be in any way illegal if you made it clear before-hand that they would be publicized. Saying it is unethical is once again only true if the baristas/judges thought going in that they would be totally secret.
Whether it is a good idea is a different question. I certainly believe that publication at the WBC level (if announced beforehand) is a good idea, but I’m not sure that holds at the national level. It can certainly be incredibly helpful for in-experienced competitors to see actual scoresheets and better understand the weighting, criteria etc. I don’t know the specifics of the finnish scene in terms of experience, size etc., but having all scoresheets public might actually make competitors feel even more intimidated by the process and also might make it hard to convince people to judge. Also, to my knowledge it would be the first national competition to take the public route, which might lead to attention being focused on the judging questions more than nurturing the competition as a whole.
Those issues might be outweighed by the benefits to transparency of judging and the understanding of competition. But perhaps workshops with past competitors going over their own scoresheets, or going over the scoresheets from the competitors you’ve listed above would serve some of the same purposes. Or as you say you could provide the option of being public this year as a stepping stone to a fuller discussion on judging transparency. It’s an interesting issue, and I’m sure we’ll be seeing a lot more conversations on the subject.
I think you’re right. As the barista competitions and the coffee scene as a whole is still fairly young concept in Finland, the question about scoresheets and judging might not be relevant at the moment. It certainly might better to focus on attracting more competitors (this year we’ll probably have about 10-12 in total) and developing the barista community rather debating over minor detais about competition scoring and feedback. I also might not be the right person to propose something like this as I haven’t received a single competition scoresheet yet…
Still, I think that it’s important to discuss this issue and, as you said, maybe rather at the WBC level than at the national level. I don’t think regarding the scoresheets as a taboo is the way to go and in order to develope the competition format and its educational aspect further we definitely need more discussion over this topic.
Would this affect the judges, would they judge like they used to if they knew that the scoresheets would be open to everyone? And some people who might compete could easily decide not to compete because they are affraid that everybody could see their scoresheet (and maybe laugh).
In a place like finland i think we should make competing as easy and “attractive” as possible and i think this would do the opposite. If someone wants to publish their scoresheet they can do like some have done already, put in the net.
There would be no point forcing anyone to publish their scoresheet, that wasn’t my original point. The idea was to create a forum and maybe encourage a little if someone would like to publish their sheets.
It’s a good point that it might discourage some people but on the other hand also make the competition more transparent and accessible by letting people know what they are expected of and where and how much do they get points from.
I think you are a perfect person to propose something like this Kalle! Every industry needs new blood, new thoughts and ideas and I think Finland would be perfect place to try some new ideas?
I have recently been watching Swedish & American versions of Masterchef, a TV show that have been a massive hit worldwide: I really enjoy the instant passionate feedback the judges provide to the contestants and viewers. Now compare that with the neutral empty faces and silence from the judges at barista competitions. I would ideally love to hear the judges opinions on the taste of the drinks while the barista still is on the stage. Not 4 months later when the barista finally decided to scan some low resolution scoresheet one can not read anything from due to bad handwriting.