Some Thoughts on 'Q'uality.

I’ve seen a few roasters recently saying things like ‘We only sell coffee that scores 86 or over’. It got me thinking about how we communicate ideas around quality in speciality coffee. 

Let’s back-track a bit. What does a score of 86 even mean? Well, you can check out the Coffee Quality Institute website here for all the details, but the idea is that individuals are trained to be able to evaluate coffee quality and then they give it a score out of 100. Coffee becomes Speciality Coffee when it scores over 80. 

Of course, rating quality is a very useful aim. It provides a language and a framework for understanding coffee potential which can help with a lot of things, like evaluating growing and processing methods at origin, establish a likely price that the coffee will fetch for the farmer, or helping a roaster to understand how a particular coffee will fit into their offering. Quantifying these things is a good idea for all those reasons and more. 

But problems arise too. First up, it’s not like we are trying to measure something that is easily measurable. What we are actually trying to capture are sensory qualities - which even if not entirely subjective, are unquestionably influenced by the preferences of the tester. 

I have no doubt that the people undertaking this task (i.e. fully qualified ‘Q’ graders) have been through a very demanding training process, and they work very hard to do a good job. Inter-rater calibration and ongoing professional development are taken seriously and the need to provide consistent and reliable scores is highly valued within that community. 

The qualified Q graders that I know are high-integrity individuals. They cup coffee in a group, with controls in place to try to ensure that they achieve an accurate result. Nevertheless,  I’d like to highlight a few issues with using these Q-grader scores before moving on to what we have decided to do about this whole issue. 

Quality scores are only ever a snapshot

Coffee quality changes over time. Any cupping score will, at best, give an indication of coffee quality at the time the coffee was assessed. Obvious eh? In our experience, those scores can start to drop off pretty quick as the coffee ages. This is why we try to get through all our lots within 3-4 months. 

Score inflation

There are those in the industry that seem to have adopted a pretty sceptical view on this. The argument being that because scores are inevitably linked to prices, then there is a perverse incentive for those that benefit from selling coffee to inflate scores. Check out Scott Rao’s post about this here. 

I’m not so sure that this is a big issue for one main reason. If the importer that I am working with inflates green scores, then they are going to lose credibility fast. We buy green coffee all the time and regardless of whether we have been through the Q grader training process, those of us with experience can easily tell the difference between a coffee that scores 88 and one that scores 85. 

If scores aren’t found to accurately reflect the coffee we buy, then they become useless - we don’t have a reliable way to discuss quality with the importer and so the relationship doesn’t work well-  and no-one benefits from that. 

High scores does not necessarily result in the ‘best’ coffee. 

Many people will find that their preference for coffee lies with beans that score maybe 84 or 85. For some people, a lack of complexity, or sweetness, or acidity is a good thing - even though those factors led to the coffee losing points when it was scored. We’ve tasted many coffees that are super-solid and delicious but which have scored as low as 84 or 85. 

High scores do not necessarily mean high prices. 

It's a good thing that higher quality coffee generally attracts higher prices but there are also a lot of other factors at play when it comes to green coffee prices. I wrote about this a few years ago. You can check out that blog post here. 

So what are we going to do about all this? 

In short, we’re going to adopt a strategy of transparency whilst encouraging our customers to be aware of the limitations associated with cupping scores. 

Essentially, this is information that we have about the coffee that we are buying and so we want to share it with you. We’ll publish cupping scores for each and every coffee that we buy from now on. When you see these scores, you can be sure of a couple of things: 

We have no intention of adopting anything like the ‘we only stock coffee which scores over 86’ stance. Why? - well, because we have tasted plenty of coffee that has scored lower and we’ve enjoyed it.

So, you can expect a range of coffee from us, just as we’ve always tried to do. From those solid and comforting beans, all the way through to the fruity bangers and the rare/exotic lots. 

You might be interested to know that we don’t generally look at cupping scores when buying coffee and we usually only see that information after we have made a decision to buy. That’s because we want to taste the coffee without preconceptions. Scores will vary but the coffees we end up buying all have one thing in common. There is something about each of them that we love. We bought them because we liked them and thought that you would like them too. 

**FWIW, we think the threshold for 'Speciality Coffee' should be raised from a score of 80 to 85. We believe that this would be a more accurate reflection of what most professionals would consider high quality coffee. 

Also in The FCR Blog