Fleiss' Kappa Agreement

Modified on Sun, 10 Oct, 2021 at 11:46 AM

What is Fleiss' Kappa?

Fleiss' Kappa is a statistical measure of agreement between at least 3 observers.

It is used to assess the interobserver reliability of several observers.

The observed variable must be categorial: either a Yes-no (binary) or List.

How to calculate Fleiss' Kappa?

  1. Go to "Test variables"
  2. Pick a first List or Yes-no variable
  3. Pick a second List or Yes-no variable
  4. Pick a third List or Yes-no variable
  5. Click on the panel titled "Find an agreement" 

Fleiss' kappa will be calculated and the interpretation according to Landis and Koch will be prompted.

Why is Fleiss' Kappa not available?

If you are not prompted with a "Fleiss' kappa" panel, please ensure that: 

  • All the chosen variables have the same type: Yes-no or List. You can not mix Yes-no and List variables
  • If you use List variables, they have the same modalities (values). For example, if you have 3 observers named "Observer 1", "Observer 2" and "Observer 3" which assess the eye colors of participants, the possible values for each observer must be identical: "blue", "green", "brown", ...
  • You have at least 3 variables selected. If there are only 2 observers, a Cohen's kappa is calculated instead.

Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select at least one of the reasons
CAPTCHA verification is required.

Feedback sent

We appreciate your effort and will try to fix the article