What is Fleiss' Kappa?

Fleiss' Kappa is a statistical measure of agreement between at least 3 observers.

It is used to assess the interobserver reliability of several observers.

The observed variable must be categorial: either a Yes-no (binary) or List.

How to calculate Fleiss' Kappa?

1. Go to "Test variables"
2. Pick a first List or Yes-no variable
3. Pick a second List or Yes-no variable
4. Pick a third List or Yes-no variable
5. Click on the panel titled "Find an agreement" 

Fleiss' kappa will be calculated and the interpretation according to Landis and Koch will be prompted.

Why is Fleiss' Kappa not available?

If you are not prompted with a "Fleiss' kappa" panel, please ensure that: 

• All the chosen variables have the same type: Yes-no or List. You can not mix Yes-no and List variables
• If you use List variables, they have the same modalities (values). For example, if you have 3 observers named "Observer 1", "Observer 2" and "Observer 3" which assess the eye colors of participants, the possible values for each observer must be identical: "blue", "green", "brown", ...
• You have at least 3 variables selected. If there are only 2 observers, a Cohen's kappa is calculated instead.