2017-09-29
Hard-won R knowledge for Inter-Rater Reliability / Inter-Annotator AgreementI have each item on a row, and then I merge my raters with one column per rater, like any sensible person would do, but of course R's irr library doesn't like that. Eyeroll.
Importing the file as a character matrix: test_matrix <- as.matrix(read.table("Documents/IRR-trial-run.txt", sep="\t", colClasses="character", na.strings = "", row.names="V1")) IRR package: > library(irr)
Error: ...
# employ profanity
> install.packages('irr')
# answer some questions
> library(irr)
Calculating:
> kripp.alpha(t(test_matrix), method="nominal")
# Note the t() that transposes the matrix. Our ever-logical friend R expects each rater to have their own row, and each item to have its own column. Because of course it does.
20:58
