| Agreement among raters is of great importance in many domains. For example, in medicine, diagnoses are often provided by more than one doctor to make sure the proposed treatment is optimal. In criminal trials, sentencing depends, among other things, on the complete agreement among the jurors. In observational studies, researchers increase reliability by examining discrepant ratings. This book is intended to help researchers statistically examine rater agreement by reviewing four different approaches to the technique.
The first approach introduces readers to calculating coefficients that allow one to summarize agreements in a single score. The second approach involves estimating log-linear models that allow one to test specific hypotheses about the structure of a cross-classification of two or more raters' judgments. The third approach explores cross-classifications or raters' agreement for indicators of agreement or disagreement, and for indicators of such characteristics as trends. The fourth approach compares the correlation or covariation structures of variables that raters use to describe objects, behaviors, or individuals. These structures can be compared for two or more raters. All of these methods operate at the level of observed variables.
This book is intended as a reference for researchers and practitioners who describe and evaluate objects and behavior in a number of fields, including the social and behavioral sciences, statistics, medicine, business, and education. It also serves as a useful text for graduate-level methods or assessment classes found in departments of psychology, education, epidemiology, biostatistics, public health, communication, advertising and marketing, and sociology. Exposure to regression analysis and log-linear modeling is helpful.
"This book would be extremely useful as a reference for researchers and academics in many fields including statistics, psychology, medicine, business, and education....Additionally, this book would be useful for a graduate level methods or assessment class in many disciplines including psychology and education....there are no other books that approach rater agreement as comprehensively as this book."
— Keith Niedermeier, Ph.D. - Pennsylvania State University
"...rater agreement is a critical issue across the behavioral and social sciences and thus a book of this nature would be very broadly utilized....The book has a number of strengths including the outstanding reputation of the author....[and] the depth of scholarship. It is comprehensive in its coverage of the manifest variable approaches."
— Todd Little, Ph.D. - University of Kansas |
