Computes confidence intervals for a 3-rater design with dichotomous ratings

Source: R/statpsych3.R

Computes adjusted Wald confidence intervals for a G-index of agreement for all pairs of raters in a 3-rater design with a dichotomous rating, and computes adjusted Wald confidence intervals for differences of all pairs of G agreement. An adjusted Wald confidence interval for unanimous G agreement among the three raters is also computed. In the three-rater design, unanimous G agreement is equal to the average of all pairs of G agreement. The G-index corrects for chance agreement.

ci.agree.3rater(alpha, f)

Arguments

alpha

alpha level for 1-alpha confidence

f

vector of frequency counts from 2x2x2 table where f = [ f111, f112, f121, f122, f211, f212, f221, f222 ], first subscript represents the rating of rater 1, second subscript represents the rating of rater 2, and third subscript represents the rating of rater 3

Value

Returns a 7-row matrix. The rows are:

  • G(1,2): G-index for raters 1 and 2

  • G(1,3): G-index for raters 1 and 3

  • G(2,3): G-index for raters 2 and 3

  • G(1,2)-G(1,3): difference in G(1,2) and G(1,3)

  • G(1,2)-G(2,3): difference in G(1,2) and G(2,3)

  • G(2,3)-G(1,3): difference in G(2,3) and G(1,3)

  • G(3): G-index of unanimous agreement for all three raters

The columns are:

  • Estimate - estimate of G-index (two-rater, difference, or unanimous)

  • LL - lower limit of adjusted Wald confidence interval

  • UL - upper limit of adjusted Wald confidence interval

References

Bonett DG (2022). “Statistical inference for G-indices of agreement.” Journal of Educational and Behavioral Statistics, 47(4), 438--458. doi:10.3102/10769986221088561 .

Examples

f <- c(100, 6, 4, 40, 20, 1, 9, 120)
ci.agree.3rater(.05, f)
#>                  Estimate          LL         UL
#> G(1,2)         0.56666667  0.46601839  0.6524027
#> G(1,3)         0.50000000  0.39564646  0.5911956
#> G(2,3)         0.86666667  0.79701213  0.9135142
#> G(1,2)-G(1,3)  0.06666667  0.00580397  0.1266464
#> G(1,2)-G(2,3) -0.30000000 -0.40683919 -0.1891873
#> G(2,3)-G(1,3) -0.36666667 -0.46222023 -0.2662566
#> G(3)           0.64444444  0.57382971  0.7068720

# Should return:
#                  Estimate          LL         UL
# G(1,2)         0.56666667  0.46601839  0.6524027
# G(1,3)         0.50000000  0.39564646  0.5911956
# G(2,3)         0.86666667  0.79701213  0.9135142
# G(1,2)-G(1,3)  0.06666667  0.00580397  0.1266464
# G(1,2)-G(2,3) -0.30000000 -0.40683919 -0.1891873
# G(2,3)-G(1,3) -0.36666667 -0.46222023 -0.2662566
# G(3)           0.64444444  0.57382971  0.7068720