Abstract:
This paper extends the methods of [10] in an attempt to handle non-standard problems in
test analysis. The approach is based on a Bayesian framework where test characteristics
are treated as random parameters for which posterior probability assessments are available.
The generality of the approach permits straightforward analyses of problems that may be
difficult using standard classical test theory and standard item response theory. We first
illustrate the methods on aviation test scores where the test outcomes are not dichotomous
(i.e. correct and incorrect responses). Instead, the approach is modified to handle questions
with answers on a five-point ordinal scale. The second problem addresses the complication
of the assessment of instructors in addition to the assessment of test questions and
students.
