Computerized classification test: Difference between revisions

Revision as of 20:49, 17 January 2025

Latest revision as of 09:10, 2 August 2025

(4 intermediate revisions by 2 users not shown)

Line 1:

A '''computerized classification test''' ('''CCT''') refers to, as its name would suggest, a [[Test (student assessment)|Performance Appraisal System]] that is administered by [[computer]] for the purpose of [[Classification rule|classifying]] examinees. The most common CCT is a mastery test where the test classifies examinees as "Pass" or "Fail," but the term also includes tests that classify examinees into more than two categories. While the term may generally be considered to refer to all computer-administered tests for classification, it is usually used to refer to tests that are interactively administered or of variable-length, similar to [[computerized adaptive testing]] (CAT). Like CAT, variable-length CCTs can accomplish the goal of the test (accurate classification) with a fraction of the number of items used in a conventional fixed-form test.

A CCT requires several components:

Line 24:

== Item selection ==

In a CCT, items are selected for administration throughout the test, unlike the traditional method of administering a fixed set of items to all examinees. While this is usually done by individual item, it can also be done in groups of items known as [[testlet]]s (Leucht & Nungester, 1996;<ref>{{cite journal|last1=Luecht|first1=R. M.|last2=Nungester|first2=R. J.|year=1998|title=Some practical examples of computer-adaptive sequential testing|journal=Journal of Educational Measurement|volume=35|pages=229-249|doi=10.1111/j.1745-3984.1998.tb00537.x}}</ref> Vos & Glas, 2000<ref>{{cite book|last1=Vos|first1=H.J.|last2=Glas|first2=C.A.W.|year=2000|chapter=Testlet-based adaptive mastery testing|editor1-last=van der Linden|editor1-first=W.J.|editor2-last=Glas|editor2-first=C.A.W.|title=Computerized Adaptive Testing: Theory and Practice|url=https://archive.org/details/computerizedadap0000unse_b0n1|doi=10.1007/0-306-47531-6_15}}</ref>).

Methods of item selection fall into two categories: cutscore-based and estimate-based. Cutscore-based methods (also known as sequential selection) maximize the [[quantities of information|information]] provided by the item at the cutscore, or cutscores if there are more than one, regardless of the ability of the examinee. Estimate-based methods (also known as adaptive selection) maximize information at the current estimate of examinee ability, regardless of the ___location of the cutscore. Both work efficiently, but the efficiency depends in part on the termination criterion employed. Because the [[sequential probability ratio test]] only evaluates probabilities near the cutscore, cutscore-based item selection is more appropriate. Because the [[confidence interval]] termination criterion is centered around the examinees ability estimate, estimate-based item selection is more appropriate. This is because the test will make a classification when the confidence interval is small enough to be completely above or below the cutscore (see below). The confidence interval will be smaller when the standard error of measurement is smaller, and the standard error of measurement will be smaller when there is more information at the theta level of the examinee.

== Termination criterion ==

There are three termination criteria commonly used for CCTs. [[Bayesian decision theory]] methods offer great flexibility by presenting an infinite choice of loss/utility structures and evaluation considerations, but also introduce greater arbitrariness. A [[confidence interval]] approach calculates a confidence interval around the examinee's current theta estimate at each point in the test, and classifies the examinee when the interval falls completely within a region of theta that defines a classification. This was originally known as adaptive mastery testing ({{harv|Kingsbury & |Weiss, |1983)}}, but does not necessarily require adaptive item selection, nor is it limited to the two-classification mastery testing situation. The [[sequential probability ratio test]] ({{harv|Reckase, |1983)}} defines the classification problem as a [[hypothesis test]] that the examinee's theta is equal to a specified point above the cutscore or a specified point below the cutscore.

== References ==

Line 48:

* {{Cite report |last=Huang |first=C.-Y. |last2=Kalohn |first2=J.C. |last3=Lin |first3=C.-J. |last4=Spray |first4=J. |year=2000 |title=Estimating Item Parameters from Classical Indices for Item Pool Development with a Computerized Classification Test |type=Research Report 2000–4 |___location=Iowa City, IA |publisher=ACT, Inc.}}

* {{Cite thesis |last=Jacobs-Cassuto |first=M.S. |year=2005 |title=A Comparison of Adaptive Mastery Testing Using Testlets With the 3-Parameter Logistic Model |type=PhD, unpublished |publisher=University of Minnesota, Minneapolis, MN}}

* {{Cite conference |last=Jiao |first=H. |last2=Lau |first2=A.C. |year=2003 |title=The Effects of Model Misfit in Computerized Classification Test |conference=Annual meeting of the National Council of Educational Measurement |___location=Chicago, IL |date=April 2003}}

* {{Cite conference |last=Jiao |first=H. |last2=Wang |first2=S. |last3=Lau |first3=C.A. |year=2004 |title=An Investigation of Two Combination Procedures of SPRT for Three-category Classification Decisions in Computerized Classification Test |conference=Annual meeting of the American Educational Research Association |___location=San Antonio |date=April 2004}}

* {{Cite journal |last=Kalohn |first=J.C. |last2=Spray |first2=J.A. |year=1999 |title=The effect of model misspecification on classification decisions made using a computerized test |journal=Journal of Educational Measurement |volume=36 |pages=47–59}}

* {{Cite report |last=Kingsbury |first=G.G. |last2=Weiss |first2=D.J. |year=1979 |title=An adaptive testing strategy for mastery decisions |type=Research report 79–05 |___location=Minneapolis |publisher=University of Minnesota, Psychometric Methods Laboratory}}

Line 62:

* {{Cite journal |last=Luecht |first=R.M. |year=1996 |title=Multidimensional Computerized Adaptive Testing in a Certification or Licensure Context |journal=Applied Psychological Measurement |volume=20 |pages=389–404}}

* {{Cite book |authorlink=Mark Reckase |last=Reckase |first=M.D. |year=1983 |chapter=A procedure for decision making using tailored testing |editor-first=D.J. |editor-last=Weiss |title=New horizons in testing: Latent trait theory and computerized adaptive testing |pages=237–254 |___location=New York |publisher=Academic Press}}

* {{Cite conference |last=Rudner |first=L.M. |year=2002 |title=An examination of decision-theory adaptive testing procedures |conference=Annual meeting of the American Educational Research Association |date=1–5 April 2002 |___location=New Orleans, LA}}

* {{Cite journal |last=Sheehan |first=K. |last2=Lewis |first2=C. |year=1992 |title=Computerized mastery testing with nonequivalent testlets |journal=Applied Psychological Measurement |volume=16 |pages=65–76}}

* {{Cite report |last=Spray |first=J.A. |year=1993 |title=Multiple-category classification using a sequential probability ratio test |type=Research Report 93–7 |___location=Iowa City, Iowa |publisher=ACT, Inc.}}

* {{Cite report |last=Spray |first=J.A. |last2=Abdel-fattah |first2=A.A. |last3=Huang |first3=C. |last4=Lau |first4=C.A. |year=1997 |title=Unidimensional approximations for a computerized test when the item pool and latent space are multidimensional |type=Research Report 97–5 |___location=Iowa City, Iowa |publisher=ACT, Inc.}}

* {{Cite report |last=Spray |first=J.A. |last2=Reckase |first2=M.D. |year=1987 |title=The effect of item parameter estimation error on decisions made using the sequential probability ratio test |type=Research Report 87–17 |___location=Iowa City, IA |publisher=ACT, Inc.}}

* {{Cite conference |last=Spray |first=J.A. |last2=Reckase |first2=M.D. |year=1994 |title=The selection of test items for decision making with a computerized adaptive test |conference=Annual Meeting of the National Council for Measurement in Education |___location=New Orleans, LA |date=5–7 April 1994}}

* {{Cite journal |last=Spray |first=J.A. |last2=Reckase |first2=M.D. |year=1996 |title=Comparison of SPRT and sequential Bayes procedures for classifying examinees into two categories using a computerized test |journal=Journal of Educational & Behavioral Statistics |volume=21 |pages=405–414}}

* {{Cite journal |last=Thompson |first=N.A. |year=2006 |title=Variable-length computerized classification testing with item response theory |journal=CLEAR Exam Review |volume=17 |issue=2}}

* {{Cite journal |last=Vos |first=H.J. |year=1998 |title=Optimal sequential rules for computer-based instruction |journal=Journal of Educational Computing Research |volume=19 |pages=133–154}}

* {{Cite journal |last=Vos |first=H.J. |year=1999 |title=Applications of Bayesian decision theory to sequential mastery testing |journal=Journal of Educational and Behavioral Statistics |volume=24 |pages=271–292}}

* {{Cite book |last=Wald |first=A. |year=1947 |title=Sequential analysis |url=https://archive.org/details/in.ernet.dli.2015.90255 |___location=New York |publisher=Wiley}}

* {{Cite journal |last=Weiss |first=D.J. |last2=Kingsbury |first2=G.G. |year=1984 |title=Application of computerized adaptive testing to educational problems |journal=Journal of Educational Measurement |volume=21 |pages=361–375}}

* {{Cite conference |last=Weissman |first=A. |year=2004 |title=Mutual information item selection in multiple-category classification CAT |conference=Annual Meeting of the National Council for Measurement in Education |___location=San Diego, CA}}

Line 80:

==External links==

* [{{cite web|url=http://edres.org/mdt/ |title=Measurement Decision Theory] by |first=Lawrence |last=Rudner|archive-url=https://web.archive.org/web/20240516104115/http://edres.org/mdt/|archive-date=16 May 2024|url-status=dead}}

* [{{cite web|archive-url=https://web.archive.org/web/20060711174833/http://www.psych.umn.edu/psylabs/catcentral/ |title=CAT Central] by |first=David J. |last=Weiss|archive-date=11 July 2006|url=http://www.psych.umn.edu/psylabs/catcentral/|url-status=dead}}

[[Category:Psychometrics]]