CAT

Multidimensional Computer Adaptive Testing (MCAT): Procedure and Process

irufano · · 13 min read

Abstract: Multidimensional Computer Adaptive Testing (MCAT) extends the classical unidimensional CAT framework to simultaneously estimate multiple latent traits. This document details the theoretical foundations, algorithmic procedures, item selection strategies, ability estimation methods, and stopping rules that govern the MCAT process, with reference to seminal and contemporary literature.


1. Theoretical Background

Multidimensional Computer Adaptive Testing (MCAT) is a psychometric framework that generalizes unidimensional CAT (UCAT) to settings where examinees possess a vector of latent traits rather than a single ability [1][2]. The fundamental motivation is that most cognitive, psychological, and educational constructs are inherently multifaceted; a single scalar cannot adequately characterize proficiency in, for example, mathematics (algebra, geometry, statistics) or language ability (reading, grammar, vocabulary) [3].

MCAT was formalized extensively by Reckase [4] through the development of Multidimensional Item Response Theory (MIRT), which provides the probabilistic measurement model underlying all MCAT procedures. The adaptive component ensures that items administered to an examinee are optimally informative given the current estimate of the ability vector θ\boldsymbol{\theta} [1][5].


2. Item Response Theory in Multiple Dimensions

2.1 The Latent Trait Vector

In MCAT, each examinee is characterized by a KK-dimensional ability vector:

θ=(θ1,θ2,,θK)RK\boldsymbol{\theta} = (\theta_1, \theta_2, \ldots, \theta_K)^{\top} \in \mathbb{R}^K

Variable Notes:

Symbol Description
θ\boldsymbol{\theta} Latent trait (ability) vector
θk\theta_k Latent ability on dimension kk
KK Total number of dimensions (latent traits)

2.2 Multidimensional Item Response Model

The most widely used model in MCAT is the Multidimensional Two-Parameter Logistic (M2PL) model [4][6]:

P(Xij=1θj,ai,di)=exp(aiθj+di)1+exp(aiθj+di)P(X_{ij} = 1 \mid \boldsymbol{\theta}_j, \mathbf{a}_i, d_i) = \frac{\exp(\mathbf{a}_i^{\top} \boldsymbol{\theta}_j + d_i)}{1 + \exp(\mathbf{a}_i^{\top} \boldsymbol{\theta}_j + d_i)}

Variable Notes:

Symbol Description
XijX_{ij} Binary response of examinee jj to item ii (1 = correct, 0 = incorrect)
θj\boldsymbol{\theta}_j Ability vector of examinee jj
ai=(ai1,ai2,,aiK)\mathbf{a}_i = (a_{i1}, a_{i2}, \ldots, a_{iK})^{\top} Discrimination parameter vector for item ii on each dimension
did_i Scalar intercept (difficulty) parameter for item ii
aiθj\mathbf{a}_i^{\top} \boldsymbol{\theta}_j Dot product: k=1Kaikθjk\sum_{k=1}^{K} a_{ik} \theta_{jk}

For the Multidimensional Three-Parameter Logistic (M3PL) model with guessing [4][7]:

P(Xij=1θj)=ci+(1ci)exp(aiθj+di)1+exp(aiθj+di)P(X_{ij} = 1 \mid \boldsymbol{\theta}_j) = c_i + (1 - c_i) \cdot \frac{\exp(\mathbf{a}_i^{\top} \boldsymbol{\theta}_j + d_i)}{1 + \exp(\mathbf{a}_i^{\top} \boldsymbol{\theta}_j + d_i)}

Variable Notes:

Symbol Description
cic_i Pseudo-guessing parameter for item ii (0ci<10 \leq c_i < 1)

2.3 Item Information in Multiple Dimensions

The Fisher Information Matrix (FIM) for item ii given ability θ\boldsymbol{\theta} is a K×KK \times K matrix [1][8]:

Ii(θ)=[Pi(θ)]2Pi(θ)Qi(θ)aiai\mathbf{I}_i(\boldsymbol{\theta}) = \frac{[P'_i(\boldsymbol{\theta})]^2}{P_i(\boldsymbol{\theta})\, Q_i(\boldsymbol{\theta})} \cdot \mathbf{a}_i \mathbf{a}_i^{\top}

where:

Pi(θ)=Pi(θ)(aiθ)=Pi(θ)Qi(θ)P'_i(\boldsymbol{\theta}) = \frac{\partial P_i(\boldsymbol{\theta})}{\partial (\mathbf{a}_i^{\top} \boldsymbol{\theta})} = P_i(\boldsymbol{\theta})\, Q_i(\boldsymbol{\theta})

Variable Notes:

Symbol Description
Ii(θ)\mathbf{I}_i(\boldsymbol{\theta}) K×KK \times K Fisher Information Matrix for item ii
Pi(θ)P_i(\boldsymbol{\theta}) Probability of correct response to item ii
Qi(θ)=1Pi(θ)Q_i(\boldsymbol{\theta}) = 1 - P_i(\boldsymbol{\theta}) Probability of incorrect response
Pi(θ)P'_i(\boldsymbol{\theta}) Derivative of PiP_i with respect to the linear predictor
aiai\mathbf{a}_i \mathbf{a}_i^{\top} Outer product of the discrimination vector (rank-1 matrix)

The cumulative FIM after administering nn items is:

I(n)(θ)=i=1nIi(θ)\mathbf{I}_{(n)}(\boldsymbol{\theta}) = \sum_{i=1}^{n} \mathbf{I}_i(\boldsymbol{\theta})

3. The MCAT Process Overview

The following diagram illustrates the complete MCAT procedure from initialization to termination:

flowchart TD A([Start: Examinee Begins Test]) --> B[Initialize Ability Estimate\n θ̂ = θ₀, e.g., 0-vector] B --> C[Select Starting Items\nfrom Item Bank] C --> D{Item Bank\nAvailable?} D -- No --> Z([Error: Insufficient Items]) D -- Yes --> E[Select Optimal Item\nusing Selection Criterion] E --> F[Administer Item to Examinee] F --> G["Record Response\nXᵢ ∈ {0, 1}"] G --> H[Update Ability Estimate\nθ̂ via MLE / MAP / EAP] H --> I[Update Fisher\nInformation Matrix I_n] I --> J{Stopping Rule\nSatisfied?} J -- No --> K{Exposure &\nContent Constraints Met?} K -- Yes --> E K -- No --> L[Apply Constrained\nItem Selection] L --> E J -- Yes --> M[Final Ability Estimate\nθ̂_final with SE] M --> N[Generate Score Report] N --> O([End: Test Complete]) style A fill:#2d6a4f,color:#fff style O fill:#2d6a4f,color:#fff style Z fill:#b5192b,color:#fff style J fill:#1d3557,color:#fff style K fill:#1d3557,color:#fff

4. Step-by-Step Procedure

Step 1: Initialization

Before any item is administered, the system establishes:

  • Prior ability distribution: θ0N(μ0,Σ0)\boldsymbol{\theta}_0 \sim \mathcal{N}(\boldsymbol{\mu}_0, \boldsymbol{\Sigma}_0), typically μ0=0\boldsymbol{\mu}_0 = \mathbf{0}, Σ0=IK\boldsymbol{\Sigma}_0 = \mathbf{I}_K (identity matrix) [2][9]
  • Item bank: A calibrated pool B\mathcal{B} of MM items with known MIRT parameters {ai,di,ci}\{\mathbf{a}_i, d_i, c_i\}
  • Starting ability estimate: θ^(0)=μ0\hat{\boldsymbol{\theta}}^{(0)} = \boldsymbol{\mu}_0

Step 2: Item Selection

At step nn, the item ii^* is selected from the remaining bank Bn=B{i1,,in1}\mathcal{B}_n = \mathcal{B} \setminus \{i_1, \ldots, i_{n-1}\} using a selection criterion S\mathcal{S}:

i=argmaxiBn  S(Ii(θ^(n1)))i^* = \underset{i \in \mathcal{B}_n}{\arg\max} \; \mathcal{S}\left(\mathbf{I}_i(\hat{\boldsymbol{\theta}}^{(n-1)})\right)

Common criteria are detailed in Section 6.

Step 3: Item Administration

Item ii^* is presented to the examinee who provides response xi{0,1}x_{i^*} \in \{0, 1\} (for dichotomous items) or xi{0,1,,mi}x_{i^*} \in \{0, 1, \ldots, m_i\} (polytomous items).

Step 4: Ability Re-estimation

The ability vector is updated using accumulated response vector x(n)=(xi1,,xin)\mathbf{x}^{(n)} = (x_{i_1}, \ldots, x_{i_n})^{\top}.

Log-likelihood function [1][6]:

(θx(n))=t=1n[xitlnPit(θ)+(1xit)lnQit(θ)]\ell(\boldsymbol{\theta} \mid \mathbf{x}^{(n)}) = \sum_{t=1}^{n} \left[ x_{i_t} \ln P_{i_t}(\boldsymbol{\theta}) + (1 - x_{i_t}) \ln Q_{i_t}(\boldsymbol{\theta}) \right]

Details of estimation methods are in Section 5.

Step 5: Update Information Matrix

I(n)(θ^(n))=t=1nIit(θ^(n))\mathbf{I}_{(n)}(\hat{\boldsymbol{\theta}}^{(n)}) = \sum_{t=1}^{n} \mathbf{I}_{i_t}(\hat{\boldsymbol{\theta}}^{(n)})

Step 6: Check Stopping Rule

Evaluate whether stopping criteria are satisfied (see Section 7). If yes → proceed to scoring; if no → return to Step 2.

Step 7: Score Reporting

Provide the final estimate θ^final\hat{\boldsymbol{\theta}}_{\text{final}} along with the standard error vector:

SE(θ^)=diag[I(n)1(θ^)]\text{SE}(\hat{\boldsymbol{\theta}}) = \sqrt{\text{diag}\left[\mathbf{I}_{(n)}^{-1}(\hat{\boldsymbol{\theta}})\right]}

Variable Notes:

Symbol Description
SE(θ^)\text{SE}(\hat{\boldsymbol{\theta}}) Vector of standard errors for each dimension estimate
diag[]\text{diag}[\cdot] Diagonal extraction operator
I(n)1\mathbf{I}_{(n)}^{-1} Inverse of the cumulative FIM (posterior covariance approximation)

5. Ability Estimation Methods

5.1 Maximum Likelihood Estimation (MLE)

MLE finds θ^\hat{\boldsymbol{\theta}} by maximizing the log-likelihood [1][10]:

θ^MLE=argmaxθ  (θx(n))\hat{\boldsymbol{\theta}}_{\text{MLE}} = \underset{\boldsymbol{\theta}}{\arg\max} \; \ell(\boldsymbol{\theta} \mid \mathbf{x}^{(n)})

The score function (gradient):

s(θ)=θ(θx(n))=t=1nxitPit(θ)Pit(θ)Qit(θ)Pit(θ)ait\mathbf{s}(\boldsymbol{\theta}) = \nabla_{\boldsymbol{\theta}} \ell(\boldsymbol{\theta} \mid \mathbf{x}^{(n)}) = \sum_{t=1}^{n} \frac{x_{i_t} - P_{i_t}(\boldsymbol{\theta})}{P_{i_t}(\boldsymbol{\theta}) Q_{i_t}(\boldsymbol{\theta})} \cdot P'_{i_t}(\boldsymbol{\theta}) \cdot \mathbf{a}_{i_t}

Solved via Newton-Raphson iteration [10]:

θ^(r+1)=θ^(r)+[I(n)(θ^(r))]1s(θ^(r))\hat{\boldsymbol{\theta}}^{(r+1)} = \hat{\boldsymbol{\theta}}^{(r)} + \left[\mathbf{I}_{(n)}(\hat{\boldsymbol{\theta}}^{(r)})\right]^{-1} \mathbf{s}(\hat{\boldsymbol{\theta}}^{(r)})

Variable Notes:

Symbol Description
rr Iteration index in Newton-Raphson
s(θ)\mathbf{s}(\boldsymbol{\theta}) Score function (gradient of log-likelihood)

⚠️ Limitation: MLE is undefined when all responses are correct or all incorrect (degenerate response patterns) [2].

5.2 Maximum A Posteriori (MAP) Estimation

MAP incorporates a prior distribution g(θ)g(\boldsymbol{\theta}) (typically multivariate normal) [2][9]:

θ^MAP=argmaxθ[(θx(n))+lng(θ)]\hat{\boldsymbol{\theta}}_{\text{MAP}} = \underset{\boldsymbol{\theta}}{\arg\max} \left[ \ell(\boldsymbol{\theta} \mid \mathbf{x}^{(n)}) + \ln g(\boldsymbol{\theta}) \right]

With a multivariate normal prior θN(μ0,Σ0)\boldsymbol{\theta} \sim \mathcal{N}(\boldsymbol{\mu}_0, \boldsymbol{\Sigma}_0):

lng(θ)=12(θμ0)Σ01(θμ0)+const\ln g(\boldsymbol{\theta}) = -\frac{1}{2}(\boldsymbol{\theta} - \boldsymbol{\mu}_0)^{\top} \boldsymbol{\Sigma}_0^{-1} (\boldsymbol{\theta} - \boldsymbol{\mu}_0) + \text{const}

The modified Newton-Raphson step becomes:

θ^MAP(r+1)=θ^(r)+[I(n)(θ^(r))+Σ01]1[s(θ^(r))Σ01(θ^(r)μ0)]\hat{\boldsymbol{\theta}}^{(r+1)}_{\text{MAP}} = \hat{\boldsymbol{\theta}}^{(r)} + \left[\mathbf{I}_{(n)}(\hat{\boldsymbol{\theta}}^{(r)}) + \boldsymbol{\Sigma}_0^{-1}\right]^{-1} \left[\mathbf{s}(\hat{\boldsymbol{\theta}}^{(r)}) - \boldsymbol{\Sigma}_0^{-1}(\hat{\boldsymbol{\theta}}^{(r)} - \boldsymbol{\mu}_0)\right]

Variable Notes:

Symbol Description
g(θ)g(\boldsymbol{\theta}) Prior density of ability vector
μ0\boldsymbol{\mu}_0 Prior mean vector (often 0\mathbf{0})
Σ0\boldsymbol{\Sigma}_0 Prior covariance matrix
Σ01\boldsymbol{\Sigma}_0^{-1} Precision matrix of the prior

5.3 Expected A Posteriori (EAP) Estimation

EAP computes the posterior mean [9][11]:

θ^EAP=E[θx(n)]=θL(x(n)θ)g(θ)dθL(x(n)θ)g(θ)dθ\hat{\boldsymbol{\theta}}_{\text{EAP}} = \mathbb{E}[\boldsymbol{\theta} \mid \mathbf{x}^{(n)}] = \frac{\int \boldsymbol{\theta} \cdot L(\mathbf{x}^{(n)} \mid \boldsymbol{\theta}) \cdot g(\boldsymbol{\theta}) \, d\boldsymbol{\theta}}{\int L(\mathbf{x}^{(n)} \mid \boldsymbol{\theta}) \cdot g(\boldsymbol{\theta}) \, d\boldsymbol{\theta}}

where the likelihood:

L(x(n)θ)=t=1nPit(θ)xitQit(θ)1xitL(\mathbf{x}^{(n)} \mid \boldsymbol{\theta}) = \prod_{t=1}^{n} P_{i_t}(\boldsymbol{\theta})^{x_{i_t}} Q_{i_t}(\boldsymbol{\theta})^{1 - x_{i_t}}

In practice, EAP is computed via Gauss-Hermite quadrature or Monte Carlo integration over a grid of quadrature points {θ(q),w(q)}\{\boldsymbol{\theta}^{(q)}, w^{(q)}\} [9]:

θ^EAPqθ(q)L(x(n)θ(q))g(θ(q))w(q)qL(x(n)θ(q))g(θ(q))w(q)\hat{\boldsymbol{\theta}}_{\text{EAP}} \approx \frac{\sum_{q} \boldsymbol{\theta}^{(q)} \cdot L(\mathbf{x}^{(n)} \mid \boldsymbol{\theta}^{(q)}) \cdot g(\boldsymbol{\theta}^{(q)}) \cdot w^{(q)}}{\sum_{q} L(\mathbf{x}^{(n)} \mid \boldsymbol{\theta}^{(q)}) \cdot g(\boldsymbol{\theta}^{(q)}) \cdot w^{(q)}}

Variable Notes:

Symbol Description
θ(q)\boldsymbol{\theta}^{(q)} qq-th quadrature point in the KK-dimensional grid
w(q)w^{(q)} Quadrature weight for point qq
L(x(n)θ)L(\mathbf{x}^{(n)} \mid \boldsymbol{\theta}) Likelihood of observed responses given θ\boldsymbol{\theta}

6. Item Selection Criteria

6.1 Maximum Determinant (D-optimality)

Select the item that maximizes the determinant of the updated FIM [1][8][12]:

i=argmaxiBn  det[I(n1)(θ^)+Ii(θ^)]i^* = \underset{i \in \mathcal{B}_n}{\arg\max} \; \det\left[\mathbf{I}_{(n-1)}(\hat{\boldsymbol{\theta}}) + \mathbf{I}_i(\hat{\boldsymbol{\theta}})\right]

Interpretation: Maximizes the volume of the confidence ellipsoid's reciprocal — reduces overall estimation uncertainty across all dimensions simultaneously.

6.2 Minimum Trace of Posterior Covariance (T-optimality / A-optimality)

i=argminiBn  tr[(I(n1)(θ^)+Ii(θ^))1]i^* = \underset{i \in \mathcal{B}_n}{\arg\min} \; \text{tr}\left[\left(\mathbf{I}_{(n-1)}(\hat{\boldsymbol{\theta}}) + \mathbf{I}_i(\hat{\boldsymbol{\theta}})\right)^{-1}\right]

Interpretation: Minimizes the sum of posterior variances across all KK dimensions [5][13].

Variable Notes:

Symbol Description
tr[]\text{tr}[\cdot] Matrix trace operator (sum of diagonal elements)
det[]\det[\cdot] Matrix determinant

6.3 Kullback-Leibler Information (KL-criterion)

Maximizes the expected Kullback-Leibler divergence between item response distributions at the current estimate and neighboring ability values [14]:

KLi(θ^)=Vx=01P(Xi=xθ^)lnP(Xi=xθ^)P(Xi=xθ)dθ\text{KL}_i(\hat{\boldsymbol{\theta}}) = \int_{\mathcal{V}} \sum_{x=0}^{1} P(X_i = x \mid \hat{\boldsymbol{\theta}}) \ln \frac{P(X_i = x \mid \hat{\boldsymbol{\theta}})}{P(X_i = x \mid \boldsymbol{\theta})} \, d\boldsymbol{\theta} i=argmaxiBn  KLi(θ^)i^* = \underset{i \in \mathcal{B}_n}{\arg\max} \; \text{KL}_i(\hat{\boldsymbol{\theta}})

Variable Notes:

Symbol Description
V\mathcal{V} Neighborhood region around θ^\hat{\boldsymbol{\theta}}
KLi(θ^)\text{KL}_i(\hat{\boldsymbol{\theta}}) KL information for item ii at current ability estimate

6.4 Mutual Information Criterion

Selects items that maximize the mutual information between the item response and the ability vector [15]:

i=argmaxiBn  I(Xi;θx(n1))i^* = \underset{i \in \mathcal{B}_n}{\arg\max} \; \mathbb{I}(X_i ; \boldsymbol{\theta} \mid \mathbf{x}^{(n-1)})

Summary Comparison

graph LR A[Item Selection Criteria] --> B[D-optimality\n det of FIM] A --> C[A-optimality\n trace of FIM⁻¹] A --> D[KL-criterion\n information gain] A --> E[Mutual Information\n Bayesian criterion] B --> F[Best for joint\n precision] C --> G[Best for average\n dimension precision] D --> H[Best for local\n discrimination] E --> I[Best for fully\n Bayesian settings]

7. Stopping Rules

7.1 Fixed Test Length

The simplest rule: terminate after exactly NmaxN_{\max} items [1]:

Stop if n=Nmax\text{Stop if } n = N_{\max}

7.2 Standard Error Threshold

Terminate when the standard error for all dimensions falls below a threshold ϵ\epsilon [2][5]:

Stop if maxk{1,,K}SE(θ^k)ϵ\text{Stop if } \max_{k \in \{1,\ldots,K\}} \text{SE}(\hat{\theta}_k) \leq \epsilon

Or alternatively for the joint criterion using the posterior covariance matrix:

Stop if tr[I(n)1(θ^)]ϵjoint2\text{Stop if } \text{tr}\left[\mathbf{I}_{(n)}^{-1}(\hat{\boldsymbol{\theta}})\right] \leq \epsilon^2_{\text{joint}}

Variable Notes:

Symbol Description
ϵ\epsilon Standard error threshold (e.g., 0.30 on the logit scale)
ϵjoint2\epsilon^2_{\text{joint}} Joint variance threshold for all dimensions

7.3 Change in Ability Estimate

Terminate when successive ability estimates converge [16]:

Stop if θ^(n)θ^(n1)2δ\text{Stop if } \left\| \hat{\boldsymbol{\theta}}^{(n)} - \hat{\boldsymbol{\theta}}^{(n-1)} \right\|_2 \leq \delta

Variable Notes:

Symbol Description
2\|\cdot\|_2 Euclidean (L2) norm
δ\delta Convergence threshold (e.g., 0.01)

7.4 Minimum-Maximum Length Rule (Hybrid)

Combines fixed and SE-based rules for practical testing [2][5]:

Stop if nNmin AND (maxkSE(θ^k)ϵ OR n=Nmax)\text{Stop if } n \geq N_{\min} \text{ AND } \left(\max_k \text{SE}(\hat{\theta}_k) \leq \epsilon \text{ OR } n = N_{\max}\right)
flowchart LR A[After each item n] --> B{n ≥ N_min?} B -- No --> F[Continue] B -- Yes --> C{SE ≤ ε\nfor all k?} C -- Yes --> D([Stop: Precision Met]) C -- No --> E{n = N_max?} E -- Yes --> G([Stop: Max Length]) E -- No --> F

8. Item Exposure Control

Uncontrolled item selection leads to overexposure of highly informative items, compromising item security. Several methods address this [17][18]:

8.1 Sympson-Hetter Method (Randomization)

Each item ii is selected with probability rir_i, where rir_i is tuned so that the exposure rate ϱiϱmax\varrho_i \leq \varrho_{\max} [17]:

ri=min(1,ϱmaxϱi)r_i = \min\left(1, \frac{\varrho_{\max}}{\varrho_i^{*}}\right)

Variable Notes:

Symbol Description
ϱi\varrho_i Observed exposure rate of item ii
ϱmax\varrho_{\max} Maximum allowable exposure rate (e.g., 0.20)
ϱi\varrho_i^{*} Unconditional selection probability of item ii
rir_i Randomization parameter for item ii

8.2 Maximum Priority Index (MPI)

Uses a priority index PIi\text{PI}_i combining information and exposure [18]:

PIi=w1S(Ii(θ^))w2ϱi\text{PI}_i = w_1 \cdot \mathcal{S}(\mathbf{I}_i(\hat{\boldsymbol{\theta}})) - w_2 \cdot \varrho_i i=argmaxiBn  PIii^* = \underset{i \in \mathcal{B}_n}{\arg\max} \; \text{PI}_i

Variable Notes:

Symbol Description
w1,w2w_1, w_2 Weights balancing information gain vs. exposure penalization
S()\mathcal{S}(\cdot) Item selection criterion value (e.g., determinant)

9. Content Balancing

Real-world tests require that items cover specified content areas C={c1,c2,,cJ}\mathcal{C} = \{c_1, c_2, \ldots, c_J\} proportionally [19]. The constrained CAT problem is:

i=argmaxiBnCjeligible  S(Ii(θ^))i^* = \underset{i \in \mathcal{B}_n \cap \mathcal{C}_j^{\text{eligible}}}{\arg\max} \; \mathcal{S}\left(\mathbf{I}_i(\hat{\boldsymbol{\theta}})\right)

where Cjeligible\mathcal{C}_j^{\text{eligible}} is the set of items from content area cjc_j that can still be administered to meet the target distribution π=(π1,,πJ)\boldsymbol{\pi} = (\pi_1, \ldots, \pi_J)^{\top} [19][20].

The Shadow Test approach [20] solves a 0-1 integer programming problem at each step to select a full-length "shadow test" that satisfies all constraints, then administers only the optimal next item from it:

MaximizeiBsiS(Ii(θ^))\text{Maximize} \quad \sum_{i \in \mathcal{B}} s_i \cdot \mathcal{S}(\mathbf{I}_i(\hat{\boldsymbol{\theta}})) subject toiCjsi=nj,j=1,,J\text{subject to} \quad \sum_{i \in \mathcal{C}_j} s_i = n_j, \quad j = 1, \ldots, J si{0,1},isi=Nmaxs_i \in \{0, 1\}, \quad \sum_{i} s_i = N_{\max}

Variable Notes:

Symbol Description
sis_i Binary decision variable (1 if item ii included in shadow test)
njn_j Required number of items from content area jj
πj\pi_j Target proportion for content area jj

10. Comparison: Unidimensional vs. Multidimensional CAT

Feature Unidimensional CAT Multidimensional CAT
Latent space Scalar θR\theta \in \mathbb{R} Vector θRK\boldsymbol{\theta} \in \mathbb{R}^K
Item information Scalar Ii(θ)I_i(\theta) Matrix Ii(θ)RK×K\mathbf{I}_i(\boldsymbol{\theta}) \in \mathbb{R}^{K \times K}
Estimation MLE/MAP (1D optimization) MLE/MAP (K-D optimization)
Item selection Maximize Ii(θ^)I_i(\hat{\theta}) Maximize det/tr1\det / \text{tr}^{-1} of FIM
Stopping rule SE(θ^)ϵ\text{SE}(\hat{\theta}) \leq \epsilon maxkSE(θ^k)ϵ\max_k \text{SE}(\hat{\theta}_k) \leq \epsilon
Computational cost Low Higher (matrix operations)
Score report Single score + SE Score profile + SE vector
Between-dimension correlation Not applicable Corr(θj,θk)\text{Corr}(\theta_j, \theta_k) estimated

References

[1] Segall, D. O. (1996). Multidimensional adaptive testing. Psychometrika, 61(2), 331–354.

[2] Reckase, M. D., & Segall, D. O. (2009). Multidimensional adaptive testing. In W. J. van der Linden & C. A. W. Glas (Eds.), Elements of Adaptive Testing (pp. 203–217). Springer.

[3] van der Linden, W. J., & Hambleton, R. K. (Eds.). (1997). Handbook of Modern Item Response Theory. Springer.

[4] Reckase, M. D. (2009). Multidimensional Item Response Theory. Springer.

[5] Mulder, J., & van der Linden, W. J. (2009). Multidimensional adaptive testing with Kullback-Leibler information item selection. In W. J. van der Linden & C. A. W. Glas (Eds.), Elements of Adaptive Testing (pp. 77–101). Springer.

[6] McKinley, R. L., & Reckase, M. D. (1983). An extension of the two-parameter logistic model to the multidimensional latent space. ETS Research Report. Educational Testing Service.

[7] Reckase, M. D. (1985). The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9(4), 401–412.

[8] Berger, M. P. F. (1992). Sequential sampling designs for the two-parameter item response theory model. Psychometrika, 57(4), 521–538.

[9] Bock, R. D., & Mislevy, R. J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6(4), 431–444.

[10] Lord, F. M. (1980). Applications of Item Response Theory to Practical Testing Problems. Lawrence Erlbaum.

[11] Mislevy, R. J. (1986). Bayes modal estimation in item response models. Psychometrika, 51(2), 177–195.

[12] Silvey, S. D. (1980). Optimal Design. Chapman and Hall.

[13] van der Linden, W. J. (1999). Multidimensional adaptive testing with a minimum error-variance criterion. Journal of Educational and Behavioral Statistics, 24(4), 398–412.

[14] Chang, H.-H., & Ying, Z. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20(3), 213–229.

[15] Cover, T. M., & Thomas, J. A. (2006). Elements of Information Theory (2nd ed.). Wiley.

[16] Weiss, D. J. (1982). Improving measurement quality and efficiency with adaptive testing. Applied Psychological Measurement, 6(4), 473–492.

[17] Sympson, J. B., & Hetter, R. D. (1985). Controlling item-exposure rates in computerized adaptive testing. Proceedings of the 27th Annual Meeting of the Military Testing Association.

[18] Leung, C. K., Chang, H.-H., & Hau, K.-T. (2002). Item selection in computerized adaptive testing: Improving the a-stratified design with the Sympson-Hetter algorithm. Applied Psychological Measurement, 26(4), 376–392.

[19] Stocking, M. L., & Swanson, L. (1993). A method for severely constrained item selection in adaptive testing. Applied Psychological Measurement, 17(3), 277–292.

[20] van der Linden, W. J. (2005). Linear Models for Optimal Test Design. Springer.


Document prepared with reference to foundational MIRT and MCAT literature. All mathematical notation follows standard psychometric conventions. LaTeX formulas are rendered in Markdown-compatible environments (e.g., Obsidian, Jupyter, Pandoc with MathJax/KaTeX).

irufano — 2026