An ABA COVRATIO calculator asks what happens to the estimated joint uncertainty of a regression's coefficients when one case is left out. It compares two covariance-matrix determinants: one from the model without case i, and one from the full model. That is a deliberately narrow question. The ratio can point to a case whose presence changes the model's coefficient-uncertainty volume, but it cannot tell you that the case is wrong or that a coefficient is clinically important.

Clinicians & ABA Professionals / Data, Outcomes and Clinical Decision-Making.

The ratio is about a covariance volume

An ordinary least-squares model estimates several coefficients at once. Their covariance matrix carries estimated variances on the diagonal and covariances off the diagonal. Its determinant is sometimes called a generalized variance because it compresses that joint uncertainty into one positive scalar when the covariance matrix is positive definite. A zero or nonpositive computed determinant requires investigation rather than interpretation as an ordinary COVRATIO input.

For case i, COVRATIO divides the determinant after deleting the case by the determinant from the full fit. A value near 1 means the two determinant values are similar on this scale. A value below 1 means the deleted fit has a smaller determinant; a value above 1 means it has a larger one. Neither direction is inherently desirable. The result reflects both the row's predictor position and how its deletion changes residual variance.

This is not DFBETAS, which reports coefficient-specific standardized changes. It is not DFFITS, which standardizes a change in the fitted value at a case. Cook's distance addresses joint coefficient movement, leverage addresses predictor geometry, and PRESS addresses leave-one-out prediction error. COVRATIO should sit beside those diagnostics when their separate questions matter.

Freeze the model before calculating

A meaningful comparison requires the same response definition, predictor meanings, coding choices, row rules, and estimator on both sides of the ratio. Complete this record before inspecting any case result.

Model itemWhat to retainStop signalResponseOperational definition, unit, denominator, direction, and observation windowThe measured quantity or scale changesDesign matrixExact labeled columns, intercept rule, categorical coding, interactions, transformations, and weightsA column cannot be reconstructedRowsStable authorized key, population, inclusion/exclusion rules, phase, and missing-data handlingFull and deletion fits use different row logicFull fitFormula, coefficient order, residuals, SSE, residual degrees of freedom, MSE, and coefficient covariance matrixAny component came from another model versionDeleted fitsDeleted row key, rank, SSE(i), degrees of freedom, si^2, and covariance matrixA deletion fit is singular or undefinedNumerical methodSoftware, version, determinant method, tolerance, and displayed precisionA rounded or unstable determinant is reusedIntended usePrespecified descriptive coefficient-uncertainty reviewResults will automatically remove a row or drive a clinical action

Preserve the original observations and the full fit. If a data correction is later verified, retain both the original and corrected analyses with a dated rationale. COVRATIO does not supply that rationale by itself.

Definition and ordinary least-squares shortcut

Let n be the number of rows and p the number of estimated coefficients, including the intercept. For a full-rank unweighted ordinary least-squares design X, define:

s^2 = SSE / (n - p)

V = Cov(beta_hat) = s^2 (X'X)^-1

After removing row i, fit the same model to X(i) and y(i):

si^2 = SSE(i) / (n - p - 1)

V(i) = Cov(betahat(i)) = si^2 [X(i)'X(i)]^-1

The covariance ratio is:

COVRATIOi = det(V(i)) / det(V)

The current statsmodels influence source implements that determinant ratio directly from leave-one-observation-out covariance estimates. For the stated OLS setting, the matrix determinant lemma provides an equivalent route:

COVRATIOi = (si^2 / s^2)^p / (1 - h_ii)

Here h_ii is the case's diagonal hat value from the full design. Agreement between the explicit covariance-matrix route and the shortcut is a useful implementation check. It is not evidence that the model, sampling unit, or interpretation is valid.

Blank coefficient-covariance influence worksheet

A completed ABA COVRATIO calculator needs the full and deleted covariance evidence on the same row. The blank table below keeps the determinant route beside the OLS shortcut so a reviewer can reconcile them before interpreting the ratio.

CaseAuthorized keyFull s^2Deleted s_i^2h_iidet(V)det(V_(i))Explicit ratioShortcut ratioContext note123...

Keep a separate model register so the determinant never becomes detached from the matrix it summarizes.

Register fieldFull fitDeletion fit for case iRow count and residual degrees of freedomParameter count and exact coefficient orderDesign rank and condition diagnosticResidual sum of squares and residual MSECovariance estimator and weight conventionLog determinant or determinant, sign, and precisionSoftware, version, code/workbook revision, analyst, and date

For very small or large determinants, compute a signed log determinant or another documented stable equivalent rather than relying on a rounded printed determinant. When both signs are positive, calculate log(COVRATIOi) = log(det(V(i))) - log(det(V)) and exponentiate only for the final displayed ratio. Numerator and denominator must use the same convention.

Fictional ten-case calculation

This arithmetic example uses no client information. The design contains an intercept and two unlabeled synthetic predictors; p = 3 and n = 10.

RowPredictor APredictor BResponse1125225733164471055396681377412881018996151010924

The full fit gives beta_hat = [0.7405797101, 1.3293075684, 0.6996779388], SSE = 23.4901771337, residual degrees of freedom 7, and s^2 = 3.3557395905. The determinant of the estimated coefficient covariance matrix is 0.0010141960.

Caseh_iiDeleted s_i^2det(V_(i))COVRATIO10.35040257653.31383242440.00150350361.482458621920.32463768123.91244105340.00237992762.346614992030.35161030603.83098086290.00232729262.294716768840.26666666673.70750988140.00186510241.838995940150.20668276973.86537659140.00195382521.926476850960.20668276973.55237547500.00151658531.495357204170.26666666673.75296442690.00193454601.907467568080.35161030603.87949280460.00241682892.382999733790.32463768123.02188311340.00109661281.0812631553100.35040257650.46075028920.00000404120.0039846282

For case 10, the shortcut calculation is:

(0.4607502892 / 3.3557395905)^3 / (1 - 0.3504025765) = 0.0039846282

An explicit deletion refit produces the same ratio. The small result means the estimated covariance determinant from the nine-row fit is much smaller than the full-fit determinant. It does not reveal why. The row might be valid, the model form might be fragile, the response might require verification, or the simple synthetic design might not represent the relation well.

Changing only the last fictional response from 24 to 20 changes the full MSE to 0.6471129515, the full covariance determinant to 0.0000072727, and case 10 COVRATIO to 0.5556640242. That comparison shows how strongly this statistic depends on the exact observed data. It does not identify which fictional value is correct.

Reading values on both sides of one

The numerator and denominator each combine a residual-variance estimate with design geometry. A ratio below one can result when deleting a case sharply lowers the residual scale, even though removing a row also reduces design information. A ratio above one can result when the information loss dominates or when the deleted residual estimate rises. Read the underlying pieces rather than attaching a generic label to the direction.

Some texts or software use investigation bands based on expressions such as 3p/n or 3p/(n-p). Those rules differ, become awkward in small samples, and are not universal clinical cutoffs. If a team uses a heuristic, preserve its exact source, formula, parameter-count convention, model scope, and prespecified role. Crossing it means “review this case and model,” not “delete this case.”

The R regression deletion documentation presents covariance ratios alongside DFBETAS, DFFITS, Cook's distance, leverage, and residual diagnostics. That placement matters: a determinant ratio does not show which coefficient moved, how fitted values changed, or whether the case has a large residual.

Invariance and reconciliation checks

CheckExpected OLS resultWhat it does not establishAdd a constant to every response with an interceptCOVRATIO values remain the sameResponse meaning is unchangedMultiply every response by a nonzero constantNumerator and denominator scale equally; ratios remain the sameUnits are clinically interchangeableApply a full-rank invertible recoding to design columnsRatios remain the same when both fits use the identical recodingCoefficient interpretations remain the samePermute complete rowsValues follow stable case keysTemporal or clustered dependence can be ignoredCompare explicit deleted covariance determinants with the leverage shortcutValues agree within a declared numerical toleranceModel assumptions are satisfiedCompare with a current trusted libraryResults agree after conventions are reconciledThe library determines clinical use

Run these checks with full precision. If a spreadsheet displays ten decimal places, keep more internally. Never copy displayed rounded determinants back into the formula.

Stop conditions are part of the calculator

Do not return a numeric ratio when any required component is undefined or misleading.

  • If n - p or n - p - 1 is not positive, there is no appropriate residual mean-square denominator for this worksheet.
  • If the full or deleted design is rank deficient, the ordinary inverse and the covariance matrix required by this calculation are not available as specified.
  • If h_ii is one or numerically indistinguishable from one, the shortcut denominator collapses; investigate the design rather than capping the value.
  • If the covariance determinant is zero, negative, nonfinite, or dominated by numerical conditioning, stop and resolve the model or numerical method.
  • If weights, robust/sandwich covariance, generalized models, penalization, mixed effects, or imputation are involved, do not insert those outputs into the unweighted OLS formula without a method-specific derivation.
  • If deletion changes coding, factor levels, transformations, imputation, row eligibility, or the estimand, the two matrices no longer answer the promised comparison.

The statsmodels COVRATIO documentation describes a leave-one-observation-out calculation. Repeated measures, clients nested in clinicians, sites, phases, or other dependent clusters may make the observation-level deletion unit inappropriate. A qualified statistician should select a model and deletion unit that reflect the data-generating structure.

From a flagged case back to clinical evidence

Begin with provenance. Verify the row key, response calculation, denominator, observation window, units, coding, missingness, and any transformation. Then inspect direct graphs and notes for phase changes, opportunity shifts, treatment integrity, schedule, staffing, setting events, measurement changes, and other context.

Review coefficient estimates and their meanings, the residual and leverage plots, DFBETAS, DFFITS, Cook's distance, and condition diagnostics as appropriate. Preserve the full fit and any prespecified sensitivity fit. A changed conclusion after a justified correction or alternative model should be documented; an undocumented search for a preferred conclusion should not be presented as validation.

COVRATIO cannot establish treatment effect, experimental control, prediction accuracy, social validity, measurement reliability, or causation. A model may have stable covariance ratios and still be poorly specified, clinically unhelpful, or based on inadequate measurement. Conversely, an unusual ratio may identify a valid and important context that deserves explanation rather than removal.

Ethics, communication, and protected data

Use the BACB Ethics Codes to structure review of professional roles, competence, confidentiality, and consumer protection. The BCBA Test Content Outline situates data-based decisions among measurement, graphing, interpretation, and experimental design. For broader questions about evidence and intended use, consult the Standards for Educational and Psychological Testing. These materials set useful boundaries; they neither validate this worksheet nor turn a determinant ratio into a clinical directive.

Explain the result in plain language: which documented model was examined, which row was omitted, how the coefficient-covariance determinant changed, what other evidence was reviewed, and what remains uncertain. Include client and caregiver perspectives when interpreting clinical relevance and accessibility needs.

Use synthetic or appropriately de-identified values for learning and software testing. Authorized identifiable information belongs only in an approved environment with documented access, minimization, retention, and disposal controls. HHS explains the distinct safeguards in its Privacy Rule summary and Security Rule summary; neither page certifies a workbook, script, or analytics environment.

Related resources

Sources