An ABA PRESS statistic calculator rebuilds the prediction question one row at a time: if a case had not helped fit the regression, how far would the model's prediction for that case have been from its observed response? The calculator keeps every leave-one-out residual visible, squares and sums them, and can express the total as a root-mean-square value in the response's units. It is an internal cross-validation diagnostic, not an independent validation sample or a clinical outcome score.

Clinicians & ABA Professionals / Data, Outcomes and Clinical Decision-Making.

PRESS changes who is allowed to train the prediction

An ordinary residual compares yi with the fitted value from a model that used row i. A PRESS residual compares yi with a model fitted to all the other rows. That distinction usually makes the PRESS contribution larger in magnitude, especially when the case has high leverage.

PRESS is commonly expanded as predictive residual sum of squares or predicted residual error sum of squares. The name matters less than an explicit definition. This page uses the squared sum of leave-one-out prediction residuals for one fixed, full-rank ordinary least-squares model.

The statistic does not answer the same question as COVRATIO, DFFITS, DFBETAS, Cook's distance, leverage, or a studentized residual. It also does not replace direct ABA measurement and graphing. Its job is to expose how a prespecified regression predicts each included row when that row is withheld from fitting.

Choose the validation unit before seeing the result

The most important input is not a number. It is the definition of a row. If rows are repeated observations from the same client, sessions nested within clinicians, or measurements ordered over time, leaving out one row may still leave very close relatives of that row in the training set. The resulting score may answer a much easier question than prediction for a new client, clinician, site, or future period.

Complete this analysis contract before calculating:

Contract fieldRequired detailPause conditionPrediction targetResponse definition, unit, denominator, direction, and windowThe target differs across rowsRow unitWhat one row represents and whether rows are independent, clustered, repeated, or orderedThe validation unit does not match the intended predictionDesignEvery predictor, coding rule, transformation, interaction, weight, and intercept conventionA feature is built differently in a deletion fitEligible dataStable authorized row keys, population, phase, inclusion/exclusion, and missing-data rulesRows are added or removed after reviewing errorsCandidate planModels to compare, scoring definition, tie rule, and any tuning procedureCandidate set grows in response to the same PRESS resultsFull fitFormula, fitted values, ordinary residuals, hat diagonal, rank, software, and versionOutputs come from different fitsIntended usePrespecified internal prediction diagnosticPRESS will be represented as causal or external validation evidence

If the intended question concerns new clients or sites, use a group-aware split. If it concerns the future, preserve time order. The scikit-learn cross-validation guidance describes leave-one-out as n training sets of n-1 samples and separately explains why dependent groups should not be shared across training and validation folds.

Formula for a full-rank OLS fit

For row i, let yi be the observed response, yhat_i the full-model fitted value, and:

ei = yi - yhati

Let h_ii be the row's diagonal hat value. In ordinary least squares, the leave-one-out or PRESS residual can be computed without manually fitting n models:

ePRESS,i = ei / (1 - h_ii)

The same quantity is obtained through the explicit deletion fit:

ePRESS,i = yi - xi' betahat_(i)

Then:

PRESS = sumi ePRESS,i^2

PRESS_RMSE = sqrt(PRESS / n)

The statsmodels influence source implements the residual shortcut and forms the error sum of squares from the residual vector. R's regression deletion documentation identifies its predictive residuals as leave-one-out cross-validation errors and verifies the same ei / (1-hii) identity for linear models.

PRESS_RMSE is a convenient rescaling of the specified sum; it divides by the number of held-out predictions n, not the residual degrees of freedom from the full fit, and has the response's unit. It is not a sampling uncertainty interval or a guarantee about a future population.

Copyable row-level worksheet

Use the ABA PRESS statistic calculator only after the response, row unit, and candidate plan are fixed. This table keeps the shortcut and explicit deletion prediction visible instead of reducing the analysis to one unexplained total.

CaseAuthorized keyObserved y_iFull fitted yhatiOrdinary e_ih_iiShortcut PRESS residualDeleted-fit predictionDifference between routesSquared contributionContext note123...

Finish with a comparison register rather than a free-floating minimum score.

Candidate modelExact response and rowsPredictor/coding versionValidation unit and splitPRESSPRESS RMSEPrespecified eligibility met?Interpretation boundary

Archive the original data version, code or workbook revision, analyst, reviewer, date, full precision, row exclusions, and any tuning performed. Models are comparable only when the target, row set, preprocessing, weighting, and validation procedure are genuinely like for like.

Fictional twelve-row example

This arithmetic-only example is not an actual client, treatment, or valid clinical model. The design has an intercept, a synthetic period number 1...12, and a binary phase column that equals 0 for the first six rows and 1 for the last six. Because those rows are ordered and phase-structured, the example is not an endorsement of row-wise leave-one-out as a valid evaluation design for an actual repeated clinical series.

CasePeriodPhaseObserved y110522073306440855096609771118811299112101011411111131212118

The full fit returns beta_hat = [3.9333333333, 0.9714285714, 0.1714285714], SSE = 11.6380952381, residual degrees of freedom 9, and MSE = 1.2931216931.

CaseFull fittedOrdinary residualh_iiPRESS residualDeleted-fit predictionSquared contribution14.90476190480.09523809520.34523809520.14545454554.85454545450.021157024825.87619047621.12380952380.23095238101.46130030965.53869969042.135398594836.8476190476-0.84761904760.1738095238-1.02593659947.02593659941.052545906047.81904761900.18095238100.17380952380.21902017297.78097982710.047969836158.79047619050.20952380950.23095238100.27244582048.72755417960.074226725169.7619047619-0.76190476190.3452380952-1.163636363610.16363636361.3540495868710.90476190480.09523809520.34523809520.145454545510.85454545450.0211570248811.87619047620.12380952380.23095238100.160990712111.83900928790.0259180094912.8476190476-0.84761904760.1738095238-1.025936599413.02593659941.05254590601013.81904761900.18095238100.17380952380.219020172913.78097982710.04796983611114.7904761905-1.79047619050.2309523810-2.328173374615.32817337465.42039126231215.76190476192.23809523810.34523809523.418181818214.581818181811.6839669421

For case 12:

2.2380952381 / (1 - 0.3452380952) = 3.4181818182

The deleted-fit prediction is 14.5818181818, so 18 - 14.5818181818 gives the same residual. Summing all twelve squared contributions yields PRESS = 22.9372966544. Dividing by 12 and taking the square root gives PRESS_RMSE = 1.3825488736 response units.

If the final fictional response is changed from 18 to 15, the refitted model has SSE = 4.1023809524, PRESS = 6.7325497080, and PRESS_RMSE = 0.7490299120. That sensitivity is not proof that 15 is correct, that the original row should be deleted, or that the refitted model will generalize.

PRESS and SSE should not be interchanged

Full-fit SSE uses predictions from a model that saw every row. PRESS uses a different deletion fit for each row. In the fictional example, SSE is 11.6380952381, while PRESS is 22.9372966544. The difference is expected; it does not by itself diagnose overfitting or model failure.

Comparing raw PRESS across different response scales or different row sets is not meaningful. A total can rise simply because there are more rows. PRESS_RMSE makes the total easier to read in one response unit, but it still requires the same target definition and a suitable validation design.

A lower PRESS among prespecified candidates may support a prediction-focused comparison when all candidates use the same rows, response, preprocessing, and validation unit. It does not establish content validity, measurement reliability, causal attribution, social importance, or superiority for a different population. Reusing the same leave-one-out results to invent and choose many models introduces selection optimism that the reported number does not reveal.

Checks for a trustworthy implementation

TestExpected resultRemaining limitAdd a constant to every response with an interceptPRESS residuals and total remain unchangedThe response retains the same meaningMultiply the response by cResiduals multiply by c, squared contributions and PRESS by c^2, RMSE by abs(c)Units are clinically comparableApply an invertible full-rank recoding of predictorsFitted and PRESS residuals remain the sameCoefficient meanings remain the samePermute complete rowsResults follow stable keys; total is unchangedOrder or dependence is irrelevantRefit explicitly without each rowyi - xi' betahat(i) agrees with ei/(1-hii)OLS assumptions and validation unit are suitableCompare with current trusted softwareValues agree after intercept, weights, and missingness conventions are reconciledSoftware output determines clinical meaning

Use unrounded residuals and leverage values in the calculation. If the explicit and shortcut routes disagree beyond a declared tolerance, stop. Common causes include row misalignment, different missingness, a changed factor level, incompatible weighting, transformed responses, regularization, or a rank-deficient deletion fit.

When row-wise leave-one-out is the wrong design

Row-wise PRESS assumes the omitted row is the meaningful unit to predict. That is often a poor match for longitudinal clinical data. If many rows come from one client, leaving out a session while training on the same client's adjacent sessions leaks person-specific and temporal information. If the intended use is prediction for a new client, all rows for that client belong together. The same principle applies to clinicians, classrooms, sites, families, or other clusters.

For future prediction, use time-respecting evaluation rather than letting later observations help predict earlier ones. For phase-structured single-case data, direct visual analysis and experimental-design logic remain central; an OLS leave-one-out score cannot create replication, verification, or experimental control.

The scikit-learn documentation notes that LeaveOneOut can be computationally expensive and a high-variance estimate of test error. Select it for the prediction question, not merely because the OLS shortcut is convenient.

Stop instead of manufacturing a number

  • Stop if any h_ii equals one or is so close that the denominator is numerically unstable.
  • Stop if deleting a row makes the design rank deficient or changes the intended coefficient space.
  • Stop if full and deletion fits do not use the same coding, transformations, weights, eligibility, or preprocessing learned only from their training rows.
  • Stop if missing values, imputation, normalization, feature selection, or tuning use information from the held-out row.
  • Stop if the data are clustered, repeated, or ordered but the worksheet treats rows as interchangeable independent units.
  • Stop if candidate models were added after seeing the same score and the selection process is not represented.
  • Stop before applying this shortcut to generalized, penalized, robust, mixed, nonlinear, or weighted models without a method-specific derivation and review.

Do not replace an undefined term with zero, cap a large residual, or omit a difficult case silently. Preserve the failure state, the affected key, and the reason review is needed.

Bring the result back to observable data

Start any follow-up with the rows that contribute most to PRESS, but do not end there. Verify operational definitions, opportunity counts, denominator changes, timing, phase, treatment integrity, setting events, staffing, schedule, data-entry provenance, and whether the predictors were available at the time a real prediction would be made.

Read the observed series and direct graph. Examine residual patterns, leverage, autocorrelation or dependence, and influence diagnostics as appropriate. Ask whether the model form misses curvature, interactions, phase changes, or other structure. Document prespecified sensitivity analyses and keep the original fit available.

The NIST regression-diagnostics reference places deleted residuals and PRESS inside a broader diagnostic workflow. A single aggregate can hide offsetting cases, so retain every signed residual and squared contribution. Even a small score cannot certify the model's clinical relevance or intended use.

Professional, communication, and privacy boundaries

The BACB Ethics Codes and BCBA Test Content Outline support competent, confidential, data-based practice grounded in measurement, graphing, interpretation, and experimental design. The Standards for Educational and Psychological Testing add broader evidence and intended-use principles. None requires PRESS or validates this calculator.

Communicate the response, row unit, population, model, split, residual convention, total, response-unit summary, major contributors, sensitivities, and unresolved limits. Say that each row was predicted from the others and whether that resembles the intended use. Include client and caregiver perspectives on relevance and accessibility.

Prefer synthetic or appropriately de-identified test data. Put authorized identifiable information only in an approved workflow with documented safeguards. HHS summarizes separate requirements in its Privacy Rule and Security Rule pages; neither certifies a spreadsheet or analytics service.

Related resources

Sources