An ABA Kendall W concordance calculator summarizes how consistently multiple raters order the same items. A result becomes interpretable only after the item set, rank direction, tie rule, rater roles and intended decision are fixed. Keep every rank beside W so reviewers can inspect the ordering that one coefficient compresses.

Clinicians & ABA Professionals / Data, Outcomes and Clinical Decision-Making.

Confirm that the input is a complete ranking design

Kendall and Babington Smith's original paper introduced the coefficient of concordance for the problem of multiple rankings. This worksheet implements the descriptive tie-corrected point estimate, without a significance test. The paper does not make Kendall W an ABA standard, approve a clinical use or supply a universal threshold.

Use this calculator when at least two raters rank the same complete set of at least three items from first through last. A defensible exercise could order fictional training priorities or prespecified implementation barriers, provided the items and rating process fit the intended decision. The direction must be identical for every rater, such as rank 1 always meaning highest priority. Each rater gives every item one position, with average ranks used for genuine ties.

Do not enter nominal categories, raw behavior counts, interval-by-interval occurrence codes, paired continuous readings or a partial top-k list. Those structures call for a different analysis. A rank discards distance: the gap between ranks 1 and 2 is not assumed to equal a measured clinical difference.

The current BACB Ethics Codes and BCBA Test Content Outline provide professional context for competent measurement and interpretation. Neither requires Kendall W. The Standards for Educational and Psychological Testing support keeping an interpretation tied to its intended use and evidence.

Register the ranking plan

Complete this design record before viewing W.

Design fieldPrespecified entryObservable construct or decisionComplete item setWhy ranking is appropriateRank directionRank 1 means:Tie ruleAverage ranks for genuine tiesRater inclusion ruleRater roles and independence procedureMissing-rank ruleStop; this calculator requires complete rankingsExclusions decided before analysisData version and extraction timePrimary statisticKendall W with tie correctionIntended consequenceQualified reviewers

Changing the item set, direction, tie handling or included raters after seeing the coefficient changes the analysis. Preserve the original version. If a sensitivity analysis is justified, document its reason before calculating and label its output separately.

Enter one row per rater

Create an m x n matrix. Rows are raters and columns are the same items in the same order. Use average ranks within each tie group.

RaterItem AItem BItem C...Item nRow sumRater 1Rater 2...

For n items, each valid rank row must sum to n(n+1)/2. Check the rank pattern as well as the sum: 1,1,4,4,5 totals 15 for five items but is not a valid average-rank encoding. If three consecutive positions are genuinely tied, replace those positions with their average rank in all three cells. Stop if any item or rater is missing.

Calculate item rank sums and dispersion

Let R_j be the sum of ranks assigned to item j across the m raters. The expected item rank sum is:

R_bar = m(n+1) / 2

Calculate the centered sum of squares:

S = sumj (Rj - R_bar)^2

Large dispersion among item rank sums indicates that raters tended to place some items consistently higher and others consistently lower. It does not show why they did so or whether the resulting order is clinically appropriate.

Calculate every tie correction

For rater i, identify each tie group g and its size t_ig. A group of two equal average ranks contributes 2^3-2=6; a group of three contributes 3^3-3=24.

Ti = sumg (tig^3 - tig)

Then sum T_i across raters. A rater with no ties contributes zero. Do not count repeated rank values that arose from an invalid row as legitimate ties; return to the source ranking and reconstruct the average ranks.

Calculate tie-corrected Kendall W

Use unrounded intermediate values:

D = m^2(n^3 - n) - m sumi Ti

W = 12S / D

If D=0, W is undefined. Report the boundary rather than forcing a value. For a valid complete-ranking design with a positive denominator, W ranges from zero to one. Round only the displayed result and retain the full-precision calculation.

Work the fictional no-tie example

Three fictional raters rank five fictional items. No person or clinical record is represented.

RaterItem AItem BItem CItem DItem ERater 112345Rater 213254Rater 321345Item rank sum4681314

Here m=3, n=5, and every row sums to 15. The expected item rank sum is Rbar=3(6)/2=9. The squared deviations are 25, 9, 1, 16 and 25, so S=76. There are no ties and sumi T_i=0.

D = 3^2(5^3-5) = 1080

W = 12(76)/1080 = 0.8444444444

The result describes concordance in this fabricated rank matrix. It does not justify a label such as acceptable or excellent.

Show why the tie correction matters

This separate fictional matrix contains one two-item tie for Rater 2.

RaterItem AItem BItem CItem DItem ERater 112345Rater 212.52.545Rater 312354Item rank sum36.58.51314

The item sums give S=83.5. The one tie group has size two, so sumi Ti=6. The corrected denominator is 1080-3(6)=1062.

W = 12(83.5)/1062 = 0.9435028249

Omitting the correction would produce 1002/1080=0.9277777778. That is not an optional alternate score. It is the wrong denominator for this tied matrix.

Check informative boundaries

Identical complete rankings produce W=1. Two exactly reversed rankings of three items produce equal item rank sums, S=0, and W=0. If every rater assigns every item the same tied rank, the tie correction exhausts the denominator and W is undefined.

These boundaries test implementation behavior. They do not establish operational cutoffs. A value near one may still conceal an unclear construct, dependent raters, a narrow item set or a ranking that has little clinical consequence.

Interpret W with the rank matrix

Use the ABA Kendall W concordance calculator as an auditable record, not a score generator. Report the item labels, rank direction, m, n, every rank row, item rank sums, tie groups, S, the corrected denominator and W. Review where rank positions differ and ask whether definitions, observation access, rater roles or context explain the pattern.

Kendall W does not identify a correct rater, preserve measured distances, demonstrate treatment integrity, establish experimental control or prove that an ordered decision benefits a client. Two panels can share W while disagreeing on different items. A qualified reviewer should connect the visible rank pattern to the prespecified use, consequences and perspectives of affected people.

Stop when the design exceeds the worksheet

Pause for missing ranks, partial lists, different item sets, pairwise-comparison input, rater weights, repeated or clustered panels, or a changed tie rule. Obtain statistical review for confidence intervals, hypothesis tests, permutation procedures, sample-size planning or generalization beyond the observed raters and items.

Weighted kappa handles two raters assigning ordered categories, while Fleiss kappa and related coefficients summarize nominal category assignments. Krippendorff alpha supports other data structures under its own assumptions. Familiar ABA IOA calculations answer still different measurement questions. Do not choose among them after seeing which number looks best.

Protect rank records and provenance

Use the minimum information needed for the review. Keep identifiable unit-level material only in approved systems, restrict access and retain an audit trail. The HHS Privacy Rule summary and HHS Security Rule summary describe federal requirements for regulated entities. State and organizational rules may add duties. This page is not legal advice.

Copyable result record

Result fieldValueData versionItem set and rank directionm raters / n itemsRater rank rowsItem rank sumsTie groups and sumi TiSCorrected denominator DKendall WMissing or excluded entriesDisagreement pattern reviewedIntended interpretation and limitsReviewer and review date

Related resources

Sources