Methodology
What has been established, and what has not.
No study has established the reliability, factor structure, invariance or predictive validity of this instrument. This page states that in numbers rather than in a sentence: the nine analyses were written down before any response existed, each with the sample it needs and the result that would count against the instrument, and every one of them shows what it has today.
- Analyses ready to run
- 0 of 9
- Consented responses
- 0
- Distinct organizations
- 0
Zero is the honest number today. A validation programme that only becomes visible once it succeeds is a marketing page, so this one is published from the start and will move as responses are contributed.
The nine analyses
Each states what is computed, the minimum sample it requires, and what result would count against the instrument. That last column is what makes this a protocol rather than a wish list: an analysis with no disconfirming outcome cannot fail, and an analysis that cannot fail establishes nothing.
A1Item exposure
0 of 30 respondents
How often each item in the adaptive bank was administered, and how far two 40-item forms overlap.
What would count against the instrument: Forms overlapping so little that no common scale links them.
A2Construct coverage in practice
0 of 30 respondents
Whether each construct really receives the three independent observations the design claims.
What would count against the instrument: Constructs routinely scored on fewer observations than the design claims.
A3Item calibration
0 of 400 respondents
Item response theory calibration per construct, on the items with enough exposure.
What would count against the instrument: Items that do not discriminate, or that fit no common dimension within their construct.
A4Dimensionality
0 of 400 respondents
Whether each construct behaves as one thing, by factor analysis on the calibrated subset.
What would count against the instrument: A construct that is not one thing: two or more factors with no substantive interpretation.
A5Internal consistency
0 of 150 respondents on one form
Omega and alpha on a fixed form, so a reported condition score has a reliability attached.
What would count against the instrument: Omega below 0.70 for a construct reported as a score.
A6Test-retest stability
0 of 20 paired engagements
Whether an organization measured twice, with no intervention between, gets the same reading.
What would count against the instrument: An intraclass correlation below 0.60, which would mean the score moves without the organization moving.
A7Team divergence
0 of 40 engagements
Whether disagreement inside one engagement is larger than the item’s own measurement error.
What would count against the instrument: Divergence indistinguishable from the error of the items it is computed on.
A8Measurement invariance
0 of 400 respondents
Whether the instrument means the same thing across sector and organization size.
What would count against the instrument: Non-invariance, which would mean scores are not comparable across the groups the product compares.
A9Criterion validity
0 of 30 engagements followed to twelve months
Whether the constraint the diagnostic indicated predicts what an organization went on to change, against measures it already kept.
What would count against the instrument: No relationship between the indicated constraint and the measures the organization already tracked, which would mean the reading describes nothing outside itself.
The thresholds, stated in advance
Omega below 0.70 for a construct reported as a score, and an intraclass correlation below 0.60 for a measurement repeated with no intervention between, are the two numbers that would fail. Both are conventional in applied organizational measurement rather than derived, and they are published here so that loosening one to accommodate a result is a visible act rather than a quiet one.
Where the sample comes from
Participants in a team diagnostic may contribute their answers after they have finished and seen their own result. Nothing is contributed without a separate, explicit choice, and the instruments that run entirely in your browser are never asked, because those pages promise your answers are never sent. What a contribution contains, and how to withdraw it.
No instrument has responses yet, so no analysis can be run on any of them.
What a green row would and would not mean
A row marked ready means the sample is large enough for the analysis to be run. It does not mean the analysis has been run, and it certainly does not mean it passed. Results land in the technical note with their sample, their date and the instrument fingerprint they were computed on, including the ones that count against the instrument.
Until then, every score this product reports is a directional reading for a conversation, which is what the technical note has said since version 1.0 and what it will keep saying until there is evidence to say otherwise.