Instrument validation
An assistant that takes you step by step, from data cleaning to the report. For every number it explains in plain words how to read it and what to do if it falls short, and at the end it gives you the text for your methods and the limitations you should report.
Your data never leave your computer. Everything happens inside your browser and AtheneaLab stores nothing. That is why, when you finish each variable, you download its PDF and its progress file: with that file you can pick it up another day.
How do you want to start?
Validate a variable
Start from scratch. You can enter the values jamovi gives you or load the AtheneaLab coded dataset so the tool calculates part of them.
Continue or correct a variable
Load the progress file (.athenealab) you downloaded: the assistant opens just as you left it.
Integrated report
Load the files of two or more variables from the same study and get the discriminant validity between them, a joint table and the unified text.
How does it work?
Validating an instrument means gathering evidence that it measures what it claims to measure and that it does so precisely in your sample. The assistant follows the order in which that work is done: each step opens when you complete the previous one, and some open or not depending on your methodological decisions.
Data and path
Your sample, your instrument and whether its structure has theoretical support. With support, you go straight to the CFA; without it, an EFA comes first.
Cleaning and jamovi
A cleaned dataset and free statistical software.
Structure
Your variable's dimensions and items, and the ones you decide to remove.
Item analysis
Item-rest correlation, α if item deleted, skewness and kurtosis.
KMO, Bartlett and EFA
Only if the structure lacks sufficient support.
CFA
Fit indices, loadings, correlations between dimensions and comparison with a one-factor model.
Reliability and validity
α, ω, AVE, Fornell-Larcker and HTMT.
Report
Text for the methods, proposed limitations and the variable's PDF.
The traffic light
Each statistic is colored according to published criteria. The color guides you; it does not decide for you: a value in amber can be reported with a justification, and one in red signals something worth reviewing. When a red result compromises validity, the assistant marks it as pending instead of turning it into a limitation.
| Statistic | Excellent | Acceptable | Not acceptable | Source |
|---|---|---|---|---|
| Cronbach's α | ≥ .90 | .70–.89 | < .70 | Nunnally and Bernstein (1994) |
| Item-rest correlation | ≥ .50 | .30–.49 | < .30 | Usual criterion |
| α if item deleted | Does not exceed α | Exceeds it by ≤ .05 | Exceeds it by > .05 | Usual criterion |
| Skewness and kurtosis | |skew| < 2 and |kurt.| < 7 | Use a robust estimator | Curran et al. (1996) | |
| KMO | ≥ .80 | .60–.79 | < .60 | Kaiser (1974) |
| Item MSA | ≥ .80 | .50–.79 | < .50 | Kaiser (1974) |
| Bartlett | p < .001 | p < .05 | p ≥ .05 | — |
| EFA loadings | ≥ .70 | .40–.69 | < .40 | Hair et al. (2019) |
| Cross-loadings | None | Difference ≥ .20 | Difference < .20 | Usual criterion |
| CFI and TLI | ≥ .95 | .90–.94 | < .90 | Hu and Bentler (1999) |
| RMSEA | ≤ .05 | .05–.08 (mediocre up to .10) | > .10 | MacCallum et al. (1996) |
| SRMR | ≤ .05 | .05–.08 | > .08 | Hu and Bentler (1999) |
| Standardized CFA loadings | ≥ .70 | .50–.69 | < .50 | Hair et al. (2019) |
| Correlation between dimensions | < .70 | .70–.84 | ≥ .85 | Brown (2015) |
| McDonald's ω | ≥ .85 | .70–.84 | < .70 | Viladrich et al. (2017) |
| AVE | ≥ .60 | .50–.59 | < .50 (retainable if CR ≥ .60) | Fornell and Larcker (1981) |
| HTMT | < .75 | .75–.84 | ≥ .85 (lenient: < .90) | Henseler et al. (2015) |
| Fornell-Larcker | √AVE > all its correlations | Some pair fails | Fornell and Larcker (1981) | |
Why go straight to the CFA when there is theoretical support?
The CFA tests a structure that has been specified in advance: how many dimensions there are and which items belong to each (Brown, 2015). The EFA is for when that structure is unknown or unclear (Fabrigar et al., 1999). If theory and prior studies already support it, running an EFA first is redundant, and going to the CFA is appropriate, especially when adapting a test with a known, clear structure (Ferrando et al., 2022).
Moreover, the EFA does not always reproduce the original structure even when it is correct: its result depends on the number of factors, the extraction method and the rotation (Lloret-Segura et al., 2014), and highly related dimensions tend to blend because the EFA lets every item load on every factor. When both analyses are used, it is best to run them in different halves of the sample (Fokkema and Greiff, 2017).
References
How to cite this tool
AtheneaLab. (2026). Instrument validation [Web application]. https://athenealab.com/en/instrument-validation/