Agent Skills
› brycewang-stanford/Awesome-Journal-Skills
› joap-data-analysis
joap-data-analysis
GitHub用于JAP期刊稿件的SEM、HLM、中介/调节及元分析。规范报告效应量、置信区间、模型拟合指标及Bootstrap间接效应，强调测量与结构模型分离、全披露及可重复性，确保符合期刊透明度和开放科学要求。
Trigger Scenarios
指定和报告主要及支持性分析
审稿人要求提供拟合指数、间接效应置信区间或透明度说明
协调预注册分析与探索性后续分析
准备用于存档的分析脚本和数据字典
Install
npx skills add brycewang-stanford/Awesome-Journal-Skills --skill joap-data-analysis -g -y
SKILL.md
Frontmatter
{
"name": "joap-data-analysis",
"description": "Use when analyzing and reporting results for a Journal of Applied Psychology (JAP) manuscript using SEM, multilevel (HLM) models, mediation\/moderation, or meta-analysis. JAP requires effect sizes with confidence intervals, model-based indirect effects with bootstrap CIs, fit indices, full disclosure, and a clean confirmatory\/exploratory split. Guides analysis norms; it does not fabricate results."
}
Data Analysis (joap-data-analysis)
JAP analyses must be model-appropriate, fully reported, and reproducible. The house toolkit is SEM/CFA, multilevel (HLM) models, mediation and moderation with proper inference, and meta-analysis. The journal expects effect sizes with confidence intervals, fit indices, bootstrap CIs for indirect effects, full disclosure of how data were handled, and a clean confirmatory vs. exploratory separation, with data and code shareable under TOP.
When to trigger
- Specifying and reporting the main and supporting analyses
- A reviewer asked for fit indices, indirect-effect CIs, robustness, or disclosure
- Reconciling preregistered analyses with exploratory follow-ups
- Preparing analysis scripts and a data/codebook for deposit
Reporting norms JAP expects
- Measurement before structure. Report the measurement model (CFA fit: χ²/df, CFI, TLI, RMSEA, SRMR) and reliability/AVE before interpreting the structural model; report measurement invariance when comparing groups or waves.
- Effect sizes + uncertainty. Give standardized and/or unstandardized estimates with confidence intervals for key paths — not just p-values and stars.
- Mediation done right. Report the indirect effect with a bootstrap (or Monte Carlo) CI (not only the Baron-Kenny steps or a Sobel z); for multilevel mediation use the appropriate (e.g., 1-1-1, 2-1-1, 2-2-1) decomposition and within/between separation.
- Multilevel correctly. Report ICC(1)/ICC(2), center predictors appropriately (group-mean vs grand-mean), model random effects, and justify the estimator; do not run OLS on nested data.
- Full disclosure. Report how sample size was determined, all exclusions (careless responding, attrition) and reasons, all conditions, and all measures. Label confirmatory vs. exploratory.
- Meta-analysis (if applicable). Report the model (random vs fixed), heterogeneity (Q, I²), artifact corrections, publication-bias checks, and a transparent coding protocol.
- Reproducibility. Provide scripts and a codebook; results should regenerate from shared data in a
fresh session (see
joap-open-science-and-transparency).
Worked micro-example (illustrative numbers)
The servant-leadership package: a multilevel mediation in the field and a causal test in the lab.
Measurement model (field, N = 612 in 74 teams)
CFA: χ²/df = 2.1, CFI = .96, TLI = .95, RMSEA = .045, SRMR = .04
Reliabilities ω: leadership .91, safety .88, performance .87
Aggregation: ICC(1) = .16, ICC(2) = .77, r_wg(j) = .85 → team-level OK
Confirmatory (preregistered) — multilevel mediation (2-2-2)
a (leadership→safety) = .42 [.27, .57]; b (safety→performance) = .31 [.14, .48]
Indirect = .13, 95% Monte Carlo CI [.05, .23] → mediation supported
Direct (leadership→performance | safety) = .09 [-.06, .24], ns
Confirmatory (preregistered) — lab experiment (causal leg)
Servant vs control on safety: d = 0.46, 95% CI [0.21, 0.71]
H3 boundary: interaction with interdependence, ΔR² = .03, CI excludes 0
Exploratory (labeled): voice as a serial L1 mediator surfaced post hoc;
reported as exploratory, flagged for confirmation in future work.
Why this passes JAP scrutiny: the measurement model is reported before structure; the indirect effect carries a bootstrap/Monte Carlo CI; nesting is modeled and aggregation justified; the experimental leg supplies causal warrant; and the post hoc serial path is honestly demoted to exploratory.
Analysis-stage reviewer pushback and the venue fix
| Reviewer pushback | What it signals | JAP fix |
|---|---|---|
| "Mediation by Sobel/steps only" | outdated inference | report indirect effect + bootstrap/Monte Carlo CI |
| "OLS on nested data" | dependence ignored | multilevel model; report ICC, centering, random effects |
| "No fit indices / measurement model" | construct validity unchecked | report CFA fit + reliability before structure |
| "Which exclusions were preregistered?" | forking-paths concern | disclosure table: rule, count, preregistered vs post hoc, result with/without |
| "Is this confirmatory?" | HARKing concern | point to the preregistration; relabel post hoc analyses exploratory |
| "Reviewer 2 couldn't rerun your code" | reproducibility gate | ship a fresh-session run log (see open-science skill) |
Calibration anchors
- Report measurement before structure: a beautiful path model on a misfitting measurement model convinces no JAP reviewer.
- Prefer estimation language with CIs to dichotomous "significant/not"; bare p-value sentences read as pre-reform.
- For mediation and cross-level effects, the interval on the carrying effect is the result — design and report so that interval is defensible.
Execution bridge (StatsPAI / Stata MCP)
Run the battery, don't just enumerate it. Full map:
execution-with-mcp. JAP is organizational psychology — multilevel survey/field data and experiments; cluster at the right level and apply mediation/moderation discipline.
- Many outcomes / specifications:
romano_wolf(step-down FWER) orbenjamini_hochberg— report the adjusted threshold. - OVB sensitivity:
oster_delta/sensemakr. - Inference:
wild_cluster_bootstrap(few clusters),twoway_cluster/conley; multilevel data → cluster at the right level. - Re-fit off one handle:
audit_result(result_id)lists the missing checks and the exactsuggest_functionfor each. - Exhibits:
etable/did_summary_to_latexfrom the handle — no retyped numbers.
Keep the decisive checks in the body and the exhaustive battery in the supplement. See the executed chain in the JF execution walkthrough.
Anti-patterns
- p-values and stars with no effect size, CI, or fit indices
- Mediation claimed via Baron-Kenny steps or Sobel z without a bootstrap/Monte Carlo CI
- Ignoring nesting (OLS on multilevel data) or aggregating without ICC/r_wg justification
- Selectively reporting conditions, measures, or exclusions (undisclosed flexibility)
- HARKing exploratory findings into confirmatory hypotheses
- Code that does not reproduce the reported numbers
Output format
【Measurement model】CFA fit + reliability + invariance reported? [Y/N]
【Main result】effect size(s) + CI(s) + meaning
【Mediation/multilevel】indirect-effect bootstrap/Monte Carlo CI; nesting modeled? [Y/N/NA]
【Disclosure】N-determination + all exclusions + all conditions + all measures? [Y/N]
【Confirmatory vs exploratory】clearly separated? [Y/N]
【Reproducible】scripts + codebook + fresh-session check? [Y/N]
【Next】joap-tables-figures
Supplementary resources
../../resources/external_tools.md— Mplus,lavaan,lme4/nlme,psych,metafor/metaSEM, bootstrap/Monte Carlo tools../../resources/official-source-map.md— statistical and disclosure requirements
Version History
- 1839142 Current 2026-07-05 13:26
