Agent Skills
› brycewang-stanford/Awesome-Journal-Skills
› cogpsych-data-analysis
cogpsych-data-analysis
GitHub用于认知心理学论文的数据分析与模型拟合。指导模型比较（AIC/BIC）、参数与模型恢复、混合模型或层级贝叶斯估计及效应量报告，确保分析可复现且符合期刊标准。
Trigger Scenarios
拟合正式模型并与竞争模型进行比较
运行行为数据分析（混合模型、层级贝叶斯等）
审稿人要求补充模型比较、恢复性检验或稳健性分析
准备用于归档的分析代码和数据字典
Install
npx skills add brycewang-stanford/Awesome-Journal-Skills --skill cogpsych-data-analysis -g -y
SKILL.md
Frontmatter
{
"name": "cogpsych-data-analysis",
"description": "Use when analyzing data and fitting\/comparing models for a Cognitive Psychology (Elsevier) manuscript. The journal expects principled model fitting and comparison (AIC\/BIC\/Bayes factors), parameter and model recovery, (generalized) linear mixed models or hierarchical Bayesian estimation where apt, and effect sizes with uncertainty — all reproducible from shared code. Guides the analysis and modeling; it does not fabricate results."
}
Data Analysis & Model Fitting (cogpsych-data-analysis)
Cognitive Psychology holds analyses to a model-based standard: fit the formal model, compare it to rivals with principled criteria, demonstrate that parameters and models are recoverable, use mixed models or hierarchical Bayesian estimation where the design demands it, and report effect sizes with uncertainty for behavioral results — all regenerable from deposited code. This is the experiment-to-model-fit loop that defines the venue.
When to trigger
- Fitting the formal model and comparing it to rival accounts
- Running the behavioral analyses (mixed models, hierarchical Bayesian, contrasts)
- A reviewer asked for model comparison, recovery, robustness, or fuller disclosure
- Preparing analysis/model code and a data dictionary for deposit
Reporting norms Cognitive Psychology expects
- Fit and compare models, don't just fit one. Report fit for your model and the rival(s) under matched flexibility; compare with AIC/BIC, cross-validation, or Bayes factors as appropriate, and say what the comparison licenses.
- Show recovery. Demonstrate parameter recovery (can the fitting procedure recover known parameters from simulated data) and model recovery (does the comparison criterion pick the generating model) — without these, a fit edge is not interpretable.
- Use the right hierarchical structure. Crossed random effects over subjects and items call for (generalized) linear mixed models; for cognitive models, hierarchical Bayesian estimation pools strength across participants. Justify the structure; don't aggregate away the variance.
- Effect sizes + uncertainty for behavior. Report standardized/unstandardized effect sizes with confidence/credible intervals for key behavioral results, not just p-values and stars.
- Confirmatory vs. exploratory. Separate pre-committed model comparisons and tests from exploratory model exploration; do not present a post hoc winning model as predicted.
- Reproducible. Model and analysis code, with seeds and pinned versions, regenerate every reported
fit, figure, and table in a fresh session (see
cogpsych-open-science-and-transparency).
Robustness
- Show the conclusion survives reasonable alternative model specifications, priors (for Bayesian fits), and exclusion choices; report sensitivity, not a single fragile fit. Report convergence diagnostics (e.g., R-hat, ESS) for Bayesian models.
Worked micro-example (illustrative numbers)
A preregistered three-experiment recognition-memory program fitting UVSD vs. DPSD to confidence-ROC data.
Model comparison (preregistered) — pooled across Exps 1-3
Fit (hierarchical Bayesian, matched flexibility):
UVSD favored: dBIC = 14 vs. DPSD; Bayes factor ~ 30 in favor of UVSD
Recovery (required): parameter recovery good (recovered d', sigma within
credible intervals); model recovery ~ 92% correct at the design's N/trials
Diagnostic signature: z-ROC slope 0.78, 95% CrI [0.72, 0.84], and linear
(no reliable curvature) — the qualitative pattern UVSD predicts and DPSD
forbids, consistent across all three experiments
Behavioral effect (mixed model)
List-strength manipulation on d': b = 0.31, 95% CI [0.18, 0.44]
Exploratory (labeled)
A small response-bias drift surfaced post hoc; reported as exploratory
Why this passes Cognitive Psychology scrutiny: the model is compared (not just fit), recovery makes the comparison interpretable, the qualitative signature corroborates the fit index, hierarchy respects subject/item variance, and the exploratory drift is honestly demoted.
Analysis-stage reviewer pushback and the venue fix
| Reviewer pushback | What it signals here | Cognitive Psychology fix |
|---|---|---|
| "You only fit your model" | one-model storytelling | fit the rival under matched flexibility; report AIC/BIC/BF and what it licenses |
| "Better fit may be overfitting" | flexibility imbalance | add model recovery + cross-validation; penalize complexity |
| "Can you recover these parameters?" | identifiability doubt | run and report parameter + model recovery simulations |
| "Aggregated means hide variance" | wrong error structure | refit with crossed-random-effects mixed model / hierarchical Bayesian |
| "Is this the model you predicted?" | post hoc selection | pre-commit the comparison; relabel post hoc fits exploratory |
| "I can't rerun your fits" | reproducibility gate | ship seeded model code + a fresh-session run log |
Calibration anchors
- A model that is fit, compared, and recovered is the unit of evidence here — a single fit with a good index but no rival and no recovery is not persuasive.
- Trust a crossed qualitative prediction over a marginal fit advantage; report both and lead with the signature the rival forbids.
- Respect the data's hierarchy: aggregating over subjects or items inflates false positives and can bias parameter estimates; use mixed/hierarchical models and justify the random-effects structure.
- For Bayesian fits, report priors, convergence, and sensitivity — a fit without diagnostics is not reproducible evidence.
Execution bridge (StatsPAI / Stata MCP)
Run the battery, don't just enumerate it. Full map:
execution-with-mcp. Cognitive Psychology is experimental — within-subject designs and mixed models dominate; report the model, the effect size, and multiple-comparison control.
- Many outcomes / specifications:
romano_wolf(step-down FWER) orbenjamini_hochberg— report the adjusted threshold. - OVB sensitivity:
oster_delta/sensemakr. - Inference:
wild_cluster_bootstrap(few clusters),twoway_cluster/conley; multilevel data → cluster at the right level. - Re-fit off one handle:
audit_result(result_id)lists the missing checks and the exactsuggest_functionfor each. - Exhibits:
etable/did_summary_to_latexfrom the handle — no retyped numbers.
Keep the decisive checks in the body and the exhaustive battery in the supplement. See the executed chain in the JF execution walkthrough.
Anti-patterns
- Fitting only your model with no rival and no comparison criterion
- Claiming a fit advantage without matched flexibility, recovery, or cross-validation
- Aggregating to cell means and ignoring crossed subject/item variance
- p-values and stars with no effect size or interval for behavioral results
- Presenting a post hoc winning model as a predicted result
- Model/analysis code that does not regenerate the reported fits
Output format
【Model comparison】rivals fit under matched flexibility + criterion (AIC/BIC/BF)? [Y/N]
【Recovery】parameter + model recovery reported? [Y/N]
【Hierarchy】mixed model / hierarchical Bayesian where apt + diagnostics? [Y/N]
【Behavioral effects】effect sizes + intervals? [Y/N]
【Confirmatory vs exploratory】separated? [Y/N]
【Reproducible】seeded code + data dictionary + fresh-session check? [Y/N]
【Next】cogpsych-tables-figures
Supplementary resources
../../resources/external_tools.md— modeling, model-comparison,lme4/brms/Stan, JAGS, recovery simulation../../resources/official-source-map.md— statistical and modeling expectations
Version History
- 1839142 Current 2026-07-05 12:37
