Agent Skills
› brycewang-stanford/Awesome-Journal-Skills
› text-retrieval-conference
text-retrieval-conference
GitHub用于评估论文是否适合TREC会议投稿。提供框架调整、证据审查、拒稿风险诊断及重写建议，确保内容符合检索评测社区标准，并提示核对最新官方指南。
触发场景
作者指定TREC为目标会议
需评估检索评估类手稿的会议适配度
将期刊或arXiv稿件重构为CS会议叙事
需要证据缺口、匿名性、重投策略诊断
安装
npx skills add brycewang-stanford/Awesome-Journal-Skills --skill text-retrieval-conference -g -y
SKILL.md
Frontmatter
{
"name": "text-retrieval-conference",
"description": "Use when targeting Text REtrieval Conference (TREC) or deciding whether a computer-science manuscript fits this venue. Encodes conference fit, framing, evidence bar, submission-cycle checks, rebuttal posture, and desk-reject risks for retrieval evaluation."
}
Text REtrieval Conference (TREC)
Conference positioning
Text REtrieval Conference (TREC) is a top computer-science conference venue for retrieval evaluation, tracks, test collections, search tasks, and NIST-style benchmarking. It rewards a retrieval paper that contributes to or learns from shared evaluation tracks and test collections. Treat this skill as a fit / venue-selection / re-framing tool for conference submission strategy, not as a substitute for the current year's CFP, author kit, ethics policy, or submission portal.
Because CS conferences change deadlines, templates, page limits, review workflow, artifact rules, AI-use policy, and rebuttal formats every cycle, always verify the live official instructions before making a submission-ready recommendation. Start from the official source anchor recorded for this venue in ../../resources/conference-roster.md and ../../resources/official-source-map.md.
When to trigger
- The author names TREC / Text REtrieval Conference as the target venue.
- A manuscript in retrieval evaluation needs a conference-fit read before being formatted or submitted.
- The paper must be re-framed from journal style or arXiv style into a selective CS conference narrative.
- The author needs an evidence-gap, anonymity, artifact, rebuttal, or re-routing diagnosis for this venue.
Scope & topic fit
- Core fit: retrieval evaluation, tracks, test collections, search tasks, and NIST-style benchmarking.
- Best submissions make a precise contribution type visible: algorithm, theorem, system, dataset, benchmark, empirical finding, design artifact, tool, or socio-technical analysis.
- The paper should explain why the result matters to TREC's reviewers, not just why it is interesting to the authors' lab or product context.
- Position related work against the most recent conference-cycle papers in this venue and its closest siblings; stale comparisons are a common early-review weakness.
- If the contribution is interdisciplinary, state which part is CS research and which part is domain evidence.
Venue-specific calibration
- Reviewer lens: Treat TREC as a retrieval evaluation venue whose reviewers expect the scope and evidence to match its own community. Do not submit a generic CS paper until the introduction names the exact subcommunity, contribution type, and proof or empirical standard.
- Contribution hook to foreground: the venue-specific contribution bar.
- Scope vocabulary to use naturally in the abstract and introduction: retrieval evaluation, tracks, test collections, search tasks, and NIST-style benchmarking.
- Distinctive fingerprint for reviewer calibration: retrieval, evaluation, tracks, test, collections, search, tasks, nist-style, benchmarking, venue-specific, contribution, trec, nist.
- Official anchor domain: trec.nist.gov. Quote annual rules only after opening that source and the current-year CFP/author kit.
Close-neighbor routing guardrail
- Use this profile only when the manuscript's central contribution is genuinely in retrieval evaluation and the author can say why TREC reviewers are the primary audience, not merely a convenient deadline.
- Closest roster neighbors to compare before final routing:
acm-sigir-conference-on-research- and-development-in-information-retrieval(SIGIR),european-conference-on-information- retrieval(ECIR),acm-sigplan-conference-on-programming-language-design-and- implementation(PLDI),acm-sigplan-symposium-on-principles-of-programming-languages(POPL). Break ties by contribution type, evidence shape, reviewer community, and the current official CFP from trec.nist.gov.
What distinguishes this venue from its closest siblings
- What TREC is. The NIST Text REtrieval Conference — an evaluation campaign building shared test collections and running tracks, not a standard paper venue.
- vs SIGIR / ECIR / CIKM. Those are research conferences; TREC produces benchmarks/tracks the IR field then evaluates against.
- Routing. Use TREC collections in your SIGIR/ECIR/CIKM paper; participate in TREC tracks for shared-task results.
TREC-specific routing detail
- Prefer TREC when the work is an information-retrieval evaluation campaign, track participation, test collection, shared task, retrieval benchmark, or evaluation methodology.
- Route spoken language technology to SLT/INTERSPEECH and user-centered search behavior to CHIIR when campaign-style text retrieval evaluation is not the core.
- TREC evidence should explain track rules, run configuration, test collection, metrics, pooling or judging assumptions, and what the evaluation teaches beyond leaderboard rank.
Method & evidence bar
- Use task-appropriate baselines, multiple datasets or languages when the claim is broad, and error analysis that explains model behavior.
- For LLM work, control for data leakage, prompt sensitivity, evaluation contamination, and human-evaluation reliability.
- For resources, document annotation, licensing, demographics, quality control, and intended use.
- For TREC, the evidence must support the venue-specific signature: a retrieval paper that contributes to or learns from shared evaluation tracks and test collections.
- Include limitations, negative results, compute/resource reporting, data provenance, and ethics details when they affect the claim.
Structure & house style
- State the language phenomenon, task, or system behavior before the model name.
- Connect examples to measured errors; reviewers dislike anecdotal examples presented as evidence.
- Use the current official template exactly; do not guess page limits, font sizes, supplement rules, anonymity exceptions, or camera-ready requirements from old cycles.
- The introduction should answer: problem, why now, what is new, why this venue, and what evidence proves the claim.
- Put the strongest result in the main paper, not only in the appendix or supplement; reviewers should not have to reconstruct the contribution.
Official-cycle checklist
- Open the live official venue page: https://trec.nist.gov/
- Re-check the current cycle's CFP, author kit, submission system, abstract/paper deadlines, page limits, supplementary-material rules, anonymity policy, dual-submission policy, ethics policy, AI-use policy, artifact/code/data expectations, rebuttal/author-response format, and camera-ready requirements.
- Confirm the review workflow and portal: OpenReview / CMT / HotCRP / PCS / START or society portal, as specified for the current cycle.
- Check whether accepted papers require in-person presentation, separate registration, artifact badges, proceedings copyright, or post-acceptance release forms.
- If the live official instructions conflict with this skill, the official instructions win.
Pre-submission self-check
- One sentence states why this manuscript belongs at TREC, using the venue's scope rather than generic "top conference" language.
- The claim is calibrated to the evidence: no broader than the datasets, proofs, systems, user studies, deployments, or threat model support.
- Related work includes the nearest current-cycle retrieval evaluation papers and explains the technical delta.
- The paper satisfies the current official template, anonymity, ethics, artifact, and rebuttal requirements.
- The main paper is self-contained enough for reviewers to evaluate novelty and correctness without hunting through external links.
Common desk-reject triggers
- Evaluation that is only a prompt table or cherry-picked generation examples.
- Missing dataset documentation, licensing, or annotation reliability.
- Claims of general language understanding from narrow English-only benchmarks.
- Formatting, anonymity, dual-submission, external-link, or supplement violations under the current-year policy.
- A contribution framed for a neighboring field while giving TREC reviewers too little technical or empirical substance.
Re-routing decision
If the paper misses TREC's bar, compare against annual-meeting-of-the-association-for-computational-linguistics / conference-on-empirical-methods-in-natural-language-processing / north-american-chapter-of-the-association-for-computational-linguistics / european-chapter-of-the-association-for-computational-linguistics. Re-route based on contribution type, not prestige: theory to a theory venue, systems to a systems venue, application-heavy work to a domain venue, and early ideas to workshops or shorter tracks when the official CFP supports them.
Output format
[Fit] High / Medium / Low (one-line reason)
[Target] Text REtrieval Conference (TREC)
[Contribution type] algorithm / theory / system / dataset / benchmark / empirical / design / security / other
[Main evidence gap] <single most important missing proof, experiment, study, artifact, or policy check>
[Official items to re-check] CFP / author kit / deadline / format / anonymity / ethics / AI-use / artifact / rebuttal / camera-ready
[Top rejection risk] <venue-specific risk>
[Re-route suggestion] <better-matched conference or journal if not a fit>
版本历史
- 1839142 当前 2026-07-05 12:46
