arXiv Research

Search and retrieve academic papers from arXiv via their REST API. No API key. Use the shipped Python 3.9+ standard-library script relative to this installed skill directory, not a hardcoded Hermes repository path:

python3 "<skill-dir>/scripts/search_arxiv.py" "GRPO reinforcement learning" --max 5 --json
python3 "<skill-dir>/scripts/search_arxiv.py" --id 1706.03762v1 --json

The script preserves versioned identifiers and full abstracts, bounds response size and retries transient HTTP errors, and rejects malformed feeds/options. unknown metadata is a coverage gap, not permission to invent dates. Use evidence-citations for quotations and office-artifacts for PDFs. Attach only requested file outputs through cloud task attach/cloud deliver; honor PKF carrier instructions for inline answers. Abstract retrieval is not full-paper reading.

Quick Reference

Searching Papers

The API returns Atom XML. Use an XML parser (the shipped script), not grep/sed.

Basic search

curl --fail --silent --show-error --max-time 30 "https://export.arxiv.org/api/query?search_query=all:GRPO+reinforcement+learning&max_results=5"

Clean output

python3 "<skill-dir>/scripts/search_arxiv.py" "GRPO reinforcement learning" --max 5 --sort date

Search Query Syntax

Boolean operators

# AND (default when using +)
search_query=all:transformer+attention

# OR
search_query=all:GPT+OR+all:BERT

# AND NOT
search_query=all:language+model+ANDNOT+all:vision

# Exact phrase
search_query=ti:"chain+of+thought"

# Combined
search_query=au:hinton+AND+cat:cs.LG

Sort and Pagination

# Latest 10 papers in cs.AI
curl --fail --silent --show-error --max-time 30 "https://export.arxiv.org/api/query?search_query=cat:cs.AI&sortBy=submittedDate&sortOrder=descending&max_results=10"

Fetching Specific Papers

# By arXiv ID
curl --fail --silent --show-error --max-time 30 "https://export.arxiv.org/api/query?id_list=2402.03300"

# Multiple papers
curl --fail --silent --show-error --max-time 30 "https://export.arxiv.org/api/query?id_list=2402.03300,2401.12345,2403.00001"

BibTeX Generation

Retrieve structured metadata using --json, retaining the complete versioned ID. Register the inspected abstract/PDF URL and title with evidence-citations, then export its escaped --style bibtex entries. For article-specific author/year/ primaryClass fields, use only returned metadata and a BibTeX serializer; leave missing fields absent rather than inventing a category or author. Compile the actual bibliography and verify rendered URLs and unresolved-citation warnings.

Reading Paper Content

After finding a paper, read it:

# Abstract page (fast, metadata + abstract)
web_load(input="https://arxiv.org/abs/2402.03300")

# Full paper: the first call returns the outline with refs, then read the sections you need
web_load(input="https://arxiv.org/pdf/2402.03300")
web_load(input=["<ref of the method section>", "<ref of the results section>"])

An outline is not the paper's text — quote only from returned content, and follow nextRef only while a section still matters. Runtimes without web_load use cloud load <url> / cloud load '<ref>' (see ingest). For local PDF processing, see office-artifacts. A failed or partial extraction is a coverage gap, not something to fill in from memory.

Common Categories

Full list: https://arxiv.org/category_taxonomy

Helper Script

The scripts/search_arxiv.py script handles XML parsing and provides clean output:

python3 "<skill-dir>/scripts/search_arxiv.py" "GRPO reinforcement learning"
python3 "<skill-dir>/scripts/search_arxiv.py" "transformer attention" --max 10 --sort date
python3 "<skill-dir>/scripts/search_arxiv.py" --author "Yann LeCun" --max 5
python3 "<skill-dir>/scripts/search_arxiv.py" --category cs.AI --sort date
python3 "<skill-dir>/scripts/search_arxiv.py" --id 2402.03300
python3 "<skill-dir>/scripts/search_arxiv.py" --id 2402.03300,2401.12345

No dependencies — uses only Python stdlib.

Semantic Scholar (Citations, Related Papers, Author Profiles)

arXiv doesn't provide citation data or recommendations. The optional Semantic Scholar API provides JSON. Check current endpoint access, authentication and rate-limit responses; do not assume a fixed public or API-key quota.

Get paper details + citations

# By arXiv ID
curl --fail --silent --show-error --max-time 30 "https://api.semanticscholar.org/graph/v1/paper/arXiv:2402.03300?fields=title,authors,citationCount,referenceCount,influentialCitationCount,year,abstract" | python -m json.tool

# By Semantic Scholar paper ID or DOI
curl --fail --silent --show-error --max-time 30 "https://api.semanticscholar.org/graph/v1/paper/DOI:10.1234/example?fields=title,citationCount"

Get citations OF a paper (who cited it)

curl --fail --silent --show-error --max-time 30 "https://api.semanticscholar.org/graph/v1/paper/arXiv:2402.03300/citations?fields=title,authors,year,citationCount&limit=10" | python -m json.tool

Get references FROM a paper (what it cites)

curl --fail --silent --show-error --max-time 30 "https://api.semanticscholar.org/graph/v1/paper/arXiv:2402.03300/references?fields=title,authors,year,citationCount&limit=10" | python -m json.tool

Search papers (alternative to arXiv search, returns JSON)

curl --fail --silent --show-error --max-time 30 "https://api.semanticscholar.org/graph/v1/paper/search?query=GRPO+reinforcement+learning&limit=5&fields=title,authors,year,citationCount,externalIds" | python -m json.tool

Get paper recommendations

curl --fail --silent --show-error --max-time 30 -X POST "https://api.semanticscholar.org/recommendations/v1/papers/" \
  -H "Content-Type: application/json" \
  -d '{"positivePaperIds": ["arXiv:2402.03300"], "negativePaperIds": []}' | python -m json.tool

Author profile

curl --fail --silent --show-error --max-time 30 "https://api.semanticscholar.org/graph/v1/author/search?query=Yann+LeCun&fields=name,hIndex,citationCount,paperCount" | python -m json.tool

Useful Semantic Scholar fields

title, authors, year, abstract, citationCount, referenceCount, influentialCitationCount, isOpenAccess, openAccessPdf, fieldsOfStudy, publicationVenue, externalIds (contains arXiv ID, DOI, etc.)

Complete Research Workflow

1. Discover: python3 "<skill-dir>/scripts/search_arxiv.py" "your topic" --sort date --max 10
2. Assess impact: curl --fail --silent --show-error --max-time 30 "https://api.semanticscholar.org/graph/v1/paper/arXiv:ID?fields=citationCount,influentialCitationCount"
3. Read abstract: web_load(input="https://arxiv.org/abs/ID")
4. Read full paper: web_load(input="https://arxiv.org/pdf/ID"), then pass the refs of the sections you need back to web_load
5. Find related work: curl --fail --silent --show-error --max-time 30 "https://api.semanticscholar.org/graph/v1/paper/arXiv:ID/references?fields=title,citationCount&limit=20"
6. Get recommendations: POST to Semantic Scholar recommendations endpoint
7. Track authors: curl --fail --silent --show-error --max-time 30 "https://api.semanticscholar.org/graph/v1/author/search?query=NAME"

Rate Limits

Notes

• arXiv returns Atom XML — use the helper script or parsing snippet for clean output
• Semantic Scholar returns JSON — pipe through python -m json.tool for readability
• arXiv IDs: old format (hep-th/0601001) vs new (2402.03300)
• PDF: https://arxiv.org/pdf/{id} — Abstract: https://arxiv.org/abs/{id}
• HTML (when available): https://arxiv.org/html/{id}
• For local PDF processing, see office-artifacts.

ID Versioning

• arxiv.org/abs/1706.03762 always resolves to the latest version
• arxiv.org/abs/1706.03762v1 points to a specific immutable version
• When generating citations, preserve the version suffix you actually read to prevent citation drift (a later version may substantially change content)
• The API <id> field returns the versioned URL (e.g., http://arxiv.org/abs/1706.03762v7)

Withdrawn Papers

Papers can be withdrawn after submission. When this happens:

• The <summary> field contains a withdrawal notice (look for "withdrawn" or "retracted")
• Metadata fields may be incomplete
• Always check the summary before treating a result as a valid paper