format-specific-extraction
GitHub提供多格式文档(Office/PDF/归档/结构化文本/邮件)的提取工作流规范,涵盖解析流程、安全预算控制及元数据提取逻辑。
Trigger Scenarios
Install
npx skills add xberg-io/xberg --skill format-specific-extraction -g -y
SKILL.md
Frontmatter
{
"name": "format-specific-extraction",
"priority": "high",
"description": "Format-specific document extraction workflows"
}
Format-Specific Extraction Workflows
Office XML (DOCX/PPTX/ODT)
ZIP archive → SecurityBudget → XML parsing → Text + tables + metadata
let budget = SecurityBudget::from_config(config);(extractors/security.rs), plusconfig.security_limits…max_files_in_archiveas the member cap. The Office path does not useZipBombValidator— that is the archive/iWork/HWPX path.- Open with
zip::ZipArchive::new(cursor)and read the parts (word/document.xml,ppt/slides/*.xml,content.xml). - Parse with
quick-xml::Reader(streaming), threading&mut budgetthrough the recursive walkers so a hostile document exhausts a budget instead of memory. - Metadata via
crate::extraction::office_metadata— see the helper table below. There is noextract_metadata(). - See
extractors/docx.rs,extractors/pptx.rs,extractors/odt.rs.
Bytes → xberg_native_pdf → Per-page text + OCR fallback → Tables → Metadata
xberg_native_pdf::PdfDocument::from_bytes(content.to_vec())?— the engine takes an ownedVec<u8>, not a slice.- OCR is forced by
config.force_ocr(whole document) orconfig.force_ocr_pages(Option<Vec<u32>>); otherwise pages with no extractable text route to OCR. config.pages: Option<PageConfig>controls per-page output — it does not gate tables.- Feature-gated
#[cfg(feature = "pdf")]; the backend isPdfConfig.backend(nativedefault,pdfiumbehindpdf-pdfium). - See
extractors/pdf/mod.rs.
Archives (ZIP/TAR/7z/GZIP)
ZipBombValidator → per-format metadata → per-format text content
ZipBombValidator::new(limits).validate(&mut archive)?before any extraction.- Metadata and content come from per-format helpers in
extraction/archive/:extract_{zip,tar,7z,gzip}_metadata,extract_{zip,tar,7z,gzip}_text_content,extract_{zip,tar,7z}_file_bytes. There is nobuild_archive_result(). - See
extractors/archive.rs,extraction/archive/{zip,tar,sevenz,gzip}.rs.
Structured Text (JSON/YAML/TOML/XML)
Single StructuredExtractor covers several MIME types: parse with the format library,
pretty-print to text. See extractors/structured.rs.
Email (EML/MSG/PST)
Parse headers → extract body (text/html) → process attachments. Message-in-message nesting is
bounded by the SecurityBudget's SecurityLimits-derived DepthValidator, the same counter
every other format uses. See extraction/email.rs, extractors/email.rs, extractors/pst.rs.
Common Helpers
| Helper | Location |
|---|---|
extract_core_properties() |
extraction/office_metadata/core_properties.rs |
extract_custom_properties() |
extraction/office_metadata/custom_properties.rs |
extract_{docx,xlsx,pptx}_app_properties() |
extraction/office_metadata/app_properties.rs |
extract_odt_properties() |
extraction/office_metadata/odt_properties.rs |
cells_to_markdown() |
extraction/markdown.rs (pub(crate)) |
SecurityBudget, SecurityLimits |
extractors/security.rs |
ZipBombValidator, DepthValidator |
extractors/security.rs |
StringGrowthValidator |
extractors/security.rs |
The security types are pub(crate): in-crate extractors can use them, out-of-crate plugin
authors cannot.
Adding a New Format
- Add one
FormatEntryto theFORMATSregistry incore/mime.rs.EXT_TO_MIMEandSUPPORTED_MIME_TYPESare derived from it — do not hand-edit either. Seemime-detection-routingfor the full procedure, including the count assertion to bump. - Create an extractor implementing
InternalDocumentExtractor(notDocumentExtractor). - Set
supported_mime_types()andpriority()(default 50). - Register in
extractors/mod.rs → register_default_extractors(). - Feature-gate if optional:
#[cfg(feature = "my-format")]. - Apply
SecurityBudget/SecurityLimitsto any user-supplied content. - Add
#[cfg_attr(alef, alef(skip))]to the extractor struct or the binding regen aborts. - Add tests with fixture files (see the
test-corpusskill for where fixtures come from).
Version History
-
d8e4815
Current 2026-08-28 18:30
更新Office路径移除ZipBombValidator;PDF后端由pdf_oxide改为xberg_native_pdf;Archive移除build_archive_result并细化文本提取。
- 531e0f7 2026-08-20 07:47


