Agent Skills
› NeverSight/learn-skills.dev
› tar-compression
tar-compression
GitHub提供tar及相关压缩工具(gzip, zip等)的使用指南，涵盖创建、解压、列表查看、追加更新、排除文件、权限保留及增量备份等操作，支持多种格式自动检测与并行压缩。
Trigger Scenarios
用户询问如何创建或解压归档文件
需要处理 tar.gz, zip, xz 等压缩格式
涉及文件打包、压缩、解压或备份操作
Install
npx skills add NeverSight/learn-skills.dev --skill tar-compression -g -y
SKILL.md
Frontmatter
{
"name": "tar-compression",
"description": "Archive and compression commands — tar, gzip, zip, zstd, xz, bzip2, 7z, and friends. Use when user mentions \"tar\", \"gzip\", \"zip\", \"unzip\", \"compress\", \"extract\", \"archive\", \"bzip2\", \"xz\", \"zstd\", \"7z\", \"7zip\", \"rar\", \"pigz\", \"pbzip2\", \".tar.gz\", \".tgz\", \".tar.xz\", \".tar.zst\", \".tar.bz2\", \"untar\", \"decompress\", \"incremental backup\", \"split archive\", \"parallel compression\", \"disk image\", \"dd\", or working with compressed files."
}
tar and compression
tar basics — create, extract, list
tar -cvf archive.tar dir/ # create .tar (no compression)
tar -czvf archive.tar.gz dir/ # create .tar.gz (gzip)
tar -cjvf archive.tar.bz2 dir/ # create .tar.bz2 (bzip2)
tar -cJvf archive.tar.xz dir/ # create .tar.xz (xz)
tar --zstd -cvf archive.tar.zst dir/ # create .tar.zst (zstd)
tar -xvf archive.tar # extract (auto-detects compression)
tar -xzvf archive.tar.gz # explicit gzip
tar -xjvf archive.tar.bz2 # explicit bzip2
tar -xJvf archive.tar.xz # explicit xz
Modern tar auto-detects compression on extraction, so tar -xvf works for all formats.
tar flags reference
| Flag | Meaning |
|---|---|
-c |
Create archive |
-x |
Extract archive |
-t |
List contents (don't extract) |
-r |
Append files to existing archive (uncompressed .tar only) |
-u |
Update — append only files newer than what's in archive |
-v |
Verbose (show files) |
-f |
Next argument is the filename — must come last before the filename |
-z |
gzip compression |
-j |
bzip2 compression |
-J |
xz compression |
--zstd |
zstd compression |
-C |
Change to directory before operating |
-p |
Preserve permissions |
-h |
Follow symlinks (archive the target, not the link) |
-f must be the last flag before the archive name. tar -cfv archive.tar fails — tar looks for a file named v.
Append, update, exclude
tar -rvf archive.tar newfile.txt # append (uncompressed .tar only)
tar -uvf archive.tar dir/ # update only newer files
tar -czvf archive.tar.gz dir/ --exclude='*.log' --exclude='.git'
tar -czvf archive.tar.gz dir/ --exclude-vcs # skip .git, .svn, .hg
tar -czvf archive.tar.gz dir/ --exclude-vcs-ignores # also honor .gitignore
tar -czvf archive.tar.gz dir/ -X exclude.txt # patterns from file
Archive inspection and partial extraction
# Inspect without extracting
tar -tzvf archive.tar.gz # list all files with sizes
tar -tzvf archive.tar.gz | grep pattern # find specific files
tar -tzvf archive.tar.gz --wildcards '*.conf' # list matching files (GNU tar)
# Extract specific files or directories
tar -xzvf archive.tar.gz path/to/file.txt # extract one file
tar -xzvf archive.tar.gz dir/subdir/ # extract one directory
tar -xzvf archive.tar.gz --wildcards '*.conf' # extract by pattern (GNU tar)
tar -xzvf archive.tar.gz -C /target/dir/ # extract to specific directory
tar -xzvf archive.tar.gz --strip-components=1 # strip top-level directory
Preserving permissions, ownership, symlinks
# Full preservation (system backups, run as root)
tar -cpzvf archive.tar.gz --same-owner dir/
# Follow symlinks instead of storing links
tar -chzvf archive.tar.gz dir/
# Restore with ownership (requires root)
tar -xpzvf archive.tar.gz --same-owner -C /restore/
# Compare archive to filesystem (check what changed)
tar -dvf archive.tar dir/
On macOS, --same-owner is default for root. On GNU/Linux, specify it explicitly.
Compression algorithms compared
| Tool | Flag | Ext | Speed | Ratio | Parallel tool |
|---|---|---|---|---|---|
| gzip | -z |
.tar.gz | Fast | Good | pigz |
| bzip2 | -j |
.tar.bz2 | Slow | Better | pbzip2 |
| xz | -J |
.tar.xz | Very slow | Best | pxz / pixz |
| zstd | --zstd |
.tar.zst | Very fast | Better | built-in -T |
| lz4 | --use-compress-program=lz4 |
.tar.lz4 | Fastest | Lower | built-in |
Rough benchmarks (1 GB mixed data, single core):
| Tool | Compress | Decompress | Compressed size |
|---|---|---|---|
| lz4 | ~2s | ~0.5s | ~55% of original |
| zstd -1 | ~3s | ~1s | ~42% |
| gzip -6 | ~12s | ~3s | ~36% |
| zstd -19 | ~90s | ~1s | ~30% |
| xz -6 | ~120s | ~8s | ~28% |
Rule of thumb: quick backup → gzip/zstd. Source distribution → xz. General purpose → zstd. Real-time → lz4. bzip2 → legacy, skip it.
gzip / gunzip
gzip file.txt # compress to file.txt.gz, removes original
gzip -k file.txt # keep original
gunzip file.txt.gz # decompress
gzip -9 file.txt # max compression (1=fast, 9=best)
gzip -l file.txt.gz # show compression ratio
zcat file.txt.gz # decompress to stdout
zstd (modern compression)
zstd file.txt # compress -> file.txt.zst (default level 3)
zstd -d file.txt.zst # decompress
zstd -19 file.txt # max standard compression (1-19)
zstd --ultra -22 file.txt # ultra mode (20-22, needs --ultra)
zstd --fast file.txt # speed over ratio
zstd -T0 file.txt # use all CPU cores
zstd --long file.txt # larger window for better ratio on large files
Dictionary compression (many small similar files)
zstd --train samples/* -o mydict.zst # train from samples
zstd -D mydict.zst file.txt -o file.txt.zst # compress with dictionary
zstd -d -D mydict.zst file.txt.zst # decompress with dictionary
Improves ratio 2-5x on small, structurally similar files (logs, JSON, configs).
Streaming
mysqldump mydb | zstd -T0 > dump.sql.zst
zstd -d < dump.sql.zst | mysql mydb
tar -cf - dir/ | pv | zstd -T0 > archive.tar.zst
Parallel compression (pigz, pbzip2, pxz)
# pigz — parallel gzip (drop-in replacement)
tar -cf - dir/ | pigz -p 8 > archive.tar.gz
pigz -d archive.tar.gz
# pbzip2 — parallel bzip2
tar -cf - dir/ | pbzip2 -p8 > archive.tar.bz2
# pxz — parallel xz
tar -cf - dir/ | pxz -T 8 > archive.tar.xz
# tar integration via --use-compress-program
tar -cf archive.tar.gz --use-compress-program='pigz -9' dir/
tar -xf archive.tar.gz --use-compress-program='pigz -d' -C /target/
zip / unzip
zip archive.zip file1 file2 # create
zip -r archive.zip dir/ # recursive (directories)
zip -r archive.zip dir/ -x '*.log' # exclude pattern
unzip archive.zip # extract to current dir
unzip archive.zip -d /target/dir/ # extract to specific dir
unzip archive.zip file.txt # extract one file
unzip -o archive.zip # overwrite without prompting
unzip -l archive.zip # list files and sizes
zipinfo archive.zip # detailed info
zip -u archive.zip updated-file.txt # update changed files in archive
Password-protected zip
zip -e archive.zip file1 file2 # prompt for password
unzip -P 'mypass' archive.zip # extract with password
zip's encryption (ZipCrypto) is weak. Use 7z with AES-256 for real security.
Split zip archives
zip -r -s 100m archive.zip dir/ # split at 100MB chunks
zip -s 0 archive.zip --out combined.zip # merge splits before extracting
unzip combined.zip
7z operations (p7zip / 7-Zip)
7z a archive.7z dir/ # create .7z
7z a archive.7z dir/ -mx=9 # ultra compression
7z a archive.7z dir/ -ms=on # solid mode (better compression)
7z a archive.7z dir/ -p'Pass' -mhe=on # AES-256 encrypt contents + filenames
7z a -v100m archive.7z dir/ # split into 100MB volumes
7z x archive.7z # extract preserving structure
7z e archive.7z # extract flat (no dirs)
7z x archive.7z -o/target/dir/ # extract to directory
7z l archive.7z # list contents
7z t archive.7z # test integrity
7z reads/writes: .7z, .zip, .tar, .gz, .bz2, .xz. Extracts .rar.
Piping tar over SSH
# Local to remote
tar -czvf - dir/ | ssh user@host 'tar -xzvf - -C /target/'
# Remote to local
ssh user@host 'tar -czvf - /remote/dir/' | tar -xzvf - -C /local/dir/
# With zstd for faster transfer
tar -cf - dir/ | zstd -T0 | ssh user@host 'zstd -d | tar -xf - -C /target/'
# With progress and bandwidth limit
tar -cf - dir/ | pv -L 10m | ssh user@host 'tar -xf - -C /target/'
Incremental backups with tar
# Level 0: full backup (creates snapshot file)
tar -czvf backup-full.tar.gz \
--listed-incremental=/var/backups/snapshot.snar dir/
# Level 1: incremental (only changes since last backup)
tar -czvf backup-inc-$(date +%Y%m%d).tar.gz \
--listed-incremental=/var/backups/snapshot.snar dir/
# Restore: apply full, then each incremental in order
tar -xzvf backup-full.tar.gz -C /restore/ --listed-incremental=/dev/null
tar -xzvf backup-inc-20260410.tar.gz -C /restore/ --listed-incremental=/dev/null
--listed-incremental=/dev/null during restore tells tar to extract everything. GNU tar only.
Split large archives
tar -czvf - dir/ | split -b 100m - archive.tar.gz.part-
cat archive.tar.gz.part-* | tar -xzvf -
Cross-platform: macOS vs GNU tar
| Feature | GNU tar (Linux) | BSD tar (macOS) |
|---|---|---|
--wildcards |
Required for patterns | Default behavior |
--exclude-vcs-ignores |
Supported | Not supported |
--zstd |
Supported | Use --use-compress-program |
--listed-incremental |
Supported | Not supported |
| Extended attributes | --xattrs |
Stored by default |
# macOS: install GNU tar for full feature set
brew install gnu-tar # use as 'gtar'
# macOS: zstd with BSD tar
tar -cf - dir/ | zstd > archive.tar.zst
# macOS: avoid resource fork ._* files
COPYFILE_DISABLE=1 tar -czvf archive.tar.gz dir/
Compression for specific use cases
# Logs (repetitive text, compresses 90%+)
gzip -9 app.log
find /var/log -name '*.log' -mtime +7 -exec gzip {} \;
zstd --train logs/*.log -o log-dict.zst && zstd -D log-dict.zst -19 app.log
# Database dumps
mysqldump mydb | zstd -T0 > mydb.sql.zst
pg_dump mydb | gzip -9 > mydb.sql.gz
pg_dump -Fc mydb > mydb.dump # pg custom format (built-in compression)
# Source code distribution
tar -cJvf project.tar.xz --exclude-vcs project/
git archive --format=tar.gz HEAD > project.tar.gz
# Disk images
dd if=/dev/sda bs=4M status=progress | zstd -T0 > disk.img.zst
zstd -d < disk.img.zst | dd of=/dev/sda bs=4M status=progress
qemu-img convert -c -O qcow2 disk.raw disk.qcow2
Common patterns
# Backup with timestamp
tar -czvf "backup-$(date +%Y%m%d-%H%M%S).tar.gz" /path/to/dir/
# Dry run — see what would be extracted
tar -tzvf archive.tar.gz | head -20
# Find largest files in archive
tar -tzvf archive.tar.gz | sort -k3 -n -r | head -20
# Create archive from file list
tar -czvf archive.tar.gz -T filelist.txt
# Create archive excluding large files
find dir/ -size -10M | tar -czvf small-files.tar.gz -T -
# Verify archive integrity
gzip -t archive.tar.gz && echo "OK" || echo "CORRUPT"
xz -t archive.tar.xz
zstd -t archive.tar.zst
Version History
- e0220ca Current 2026-07-05 20:45
