udf

udf is a Go CLI tool that extracts content from archives, virtual disk images and filesystem images, and lists (or merges into a single rootfs) image filesystems.

One binary speaks two faces: a traditional CLI and a networked agent. Every command is defined once (xyz-go) and automatically available as a CLI subcommand, an HTTP REST route (with an OpenAPI document) and an MCP tool. All inputs are local file paths on the machine running udf — nothing is uploaded; the operation always happens where the program runs.

中文说明见 README.zh-CN.md.

Table of Contents

• Features
• Interfaces
• Installation
• CLI Usage
• Input Modes
• Subcommands
• Output Rules
• Options
• Multi-image Archives
• Generated Files
• Error Handling
• Technical Notes
• Known Limitations
• Use as a Library
• Current Scope
• License

Features

Three interfaces, one definition:

• CLI subcommands, an HTTP REST service (/openapi.json included) and an MCP tool server, generated from the same command definitions
• Local-path semantics: archive / dest are paths on the host running udf; no file uploads
• Bearer-auth, TLS and CORS built in for the network modes

Archive handling:

• Extract image archives into a merged rootfs; a batch of archives is extracted with one archive per core, results in input order
• Random-access index for large compressed archives — gzip and zstd, docker-save images and plain tar.gz/.tar.zst bundles alike: the first command builds it in one pass with a block-aware deflate scanner (~500 MB/s decompressed), picking the layer directories out of that same pass and recording every member's header fields, after which ls/cp/cat/extract drop to well under a second instead of decompressing the stream again (on a 26 GB upgrade bundle: 69 s per command → 61 s once, then 0.02 s to list and 2.5 s to extract a 1.95 GB member)
• Outer archive formats: .tar, .tar.gz, .tgz, .tar.xz, .tar.bz2, .tar.zst, .tar.lz4, .zip, .7z, .rar, .cpio (and .cpio.gz/.cpio.xz/.cpio.zst, e.g. initramfs), .asar (Electron), .rpm, .deb/.ipk, .cab/.msi-embedded cabs, .nar (Nix), .xar/.pkg (macOS installer) (and .ppkg Windows provisioning packages, an OPC/ZIP)
• Executable bundle formats: PyInstaller onefile (CArchive + PYZ, .pyc reconstruction), py2exe (PE resources, the bootstrap PYTHONSCRIPT script rebuilt as a .pyc, and the module archive appended to the exe), Nuitka onefile (appended and embedded payloads), .NET single-file apps (bundle v1/v2/v6, incl. deflate), ZIP self-extracting executables
• Android and game containers: boot.img (v0–v4: kernel, ramdisk, dtb, plus every file inside the ramdisk's cpio, including LZ4-legacy-compressed ramdisks such as the emulator's ramdisk.img), sparse images (img, expanded on the fly as a disk container), super partitions (dynamic partitions — system, product, vendor, … — read through their extent mappings, standalone or inside a GPT disk) and Unity AssetBundles (UnityFS: per-node decompression — stored, LZ4/LZ4HC, LZMA — with entry names and byte-exact contents); binary Android XML (AXML) inside APKs — AndroidManifest.xml, layouts — and the compiled resource table resources.arsc are decoded to readable text, resource names, configurations and @package/name references resolving against the table itself
• Virtual disk images: qcow2, QCOW v1, VMDK, VHD/VHDX, VDI, QED, Parallels, WIM, ESD, SWM, FFU, raw/.img/.ami (MBR- and GPT-partitioned disks included), OVA, OVF, VMA, SIF, AppImage, Android sparse (ext4/xfs/btrfs/NTFS/squashfs/ISO9660/UDF/exFAT/EROFS/FAT, including LVM2 logical volumes and Android dynamic partitions)
• Common image layouts: flat manifest.json + config.json + layers/..., classic docker save (<layer-id>/layer.tar), OCI image layout directories, and single-file OCI image archives (.oci.tar, incl. flatpak bundles)
• Input may be a single archive, a glob pattern, or a directory (top level scanned)

Extraction correctness:

• Layers merged in manifest.json order
• Whiteout files and opaque directories handled
• Symlinks and hardlinks supported
• Fallback to file copy when the target filesystem does not support hardlinks
• Directory metadata restored after extraction

Inspection:

• List merged contents without extracting (udf ls, ls -al-style information both as a CLI table and as structured JSON)
• Extract a single file or directory only (udf cp)
• Show image metadata: tags, layers, working dir, entrypoint (udf info)
• Write a single file's raw bytes to stdout, pipe-friendly (udf cat)
• Hex-dump the first bytes of a file to inspect its header (udf xxd)

Convenience:

• Export config.json as readable config.yaml
• One error taxonomy: CLI exit code, HTTP status and MCP error code stay aligned
• Importable as a Go library

Containers and disk images

Beyond container layer archives, udf can open a virtual disk image and extract a filesystem through the same commands. info, ls, cp and extract detect disk inputs automatically:

./udf info ./disk.qcow2                        # list disks, volumes + filesystems
./udf ls ./disk.qcow2                          # list the volumes (partitions + LVs)
./udf ls ./disk.qcow2 /vg1/root                # list a volume's root directory
./udf ls ./disk.qcow2 /vg1/root/etc            # list /etc inside a volume
./udf cp ./disk.qcow2 /vg1/root/etc/hostname ./hostname
./udf extract ./disk.qcow2                     # extract every filesystem volume
./udf ls ./appliance.ova                       # list the vmdk disks in an OVA
./udf ls ./appliance.ova /disk1.vmdk           # then its volumes

Android images work through the same commands: a boot.img (whose ramdisk is unpacked in turn), an emulator or device system.img — a GPT disk whose super partition holds the dynamic partitions — and a ramdisk (ramdisk.img, initrd) as a plain cpio:

./udf ls ./boot.img                            # kernel, ramdisk, dtb, and the ramdisk's files
./udf ls ./system.img                          # a GPT disk: p1, p2, and inside p2 the dynamic partitions
./udf ls ./system.img p2/system                # the system partition's root
./udf cat ./system.img p2/system/system/build.prop
./udf ls ./super.img                           # a standalone super.img: system, product, vendor, …
./udf ls ./ramdisk.img                         # a ramdisk (LZ4-legacy cpio) as an archive
./udf cp ./app.apk resources.arsc ./res.txt    # the compiled resource table, decoded

An archive holding several images (or several platforms of one image) is picked from with --tag, --index or --platform, and verify recomputes the digests of the config, the manifest and every layer:

./udf info ./images.tar.gz                     # one line per image: os, arch, variant, layers
./udf ls --platform linux/arm64 ./multi.tar /etc
./udf cp --tag app:1.2 --platform linux/amd64 ./multi.tar /etc/hosts ./hosts
./udf verify ./images.tar.gz                   # config/manifest/layer digests, and what is missing
./udf verify --fast ./images.tar.gz            # only check that every layer is present

Supported disk container formats:

• qcow2 (v2/v3), read-only and streamed
• VMDK sparse extents — both monolithicSparse (flat or deflate) and streamOptimized
• OVA archives (a tar of an OVF descriptor plus one or more .vmdk disks)
• OVF standalone descriptors (.ovf referencing sibling .vmdk disks)
• VHD/VHDX virtual disks (fixed, dynamic and differencing)
• QED QEMU Enhanced Disk images (two-level page table; little-endian)
• QCOW v1 legacy QEMU disk images (two-level page table; compressed clusters)
• WIM Windows imaging files (the first image; XPRESS and LZX compression)
• ESD Windows "Electronic Software Download" files (WIM with LZMS compression)
• SWM split WIM images (the first part; sibling *.swm2/3/… parts are read automatically)
• FFU Full Flash Update images (locates the embedded disk by scanning past the security header — heuristic, not validated against a real Microsoft FFU)
• VMA Proxmox vzdump archives (one or more raw disks)
• SIF Singularity container images (the system partition is extracted)
• VDI VirtualBox disk images
• Parallels disk images (.parallels/.hds)
• raw disk and filesystem images (.img/.raw/.dd/.ext4/.xfs, …) — bytes are the device itself

The virtual disk is addressed as a path tree: / lists disks (for a multi-disk OVA) or volumes, /<volume> enters a volume, and the rest is a path inside it. Volumes are named p1…pN for partitions and vg/lv for LVM logical volumes.

• MBR/GPT partition tables are parsed, plus LVM2 physical volumes in-process: each logical volume is exposed as a selectable volume, in addition to plain partitions
• Filesystems supported for extraction are ext4, xfs, btrfs, NTFS, squashfs, ISO9660, UDF, exFAT, EROFS and FAT (fat12/16/32)
• When a disk holds several filesystems, ls and cp require a volume prefix (or start from / and descend); a single filesystem volume is used implicitly, so ./udf ls img.qcow2 /etc still works for simple images
• extract writes every filesystem volume; with several, each lands in its own subdirectory named after the volume
• Regular files, directories and symlinks are recreated; hardlinks become independent copies, and device nodes, FIFOs and sockets are skipped

Interfaces

# CLI
./udf ls ./image.tar /etc

# HTTP — every routed command answers on the same port
./udf serve --addr 127.0.0.1:8080
curl -s 'http://127.0.0.1:8080/ls?archive=/data/image.tar&path=/etc'
curl -s -X POST 'http://127.0.0.1:8080/cp' -H 'Content-Type: application/json' \
  -d '{"archive":"/data/image.tar","source":"/etc/passwd","dest":"/tmp/passwd"}'
curl -s http://127.0.0.1:8080/openapi.json

# MCP — commands become tools (stdio / SSE / streamable HTTP)
./udf mcp stdio
./udf mcp http --addr 127.0.0.1:9000 --bearer s3cret

Route overview: GET /info?archive=…, GET /ls?archive=…&path=…, POST /cp, GET /cat?archive=…&source=…, GET /xxd?archive=…&source=…, POST /extract, plus /healthz and /openapi.json. Each route takes the same selection parameters as the CLI (tag=, repo-tag=, index=, image-index=), so ?tag=safeline-mgt:latest works over HTTP too.

In MCP clients, register udf as a stdio server:

{"command": "udf", "args": ["mcp", "stdio"]}

Security: every path argument refers to the machine that runs udf. When you expose serve or mcp http/sse beyond the loopback interface, that means callers can read and write server-local files. Protect those modes with --bearer, TLS (--tls-cert/--tls-key) and CORS allowlists — see the xyz-go built-in configuration.

Installation

Install with Homebrew (macOS)

brew install ejfkdev/tap/udf

The formula lives in the homebrew-udf tap and packages the binaries published by this repository's release workflow.

Install with go install

go install github.com/ejfkdev/udf@latest

Build from source

git clone https://github.com/ejfkdev/udf.git
cd udf
go build -o udf .

CLI Usage

./udf <command> [arguments]

Examples:

./udf info ./image.tar
./udf ls ./image.tar /etc
./udf cp ./image.tar /etc/passwd ./passwd
./udf extract ./image.tar
./udf extract -o ./output -t repo/app:latest './repo/*.tar'
./udf serve --addr 127.0.0.1:8080
./udf mcp stdio

extract is the default command, so the classic invocation keeps working:

./udf ./image.tar                    # same as: ./udf extract ./image.tar
./udf './repo/*.tar' -o ./output -t repo/app:latest

When using the shorthand form, put flags after the archive path; to lead with flags, name the subcommand explicitly (./udf extract -o ./out ./image.tar).

Built-in conveniences: -h per-command help, -v version, --json for machine-readable CLI output, and completion bash|zsh|fish.

Input Modes

Every command takes one input expression for the archive:

• a single archive file (.tar/.tar.gz/.tgz/.zip/.7z/.rar/.cpio/.asar/.rpm/PyInstaller/py2exe/Nuitka/.NET single-file executable, Android boot.img, Unity AssetBundle) or a disk image (.qcow2/.vmdk/.vhd/.vhdx/.vdi/.img/.raw/.dd/.ova/.vma, Android sparse img) — required by info, ls and cp
• a glob pattern, or a directory (top level scanned, not recursive) — extract also accepts these and expands them into a batch

Examples:

./udf extract ./images
./udf extract './images/*.tar'
./udf ls ./image.tar /etc

Subcommands

info — image archive metadata

./udf info ./image.tar                     # the only image, or one selected with -t/-i
./udf info ./docker-save.tar.gz            # several images: one line per image
./udf info -t demo/app:latest ./image.tar   # the detail of one image

One image (also the form used when one is selected with --tag or --index):

index           0
total_images    1
repo_tags       [demo/app:latest]
config_path     config.json
os              linux
architecture    amd64
created         2026-04-15T12:50:36.123456789Z
docker_version  27.5.1
working_dir     /app
entrypoint      [/docker-entrypoint.sh]
cmd             [/bin/sh -c sh /app/entrypoint.sh]
layer_count     2
size            75.9 MiB
layers          [layer1.tar layer2.tar]

An archive holding several images and no selection is answered with a listing of them instead of an error — the file is what it is, and which image it holds is exactly the information missing:

Index  Image                          OS     Architecture  Created                        Layers  Size
-----  -----------------------------  -----  ------------  -----------------------------  ------  ---------
0      demo/app:latest, app:latest    linux  amd64         2026-04-15T12:50:36.123456789Z  2      75.9 MiB
1      demo/db:9.1-lts                linux  amd64         2026-03-02T08:11:02.987654321Z  5      412.3 MiB

--json returns the same as an array. Size is how much space the image's layers take in the archive, and is empty when the archive's members were not indexed (an uncompressed or small archive).

A disk image answers with a disks array (one entry per disk, with its volumes and filesystems) and a plain archive with a plain object (format, file count, total size); neither carries the image fields above.

Cache and temporary files

Everything udf derives lives in one per-user directory inside the system temporary directory:

<temp>/ejfkdev/udf/          macOS /var/folders/…/T/ejfkdev/udf, Linux /tmp/ejfkdev/udf, Windows %TEMP%\ejfkdev\udf
├── *.json                   derived values: input class, image metadata, listings, image summaries
├── idx/*.idx                random-access indexes for large gzip archives
└── tmp/*                    per-run scratch (disks unpacked from an OVA/VMA/VM export)

Nothing here is needed for correctness: every entry is rebuilt from its input file, and scratch is removed by the run that made it. The location is chosen so each system reclaims it on its own terms — macOS purges entries unused for three days, Linux clears /tmp at boot or after ten days — and udf prunes what the OS leaves behind (derived values after seven days without being rewritten, interrupted scratch after a day), so it stays bounded on Windows as well. UDF_CACHE_DIR points it somewhere else (a specific volume, or /dev/null-style non-persistence); deleting the directory by hand is always safe.

ls — list directory contents without extracting

udf ls builds the merged filesystem view in memory and renders it like ls -al, without writing anything to disk. The CLI prints an aligned table; HTTP and MCP return the same data as structured JSON.

./udf ls ./image.tar          # the image root
./udf ls ./image.tar /etc     # one directory

name    type  mode        size  mod_time                    target
------  ----  ----------  ----  --------------------------  ------
group   file  -rw-r--r--  10    2026-08-22T00:04:25+08:00
passwd  file  -rw-r--r--  30    2026-08-22T00:04:25+08:00

Details:

• Layers are merged with the same whiteout and opaque-directory semantics as a full extract
• Columns carry type (dir|file|symlink|hardlink), mode, size, mtime and the symlink target
• Leading / in the path is optional; / or . lists the image root
• A file path lists that single entry

cp — extract a single file or directory

./udf cp ./image.tar /etc/passwd ./passwd
./udf cp ./image.tar /etc/nginx ./nginx
./udf cp ./image.tar / ./rootfs

Destination semantics mirror cp:

• Directory source into an existing directory lands at <dest>/<basename>
• Directory source into a non-existent path copies into that path directly
• File or symlink source goes to <dest> as a file, or into <dest>/<basename> when <dest> is an existing directory
• Extracting / (the image root) puts the contents directly into <dest>
• Whiteout-processed entries are skipped, symlinks are recreated as symlinks, and hardlinks are preserved when the source stays inside the selection (content is copied otherwise)

cat — write a single file to stdout

Like the OS cat, udf cat streams one file's raw bytes to stdout (binary-safe, no trailing newline), so it can be piped into other programs:

./udf cat ./image.tar /etc/passwd | grep root
./udf cat ./disk.qcow2 /p1/etc/hostname

Symlinks and hardlinks inside the image are followed to their target file. Prefer cat for small text files; for large or binary files use xxd to preview a bounded prefix instead of dumping everything.

xxd — hex-dump a file header

Like xxd -l N / hexdump -C, udf xxd prints the first bytes of a file as hex + ASCII so you can identify it by its header:

./udf xxd ./image.tar /etc/passwd                        # first 256 bytes
./udf xxd ./image.tar /etc/passwd -n 64 -s 4096          # 64 bytes, skipping 4096

-n/--bytes sets the byte count (default 256) and -s/--offset skips bytes from the start before dumping.

extract — extract the merged rootfs

./udf extract ./image.tar
./udf ./image.tar                     # extract is the default command
./udf extract -o ./output -f -t repo/app:latest './repo/*.tar'

extract expands its input expression into one or more archives and processes them in order. The result is one row per archive (table on the CLI, JSON array over HTTP/MCP), and a row carries its own message when one archive fails without stopping the rest:

archive      output_dir                     layers  error
-----------  -----------------------------  ------  -----
./image.tar  /tmp/out/image                 2

Output Rules

For extract, if -o/--output is not specified the output goes beside each input archive; with -o it goes under the given parent directory:

• single-image archive:
  • {file_name}/
• multi-image archive:
  • {file_name}/{repo_tag}/
  • if no tag exists, fallback to {file_name}/index-{n}/

Examples:

input:  /data/demo/tempest.tar
output: /data/demo/tempest

input:  /data/demo/bundle.tar
tags:   repo/app:1.0, repo/app:latest
output: /data/demo/bundle/repo_app_1.0
        /data/demo/bundle/repo_app_latest

Options

Command flags:

• -t, --repo-tag (preferred: --tag) — select the image by tag: a full RepoTags entry, name:tag, a repository name or a bare tag, as long as the name is unambiguous (info, ls, cp, cat, xxd, extract)
• -i, --image-index (preferred: --index) — select the image by its index in the manifest.json array (all commands)
• -o, --output — output parent directory (extract)
• -f, --force — write into an existing non-empty target directory (extract)
• -b, --buffer-size — copy buffer size in bytes (cp, cat, extract)
• -n, --bytes — number of bytes to dump (xxd, default 256)
• -s, --offset — skip this many bytes from the start before dumping (xxd)

Built-in flags come from xyz-go: -h/--help, -v/--version, --json, --xyz.lang en|zh-CN (interface language, defaults to LANG/LC_ALL detection), and completion bash|zsh|fish. The serve and mcp modes add --addr, --bearer, --cors, --tls-cert/--tls-key, --timeout, --log-level (plus --versions and --session-timeout for mcp) — details in the xyz-go README.

Multi-image Archives

If an archive contains only one image:

• you do not need to specify -t or -i

If an archive contains multiple images:

• info prints the images (one line per image); the commands that need a target (ls, cp, cat, xxd, extract) print an error that lists the available options
• select one with --tag (also -t / --repo-tag) by name, or --index (also -i / --image-index) by position

--tag accepts any name an image is listed under, as long as it is unambiguous — a full repo tag, name:tag, a repository name or a bare tag:

./udf info --tag swr.example/chaitin/safeline-mgt:latest ./image.tar   # as listed
./udf info --tag chaitin/safeline-mgt:latest ./image.tar               # tail of the path
./udf info --tag safeline-mgt:latest ./image.tar                       # name:tag
./udf info --tag safeline-mgt ./image.tar                              # repository name
./udf info --tag 15.18 ./image.tar                                     # bare tag, if only one image has it
./udf info --index 3 ./image.tar                                      # by position
./udf info --tag safeline-luigi:9.1.0-lts ./image.tar                  # a tag with its version

A name that matches more than one image is refused with the list of candidates ("latest" matches more than one image: [1]=…), so a selection is never a guess. -t/--repo-tag resolve the same way, so full tags keep working and shorter ones now work too.

Generated Files

For each extracted image, udf writes:

• merged rootfs
• config.yaml

config.yaml is generated from the original config.json, with strings formatted for readability where possible.

Error Handling

One error taxonomy drives all three interfaces:

Kind	CLI exit	HTTP status	MCP code
invalid input	2	400	-32602
not found	1	404	-32001
conflict	1	409	-32009
unauthorized / forbidden	1	401 / 403	-32010 / -32011
unavailable	1	503	-32603
internal	1	500	-32603

Batch extract keeps going after one archive fails: failures land in the row's error column, and the command only reports an error itself when nothing succeeded. Low-level system errors are preserved as-is for diagnostics.

Technical Notes

• Layer application follows manifest.json
• Whiteout files are handled during extraction
• Inner layer streams are auto-detected: gzip, bzip2 and uncompressed tar are supported
• Directory metadata is restored after extraction to avoid intermediate permission issues
• Memory usage is kept low by streaming layer extraction instead of unpacking all layers to disk first

Known Limitations

• zstd-compressed inner layers are not supported and fail with an explicit error
• A disk image must contain a supported filesystem (ext4/xfs/btrfs/NTFS/squashfs/ISO9660/UDF/exFAT/EROFS/FAT) (whole disk, partitions, or LVM2 logical volumes); other filesystems (e.g. JFS/ReiserFS) are reported by info but not extracted
• Disk extraction recreates regular files, directories and symlinks; file ownership is not preserved (files are written as the current user)
• Directory input only scans the top level and is not recursive
• Multi-image archives require an explicit -t/-i selection; udf never prompts interactively
• One input expression per command; use a directory or a glob for batches
• The interface language follows LANG/LC_ALL, override with --xyz.lang en|zh-CN; the Go library errors still carry stable i18n keys

Use as a Library

Everything is importable as a Go library:

package main

import (
	"fmt"
	"log"

	"github.com/ejfkdev/udf/image"
)

func main() {
	meta, err := image.ScanImageMetadata("./app.tar", image.Selection{RepoTag: "demo/app:latest"})
	if err != nil {
		log.Fatal(err)
	}

	// Print /etc the way `ls -al` would, without extracting anything.
	tree, err := image.BuildFileSystem("./app.tar", meta)
	if err != nil {
		log.Fatal(err)
	}
	listing, err := image.FormatListing(tree, "/etc")
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(listing)

	// The same view as structured data.
	entries, err := image.ListEntries(tree, "/etc")
	if err != nil {
		log.Fatal(err)
	}
	_ = entries

	// Extract a single file out of the image.
	if _, err := image.ExtractPath("./app.tar", meta, "/etc/passwd", "./passwd", 1<<20); err != nil {
		log.Fatal(err)
	}
}

Public packages:

• github.com/ejfkdev/udf/image — high level: metadata scanning, merged file system view, listing, selective and full extraction
• github.com/ejfkdev/udf/fsview — whiteout-aware in-memory merged filesystem tree
• github.com/ejfkdev/udf/layer — single layer application and compressed stream detection
• github.com/ejfkdev/udf/fsutil — escape-safe path resolution and file writing helpers
• github.com/ejfkdev/udf/types — shared data structures
• github.com/ejfkdev/udf/i18n — bilingual message bundles

Selection and not-found errors implement *i18n.LocalizedError with a stable .Key you can match on; Error() renders readable English by default.

Current Scope

udf is a general "extract the contents of an encapsulation format" tool: it detects the input by its content (magic bytes), not its extension, and opens it through one of three pipelines — an archive, a virtual disk container, or a raw filesystem image.

Supported:

• archive extraction (tar, tar.gz/tgz, tar.xz, tar.bz2, tar.zst, tar.lz4, zip, 7z, rar, cpio, cpio.gz/xz/zst, asar, rpm, deb/ipk, cab, nar, xar/pkg, pyinstaller, py2exe, nuitka, .NET single-file, zip SFX, OCI layout, oci-archive/flatpak, docker save, Android boot.img, Unity AssetBundle), batch via directory or glob
• disk / VM images: qcow2, QCOW v1, VMDK, VHD/VHDX, VDI, QED, Parallels, WIM/ESD/SWM, FFU, VMA, SIF, OVA/OVF, raw/.img/.ami, Android sparse img
• filesystems (whole disk, partitions, or LVM2 logical volumes): ext2/3/4, xfs, btrfs, NTFS, squashfs, ISO9660, UDF, exFAT, EROFS (uncompressed), FAT12/16/32
• listing without extracting (ls), selective extraction (cp), metadata (info), single-file read to stdout (cat), hex header dump (xxd), full extraction (extract)
• one binary, three interfaces: CLI, HTTP (REST + OpenAPI), MCP tools
• config YAML export
• importable as a Go library

Not intended as:

• a Docker replacement
• a container runtime
• an OCI registry client

License

This project is licensed under the MIT License. See LICENSE.