Go library for queuing and extracting compressed archives, and for splitting FLAC and APE images that have a CUE sheet. Can also be used ad-hoc for direct decompression and extraction. See GoDoc.
- GoDoc
- Works on Linux, Windows, FreeBSD and macOS without Cgo.
- Supports 32 and 64 bit architectures.
- Decrypts RAR and 7-Zip archives with passwords.
- Extracts ISO images (ISO9660 and UDF volumes) and Electron ASAR archives.
- Splits FLAC+CUE and APE+CUE sheets into tracks. APE output can be APE, WAV, or FLAC.
ConvertAPEwrites one APE file to APE, WAV, or FLAC when there is no CUE sheet.- Detects non-UTF8 zip filenames automatically.
This library provides a queue, and a common interface to extract files. It does not do the heavy lifting, and relies on these libraries to extract files:
- RAR: nwaples/rardecode
- 7-Zip: bodgit/sevenzip
- ISO: Unpackerr/iso9660
- UDF: golift/udf
- ASAR: golift/asar
- APE: golift/ape
- FLAC: mewkiz/flac
- CPIO: cavaliergopher/cpio
- RPM: cavaliergopher/rpm
- ar / deb: peterebden/ar
- Brotli: andybalholm/brotli
- LZ4: pierrec/lz4
- XZ: therootcompany/xz
- LZMA: ulikunitz/xz
- Zstandard, S2, Snappy, Zlib: klauspost/compress
- LZW: sshaman1101/dcompress
Zip, Gzip, Tar and Bzip are all handled by the standard Go library.
package main
import (
"log"
"os"
"strings"
"golift.io/xtractr"
)
// Logger satisfies the xtractr.Logger interface.
type Logger struct {
xtractr *log.Logger
debug *log.Logger
info *log.Logger
}
// Printf satisfies the xtractr.Logger interface.
func (l *Logger) Printf(msg string, v ...interface{}) {
l.xtractr.Printf(msg, v...)
}
// Debug satisfies the xtractr.Logger interface.
func (l *Logger) Debugf(msg string, v ...interface{}) {
l.debug.Printf(msg, v...)
}
// Infof printf an info line.
func (l *Logger) Infof(msg string, v ...interface{}) {
l.info.Printf(msg, v...)
}
func main() {
log := &Logger{
xtractr: log.New(os.Stdout, "[XTRACTR] ", 0),
debug: log.New(os.Stdout, "[DEBUG] ", 0),
info: log.New(os.Stdout, "[INFO] ", 0),
}
q := xtractr.NewQueue(&xtractr.Config{
Suffix: "_xtractd",
Logger: log,
Parallel: 1,
FileMode: 0644, // ignored for tar files.
DirMode: 0755,
})
defer q.Stop() // Stop() waits until all extractions finish.
response := make(chan *xtractr.Response)
// This sends an item into the extraction queue (buffered channel).
q.Extract(&xtractr.Xtract{
Name: "my archive", // unused by this library.
Path: "/tmp/archives", // can also be a direct file.
CBChannel: response, // queue responses are sent here.
})
// Queue always sends two responses. 1 on start and again when finished (error or not)
resp := <-response
log.Infof("Extraction started: %s", strings.Join(resp.Archives.List(), ", "))
resp = <-response
if resp.Error != nil {
// There is possibly more data in the response that is useful even on error.
// ie you may want to cleanup any partial extraction.
log.Printf("Error: %v", resp.Error)
}
log.Infof("Extracted Files:\n - %s", strings.Join(resp.NewFiles, "\n - "))
}This example shows ExtractFile() with a very simple XFile.
You can choose output path, as well as file and dir modes.
Failing to provide OutputDir results in unexpected behavior.
ExtractFile() attempts to identify the type of file. If you
know the file type you may call the direct method instead:
ExtractZIP(*XFile),ExtractRAR(*XFile),Extract7z(*XFile)ExtractTar(*XFile),ExtractTarGzip(*XFile),ExtractTarBzip(*XFile),ExtractTarXZ(*XFile)ExtractGzip(*XFile),ExtractBzip(*XFile),ExtractXZ(*XFile),ExtractZstandard(*XFile)ExtractISO(*XFile),ExtractASAR(*XFile),ExtractRPM(*XFile),ExtractCPIO(*XFile)ExtractCUE(*XFile)for a.cueor.cue.txtnext to a FLAC or APE imageConvertAPE(*XFile)for one APE file and no CUE sheet
A lone .ape is not an archive. ExtractFile does not convert it.
ExtractFile does follow a .cue whose FILE line names a FLAC or APE image.
package main
import (
"log"
"strings"
"golift.io/xtractr"
)
func main() {
x := &xtractr.XFile{
FilePath: "/tmp/myfile.zip",
OutputDir: "/tmp/myfile", // do not forget this.
}
// size is how many bytes were written.
// files may be nil, but will contain any files written (even with an error).
size, files, _, err := xtractr.ExtractFile(x)
if err != nil || files == nil {
log.Fatal(size, files, err)
}
log.Println("Bytes written:", size, "Files Extracted:\n -", strings.Join(files, "\n -"))
}XFile embeds APEOpts. ExtractCUE and ConvertAPE both read it.
An empty Output re-encodes APE. Compression 0 is normal (2000).
WAV and FLAC ignore compression.
size, files, err := xtractr.ConvertAPE(&xtractr.XFile{
FilePath: "/tmp/song.ape",
OutputDir: "/tmp/song",
APEOpts: xtractr.APEOpts{Output: xtractr.AudioFormatFLAC},
})An APE CUE split this package can decode is cut on the cue sample. Files it cannot decode stay on a frame copy, and only when the output is still APE. That copy keeps whole frames, so a cue in the middle of a frame overlaps the next track by at most one frame.
// XFile defines the data needed to extract an archive.
type XFile struct {
// APEOpts selects the container and APE compression for ConvertAPE and for
// CUE splits of APE images. The zero value re-encodes APE at normal (2000).
APEOpts
// Path to archive being extracted.
FilePath string
// Folder to extract archive into.
OutputDir string
// Write files with this mode.
FileMode os.FileMode
// Write folders with this mode.
DirMode os.FileMode
// Suffix brands cross-device copy siblings as a known extra extension
// (e.g. movie.mkv.xtractr_partial). Empty uses DefaultSuffix.
Suffix string
// (RAR/7z) Archive password. Blank for none. Gets prepended to Passwords, below.
Password string
// (RAR/7z) Archive passwords (to try multiple).
Passwords []string
// FileWorkers controls how many files within a single archive are extracted
// concurrently. Only effective for random-access formats (ZIP, 7z, ASAR).
// Streaming formats ignore this. 0 or 1 = sequential.
// Total concurrent I/O when using the queue = Config.Parallel * FileWorkers.
FileWorkers int
// MaxBytes is the maximum uncompressed bytes written for this archive.
// 0 means unlimited.
MaxBytes uint64
// MaxFiles is the maximum files, directories, and symlinks created for this
// archive. 0 means unlimited.
MaxFiles int
// MaxRatio is the maximum bytesWritten / archiveFileSize. 0 means unlimited.
MaxRatio float64
// AllowSymlinks allows FilePath to be a symbolic link to an archive.
AllowSymlinks bool
// Progress is called periodically during file extraction.
// Contains info about the progress of the extraction.
// This is not called if an Updates channel is also provided.
Progress func(Progress)
// If an Updates channel is provided, all Progress updates are sent to it.
// Contains info about the progress of the extraction.
Updates chan Progress
// If the archive only has one directory in the root, then setting
// this true will cause the extracted content to be moved into the
// output folder, and the root folder in the archive to be removed.
SquashRoot bool
// SkipOnRecursion, if set by an extractor, lists paths that were copied into
// the output (e.g. a CUE sheet) and must not be re-extracted when recursing.
SkipOnRecursion []string
}Extraction caps (0 means unlimited):
MaxBytes— uncompressed bytes writtenMaxFiles— files, directories, and symlinks createdMaxRatio—bytesWritten / archiveFileSize
On a queue job (Xtract/Config) these apply per top-level archive. Two sibling rips in one folder each get a full cap. Extras from that folder share the tighter leftover (smallest remaining byte/ratio room, then files). MaxRatio keeps the parent archive size, not child compressed sizes. Bytes of an intermediate archive that same tracker wrote are left out of the ratio numerator when that archive is extracted; MaxBytes still counts them. Standalone XFile stays per-archive.
Queue extras caps on Xtract and Config (0 inherits Config, then unlimited):
MaxNested— archives extracted from one source folder's extras passExtrasMaxDepth— how deep that extras walk goes
AllowSymlinks is on Filter/Xtract and XFile (default false). When set, the initial search may include
symlink-named archives. The extras pass never follows archive-member links.
Exceeding MaxBytes/MaxFiles/MaxRatio returns those errors and stops the extract (and extras if the tighter leftover is already used).
Exceeding MaxNested returns ErrMaxNested and deletes that folder's extract output.
Queue jobs inherit Config values when the Xtract field is 0.
MaxFiles counts claimed-plus-created entries, not just created. An archive whose header claims more entries
than MaxFiles fails fast with ErrMaxFiles even when every entry already exists on disk and nothing new would
be created. This fail-closed default is intentional for a security cap: headers can understate, so the runtime
write path always re-checks regardless of what the header claimed.