jetget

JetGet

JetGet is an open-source, thread-safe Go library for high-speed concurrent file downloading over HTTP. It features resumable downloads, real-time progress tracking, and optimized disk I/O handling via pre-allocation and concurrent file writing.

Features

• Concurrent Downloading: Splits the file into chunks and downloads them in parallel, reducing overall download time.
• Resumable Downloads: Saves internal state periodically to a .jetget JSON file, allowing downloads to be resumed even after application crashes or network interruptions.
• Pre-allocation: Truncates the file to the final size before downloading to prevent disk fragmentation.
• Zero File Merging: Uses os.File.WriteAt to write chunks directly into the final file concurrently, avoiding expensive post-download merging.
• Functional Options: Clean API design pattern for client configuration.
• HTTP/3 (QUIC) Support: Next-generation HTTP protocol support for faster downloads on unreliable networks.
• Real-time Progress: Channel-based, non-blocking progress updates for building progress bars or UI.

Installation

go get github.com/Locon213/jetget

Quick Start

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/Locon213/jetget"
)

func main() {
	// 1. Initialize the client with functional options
	client := jetget.NewClient(
		jetget.WithConcurrency(8),
		jetget.WithOutputDir("./downloads"),
		jetget.WithUserAgent("MyCustomDownloader/1.0"),
		jetget.WithHTTP3(), // Enable HTTP/3 for faster connections
	)

	url := "https://example.com/large-file.zip"
	ctx := context.Background()

	// 2. Start the download in the background
	task, err := client.Download(ctx, url)
	if err != nil {
		log.Fatalf("Download failed to start: %v", err)
	}

	// 3. (Optional) Track progress in real-time
	go func() {
		for stats := range task.Progress() {
			if stats.TotalBytes > 0 {
				percent := float64(stats.DownloadedBytes) / float64(stats.TotalBytes) * 100
				fmt.Printf("\rDownloaded: %d / %d bytes (%.2f%%)", stats.DownloadedBytes, stats.TotalBytes, percent)
			} else {
				fmt.Printf("\rDownloaded: %d bytes (unknown total size)", stats.DownloadedBytes)
			}
		}
	}()

	// 4. Wait for the download to finish
	if err := task.Wait(); err != nil {
		log.Fatalf("\nDownload error: %v", err)
	}

	fmt.Printf("\nDownload completed successfully! File saved to: %s\n", task.FilePath)
}

How It Works

1. Initialization: The client sends an HTTP HEAD request to query the Content-Length and checks if the server supports Accept-Ranges: bytes.
2. Pre-allocation: An empty file of the exact Content-Length is created via os.Truncate.
3. Chunking: The file goes through mathematical chunking and is downloaded concurrently by n configured goroutines.
4. Resumability: Every second, a background goroutine exports ChunkState to a sidecar JSON file (e.g., file.zip.jetget). Upon success, this state file is pruned. If the application restarts, it restores the state from this file and continues from previous offsets.

Dependencies

JetGet strictly relies on the Go Standard Library with additions for robust concurrency and protocol handling:

• golang.org/x/sync/errgroup
• github.com/quic-go/quic-go/http3

License

MIT License