Documentation
¶
Overview ¶
Package sstable implements a Sorted String Table (SSTable) for efficient storage and retrieval of key-value pairs. An SSTable is an immutable, ordered file format that maps keys to values, optimized for high-throughput sequential read/write operations.
The SSTable format includes:
- A header with magic number and version
- A sequence of sorted key-value records
- A sparse index for efficient key lookups
- A footer containing the index offset and magic number
Key features:
- Immutable: Once written, records cannot be modified
- Sorted: Records are stored in key order
- Efficient: Binary search enabled through sparse indexing
- Sequential: Optimized for sequential read/write patterns
Basic usage:
// Writing records
writer, err := sstable.OpenWriter(file, nil)
if err != nil {
log.Fatal(err)
}
record := partition.RecordImpl{
ID: "key1",
PartitionKey: "partition1",
Timestamp: time.Now(),
Data: []byte("value1"),
}
if err := writer.Write(&record); err != nil {
log.Fatal(err)
}
writer.Close()
// Reading records
reader, err := sstable.OpenReader(file, nil)
if err != nil {
log.Fatal(err)
}
defer reader.Close()
// Get specific record
record, err := reader.Get("key1")
// Iterate all records
for record := range reader.All() {
// Process record
}
File Format:
- Header (16 bytes):
- Magic number (8 bytes, "SSTB" in hex)
- Format version (8 bytes)
- Records:
- Sequence of variable-length records in sorted order
- Sparse Index:
- Count of index entries (8 bytes)
- Sequence of (key, offset) pairs
- Footer (16 bytes):
- Index offset (8 bytes)
- Magic number (8 bytes, "ENDB" in hex)
The sparse index enables efficient key lookups by maintaining a subset of keys and their file offsets, reducing the amount of data that needs to be read during binary search operations.
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( ErrTableClosed = errors.New("sstable: table already closed") ErrInvalidKey = errors.New("sstable: invalid key") ErrKeyNotFound = errors.New("sstable: key not found") ErrCorruptedTable = errors.New("sstable: corrupted table data") ErrWriteError = errors.New("sstable: records must be written in sorted order") )
Common errors that can be returned by SSTable operations.
Functions ¶
This section is empty.
Types ¶
type BufferReaderSeeker ¶
type BufferReaderSeeker struct {
// contains filtered or unexported fields
}
BufferReaderSeeker stores pointers to a io.Reader and a io.Writer and a io.Seeker. It implements [ReadWriteSeeker].
func NewReadSeeker ¶
func NewReadSeeker(r io.ReadSeeker, size int) *BufferReaderSeeker
type Options ¶
type Options struct {
// BufferSize is the size of the read/write buffer.
BufferSize int
}
Options configures the behavior of an SSTable.
type TableReader ¶
type TableReader struct {
// contains filtered or unexported fields
}
TableReader represents the reading component of an SSTable.
func OpenReader ¶
func OpenReader(rs io.ReadSeeker, opts *Options) (*TableReader, error)
OpenReader initializes a new SSTableReader using the provided ReadSeeker.
Example ¶
ExampleOpenReader demonstrates reading an sstable back.
package main
import (
"bytes"
"fmt"
"time"
"github.com/davidvella/xp/partition"
"github.com/davidvella/xp/sstable"
)
func main() {
// Create an in-memory buffer to act as the file.
var buf bytes.Buffer
// Initialize the SSTable writer.
writer, err := sstable.OpenWriter(&buf, nil)
if err != nil {
fmt.Printf("Error opening writer: %v\n", err)
return
}
// Create a new record.
record := partition.RecordImpl{
ID: "exampleKey",
PartitionKey: "examplePartition",
Timestamp: time.Now(),
Data: []byte("exampleValue"),
}
// Write the record to the SSTable.
if err := writer.Write(&record); err != nil {
fmt.Printf("Error writing record: %v\n", err)
return
}
// Close the writer to flush data.
if err := writer.Close(); err != nil {
fmt.Printf("Error closing writer: %v\n", err)
return
}
// Initialize the SSTable reader.
readerSeeker := bytes.NewReader(buf.Bytes())
reader, err := sstable.OpenReader(readerSeeker, nil)
if err != nil {
fmt.Printf("Error opening reader: %v\n", err)
return
}
defer reader.Close()
// Retrieve the record by key.
readRecord, err := reader.Get("exampleKey")
if err != nil {
fmt.Printf("Error reading record: %v\n", err)
return
}
// Print the retrieved record's data.
fmt.Println(string(readRecord.GetData()))
// Read all records
for rec := range reader.All() {
fmt.Println(rec.GetID())
}
}
Output: exampleValue exampleKey
type TableWriter ¶
type TableWriter struct {
// contains filtered or unexported fields
}
TableWriter represents the writing component of an SSTable.
func OpenWriter ¶
func OpenWriter(rw io.Writer, opts *Options) (*TableWriter, error)
OpenWriter initializes a new SSTableWriter using the provided WriteSeeker.
Example ¶
ExampleOpenWriter demonstrates creating an SSTable, writing a record, and reading it back.
package main
import (
"bytes"
"fmt"
"time"
"github.com/davidvella/xp/partition"
"github.com/davidvella/xp/sstable"
)
func main() {
// Create an in-memory buffer to act as the file.
var buf bytes.Buffer
// Initialize the SSTable writer.
writer, err := sstable.OpenWriter(&buf, nil)
if err != nil {
fmt.Printf("Error opening writer: %v\n", err)
return
}
// Create a new record.
record := partition.RecordImpl{
ID: "exampleKey",
PartitionKey: "examplePartition",
Timestamp: time.Now(),
Data: []byte("exampleValue"),
}
// Write the record to the SSTable.
if err := writer.Write(&record); err != nil {
fmt.Printf("Error writing record: %v\n", err)
return
}
// Close the writer to flush data.
if err := writer.Close(); err != nil {
fmt.Printf("Error closing writer: %v\n", err)
return
}
}
func (*TableWriter) Close ¶
func (w *TableWriter) Close() error
Source Files