README
¶
stringy
Unicode string encoding handling for Go. Provides helpers for working with UTF-8, UTF-16, and UTF-32 string representations, including code unit and code point counting.
Why does this library exist?
Prior to Go 1.23, there was no built-in way to count UTF-16 code units in a string. This is important for interoperability with systems and protocols (such as JavaScript, Windows APIs, and some network protocols) that use UTF-16 as their string encoding. Go's built-in string type is UTF-8, and while it is easy to work with runes (Unicode code points), handling UTF-16 code units correctly is non-trivial, especially for characters outside the Basic Multilingual Plane (BMP) that require surrogate pairs.
This library provides types and helpers that "just work" for handling UTF-16, UTF-32, and UTF-8 string representations, including correct code unit and code point counting, and conversion between representations.
How it works
Utf16is a slice ofuint16representing a UTF-16 encoded string. It implements methods for appending, writing, and converting to/from Go strings, as well as counting code units and code points.Utf32is a slice ofrune(Unicode code points), with similar methods for manipulation and conversion.Utf8is a wrapper around Go's built-in string type, with helpers for code unit and code point counting.
All types implement the String interface, which provides methods for getting the string representation, code unit count, and code point count.
Example usage
import "github.com/R167/stringy"
var s stringy.Utf16
s.WriteString("hello 𝄞 world")
fmt.Println(s.String()) // prints: hello 𝄞 world
fmt.Println(s.CodeUnits()) // prints: number of UTF-16 code units
fmt.Println(s.CodePoints()) // prints: number of Unicode code points (runes)
Documentation
¶
Overview ¶
stringy provides utilities for working with UTF-8, UTF-16, and UTF-32 encoded strings.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Utf16CodeUnits ¶
Utf16CodeUnits returns the number of code units in the UTF-16 encoding of the string.
Types ¶
type String ¶
type String interface {
// String returns the string representation of the string.
String() string
// CodeUnits returns the number of code units in the string.
CodeUnits() int
// CodePoints returns the number of code points in the string.
// This is the number of runes in the string and what you would use for character-based operations.
CodePoints() int
}
type Utf16 ¶
type Utf16 []uint16
Utf16 is a slice of uint16 values that represents a UTF-16 encoded string.
*Utf16 implements the io.StringWriter and io.Writer interfaces.
func (*Utf16) AppendRunes ¶
AppendRunes appends a rune slice to the Utf16 slice.
func (Utf16) CodePoints ¶
CodePoints returns the number of code points in the Utf16 slice.
type Utf32 ¶
type Utf32 []rune
Utf32 is a slice of runes that represents a UTF-32 encoded string.
*Utf32 implements the io.StringWriter and io.Writer interfaces.
func (*Utf32) AppendRunes ¶
AppendRunes appends a rune slice to the Utf32 slice.
type Utf8 ¶
type Utf8 string
Utf8 is a string type that supports UTF-8 encoding.
It is implemented as a string since Go's built-in string type is already UTF-8 encoded. Because of this, Utf8 does not implement the same interfaces for updating as the other types.
func (Utf8) CodePoints ¶
CodePoints returns the unicode charachter length of this string.
Source Files