Skip to content

Latest commit

 

History

History
122 lines (93 loc) · 4.33 KB

package-layout.md

File metadata and controls

122 lines (93 loc) · 4.33 KB
Raw

Package layout

This document describes the package layout within franz-go. Reading this document should help you to understand how this library is supposed to be used.

franz-go consists of multiple Go packages, where each serves its own purpose. The primary package is kgo, but supplementary packages are to be used as necessary (for example, for sasl authentication).

Package kgo

The package kgo is the main package as it provides all the core functionality to interact with Kafka. It has support for transactions, regex topic consuming, the latest partitioning strategies, data loss detection, closest replica fetching, and more. If a client KIP exists, this library aims to support it.

This library attempts to provide an intuitive API while interacting with Kafka the way Kafka expects (timeouts, etc.). In some cases, this makes the tricky corners of Kafka more explicit and manual to deal with correctly. In the general case, though, I hope this library is satisfactory.

Package kmsg

The main package is kgo, but kmsg also exists, containing autogenerated code for every request and response type. kmsg implements the messages as defined in the Kafka protocol. You will want to use this package if you need more control how to access Kafka, or you are using an API within the kgo package that requires you to process a pure Kafka struct definition. A common use case is constructing and sending admin api requests.

Usage:

req := kmsg.NewMetadataRequest()
res, err := req.RequestWith(ctx, client)
if err != nil {
    panic(err) // error during request has happened (e. g. context cancelled)
}

Package sasl

Package sasl specifies interfaces that any sasl authentication (PLAIN, GSSAPI, SCRAM, ...) must provide to interop with Kafka SASL. This package is more specific to SASL and not to Kafka - hence it has its own package, and hence the lack of a k prefix in the package name.

Usage:

seeds := []string{"localhost:9092"}

// SASL Plain credentials
user := ""
password := ""

opts := []kgo.Opt{
    kgo.SeedBrokers(seeds...),
    // SASL Options
    kgo.SASL(plain.Auth{
        User: user,
        Pass: password,
    }.AsMechanism()),
}
client, err := kgo.NewClient(opts...)

Package kversion

Package kversion specifies api key versions corresponding to Kafka versions. Kafka technically has internal broker versions that bump multiple times per release. The kversion package defines only releases and tip (and stable, which matches the most recent release).

You usually use this package to set the max or min versions that are acceptable for requests issued with the kgo package, however, there are additional miscellaneous useful functions related to Kafka api key versioning.

Setting a max version may be important to you if you use any kmsg type directly and do not call the type's Default function. If you do this, then you may accidentally opt in to a new field's default 0 value being important and triggering unforeseen behavior. If you pin the client with a MaxVersions option, you can prevent this.

Additionally, if you are a client talking to brokers of unknown version, you may want to avoid downgrading a request too much to avoid a field in the request being silently elided.

Usage:

client, err := kgo.NewClient(
    kgo.SeedBrokers(seeds...),
    kgo.MaxVersions(kversion.V2_4_0()),
    kgo.MinVersions(kversion.V1_0_0()),
)

Package kerr

Package kerr is dedicated to all Kafka errors. Most responses from Kafka contain an ErrorCode field somewhere within it. The kerr package can be used to translate that int16 error code into meaningful text.

Usage:

err := kerr.ErrorForCode(response.ErrorCode)
if err != nil {
    // Handle error
}

Package kbin

Package kbin is a small utility package that is for speaking the binary protocol that Kafka speaks. This is useful for reading from and writing to Kafka.