fastlog

A high performance Rust library for asynchronous logging

Version 0.3 adds opt-in worker rendering with reusable storage per producer thread. Existing logging calls and the default producer formatter remain available. Message arguments still format on the producer; the worker can render the common line header and write the file. See the changelog for the release's additions and compatibility details.

Usage

Fastlog requires a minimum rustc version of 1.85.0.

To use fastlog, first add this to your Cargo.toml:

[dependencies]
fastlog = "0.3"
log = "0.4"

Then initialize the logger and use the log macros.

Example

fn main() {
    fastlog::LogBuilder::new().build().unwrap().init().unwrap();
    log::info!("Hello, world.");
    log::logger().flush();
}

The final flush matters: the installed logger is never dropped, so records still queued on the worker when the process exits are lost. Flush before returning from main and on every early exit path.

More examples can be found under examples directory.

Worker rendering for ordinary messages

Existing format and format_with_capacity callbacks run on the producer and retain the global message-count queue. To use reusable producer storage and worker-side header rendering, select worker_format during initialization:

use std::fmt::Write as _;
use fastlog::{LogBuilder, WorkerConfig};

let mut builder = LogBuilder::new();
builder.worker_format(WorkerConfig::default(), |output, record| {
    write!(output, "{:?} {} {}: {}",
        record.timestamp(),
        record.thread_name().unwrap_or_default(),
        record.level(),
        record.message())
});
builder.build().unwrap().init().unwrap();
fastlog::prepare_current_thread().unwrap();

log::info!("processed {} messages", 42);
log::logger().flush();

Logging callsites keep using log. Message Display and Debug formatting still run on the producer. The worker receives owned message bytes and the original timestamp, thread name, level, target, module, file, and line. The renderer receives an empty reusable String; Fastlog adds one newline after the callback. Arbitrary old formatter callbacks are not moved across threads. Structured key-value fields are outside this ordinary-message interface.

Each producer gets a pre-touched 512 KiB byte queue and 4 KiB initial scratch buffer by default. Call prepare_current_thread on each sensitive thread after logger initialization to move registration and allocation before its hot loop. Other threads register on their first log call. Preparation does not lock pages in memory or warm arbitrary application formatting code.

Normal publication does not notify the worker. When idle, the worker parks for 1 ms by default; scheduling and output work can extend delivery time. Full queues block the producer and wake the worker. Oversized records use an owned allocation and an ordered marker in the same byte queue. Each lane has one overflow slot, in addition to producer and worker buffers. Scratch buffers can grow and retain their capacity, so byte queue capacity is not a total memory limit. Recursive producer formatting and logging during thread-local teardown have allocation-capable fallback paths.

Use WorkerConfig::new(queue_bytes, scratch_bytes, idle_interval) to choose different settings. These are bytes per producer, while capacity(n) remains the legacy global message count, including rendezvous behavior at zero. Mixing an explicit capacity(n) with a worker renderer is a build error.

The worker keeps publication order within a producer and budgets each lane's normal drain so a busy producer cannot monopolize the worker. Cross-thread ordering may differ from the legacy shared queue. Flush takes a finite snapshot of each registered queue and waits for that work to be written. Concurrent flush requests have separate replies. File flushing does not imply fsync or durable storage.

Logger::try_flush and Logger::shutdown return worker errors. Shutdown consumes an uninstalled logger and joins its worker; the global log logger remains owned by the facade and should be flushed before application exit. Standard logging and flush methods panic on errors. A worker panic is reported when panic unwinding is enabled; a panic = "abort" build retains Rust's abort behavior. A renderer must not log to or flush its own logger; Fastlog detects that case. Drop drains and joins, and avoids a second panic during unwinding.

Performance benchmarking

Run the Rust benchmark targets from the repository root:

cargo bench --all-features --bench logging_latency
cargo bench --all-features --bench producer_latency
cargo bench --all-features --bench allocation_counts

logging_latency uses Criterion for aggregate measurements. producer_latency records individual logging-call durations, including waits when the queue fills. allocation_counts measures allocations in a separate process so its instrumentation does not affect the latency measurements. The benchmarks use the platform null device by default; they do not measure storage durability.