ThreadSafeQueueLib

Exploring Concurrency and Lock-Free Data Structures in Modern C++

A group learning project by Badri Bishal Das , Sujal Patnaik , Sudipto Ghosh , Biswabhusan Samal at IIT Guwahati

Read Blog Here

---

Hello!

ThreadSafeQueueLib is a group project created to dive deep into C++ and concurrency concepts. Instead of just reading about threads, locks, and atomics, we decided to build a family of thread-safe queues from scratch.

Our goal wasn't to write the next big production library, but to really understand what makes concurrent programming so challenging and rewarding. A standard std::queue isn't safe to use when multiple threads are pushing and popping at the same time. To solve this, we implemented both blocking (mutex-based) and lock-free queues.

---

What We Built

We explored different queue designs depending on how many threads are involved:

• spsc (Single-Producer Single-Consumer) - Lock-free versions (both bounded and unbounded).
• mpsc (Multi-Producer Single-Consumer) - Unbounded lock-free.
• mpmc (Multi-Producer Multi-Consumer) - Both traditional locking and lock-free implementations.

We used C++ templates to write this as a header-only library, allowing the compiler to optimize the queues at build time depending on the types of data being stored.

How to use this?

prerequisites : C++23, CMake

Integration

You can easily pull this into your own CMake projects, or clone the Git repo directly:

include(FetchContent)
FetchContent_Declare(
    tsqlib
    GIT_REPOSITORY https://github.com/badri41/ThreadSafeQueueLib.git
    GIT_TAG main
)
FetchContent_MakeAvailable(tsqlib)

target_link_libraries(your_target PRIVATE ThreadSafeQueueLib)

A Quick Example

#include <iostream>
#include <thread>
#include <tsfqueue.hpp> // Main header

int main() {
    // A queue where one thread pushes and one thread pops
    tsfqueue::spscUnbounded<int> queue;

    std::thread producer([&]() {
        for (int i = 0; i < 1000; ++i) {
            queue.push(i);
        }
    });

    std::thread consumer([&]() {
        int value;
        for (int i = 0; i < 1000; ++i) {
            // Keep trying to pop until we get a value
            while (!queue.tryPop(value)) {
                std::this_thread::yield(); 
            }
        }
    });

    producer.join();
    consumer.join();

    std::cout << "All done without data races!\n";
    return 0;
}

---

Building and Testing

We used GoogleTest and Clang's ThreadSanitizer to make sure our lock-free logic actually works and doesn't contain hidden data races.

mkdir build && cd build
cmake ..
cmake --build . -j4
ctest --output-on-failure -j4

---

Benchmarking

benchThroughput.cpp To measure the raw throughput (operations per second) of the different queue architectures, we included a standalone, dependency-free benchmarking tool. It evaluates how each queue scales under different levels of thread contention by varying producers and consumers from 1 up to 16 threads.

To run the benchmarks with maximum performance, compile the source using the -O3 optimization flag and the C++23 standard:

# Compile the benchmark
g++ -O3 -std=c++23 -pthread -I./include benchmarking/benchThroughput.cpp -o benchThroughput.exe

# Run the executable
.\benchThroughput.exe  //In powershell

benchLatency.cpp While throughput measures how many items are processed, latency measures how fast a single item travels from a producer to a consumer. To measure this, our benchmark pushes ultra-precise nanosecond timestamps through the queues and calculates the transit time.

Because averages can be heavily skewed by OS-level background noise or thread context-switching, the benchmark sorts millions of operations to provide industry-standard Percentile Metrics (measured in microseconds, $\mu s$):

• p50 (Median): The typical, everyday performance of the queue.
• p99 & p99.9 (Tail Latency): The absolute worst-case scenarios. In blocking queues, this number spikes massively due to "Lock Convoys" (the OS pausing threads to wait for a std::mutex). In lock-free queues, this number stays incredibly low.

# Compile the benchmark
g++ -O3 -std=c++23 -pthread -I./include benchmarking/benchLatency.cpp -o benchLatency.exe

# Run the executable
.\benchLatency.exe  //In powershell

The commands will print live throughput results to the terminal and save a formatted table to benchmarkResults.txt and latencyResults.txt in the current directory.

Benchmark Results Summary

Here are the benchmarking results generated on our test system.

1. spsc (Single-Producer Single-Consumer)

Because there is no contention between multiple producers or multiple consumers, SPSC architectures achieve incredibly high throughput. However, to truly prove the value of lock-free programming, we benchmarked our lock-free implementations against a traditional std::mutex + std::queue.

The Lock-Free Advantage (Payload Scalability):

• For very small payloads (4 Bytes), a traditional std::mutex queue performs extremely well (up to 20M Ops/sec) due to modern OS futex optimizations and std::deque block allocation.
• However, as payload size increases to 1KB, the critical section inside the mutex grows, causing the producer and consumer to lock each other out during data copies.
• In contrast, our Bounded Lock-Free SPSC queue shines under heavy payloads, sustaining ~11.2M Ops/sec. By eliminating locks, the producer and consumer can copy massive 1KB payloads into the ring buffer completely independently, resulting in nearly a 2x performance increase over the mutex baseline (6.8M Ops/sec).
• (Note: The Unbounded Lock-Free queue drops to ~5.8M Ops/sec at 1KB payloads due to the overhead of the OS dynamically allocating new/delete nodes).

2. mpsc Unbounded (Multi-Producer Single-Consumer)

As the number of concurrent producers increases, contention on the queue's tail pointer increases, leading to more CAS (Compare-And-Swap) retries. This naturally decreases overall throughput and increases tail latency.

Throughput (1,000,000 Operations):

• 1 Producer: 1.09e+07 Ops/sec
• 2 Producers: 6.62e+06 Ops/sec
• 4 Producers: 5.91e+06 Ops/sec
• 8 Producers: 4.63e+06 Ops/sec
• 16 Producers: 3.71e+06 Ops/sec

Latency (Transit Time - 1,000,000 Operations):

• 1 Producer: p50: 0.70 us | p99: 123.50 us
• 2 Producers: p50: 7207.30 us | p99: 21634.40 us
• 4 Producers: p50: 39352.60 us | p99: 65887.70 us
• 8 Producers: p50: 68444.10 us | p99: 107720.30 us

(Notice how tail latency spikes significantly as threads compete for the lock-free enqueue).

---

Acknowledgements

This group project was built under the awesome mentorship and guidance of Toshit Jain Bhaiya as part of the Coding Club, IIT Guwahati.

• Contributors: Badri Bishal Das, Sujal Patnaik, Sudipto Ghosh & Biswabhusan Samal
• Blog Write-up: Read Here