Object Store

A high-performance, multi-threaded object store implementation in C that provides persistent storage and retrieval of binary data blocks. Built as a client-server system using Unix domain sockets and POSIX threads, it's designed to handle concurrent requests from multiple clients efficiently.

🎯 Problem Statement

Modern applications often need to store and retrieve arbitrary binary data (files, images, serialized objects) in a simple, efficient manner. This object store provides:

• Isolated storage spaces for each client
• Concurrent access via multi-threading
• Simple API for store/retrieve/delete operations
• Persistent storage using the filesystem
• Graceful shutdown with signal handling

✨ Features

• 🚀 Multi-threaded Architecture: Thread pool handles multiple concurrent clients
• 🔒 Per-client Namespacing: Each client has an isolated storage space
• 📡 Unix Domain Sockets: Efficient local IPC communication
• 💾 File-based Persistence: Objects stored in filesystem for durability
• 📊 Runtime Statistics: SIGUSR1 signal provides server metrics
• 🛡️ Graceful Shutdown: Handles SIGINT/SIGQUIT without data corruption
• 🧪 Comprehensive Tests: Automated test suite with 50+ concurrent clients
• 📝 Simple Protocol: Text-based protocol for easy debugging

🏗️ Architecture

┌─────────────────────────────────────────────────────┐
│                   Client Application                │
├─────────────────────────────────────────────────────┤
│              Client Library (libstore.a)            │
│  os_connect() | os_store() | os_retrieve() | ...   │
└────────────────────┬────────────────────────────────┘
                     │ Unix Socket
                     │ (objstore.sock)
┌────────────────────▼────────────────────────────────┐
│                  Object Store Server                │
│  ┌────────────────────────────────────────────┐    │
│  │         Thread Pool (Worker Threads)       │    │
│  │  ┌──────┐  ┌──────┐  ┌──────┐  ┌──────┐   │    │
│  │  │Client│  │Client│  │Client│  │ ...  │   │    │
│  │  │Handler│ │Handler│ │Handler│ │      │   │    │
│  │  └───┬──┘  └───┬──┘  └───┬──┘  └──────┘   │    │
│  └──────┼─────────┼─────────┼────────────────┘    │
│         │         │         │                      │
│  ┌──────▼─────────▼─────────▼───────────────┐     │
│  │         Hash Table (Client Management)    │     │
│  └──────────────────┬────────────────────────┘     │
└─────────────────────┼──────────────────────────────┘
                      │
           ┌──────────▼──────────┐
           │   Filesystem         │
           │   data/              │
           │   ├─ client1/        │
           │   │  ├─ object1      │
           │   │  └─ object2      │
           │   └─ client2/        │
           │      └─ object1      │
           └─────────────────────┘

📋 Table of Contents

• Quick Start
• Installation
• Usage
• API Reference
• Protocol Specification
• Testing
• System Requirements
• Project Structure
• License

🚀 Quick Start

Get the object store running in under 30 seconds:

# Clone the repository
git clone https://github.com/GennaroBaratta/objectStoreSOL.git
cd objectStoreSOL

# Build the project
make all

# Start the server (in background)
./server &

# Run a test client
./client myclient 1  # Store 20 objects of increasing sizes

# Get server statistics
killall -SIGUSR1 server

# Graceful shutdown
killall -SIGINT server

📦 Installation

Prerequisites

• GCC compiler with C99 support
• POSIX-compliant operating system (Linux, macOS, BSD)
• pthread library
• Make build system

Build from Source

# Clone the repository
git clone https://github.com/GennaroBaratta/objectStoreSOL.git
cd objectStoreSOL

# Build all components (server, client library, test client)
make all

# Run automated tests
make test

# Clean build artifacts
make clean

Build Artifacts

After building, you'll have:

• server - The object store server executable
• client - Test client for basic operations
• imgclient - Example client for storing/retrieving images
• libstore.a - Static library for linking with your applications

💻 Usage

Starting the Server

./server

The server will:

• Create a Unix socket at ./objstore.sock
• Create a data/ directory for persistent storage
• Start listening for client connections
• Handle signals for statistics (SIGUSR1) and shutdown (SIGINT, SIGQUIT)

Using the Test Client

The test client supports three modes:

# Mode 1: Store 20 objects (100 bytes to 100,000 bytes)
./client client_name 1

# Mode 2: Retrieve and verify previously stored objects
./client client_name 2

# Mode 3: Delete all stored objects
./client client_name 3

Example:

# Store objects
./client alice 1
# Output:
# ---Client: alice
# ---Stores: 21
# ---Retrives: 0
# ---Deletes: 0
# ---Successes: 21
# ---Failures: 0
# ---Stored bytes: 1050105
# ---Retrived bytes: 0

# Retrieve and verify
./client alice 2

# Clean up
./client alice 3

Server Statistics

Get real-time server statistics:

killall -SIGUSR1 server

This prints to server stdout:

• Number of connected clients
• Total number of objects stored
• Total storage size

Graceful Shutdown

# Graceful shutdown (waits for active operations)
killall -SIGINT server

# Or using Ctrl+C if running in foreground

📚 API Reference

Client Library Functions

Link your application with libstore.a and include clientLibrary.h:

#include <clientLibrary.h>

int os_connect(char *name)

Establishes a connection to the object store and registers the client.

• Parameters:
  • name - Unique client identifier (must be valid POSIX filename)
• Returns:
  • 0 on success
  • -1 on failure
• Note: Must be called before any other operations

Example:

if (os_connect("my_client") == 0) {
    printf("Connected successfully\n");
} else {
    fprintf(stderr, "Connection failed\n");
    exit(1);
}

int os_store(char *name, void *block, size_t len)

Stores a binary data block in the object store.

• Parameters:
  • name - Object name (unique within client namespace)
  • block - Pointer to data buffer
  • len - Size of data in bytes
• Returns:
  • 0 on success
  • -1 on failure

Example:

const char *data = "Hello, Object Store!";
if (os_store("greeting", (void*)data, strlen(data)) == 0) {
    printf("Stored successfully\n");
}

void *os_retrieve(char *name)

Retrieves a previously stored object.

• Parameters:
  • name - Object name to retrieve
• Returns:
  • Pointer to newly allocated buffer containing the data (caller must free)
  • NULL on failure
• Note: Caller is responsible for freeing the returned memory

Example:

char *data = os_retrieve("greeting");
if (data != NULL) {
    printf("Retrieved: %s\n", data);
    free(data);  // Don't forget to free!
} else {
    fprintf(stderr, "Retrieval failed\n");
}

int os_delete(char *name)

Deletes an object from the store.

• Parameters:
  • name - Object name to delete
• Returns:
  • 0 on success
  • -1 on failure

Example:

if (os_delete("greeting") == 0) {
    printf("Deleted successfully\n");
}

int os_disconnect()

Closes the connection to the object store.

• Returns:
  • 0 on success
  • -1 on failure
• Note: Should be called before application exit

Example:

os_disconnect();

Complete Example

#include <clientLibrary.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main() {
    // Connect to object store
    if (os_connect("my_app") != 0) {
        fprintf(stderr, "Failed to connect\n");
        return 1;
    }

    // Store some data
    const char *message = "Hello, World!";
    if (os_store("msg1", (void*)message, strlen(message)) == 0) {
        printf("Stored: %s\n", message);
    }

    // Retrieve the data
    char *retrieved = os_retrieve("msg1");
    if (retrieved != NULL) {
        printf("Retrieved: %s\n", retrieved);
        free(retrieved);
    }

    // Delete the object
    os_delete("msg1");

    // Disconnect
    os_disconnect();

    return 0;
}

Compile and link:

gcc -o my_app my_app.c -L. -lstore -pthread

🔌 Protocol Specification

The object store uses a simple text-based protocol over Unix domain sockets.

Message Format

All messages consist of:

1. Header: ASCII text line terminated by \n
2. Data (optional): Binary data of length specified in header

Client → Server Requests

Operation	Request Format	Description
REGISTER	REGISTER <name>\n	Register client with given name
STORE	STORE <name> <len>\n<data>	Store binary data block
RETRIEVE	RETRIEVE <name>\n	Retrieve stored object
DELETE	DELETE <name>\n	Delete stored object
LEAVE	LEAVE\n	Disconnect from server

Server → Client Responses

Response	Format	Meaning
OK	OK\n	Operation succeeded
KO	KO <message>\n	Operation failed with error message
DATA	DATA <len>\n<data>	Retrieved data (RETRIEVE response)

Example Session

Client                          Server
  |                               |
  |------ REGISTER alice -------->|
  |<---------- OK ---------------|
  |                               |
  |-- STORE file1 13 ------------>|
  |   Hello, World!               |
  |<---------- OK ---------------|
  |                               |
  |---- RETRIEVE file1 ---------->|
  |<--- DATA 13 -----------------|
  |     Hello, World!             |
  |                               |
  |------ DELETE file1 ---------->|
  |<---------- OK ---------------|
  |                               |
  |-------- LEAVE -------------->|
  |<---------- OK ---------------|
  |                               |

🧪 Testing

Automated Test Suite

Run the full test suite:

make test

This executes:

1. First wave: 50 concurrent clients storing objects (mode 1)
2. Second wave:
   • 30 clients retrieving and verifying objects (mode 2)
   • 20 clients deleting objects (mode 3)
3. Outputs results to testout.log
4. Runs analysis script testsum.sh for summary

Manual Testing

# Start server
./server &
SERVER_PID=$!

# Store objects with multiple clients
./client alice 1 &
./client bob 1 &
./client charlie 1 &
wait

# Retrieve objects
./client alice 2
./client bob 2

# Get statistics
killall -SIGUSR1 server

# Cleanup
./client alice 3
./client bob 3
./client charlie 3

# Shutdown server
kill -SIGINT $SERVER_PID

Test Image Client

An example client for storing/retrieving images:

# Stores img.jpg and retrieves it as imgOut.jpg
./imgclient

Test Output Analysis

The testsum.sh script analyzes testout.log and provides:

• Total clients launched
• Success/failure counts per test mode
• Total bytes stored/retrieved
• Anomalies detected

💾 System Requirements

Operating System

• Linux (tested)
• macOS / BSD (should work, Unix domain sockets required)
• POSIX-compliant system

Dependencies

• GCC (or compatible C compiler with C99 support)
• pthread library
• Make build system
• Standard POSIX libraries: socket, unistd, fcntl, signal, etc.

Hardware Requirements

• Minimal: Any system capable of running a POSIX OS
• Recommended: Multi-core processor for concurrent client handling
• Disk space: Depends on stored objects (managed in data/ directory)

Build Flags

The project uses these compiler flags (see Makefile):

• -std=c99 - C99 standard
• -Wall -Wextra -pedantic - Strict warnings
• -g - Debug symbols
• -fsanitize=address - Address sanitizer for memory safety
• -pthread - POSIX threads support

📁 Project Structure

objectStoreSOL/
├── server.c              # Main server implementation
├── client.c              # Test client application
├── imgclient.c           # Image storage example client
├── clientLibrary.c       # Client API implementation
├── clientLibrary.h       # Client API header
├── communication.c       # Socket communication utilities
├── communication.h       # Communication header
├── request.c             # Request handling logic
├── request.h             # Request handling header
├── icl_hash.c            # Hash table implementation
├── icl_hash.h            # Hash table header
├── Makefile              # Build configuration
├── test.sh               # Automated test script
├── testsum.sh            # Test results analysis script
├── README.md             # This file
├── LICENSE               # MIT License
└── data/                 # Created at runtime for object storage
    ├── client1/          # Per-client directories
    │   ├── object1       # Stored objects
    │   └── object2
    └── client2/
        └── object1

Key Components

• Server (server.c): Multi-threaded server with worker pool, signal handling, and client management
• Client Library (clientLibrary.c/h): API for applications to interact with object store
• Communication Layer (communication.c/h): Socket operations and protocol implementation
• Request Handlers (request.c/h): REGISTER, STORE, RETRIEVE, DELETE, LEAVE operations
• Hash Table (icl_hash.c/h): Efficient client lookup and management
• Test Clients (client.c, imgclient.c): Example applications and test suite

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

MIT License

Copyright (c) 2025 Gennaro Baratta

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

👤 Author

Gennaro Baratta

• GitHub: @GennaroBaratta

🎓 Academic Context

This project was developed as part of the Operating Systems Laboratory course (a.a. 2018/19). It demonstrates:

• Multi-threaded programming with POSIX threads
• Inter-process communication via Unix sockets
• Signal handling for process management
• Concurrent data structure access with synchronization
• Client-server architecture design
• Protocol design and implementation

🙏 Acknowledgments

• ICL Hash table implementation
• Operating Systems course instructors
• POSIX standards committee for excellent documentation

---

Note: If you find this project useful, please give it a ⭐ on GitHub!

For questions or issues, please open an issue.