Samsa — A Kafka Clone Built from Scratch in Go

Why would anyone build a Kafka clone in Go from scratch? To truly understand a distributed system, you have to feel the friction of its gears. You have to handle the bytes, manage the file descriptors, and worry about the race conditions yourself.

Samsa is a lightweight, high-performance Kafka clone built from the ground up in Go. It’s not just a mock; it implements the core Kafka binary protocol, uses an append-only log for durability, and features a unified CLI for both server management and client operations.

Key Features

• Kafka-Compatible Protocol: Implements a custom binary parser (avoiding reflection for performance) to speak the real Kafka protocol.
• Append-Only Storage: High-performance disk I/O using sequential writes and explicit fsync for data durability.
• In-Memory Indexing: Maps logical offsets to physical byte positions for O(1) message retrieval.
• Hexagonal Architecture: Complete decoupling of the network layer, protocol engine, and storage logic.
• Unified CLI: A single binary built with Cobra that acts as both the broker and the administrative client (Topic management, Produce, Consume).
• Graceful Shutdown: Robust signal handling to ensure all in-flight requests complete and file handles are synced before exiting.

Why I Built It

I wanted to demystify the “black box” of message brokers. Building Samsa forced me to confront low-level systems programming challenges: Big Endian byte ordering, manual buffer management, and the complexities of building a “live stream” experience over a request-response protocol.

Tech

Go · Kafka Protocol · TCP Sockets · Append-only Logs · Distributed Systems · Cobra · Slog

GitHub