Protocol Buffers

JSON is everywhere. Human-readable, universally supported, trivially easy to parse. But it has costs that are invisible at small scale and crushing at large scale.

A simple user object in JSON: {"id":123,"name":"Roni","email":"roni@example.com","active":true}. 62 bytes. Field names repeated in every single message. Quotes around strings. No type information. Every consumer must parse the text and infer types.

The same object in Protocol Buffers: ~30 bytes. No field names in the payload, just field numbers and values. Binary encoding, no text parsing. Types defined in the schema and enforced at compile time.

Google created Protocol Buffers in 2001 for internal use. By 2008, they open-sourced it. Today, Protobuf is the serialization format behind gRPC, Google's entire internal infrastructure, and thousands of high-performance systems. LinkedIn's data infrastructure team measured their Protobuf encoding at 20x faster than JSON for their messaging pipelines, processing 7 trillion messages per day.

1. Define the schema (.proto file):

syntax = "proto3";

message User {
  int32 id = 1;     // field number, not default value
  string name = 2;
  string email = 3;
  bool active = 4;
  repeated string tags = 5;  // list
}

2. Compile to target language:

protoc --python_out=. --java_out=. --go_out=. user.proto

This generates type-safe classes in Python, Java, Go (and 10+ other languages). Each class has getter/setter methods for every field, serialization methods, and builder patterns.

3. Serialize and deserialize:

user = User(id=123, name="Roni", email="roni@example.com", active=True)
binary = user.SerializeToString()  # ~30 bytes of binary
user2 = User.FromString(binary)    # type-safe deserialization

Why field numbers matter. In JSON, the key "name" is sent with every message. In Protobuf, the field number 2 (1 byte) replaces the key. This is why Protobuf is smaller: field names exist only in the schema, never in the serialized data. Changing a field name in the .proto file does not break wire compatibility because the number stays the same.

Schema evolution rules:

• Adding new fields: safe (old readers skip unknown fields)
• Removing fields: safe if you reserve the number (new readers use defaults for missing fields)
• Changing field numbers: BREAKS EVERYTHING (the wire format uses numbers, not names)
• Changing field types: often unsafe (int32 to string is a breaking change)

The golden rule: never reuse a field number, even after deleting a field. Use the reserved keyword to prevent accidental reuse.