Cap’n Proto is a data serialization format and Remote Procedure Call (RPC) framework for exchanging data between computer programs. The high-level design focuses on speed and security, making it suitable for network as well as inter-process communication. Cap'n Proto was created by the former maintainer of Google's popular Protocol Buffers framework (Kenton Varda) and was designed to avoid some of its perceived shortcomings.
| Cap'n Proto | |
|---|---|
| Original author(s) | Kenton Varda |
| Stable release | 1.2.0[1] 
/ 15 June 2025 |
| Repository | github |
| Written in | C++ |
| Type | Remote procedure call framework, serialization format and library, IDL compiler |
| License | MIT License |
| Website | capnproto |
Technical overview
editIDL Schema
editLike most RPC frameworks dating as far back as Sun RPC and OSF DCE RPC (and their object-based descendants CORBA and DCOM), Cap'n Proto uses an Interface Description Language (IDL) to generate RPC libraries in a variety of programming languages - automating many low level details such as handling network requests, converting between data types, etc. The Cap'n Proto interface schema uses a C-like syntax and supports common primitives data types (booleans, integers, floats, etc.), compound types (structs, lists, enums), as well as generics and dynamic types.[2] Cap'n Proto also supports object-oriented features such as multiple inheritance, which has been criticized for its complexity.[3]
@0xa558ef006c0c123; # Unique identifiers are manually or automatically assigned to files and compound types
struct Date @0x5c5a558ef006c0c1 {
year @0 :Int16; # @n marks order values were added to the schema
month @1 :UInt8;
day @2 :UInt8;
}
struct Contact @0xf032a54bcb3667e0 {
name @0 :Text;
birthday @2 :Date; # fields can be added anywhere in the definition, but their numbering must reflect the order in which they were added
phones @1 :List(PhoneNumber);
struct PhoneNumber { # Compound types without a static ID cannot be renamed, as automatic IDs are deterministically generated
number @0 :Text;
type @1 :PhoneType = mobile; # Default value
enum PhoneType {
mobile @0;
landline @1;
}
}
}
Values in Cap'n Proto messages are represented in binary, as opposed to text encoding used by "human-readable" formats such as JSON or XML. Cap'n Proto tries to make the storage/network protocol appropriate as an in-memory format, so that no translation step is needed when reading data into memory or writing data out of memory.[note 1] For example, the representation of numbers (endianness) was chosen to match the representation the most popular CPU architectures.[4] When the in-memory and wire-protocol representations match, Cap'n Proto can avoid copying and encoding data when creating or reading a message and instead point to the ___location of the value in memory. Cap'n Proto also supports random access to data, meaning that any field can be read without having to read the entire message.[5]
Unlike other binary serialization protocols such as XMI, Cap'n Proto considers fine-grained data validation at the RPC level an anti-feature that limits a protocol's ability to evolve. This was informed by experiences at Google where simply changing a field from mandatory to optional would cause complex operational failures.[6][note 2] Cap'n Proto schemas are designed to be flexible as possible and pushes data validation to the application level, allowing arbitrary renaming of fields, adding new fields, and making concrete types generic.[7] Cap'n Proto does, however, validate pointer bounds and type check individual values when they are first accessed.[5]
Enforcing complex schema constraints would also incur significant overhead,[note 3] negating the benefits of reusing in-memory data structures and preventing random access to data.[8] Cap'n Proto protocol is theoretically suitable[9] for very fast inter-process communication (IPC) via immutable shared memory, but as of October 2020 none of the implementations support data passing via shared memory.[10] However, Cap'n Proto is still generally considered faster than Protocol Buffers and similar RPC libraries.[11][12]
Networking
editCap'n Proto RPC is network aware: supporting both handling of disconnects and promise pipelining, wherein a server pipes the output of one function into another function. This saves a client a round trip per successive call to the server without having to provide a dedicated API for every possible call graph. Cap'n Proto can be layered on top of TLS[13] and support for the Noise Protocol Framework is on the roadmap.[14] Cap'n Proto RPC is transport agnostic, with the mainline implementation supporting WebSockets, HTTP, TCP, and UDP.[15]
Capability security
editThe Cap'n Proto RPC standard has a rich capability security model based on the CapTP protocol used by the E programming language.[16]
This section needs expansion. You can help by adding to it. (March 2021) |
As of October 2020, the reference implementation only supports level 2.[14]
Comparison to other serialization formats
editCap'n Proto is often compared to other zero-copy serialization formats, such as Google's FlatBuffers and Simple Binary Encoding (SBE).[8][17]
Adoption
editCap'n Proto was originally created for Sandstorm.io, a startup offering a web application hosting platform with capability-based security. After Sandstorm.io failed commercially, the development team was acqui-hired by Cloudflare,[18] which uses Cap'n Proto internally.[19]
Notes
edit- ^ Unlike Apache Arrow, Cap'n Proto's in-memory values are not suited for sharing mutable data
- ^ Marking a field as required was removed from Protocol Buffers 3.
- ^ Assuming the data has already been allocated (e.g. in network buffers, read from disk) access becomes O(1). Additional serialization/deserialization steps (as required to inspect values) would limit performance to O(n).
References
edit- ^ "Release 1.2.0". 15 June 2025. Retrieved 13 July 2025.
- ^ Varda, Kenton. "Cap'n Proto Schema Language". Archived from the original on 2015-03-17. Retrieved 2020-09-05.
- ^ Denhardt, Ian (June 2019). "A Critique of the Cap'n Proto Schema Language". zenhack.net. Archived from the original on 2019-06-26. Retrieved 2020-10-10.
- ^ Varda, Kenton. "Cap'n Proto: Introduction". Cap'n Proto Homepage. Archived from the original on 2015-03-17. Retrieved 2020-11-09.
- ^ a b Varda, Kenton. "Cap'n Proto: Encoding Spec". Cap'n Proto. Archived from the original on 2015-03-17.
- ^ Varda, Kenton. "FAQ § How do I make a field "required", like in Protocol Buffers?". Cap'n Proto. Archived from the original on 2015-03-18. Retrieved 2020-09-05.
- ^ "Cap'n Proto: Schema Language". capnproto.org. Retrieved 2020-10-10.
- ^ a b "Cap'n Proto: Cap'n Proto, FlatBuffers, and SBE". capnproto.org. Retrieved 2020-10-10.
- ^ Richardson, Corey (October 2016). "Robigalia: An Operating System for the Modern Era". robigalia.gitlab.io. Archived from the original on 2018-09-15. Retrieved 2020-10-10.
- ^ Kenton, Varda (May 3, 2017). "Why is not intended that in-memory state be in Cap'n Proto / Protobuf objects?". Hacker News (news.ycombinator.com). Retrieved 2020-10-10.
- ^ Naughton, Chris (Aug 24, 2018). "Protocol Benchmarks". Github. Archived from the original on 2018-08-30. Retrieved 2020-09-05.
- ^ Parimi, Dinesh (2019). "Datacenter Tax Cuts: Improving WSC Efficiency Through Protocol Buffer Acceleration" (PDF). Archived (PDF) from the original on 2020-09-06. Retrieved 2020-09-05.
- ^ "Cap'n Proto: Road Map". capnproto.org. Retrieved 2020-10-10.
- ^ a b "Roadmap". Cap'n Proto. 2021-03-13. Archived from the original on 2015-03-17.
- ^ "Cap'n Proto: C++ RPC". capnproto.org. Retrieved 2020-10-10.
- ^ "RPC Protocol". Cap'n Proto. Archived from the original on 2015-03-18.
- ^ "Why flatbuffers instead of capnp?".
- ^ Varda, Kenton (13 Mar 2017). "The Sandstorm Team is joining Cloudflare". Sandstorm.io. Archived from the original on 2017-03-13. Retrieved 2020-09-05.
- ^ Zhi, Jiale (2013). "Introducing lua-capnproto: better serialization in Lua". Archived from the original on 2014-03-06. Retrieved 2020-09-05.