The Petrichor Protocol Primitives

This is an idea I have for a basic set of primitives that let you build
protocols quickly and easily.  It is similar to ZeroMQ except:

1. It's transparent and you can actually understand it.
2. It could be implemented in any language, not just C++.
3. It doesn't blow up your server with any of 492 asserts.
4. You can put it on the internet without worrying about it.
5. It can transmit actual data structures or raw binary data.
6. You can put it in an SSL socket or any other crypto protocol.
7. It will work with any socket system, events, or threads.

Getting The Code

You can get the code at
which contains a simple Python proof-of-concept to see what people think about the idea.

Here's what's in this first version:

* Look at the and to see an example of an advanced echo server.
* Look at to see what it takes to make it work.
* Look at and to see the data you can send.

Planned Features

* Clean up the implementation and make it a real python package.
* Implement the Dedupe, SSL, and Compression helpers.
* Create samples for different existing protocol semantics.
* Write versions for Ruby and C.
* Create a literate document teaching you how to implement Petrichor yourself as the spec.
* Get a decent website that isn't a PRE tag.

The Entire Protocol So Far

Petrichor is simply sequences of paired TNetstrings as HEADER,BODY.

The main API is:

    def read_chunk(conn):
        payload = read_payload(conn)
        if payload:
            data, _ = tnetstrings.parse(payload)
            return data
            return None
    def recv(conn):
        header = read_chunk(conn)
        if header == None or type(header) != dict:
            return None, None
        data = read_chunk(conn)
        return header, data
    def send(conn, header, body):
        data = tnetstrings.dump(body)
        conn.write(tnetstrings.dump(header) + data)

Then there's a few helper functions for common stuff:

    def connect(host, port, header, body="", timeout=20):
        sock = socket.create_connection((host, port), timeout)
        conn = sock.makefile()
        send(conn, header, body)
        header, body = recv(conn)
        return sock, conn, header, body
    def connect_allowed(header):
        return header.get('allowed', True)
    def accept(conn):
        return recv(conn)
    def peer_allowed(conn, header, msg):
        header['allowed'] = True
        send(conn, header, msg)
    def peer_denied(conn, header, msg):
        header['allowed'] = False
        send(conn, header, msg)
    def request(conn, header, body):
        send(conn, header, body)
        header, body = recv(conn)
        return header, body

Other than that there's just some extra gear for reading TNetstrings
off the wire, which will probably go into the tnetstrings library.

Sample Client For The Lazy

The following is an echo client with probably more features than
an echo client should have:

   import petrichor
   import sys
   from uuid import uuid4
   VERSION = "0.1"
   HOST, PORT, USER = sys.argv[1], int(sys.argv[2]), sys.argv[3]
   IDENT = {"user": USER, "ident": uuid4().hex, 'version': VERSION}
   sock, conn, header, body = petrichor.connect(HOST, PORT, IDENT)
   if petrichor.connect_allowed(header):
       print "CONNECTED", body
       print "YOU WERE DENIED", body
   while header != None:
       msg = raw_input("> ")
       header, reply = petrichor.request(conn, {}, msg)
       print reply

Sample Server For The Lazy

Here's the echo server, again with things like identifying connections
and users, which most echo servers wouldn't have:

    import SocketServer
    import petrichor
    from uuid import uuid4
    import sys
    IDENT = uuid4().hex
    VERSION = "0.1"
    class PetrichorServer(SocketServer.StreamRequestHandler):
        def handle(self):
            ident, body = petrichor.accept(self.rfile)
            # at this point you'd some kind of auth, we just fake it
            reply = {"ident": IDENT, 'version': VERSION}
            petrichor.peer_allowed(self.wfile, reply, "WELCOME")
            header = {}
            while header != None:
                # just keep looping until you get a None which is returned on close
                header, body = petrichor.recv(self.rfile)
                print "RECEIVED", ident['user'], ident['ident'], header, body
                if header != None:
                    petrichor.send(self.wfile, {}, body)
            print "ALL DONE"
    if __name__ == "__main__":
        HOST, PORT = sys.argv[1], int(sys.argv[2])
        server = SocketServer.TCPServer((HOST, PORT), PetrichorServer)
        print "RUNNING", HOST, PORT