Protocol Overview

More...

Classes
struct	caf::io::basp::header
	The header of a Binary Actor System Protocol (BASP) message. More...

struct	caf::io::basp::routing_table::route
	Describes a routing path to a node. More...

class	caf::io::basp::routing_table
	Stores routing information for a single broker participating as BASP peer and provides both direct and indirect paths. More...

class	caf::io::basp::instance::callee
	Provides a callback-based interface for certain BASP events. More...

class	caf::io::basp::instance
	Describes a protocol instance managing multiple connections. More...

Typedefs
using	caf::io::basp::buffer_type = std::vector< char >
	Storage type for raw bytes.

Enumerations
enum	caf::io::basp::message_type : uint32_t { caf::io::basp::message_type::server_handshake = 0x00, caf::io::basp::message_type::client_handshake = 0x01, caf::io::basp::message_type::dispatch_message = 0x02, caf::io::basp::message_type::announce_proxy_instance = 0x03, caf::io::basp::message_type::kill_proxy_instance = 0x04 }
	Describes the first header field of a BASP message and determines the interpretation of the other header fields. More...

enum	caf::io::basp::error : uint64_t { caf::io::basp::error::no_route_to_destination = 0x01, caf::io::basp::error::loop_detected = 0x02 }
	Describes an error during forwarding of BASP messages. More...

enum	caf::io::basp::connection_state { caf::io::basp::await_header, caf::io::basp::await_payload, caf::io::basp::close_connection }
	Denotes the state of a connection between to BASP nodes. More...

Functions
void	caf::io::basp::read_hdr (serializer &sink, header &hdr)
	Deserialize a BASP message header from `source`.

void	caf::io::basp::write_hdr (deserializer &source, const header &hdr)
	Serialize a BASP message header to `sink`.

bool	caf::io::basp::is_handshake (const header &hdr)
	Checks whether given header contains a handshake.

std::string	to_string (const header &hdr)

void	read_hdr (deserializer &source, header &hdr)

void	write_hdr (serializer &sink, const header &hdr)

bool	operator== (const header &lhs, const header &rhs)

bool	operator!= (const header &lhs, const header &rhs)

bool	valid (const header &hdr)
	Checks whether given BASP header is valid.

Variables
constexpr uint64_t	caf::io::basp::version = 1
	The current BASP version. More...

constexpr size_t	caf::io::basp::header_size
	Size of a BASP header in serialized form. More...

Detailed Description

Protocol Overview

The "Binary Actor Sytem Protocol" (BASP) is not a network protocol. It is a specification for the "Remote Method Invocation" (RMI) interface used by distributed instances of CAF. The purpose of BASP is unify the structure of RMI calls in order to simplify processing and implementation. Hence, BASP is independent of any underlying network technology, and assumes a reliable communication channel.

The RMI interface of CAF enables network-transparent monitoring and linking as well as global message dispatching to actors running on different nodes.

The figure above illustrates the phyiscal as well as the logical view of a distributed CAF application. Note that the actors used for the BASP communication ("BASP Brokers") are not part of the logical system view and are in fact not visible to other actors. A BASP Broker creates proxy actors that represent actors running on different nodes. It is worth mentioning that two instances of CAF running on the same physical machine are considered two different nodes in BASP.

BASP has two objectives:

Forward messages sent to a proxy to the actor it represents

Whenever a proxy instance receives a message, it forwards this message to its parent (a BASP Broker). This message is then serialized and forwarded over the network. If no direct connection between the node sending the message and the node receiving it exists, intermediate BASP Brokers will forward it until it the message reaches its destination.
Synchronize the state of an actor with all of its proxies.

Whenever a node learns the address of a remotely running actor, it creates Ma local proxy instance representing this actor and sends an announce_proxy_instance to the node hosting the actor. Whenever an actor terminates, the hosting node sends kill_proxy_instance messages to all nodes that have a proxy for this actor. This enables network-transparent actor monitoring. There are two possible ways addresses can be learned:
- A client connects to a remotely running (published) actor via remote_actor. In this case, the server_handshake will contain the address of the published actor.
- Receiving dispatch_message. Whenever an actor message arrives, it usually contains the address of the sender. Further, the message itself can contain addresses to other actors that the BASP Broker will get aware of while deserializing the message object from the payload.

Node IDs

The ID of a node consists of a 120 bit hash and the process ID. Note that we use "node" as a synonym for "CAF instance". The hash is generated from "low-level" characteristics of a machine such as the UUID of the root file system and available MAC addresses. The only purpose of the node ID is to generate a network-wide unique identifier. By adding the process ID, CAF disambiguates multiple instances running on the same phyisical machine.

Header Format

Operation ID: 4 bytes.

This field indicates what BASP function this datagram represents. The value is an uint32_t representation of message_type.
Payload Length: 4 bytes.

The length of the data following this header as uint32_t, measured in bytes.
Operation Data: 8 bytes.

This field contains.
Source Node ID: 18 bytes.

The address of the source node. See Node IDs.
Destination Node ID: 18 bytes.

The address of the destination node. See Node IDs. Upon receiving this datagram, a BASP Broker compares this node ID to its own ID. On a mismatch, it selects the next hop and forwards this datagram unchanged.
Source Actor ID: 4 bytes.

This field contains the ID of the sending actor or 0 for anonymously sent messages. The full address of an actor is the combination of the node ID and the actor ID. Thus, every actor can be unambigiously identified by combining these two IDs.
Destination Actor ID: 4 bytes.

This field contains the ID of the receiving actor or 0 for BASP functions that do not require

Example

The following diagram models a distributed application with three nodes. The pseudo code for the application can be found in the three grey boxes, while the resulting BASP messaging is shown in UML sequence diagram notation. More details about individual BASP message types can be found in the documentation of message_type below.

Enumeration Type Documentation

enum caf::io::basp::connection_state

Denotes the state of a connection between to BASP nodes.

Enumerator
await_header	Indicates that a connection is established and this node is waiting for the next BASP header.
await_payload	Indicates that this node has received a header with non-zero payload and is waiting for the data.
close_connection	Indicates that this connection no longer exists.

enum caf::io::basp::error : uint64_t

strong

Describes an error during forwarding of BASP messages.

Enumerator
no_route_to_destination	Indicates that a forwarding node had no route to the destination.
loop_detected	Indicates that a forwarding node detected a loop in the forwarding path.

enum caf::io::basp::message_type : uint32_t

strong

Describes the first header field of a BASP message and determines the interpretation of the other header fields.

Enumerator
server_handshake	Send from server, i.e., the node with a published actor, to client, i.e., node that initiates a new connection using remote_actor().
client_handshake	Send from client to server after it has successfully received the server_handshake to establish the connection.
dispatch_message	Transmits a message from `source_node:source_actor` to `dest_node:dest_actor`.
announce_proxy_instance	Informs the receiving node that the sending node has created a proxy instance for one of its actors. Causes the receiving node to attach a functor to the actor that triggers a kill_proxy_instance message on termination.
kill_proxy_instance	Informs the receiving node that it has a proxy for an actor that has been terminated.

Function Documentation

bool operator!=	(	const header &	lhs,
		const header &	rhs
	)

bool operator==	(	const header &	lhs,
		const header &	rhs
	)

void read_hdr	(	deserializer &	source,
		header &	hdr
	)

void write_hdr	(	serializer &	sink,
		const header &	hdr
	)

constexpr size_t caf::io::basp::header_size

Initial value:

=
  node_id::host_id_size * 2 + sizeof(uint32_t) * 2 +
  sizeof(actor_id) * 2 + sizeof(uint32_t) * 2 + sizeof(uint64_t)

Size of a BASP header in serialized form.

constexpr uint64_t caf::io::basp::version = 1

The current BASP version.

Different BASP versions will not be able to exchange messages.

Protocol Overview

Classes

Typedefs

Enumerations

Functions

Variables

Detailed Description

Protocol Overview

Node IDs

Header Format

Example

Enumeration Type Documentation

Function Documentation

Variable Documentation