Module core

Expand description

Core data structures and protocol implementations for building a swarm.

§Network builder

To build a network or a node, you first need to create a CoreBuilder object using the CoreBuilder::with_config method to create a bootstrap node, then you can simply call CoreBuilder::build to set up the network. This will create a Core struct which serves as the interface between an application and the network. It contains several methods used to interact and perform activities on the network.

The CoreBuilder::with_config method takes a parameter - BootstrapConfig, to pass in a node’s bootstrap configuration.

§Default setup

Here’s how you would build a bootstrap node with the default library settings

// Default config
let config = BootstrapConfig::default();
let mut network = CoreBuilder::with_config(config)
	.build()
	.await
	.unwrap();

§Overriding the default network configuration

You can explicitly overrride the default values of CoreBuilder::with_config by calling the methods like the following before building the network:

CoreBuilder::with_transports: Configures a custom transport to use, specified in TransportOpts.
CoreBuilder::with_network_id : Configures the network ID or name e.g. /your-protocol-name/1.0.
CoreBuilder::listen_on : Configures the IP address to listen on e.g. IPv4(127.0.0.1).
CoreBuilder::with_idle_connection_timeout: Configures a timeout for keeping a connection alive.
CoreBuilder::with_replication: Configures the node for replication.
etc.

For example:

		// Default config
		let config = BootstrapConfig::default();

		// Create a default network core builder
		let default_node = CoreBuilder::with_config(config);

		// Override default with custom configurations
		// Network Id
		let mut custom_network_id = "/custom-protocol/1.0".to_string();		
		// Transport
		let mut custom_transport = TransportOpts::TcpQuic {		
			tcp_config: TcpConfig::Custom {
				ttl: 10,
				nodelay: true,
				backlog: 10,
			},
		};
		// Keep-alive	
		let mut custom_keep_alive_duration = 20;	
		// IP address
		let mut custom_ip_address = IpAddr::V6(Ipv6Addr::new(0, 0, 0, 0, 0, 0, 0, 1));	

		// Build a custom configured node
		let custom_node = default_node
			.with_network_id(custom_network_id.clone())
			.with_transports(custom_transport.clone())
			.with_idle_connection_timeout(custom_keep_alive_duration.clone())
			.listen_on(custom_ip_address.clone())
			.with_replication(ReplNetworkConfig::Default)
			.build()
			.await.
			.unwrap();

§Application Interaction

The core library provides very simple interfaces to communicate with the network layer to trigger and drive network behaviour or to make enquiries about internal network state. This is achieved by constructing a request with the AppData struct and passing it to the network. An AppResponse structure is returned, containing the network’s response to the applications request.
There are two ways of querying the network layer with each having its own peculiarities and use-cases:

Using the Core::query_network() method: This method aims to complete its operations atomically and blocks until the network returns a reponse or if the request times out. It is useful when the response from the network is important for the application logic to continue.

    // Default config
    let config = BootstrapConfig::default();
    // Default handler
    let handler = DefaultHandler;

    // Create a default network core builder
    let node = CoreBuilder::with_config(config, handler);

    // Join a network (subscribe to a topic)
    let gossip_request = AppData::GossipsubJoinNetwork(MY_NETWORK.to_string());

    // Blocks until response is returned from the network layer
    if let Ok(_) = node.query_network(gossip_request).await {
        println!("Subscription successfull");
    }

Using the Core::send_to_network() and Core::recv_from_network() method: This method does not block and is split into two parts - sending and recieving. When the request is recieved by the network layer through the Core::send_to_network() method, a StreamId is immediately returned and the request is handled. When there is a response (or a timeout), it is stored internally in a response buffer until it is returned by explicitly polling the network through the Core::recv_from_network() method which takes in the StreamId returned earlier. The StreamId helps to track the requests and their corresponding responses internally in the network layer.

    // Default config
    let config = BootstrapConfig::default();
    // Default handler
    let handler = DefaultHandler;

    // Create a default network core builder
    let node = CoreBuilder::with_config(config, handler);

    // Join a network (subscribe to a topic)
    let gossip_request = AppData::GossipsubJoinNetwork(MY_NETWORK.to_string());

    // Send request to the application layer
    let stream_id = node.send_to_network(gossip_request).await.unwrap();

    // ...
    // Run other application logic
    //...

    // Explicitly retrieve the response of our request
    if let Ok(result) = node.recv_from_network(stream_id).await {
        println!("Subscription successfull");
        assert_eq!(AppResponse::GossipsubJoinSuccess, result);
    }

Note: The internal buffer is limited in capacity and pending responses should be removed as soon as possibe. A full buffer will prevent the network from recieving more requests.

§Event Handling

Network events are stored in an internal buffer in the network layer as they occur. They have to be polled and consumed explicitly and are exposed by Core to the application layer by two functions:

[Core::next_event()]: This function returns the next event in the queue and returns None` if the internal event buffer queue is empty.

    // ...
    // Read events generated at setup
	while let Some(event) = node.next_event().await {
		match event {
			NetworkEvent::NewListenAddr {
				local_peer_id,
				listener_id: _,
				address,
			} => {
				// Announce interfaces we're listening on
				println!("Peer id: {}", local_peer_id);
				println!("We're listening on {}", address);
			},
			NetworkEvent::ConnectionEstablished {
				peer_id,
				connection_id: _,
				endpoint: _,
				num_established: _,
				established_in: _,
			} => {
				println!("Connection established with peer: {:?}", peer_id);
			},
			_ => {},
		}
	}
    // ...

Core::events(): This function drains all the buffered events and returns it to the application layer as an Iterator object;

    // ...
    let _ = events
		.map(|e| {
			match e {
				NetworkEvent::NewListenAddr {
					local_peer_id,
					listener_id: _,
					address,
				} => {
					// Announce interfaces we're listening on
					println!("[[Node {}]] >> Peer id: {}", game_state.node, local_peer_id);
					println!(
						"[[Node {}]] >> We're listening on the {}",
						game_state.node, address
					);
				},
				NetworkEvent::ConnectionEstablished {
					peer_id,
					connection_id: _,
					endpoint: _,
					num_established: _,
					established_in: _,
				} => {
					println!("Connection established with peer: {:?}", peer_id);
				},
				_ => {},
			}
		})
		.collect::<Vec<_>>();
    // ...

It is important to consume critical events promptly to prevent loss if the buffer becomes full.

§Replication

After configuring your node for replication, you can participate in replication activities accross the network by calling methods exposed to the application layer:

Core::replicate: Replicates data across replica nodes on the network.
Core::replicate_buffer: Clone a remote node’s replica buffer.
Core::join_repl_network: Join a replica network.
Core::leave_repl_network: Exit a replica network.
Etc.

Modules§

gossipsub_cfg
Module containing important state relating to the Gossipsub protocol.
ping_config
Module that contains important data structures to manage Ping operations on the network.
replication
Module that contains important data structures to manage replication operations on the network.
sharding
Module that contains important data structures to manage sharding operations on the network.

Structs§

Core
The core interface for the application layer to interface with the networking layer.
CoreBuilder
Structure containing necessary data to build Core.
IdentifyInfo
The struct that contains incoming information about a peer returned by the Identify protocol.
StreamId
A simple struct used to track requests sent from the application layer to the network layer.

Enums§

AppData
Request sent from the application layer to the networking layer.
AppResponse
Response to requests sent from the application to the network layer.
NetworkError
Network error type containing errors encountered during network operations.
NetworkEvent
Enum that represents the events generated in the network layer.
RpcConfig
The configuration for the RPC protocol.

Constants§

NETWORK_READ_TIMEOUT
The duration (in seconds) to wait for response from the network layer before timing out.
TASK_SLEEP_DURATION
The time it takes for the task to sleep before it can recheck if an output has been placed in the response buffer.

Type Aliases§

AppResponseResult
Type that represents the response of the network layer to the application layer’s event handler.
ByteVector
Type that represents a vector of vector of bytes.
NetworkResult
Type that represents the result for network operations.
Nonce
Type that represents a nonce.
RpcData
Type that represents the data exchanged during RPC operations.
ShardId
Type that represents the id of a shard.
StringVector
Type that represents a vector of string.

Module coreCopy item path