IPFS Cluster architecture

This document gives an overview of the IPFS Cluster architecture.

Modularity

IPFS Cluster architecture attempts to be as modular as possible, with interfaces between its modules (components) clearly defined, in a way that they can:

Components

IPFS Cluster consists of:

Components perform a number of functions and need to be able to communicate with eachothers: i.e.:

RPC API

Communication between components happens through the RPC API: a set of functions which stablishes which functions are available to components (rpc_api.go).

The RPC API uses go-libp2p-gorpc. The main Cluster component runs an RPC server. RPC Clients are provided to all components for their use. The main feature of this setup is that Components can use go-libp2p-gorpc to perform operations in the local cluster and in any remote cluster node using the same API.

This makes broadcasting operations and contacting the Cluster leader really easy. It also allows to think of a future where components may be completely arbitrary and run from different applications. Local RPC calls, on their side, do not suffer any penalty as the execution is short-cut directly to the correspondant component of the Cluster, without network intervention.

On the down-side, the RPC API involves “reflect” magic and it is not easy to verify that a call happens to a method registered on the RPC server. Every RPC-based functionality should be tested. Bad operations will result in errors so they are easy to catch on tests.

Code layout

Components are organized in different submodules (i.e. pintracker/maptracker represents component PinTracker and implementation MapPinTracker). Interfaces for all components are on the base module. Executables (ipfs-cluster-service and ipfs-cluster-ctl are also submodules to the base module).

Configuration

A config module provides support for a central configuration file which provides configuration sections defined by each component by providing configuration objects which implement a ComponentConfig interface.