hi
i want to know if ceramic network implemnets the load balancing among its nodes.
if so, does it inherit such proprty from IPFS archtecture or it considered its own plan?
does ceramic considered that in all network types?
if i want to have a private network, is it needed to implement load balancing by myself?
thanks anyone can help
I think this thread is related Scaling IPFS Cluster or Elastic IPFS? - #14 by jthor
thanks dear friend
hope it can help
Hi @Ati - Ceramic nodes do not currently support being load balanced across, though it is something we are working to support and plan to support for ComposeDB-only deployments soon!
thank you @spencer for your reply.
so, if i create some ceramic nodes to handle my dapp functionalities, is it permitted, possible and even neccessory for me to balance load between them in the network ?
The problem with load balancing across multiple nodes today is most severe if you are using TileDocument or Caip10Link streamtypes. Imagine a scenario where a stream is created on Node A, and then an update to that stream is routed to Node B. Since node B doesn’t have any data for that stream it needs to query the network to load the current state of the stream from Node A before it can apply the update. This is problematic for two reasons. One, it introduces additional latency into your application flow as your app needs to wait on Node B to sync the stream from Node A. Two, and worse, is that this generates extra load on the entire network to handle the query message that Node B needs to send (this concern will go away in the future with the more efficient query mechanisms we are building now, but for now querying the network to load a stream is pretty inefficient and causes load to the whole network).
Many of these problems can be avoided, however, by using ComposeDB. If you have two nodes that are configured to index the same set of Models, and you only ever use ComposeDB queries and never use TileDocument, Caip10Link, or the 3ID DID method, then load balancing across multiple nodes is safe. This is because with ComposeDB the nodes proactively sync all the streams in the Model, so nodes won’t need to query the network on demand to satisfy incoming queries.
If your app is ComposeDB-only, you can load balance across multiple nodes so long as you are careful. All nodes must have the same set of Models indexed. You also need to ensure the underlying IPFS nodes backing the Ceramic nodes are well connected. You can use ipfs swarm connect to force them to connect to each other. We also recommend running with Historical Data Sync enabled, which will help ensure that all nodes in the load balanced cluster stay in sync and don’t miss any messages from each other. It’s also still probably a good idea to use “sticky sessions” in your load balancer config to avoid switching between nodes regularly.
We are planning a project to improve the documentation here as well as add some features that will make configuring a load balanced setup easier (such as handling the ipfs peer swarm connections). This is why I said load balancing isn’t yet recommended - not because it’s not possible, but just because there are a lot of little details to get right and we haven’t fully documented the process or built the tools to make it easy yet.
many thanks dear @spencer for your great clarification. 