If we do not distribute CAS, then CAS will still be responsible for creating one Time Event for every Data Event on the network. This means total network throughput will be limited to CAS throughput, since CAS will have to ingest, anchor, and provide Time Events for every Data Event.
The following proposal attempts to break the symmetry between CAS and the rest of the network by offloading some work from CAS to Ceramic nodes.
Semi-Distributed Anchoring
This proposal takes advantage of the fact that CAS anchors trees.
If each node collects all of its events for a period of time, it can build its own Merkle tree for all of the Data Events that it wishes to anchor. It can then send an anchor request to the CAS for only the root of this tree.
The CAS, similarly, collects one tree root per node per anchor period and includes them in its own Merkle tree. It then anchors the root of this parent Merkle tree to the blockchain.
Once the tree is anchored, the CAS responds to each node with the proof and the path of the node’s subtree within the parent tree.
Since each node already maintains its own list of Data Events pending anchoring, it can take the proof and the node’s path through the parent tree to then create and distribute the corresponding Time Events.
The path in a Time Event will be the concatenation of the path of the node’s subtree in the parent tree and the path of the Data Event within the node’s subtree.
This approach reduces the work on the CAS from the order of the number of Data Events on the network to the order of the number of Ceramic nodes on the network.
Each node need only submit one request per anchor period, and receive one response per anchor period. Moreover, each node is now responsible for the creation and dissemination of Time Events for Data Events it originated.
Intermediate Steps
Anchor Proof Polling
Today, Ceramic nodes poll the CAS for every stream that is pending an Anchor. But if no new anchors have occurred, then none of these streams have been anchored and will all receive the same answer.
If, instead, Ceramic nodes polled for the most recent anchor proof, they could limit themselves to one request per polling period and not one request per stream in a loop across the anchor request store, as we do today.
Ceramic nodes can poll for the Time Events of individual streams only after a new anchor has occurred. In the meantime, it can poll at a much lower frequency for the latest anchor proof.
Note that at this stage, CAS is still responding to Ceramic nodes with Time Events versus just Merkle tree nodes.
Give Out Sub-Trees
Our CAS makes no promises about the order of events in the tree. If it sorts all of the Data Events by the node making the requests, then when that node polls, the CAS can respond with the proof, the path from the proof to the node’s part of the tree, and the node’s subtree.
At this stage, the CAS’s response does not include the actual Time Events, just the Merkle tree leaves and paths that are relevant to the requesting node. The node then walks the tree, creates and distributes the Time Events, and removes the Data Events from the anchor request store.
After the tree is processed, whatever is remaining in the node’s anchor request store can be polled or resubmitted to the CAS.