Ceramic and IPFS
A Ceramic node running development or in production requires a companion IPFS node for data persistence. We are strongly recommending that all Ceramic node operators upgrade their IPFS node ASAP.
Why upgrade to Kubo v0.18.1?
Over the last year or so, we have observed several periods of extreme flooding of the Pubsub network with duplicate messages (see issues here and here). We do not yet fully understand the reason that such a large number of duplicate messages are able to propagate through the network but are continuing to investigate.
Even though we don’t know the exact root cause of the Pubsub flooding, we have worked with the teams at Protocol Labs to implement 2 fixes in Kubo that we believe will help protect against these types of issues going forward. Both fixes are included in Kubo v0.18.1:
- We fixed a bug in the Kubo Pubsub implementation that exacerbated the problem of duplicate messages.
- We added a new option that allows setting the maximum time-to-live for the Pubsub cache, which we have used successfully to dampen the propagation of duplicates.
How to upgrade?
We strongly recommend that you upgrade to using our newest Kubo Docker image. This image comes pre-configured with defaults that will protect your node against Pubsub floods that could disrupt the functioning of your node, and also sets up bootstrap peers that will ensure reliable connectivity with the rest of the network.
If your deployment uses the vanilla Kubo distribution (whether CLI or Docker image) from Protocol Labs, we recommend applying the same defaults to your distribution as we do by running the commands that our pre-configured Docker image runs on startup. Especially important is the line that increases the size of the SeenMessage cache: ipfs config Pubsub.SeenMessagesTTL 10m
