Hello everyone. As a community, we always want to be completely transparent with you all about any major incidents that happen. Unfortunately, we have to inform you about an incident that occurred over the holiday period.
On December 29th at 9:38 UTC the DNS record for the Ceramic Anchor Service (CAS) on mainnet was unintentionally deleted, causing anchor requests from all nodes on mainnet to fail.
A user reported a DNS issue while starting their Ceramic node at 5:45 UTC on Jan 3rd, alerting us to the problem. The DNS record was restored at 9:40 UTC, production CAS requests resumed at 13:45 UTC, and the anchor backlog was cleared as of 20:43 UTC.
Who’s this the most relevant to? Projects running on mainnet that have a short CACAO expiry window are subject to some potential data loss. This includes 24 hour CACAO expiration windows, which were the default until recently. We recommend users update to the newest version of js-did, which now has a week-long CACAO expiry by default, in order to defend against impacts from network issues going forward.
We’re sorry for the inconvenience. Let us know if you have any questions or concerns.