The Future of Ceramic: Focusing on Recall

Important update about the future of Ceramic. Please see this blog post for more information: The Future of Ceramic: Focusing on Recall

1 Like

I’ve managed to get ceramic-one up and running, but now I’d like to create and deploy composites. From what I see, this would require using ComposeDB, which is being deprecated, because I don’t see any other way to create composites or deploy them. Does it still make sense to go this route? checking the ceramic-one options in the client terminal, but I dont see anything

Apologies if this is already covered somewhere, I haven’t found much up-to-date documentation (the migration guide from Ceramic to Ceramic One is 11 months old and still references js-ceramic, which you mentioned will also be deprecated) and also in the website there is still references to composedb and js-ceramic.

I don’t want to take up too much of your time, I’ve used Ceramic before and I’m familiar with ComposeDB and production deployments, I just want to make sure I’m heading in the right direction, or get any small suggestions you might have.

Thanks a lot again!

Composites are really just groups of Models, so now with ceramic-one you can just create Models directly using json-schema. @mohsin may be able to point you to the most up-to-date docs we have here. Thanks for bearing with us during the rough edges of this transition!

1 Like

@mohsin any suggestion here for now?

I am checking the different options I have in the terminal using ceramic-one, but I don’t see any command to create models unless I am wrong.

Thanks in advance!

Hi @Eloy, apologies for the late response. I was traveling last week and am just catching up with messages.

Ceramic One doesn’t have a CLI like ComposeDB did. You can use the C1 TypeScript SDK to create models and model instance documents, as shown in the examples here.

I would recommend waiting for a few more days till this PR gets merged though, and a new version of the SDK is published. It fixes an important bug in how the SDK creates new streams.

Let me know if you have any more questions.

1 Like

Ok, perfect! Thanks a lot @moshin, no worries a lot about the delay in replying :slight_smile:

I am going to create the scripts to run them and create the models and deploy them, and I will run these once the PR was merged, that should not take that much from what I infer from your previous message.

I did not want to be blocked too much time, so I was thinking about using directly composedb as it is not deprecated yet, but if it is going to be matter of days until the merge of the PR, it would be worth it the wait, because i would like to have everything up to date.

And by the way, if you have any other relevant documentation of these new tools would be great if you can send it to me.

Thanks a lot again!

Also minor question @spencer @mohsin:

Does Ceramic provide a way to receive updates in a pub/sub-like manner, that is, being notified of changes without having to periodically query for them?

For example, after going through the documentation shared by @mohsin:
Imagine I have a model representing a list of posts in a forum. If any user creates or modifies a post, is there a way for other users to automatically receive those updates (without polling), similar to a pub/sub system?

Let me know if anything is unclear, and thanks in advance!

Hi everyone,

I’m the guy who was building a dapp with Ceramic for my bachelor’s thesis project—maybe you remember my previous posts/questions!

I have a question regarding the recent announcement about the deprecation of js-ceramic and ComposeDB.

My application is built using OrbisDB, which relies on js-ceramic (as i understand), but I do not use ComposeDB at all. Given that OrbisDB depends on js-ceramic for stream creation, reading, and authentication, I am wondering:

  • Will OrbisDB-based applications (without ComposeDB) continue to work for some time after js-ceramic is deprecated?
  • Is there any timeline for when js-ceramic nodes or related infrastructure will be shut down entirely?
  • Are there migration paths, or will OrbisDB need to be rewritten to support ceramic-one?

Any clarification would be greatly appreciated, as I am assessing my options for maintaining or migrating my project.

Just for context: I am planning to deliver my thesis project in October this year, and it is almost finished at this point. This is why I am a bit worried about possible sudden shutdowns or breaking changes, as I wouldn’t have enough time to re-migrate or rewrite everything before my deadline.

Thanks in advance!

Does Ceramic provide a way to receive updates in a pub/sub-like manner, that is, being notified of changes without having to periodically query for them?

Yes, ceramic-one has a feed API built on DataFusion that can provide a subscribable feed. Not sure what we have in terms of docs or examples of how to use it though, will have to defer to @mohsin for that

1 Like

@DavidRs I would recommend reaching out to the OrbisDB team directly about what they recommend.

Hi @Eloy. Yes, as @spencer mentioned, you can use the Ceramic One feed API.

Please take a look at this page and let me us know what you think.

1 Like

Perfect, thanks a lot @mohsin @spencer.

I think what I am looking for, similar to a pub/sub would be the “Query the pipeline” section, right?

By continuously processing incoming events, streaming systems allow for the dynamic creation and updating of table views, providing up-to-the-moment insights without the need for batch processing.

Just to confirm with you and be sure that I am right and following the right path.

By the way, I was able to create model using the repository directly with the non-merged changes (I installed it from cloned repository in my machine), and worked for me.

Thanks a lot again! :slight_smile:

1 Like

Hello @mohsin @spencer,

I was able to use the Flight SQL library successfully by following the “Query the pipeline” section, and I’m able to query events related to a specific stream ID.

However, when I modify the stream from a different terminal, I don’t see any new events appearing in the terminal that’s running the query. It seems like I need to manually re-run the query to get updates, rather than maintaining a persistent connection that receives new events as they arrive.

Am I missing something? Or is it currently not possible to use this in a pub/sub fashion, where I would subscribe once and automatically receive updates?

Thanks in advance!

This is the code I am using in case it was useful for you:

async function pollEventsByModelId(modelId: string) {
    const modelStreamId = StreamID.fromString(modelId);
    const modelCid = modelStreamId.cid.toString();

    const processedEventCids = new Set<string>();

    console.log(`Polling for new events for model_id = ${modelCid}`);

    const client: FlightSqlClient = await createFlightSqlClient({
                headers: new Array(),
                username: undefined,
                password: undefined,
                token: undefined,
                tls: false,
                host: "127.0.0.1",
                port: 5102,
            });

    // This is the main polling loop.
    while (true) {
        try {
            // The query remains the same, fetching all events for the model.
            const feed = await client.preparedFeedQuery(
                "SELECT * FROM conclusion_events WHERE stream_cid = $1",
                [["$1", modelCid]]
            );

            // The inner loop processes the batch of results from the query.
            while(true) {
                const batchBuffer = await feed.next();
                if (batchBuffer === null) {
                    break; // The entire result set has been processed.
                }

                const eventTable = tableFromIPC(batchBuffer);

                for (let i = 0; i < eventTable.numRows; i++) {
                    const row = eventTable.get(i);
                    if (!row || !row.event_cid || !row.data) continue;

                    const eventCid = row.event_cid.toString();

                    // Check if we have already processed this event CID.
                    if (!processedEventCids.has(eventCid)) {
                        // If not, it's a new event.
                        const jsonString = Buffer.from(row.data).toString();
                        const eventPayload = JSON.parse(jsonString);

                        console.log("🆕 New event received:", eventPayload);

                        // IMPORTANT: Add the CID to our set so we ignore it in future polls.
                        processedEventCids.add(eventCid);
                    }
                }
            }
        } catch (error) {
            console.error("Error during polling cycle:", error);
        }

        // Wait for 5 seconds before starting the next polling cycle.
        await new Promise((resolve) => setTimeout(resolve, 5000));
    }
}
1 Like

Oh, interesting. Let me play with your code and see what’s up. Thanks for testing!