For those unable to make the core devs call on October 26, here’s a summary.
Agenda
- CIP discussions
- Ecosystem updates
- Us3r network
- Orbis
- Intuition
- Hirenodes
- 3Box Labs
- ComposeDB node separation
- Open floor
Attendees
Aaron Goldman
Joel Thorstensson
Michael Sena
Nathaniel Cook
Spencer Brody
Liang Qiao
Zhang Yunhe
James Pham
Donat
Mohsin Zaidi
Danny Browning
Paul Le Cam
John McClure
Sergey Ukstv
Auto notes
Apologies in advance if the notes for this meeting aren’t the most clear. We experimented with using AI-generated summaries.
Database Updates and Field Discussions
Joel and Michael organized a meeting with discussions on a variety of topics including updates on their shared databases and the possibility of using an application-specific field in the event ID. The decision was made to trial having notes from the Zoom summary function to make note-taking easier. Aaron discussed the tension between ordering events for efficiency and maintaining time-based order for cost-effectiveness. Spencer and Baptiste introduced the idea of adding an application-specific field to the event ID, which was initially referred to as a ‘namespace’ or ‘context’. Nathaniel pointed out the implications of using a fixed-length byte field for this data, suggesting it might limit the scope of their applications. The team agreed that the recon needs to be fixed with the context and might need to do the hashing themselves to maintain any kind of ordering.
Model Complexities and Data Handling Discussed
Liang discussed the complexities of using the same models in different applications and suggested that either a field in a model or an index level namespace could be used to solve this issue. He also mentioned an infrastructure upgrade that improved their data handling. Baptiste and Joel discussed the possibility of comparing data streams and Joel shared about an internal dashboard that shows individual rights of data production. Aaron and Spencer raised questions about the efficiency of data loading.
Cows, Shipping, and New Project Discussion
Spencer, Aaron, Baptiste, Liang, Joel, and Michael discussed their work, specifically about the pace of their cows, with Spencer expressing surprise. The team then shifted their focus to a new project that Liang proposed, which involves shipping a ceramic note as a service. This idea aims to improve developer experience by enabling users to spin up their own node for testing or development. Baptiste suggested integrating this service into their UI, providing a new note for the user. The team agreed to finalize the specifications for this new project in early November.
Modeling Framework for Applications and Databases
Michael and Baptiste discussed the modeling framework for applications, protocols, and composable data across databases. They emphasized the need for backwards compatibility and the use of instance documents. The team, consisting of Baptiste, Spencer, Aaron, Liang, Michael, Nathaniel, Joel, and John, deliberated on the handling of arrays in documents, developer experience, and the challenges of building databases on top of indexing and query systems. The conversation also touched on composite models and the concept of individual models. Towards the end, John from Intuition introduced himself and highlighted his work on leveraging attestations for subjective data. He identified some issues related to aggregate models and the need for a standardized procedure for ingesting data uniformly.
Progress, Deployment, Migration, Newsletter, and Feedback
John discussed the progress of their project, Nathaniel provided an update on the deployment of Restoramic in their QA environment, and outlined their plans to push it to Clay and then to main net. Nathaniel also mentioned that the migration process is working smoothly and can work with anyone who wants to migrate a node. Joel’s microphone was not working well. Michael discussed his idea of creating an ecosystem newsletter to highlight the work of all teams, and asked for input on how to gather and share information. James asked for feedback on a presentation about the composed of B mode separation.
Developing Custom Indexing Solution for Ceramic
The meeting discussed the development of a custom indexing solution for Ceramic. Sergi explained that the aim is to provide a simplified method for users to build their own solutions on top of Ceramic, without needing access to its internals. He also outlined different layers involved in this process, including a streaming layer, an aggregation layer, and an index solution. Sergi highlighted the importance of providing some level of introspection into what’s happening internally, such as exposing a feed of stream states and Ceramic events. Spencer suggested that focusing on the feed of changes to documents might be sufficient for most users for now, and deferring the lower level of actual events to a later stage. Joel asked how this relates to the Ceramic API, and Sergi responded that it is directly related but they are approaching it from a different angle.
Document vs Event Usage in System: Debate Continues
Aaron, Spencer, Liang, SU, and Nathaniel discussed the use of events and documents in their system. Liang highlighted that the users primarily care about the documents and may not be aware of events or commits. However, for building a new database, events could be more useful. The group also debated whether to push everything needed for processing from the firehose or just the stream ID or C id. SU explained that the idea is to push the stream state, which includes processed events, to eliminate unnecessary requests. Nathaniel pointed out a trust relationship that changes when events are not sent but documents are, emphasizing the need to verify the document’s construction from the events. Spencer countered that this doesn’t alter the current trust relationship with Ceramic.
Explanation of Ceramic Node System for Document State and Event Logging
Spencer explained how the current system works, where the ceramic node provides the current state of a document when asked. Nathaniel agreed with Spencer’s explanation. Spencer emphasized that the system also includes metadata about the log of events that went into computing the state. John found all proposed solutions interesting and expressed familiarity with constructing a database from a change data capture feed. However, he indicated that they would not dive down to the event level to verify the integrity of the document. It was determined that stream state changes would be sufficient for their needs. Spencer also mentioned the need to determine the right API for connecting to the node and accessing the feed, but time ran out for further discussion. John clarified that they only operate in a private networking context and do not run much off-browser.