Querying all active streams and tiles

Hey,

Im building a Next application using the ceramic http client.

Is there a way to query all active streams or tiles for a given / connected did ? I’ve read through the documentation and it seems that each time you need either a specific stream or commit id.

I’m a bit confused about how to do a general query, I’ve also looked at loading a document via deterministic for a given did and it seems to only bring back the latest tile created with the tags/family metadata specified even when multiple tiles have been created including the same tags/family.

Ultimately I’m looking to do a general query from a did. Is this possible?

2 Likes

Having the exact same problem. Have been trying to get all tiles through the IDX index but that didn’t work as I need it to

1 Like

Using TileDocument there isn’t a way to query across the whole network. But using the IDX protocol through the DIDDataStore library allows you to associate data to a given DID for a given data model/schema. Kind of like a simple key-value store tied to the user identity. So if you know the DID of the user you want data for, then DIDDataStore should work well for you. If you want to query across all user DIDs in the network, that’s something that isn’t easily possible with TileDocument/DIDDataStore. For queries across users, I’d recommend checking out ComposeDB, the new graph database built on top of Ceramic that has been the big focus of the Ceramic dev team for the last several months.

1 Like

if we create a model/schema does this also mean that any other platform could call that same schema and edit the data, similar to how basic profile can be called by any app and the user can update it?

is there any code samples you could share regarding the IDX protocol / DIDDataStore… i think that would be most suited for me, I only want to query for the connected user’s DID not the whole network/across users. thanks!

yep, that’s right! Streams are controlled by the end user’s DID, so an end user can edit their own data regardless of what app they are using to access it. Multiple apps can be built to read/write the same data models.

I think @avi or @sterahi should be able to point you towards the best place to get started with DIDDataStore.

Actually going to go ahead & tag in @paul & @zfer as they have more experience with DIDDataStore! That being said if you’re looking for a simple app that is using DIDDataStore I’d suggest taking a look at https://www.npmjs.com/package/@ceramicnetwork/create-ceramic-app.

cool thanks! had a look at the app. I’m still a bit confused of how we can query all tiles in a stream from a schema? there doesn’t seem to be anything in the documentation or examples. Is there a way to be able to see all of the “version history” of a stream from the datastore?

I’m trying to make it where after I create a model and deploy it a user can have multiple entries of that model, for example like in mongo when you have a “collection” you can add multiple entries to that collection, but it seems that each tile a model is updated by the user it overrides the previous version (which makes sense when thinking about the model from the “basic profile” example). Is there anyway to add on multiple entries to a model or I’d have to deploy a new model each time and manage it this way?

@avi or @sterahi , is it possible to get any insight on this? quite blocked atm

With DIDDataStore, you can only map a DID to a single TileDocument for a given Model. If you want to have a list that grows, one approach is to have the document in the DIDDataStore contain an array of StreamIDs of other TileDocuments, which you can then go and load directly. This can work well so long as the list doesn’t get too large. If the list is going to grow arbitrarily large, then you’re likely to start having issues when the “index” document gets too big.

Better support for growing lists/collections like this is one of the main reasons we’ve built ComposeDB. ComposeDB is live on our Clay testnet now, and Beta support will be launching on Mainnet next month. If that works for your release timeline, I’d encourage you to check out ComposeDB as it is significantly more powerful and performant than DIDDataStore, and is where most of the resources from the Ceramic core development team are going right now.

It seems like composedb would be best suited as the list would grow arbitrarily wrong, are there any repos with example implementations I can check out?

I believe https://composedb.js.org/ is the best resource for getting started with ComposeDB. I don’t think there’s a repo with an example implementation yet, though @avi would know for sure.

2 Likes

maybe someone who is further along in the intro cohort could share an intro app? I could use the same, as it appears currently there is still some glue between old ceramic and composeDB which holds this together, which isn’t fully documented yet.