We should be able to answer the question “is this data valid for this stream, or is it mistaken/malicious”.
I still don’t see how validation helps us answer this question much.
It seems like there are three options:
- Supports one, or a few ways to validate data
- Support arbitrary ways to validate data through some programmable logic
- Don’t validate data
(1) greatly limits what can be built on the protocol, (2) would be great, but hard to get right. So to me (3) seems like the best option.
There are a few use cases, which we’ve heard desire for from users, that makes me convinced that (3) is the best option. The ability to point to files and folders (e.g. various ipld DAG structures), possibility to deleting data, and storing encrypted data. These features all requires a “detached payload”, e.g. the event only contains the CID of the payload.
Yes, this is definitely needed. I think that drawing a clear boundary of what the rust-ceramic code base does is the first step to help clarify some of this!