Simplify event creation

The choice of JOSE for representing signatures in Ceramic was made in order to make it easy for sign and create events. However, due to an unfortunate decision made in the design process of dag-jose (an IPLD codec for JOSE) the creation of events requires specialized libraries that can encode data as dag-cbor, as well as producing CIDs. This means that off-the-shelf JWT libraries are not sufficient for creating events, which would be desirable because it would make it trivial for anyone to create custom clients for Ceramic.

Current event structure

DataEvents currently have the following layout,

type Event InitEvent | DataEvent | TimeEvent

type DataEventPayload struct {
  id &InitEvent
  prev &Event
  header optional struct {
    controllers optional [String]
  }
  data Any 
}

type DataEvent struct {
  payload String
  signatures [Signature]
  link: &DataEventPayload
}

In order to create an event a client needs to do the following:

  1. Create the DataEventPayload and encode it using dag-cbor
  2. Compute the CID of the data in (1)
  3. Encode the CID as base64url and sign over it using a JWS algorithm
  4. Create a wrapping object structured as a DagJWSResult, put the resulting jws in this object, encode the data from (1) as base64 and put it in the linkedBlock field

Note that for now I’m skipping the extra complexity of creating a commit that includes a CACAO object-capability.

As we can see above the process of creating an event is quite complicated and not something that any developer can do in a few minutes as a small part of a larger application.

Simplifying data events

What if the process of creating an event was as simple as signing a JWT? Well, not only is this possible but the ground work has already been started. Let’s have a look at how this could work.

First we would need to sign the event payload directly, removing the step of dag-cbor encoding. JWTs always sign JSON data, so we can easily structure our payload as dag-jose manually without any third party library (that is as long as we know the CID of id and prev, which we should have already from interacting with Ceramic).

{
  "id": { "/": "bafy2bzaced..." },
  "prev": { "/": "bagcqcerals..." },
  "data": "The data of my event"
}

This payload can then be signed as a JWT, or plainly a JWS. Ceramic’s apis could be updated to accept this JWT without any additional data.

IPLD representation

One problem remains however. How do we would represent this in IPLD? dag-jose can not be used since it requires the payload to always be a CID, which is not the case here. The resulting JWT will be encoded in the compact serialization, e.g. eyJhVCJ9.eyJzIyfQ.Sflsw5c, so we couldn’t just store this data using something like dag-cbor without loosing the ability to inspect it like any other IPLD object.

Varsig is a new standard being developed at CASA. It provides a way to represent any type of signature as a byte string that includes information about how to parse and verify it. What this means in practice is that we can store the payload from above, appending an additional field _sig that contains the varsig bytes, directly in IPLD using any encoding.

type Event InitEvent | DataEvent | TimeEvent
type Varsig Bytes

type DataEvent struct {
  id &InitEvent
  prev &Event
  header optional struct {
    controllers optional [String]
  }
  data Any
  _sig Varsig
}

For example, in our protocol implementation we might chose to store the above IPLD object using dag-cbor. This would mean that DataEvent’s can be easily observed in the IPLD explorer or through any IPFS gateway. While the explorer actually did get support for dag-jose merged, a single object DataEvent will greatly increase understandability of the Ceramic event log.

Additional event formats

While supporting plain JWTs as events is likely the biggest unlock for Ceramic in terms of what varsig provides, there are additional features that could be enabled. In particular we could support multiple new event formats. The ones that comes to mind are EIP-712, W3C Verifiable Credentials, and IETF Verifiable Credentials. Adding support for one or multiple of these could make Ceramic more accessible to more developers, and potentially make already existing data compatible with Ceramic without any extra effort of the data creators.
It’s however worth noting that the extra complexity of supporting various different event formats might prove to be more effort than the value it provides. Each new format we plan to add should be evaluated carefully.

5 Likes

There’s actually a much simpler way to make events creating using JWTs (and by extension sd-jwt-vc) possible without the need to implement varsig. We can simply extend the dag-jose spec to support decoding payloads.

Current limitations of dag-jose

Currently the payload of a dag-jose object can only be a base64-url encoded CID. This is the main thing that is causing the problem described above since the user creating an event needs to understand how to encode their event data using dag-cbor. Currently the dag-jose codec reads the CID from the payload field and outputs it as a CID instance on a new link field. We could fairly trivially extend dag-jose to detect if the payload is not a CID and decode it properly.

Proposal to extend dag-jose

We extend dag-jose to also support arbitrary json data. When the dag-jose decoder parses the IPLD block that has json data instead of a CID in its payload we create a new field pld where the decoded data is added (note that we need to retain the base64-url encoded data in the payload field for the object to be easily validated with standard JWS tooling).

One key consideration for how to decode the payload is how to deal with CIDs. Normally, e.g. in dag-json, CIDs are represented as follows:

{
  "wikipedia": { "/": "bafybeiaysi4s6lnjev27ln5icwm6tueaw2vdykrtjkwiphwekaywqhcjze" }
}

However, this causes issues with the sd-jwt-vc spec because the object approach to the CID encoding is not a valid url. We can diverge from dag-json and instead use ipfs urls for CIDs:

{
  "wikipedia": "ipfs://bafybeiaysi4s6lnjev27ln5icwm6tueaw2vdykrtjkwiphwekaywqhcjze"
}

Drawbacks

The main limitation of this approach is that we wouldn’t have a future proof way of representing signature formats besides JWS, e.g. eip712 or W3C VCDM.

Adding support in Ceramic

Extending Ceramic to support the new format would be fairly straight forward. We would need to update the dag-jose library, then add support for getting the inline payload from the pld field, which should be a fairly minor change.

It would break backwards compatibility, right? We lose interop with dag-jose on kubo/IPFS. Probably all right if we are in ReCon land. Is this half-compatible step worth it though?

1 Like

Not necessarily, Kubo could also be upgraded to use the new dag-jose codec.

Fwiw, if we can support plain JWS signatures with dag-jose, I’m not sure we are gonna need varsig for a very long time.

Would it be simpler to propose a new multicodec called dag-jose-flattened or something like that, to signal that a conversion-step was done to flatten IPLD-style native link objects to SD-JWT-friendlier string-leafs? i’m thinking by analogy to the RDFC and JCS entries, which are basically identical to others except that they warn the consumer (in-band) about an extra canonicalization step versus the pre-existing multicodecs which are their raw/not-necessarily-canonicalized equivalents.

1 Like

that way you could at least segment the system into the backwards-compatible portion of the network that converts dag-jose to dag-jose-flattened before doing SD-JWT things, and the yolo portion of the network that just drops dag-jose on sight

1 Like

Interesting idea. However, I think the upgrade process in this case would actually be more complicated, since developers would now need to support two different codecs.

Two key things to note:

  1. The change to dag-jose is completely backwards compatible, all old data would remain loadable with the new version of the codec
  2. The dag-jose spec is still in draft so a slight upgrade should be fine

Update!