More usecases for CALM (Consistency as Logical Monotonicity)

[abrahampalmer]

See the first part of How are timestamps and ordered timelines of events achieved on Holochain? for background.

I’d be interested on further comments regarding what CALM means for other hApp scenarios. The paper is a bit dense so I wanted to see if I was translating it properly. For instance I find the following two application type of examples in the paper:

The Amazon Dynamo has a shopping cart where items can be added and deleted as this data is synced. This eventual consistency of the state isn’t a problem until you go to do a checkout operation. Here the checkout needs to include a manifest of all the update IDs from the client that preceded it’s checkout. A shipping agent can then know to wait to receive all these updates before processing the checkout.

Another non-obvious use of CALM analysis is to identify when to compensate (“apologize” [28]) for inconsistency, rather than prevent it via coordination. For example, when a retail site allows you to purchase an item, it should decrement the count of items in inventory. This non-monotonic action suggests that coordination is required, e.g., to ensure that the supply is not depleted before an item is allocated to you. In practice, this requires too much integration between systems for inventory, supply chain, and shopping. In the absence of such coordination, your purchase may fail non-deterministically after checkout. To account for this possibility, additional compensation code must be written to detect the out-of-stock exception, and handle it by—for example—sending you an apologetic email with a loyalty coupon. Note that a coupon is not a clear mathematical inverse of any action in the original program; domain-aware compensation often goes beyond typical type system logic.

If I understand it right, I believe I have heard the “apologize” approach used as an example in Holochain where you have a low probability event and can safely deal with it after the fact if it happens. I think this applies for things like selection of a unique user handle or reversing a transaction found to be fraudulent. I’ve also heard of setting up verification functions to wait for all the information to be available before running which seems a version of the first case.

Please chime in or correct my examples of how to Keep CALM and be hApp’y (Maybe that needs to be a T-shirt )

[pauldaoust]

Heh, I was just talking with Art a week or so ago about whether it’d be useful to do a write-up on the general concept of logical monotonicity and how it might apply to app development. Seems that there is some interest…

I think there’s the hard logic aspect and the soft “what to do about the consequences” UX aspect. Both are pretty interesting IMO.

I think you’ve interpreted both passages correctly, and articulated them beautifully from a Holochain perspective. Wait for validation dependencies to be satisfied; apologise for the consequences of non-monotonic parts of your program. The nice thing about building everything on a system that’s logically monotonic by design is that the system itself can reason about these scenarios and trigger events accordingly. This allows you, as an app developer, to simply define the conditions and the handler functions and let the system do the hard work.

Holochain has already implemented the one — a validator can’t validate a link until it’s also received and validated the link’s base. I believe there’s the intention to extend this to validation dependencies in general — e.g., a transaction’s validity depends on the previous transaction from each counterparty’s chain, so it can’t be validated until both of those transactions have propagated and been validated. This allows for a nice inductive proof: if the previous transactions have been validated, then it means that some other validator has validated the transactions before them, and so on and so on until transaction 0. So all you have to do, as a validator, is pull the previous transactions, check the validation attestations on them, decide whether you trust those attestations, then validate the new transaction. Super speedy, and eventually this will all happen automatically FWIU.

It also opens up the possibility of setting boundary conditions that cause you to not trust the attestations on the previous entries and trigger a manual revalidation — say, for instance, the spread of validator addresses was wider than you’d statistically expect given your knowledge of the DHT, so you suspect the validators were in an evil partition. And all this could theoretically happen in the background without bugging the app developer or user at all.

Handling the latter — the conflict resolution or apology scenarios — is on the roadmap but not implemented yet. It’ll probably look like a set of automatic conflict resolution functions (CRDTs) that pull the problem into the domain of the monotonic, plus a conflict resolution handler that enacts some sort of human process. Essentially a function that gets called when the conflict couldn’t be resolved automatically and lets the app give the user a loyalty coupon, start an auction for the username, or allow the user to rollback a fraudulent transaction that they innocently accepted.

My interpretation of CALM is “you can’t unsay what’s been said, but you can say new things that create the basis for new truths”.

An interesting observation on the real-world impact on coordination conflicts, from this excellent article about local-first apps:

Conflicts are not as significant a problem as we feared.

We are often asked about the effectiveness of automatic merging, and many people assume that application-specific conflict resolution mechanisms are required. However, we found that users surprisingly rarely encounter conflicts in their work when collaborating with others, and that generic resolution mechanisms work well. The reasons for this are:

[…] Users have an intuitive sense of human collaboration and avoid creating conflicts with their collaborators. For example, when users are collaboratively editing an article, they may agree in advance who will be working on which section for a period of time, and avoid concurrently modifying the same section.