[UPDATE]: since I wrote this, I’ve had some conversations that illuminated some issues with my validation code, although it is not exactly the part of my write-up todo with how Atomic Commits work. That part is still right. Just the specifics of the 0 vs _ matching to check whether a project meta exists. Don’t follow that pattern.
I had an interesting experience today while working on testing my validation rules…
Here is my scenario:
- I have a type of entry called a ProjectMeta, which should ideally only have 1 per DNA/space.
- During the creation of one of these ProjectMeta entries, I would like to check that the creator does not have knowledge of some pre-existing ProjectMeta entry, in order that if it finds one, it will fail validation. However, I know that in a distributed system which has the nature of being “eventually consistent” (and only if “islands” of the network eventually discover one another), and so we can’t be sure that none actually exist. We can only be sure that none exist “that we know about”. It sounds obvious to state, but we know that "if we know one exists, one exists, but if we know of none that exists, one still may exist)
- In order to find an existing ProjectMeta within a DNA, we use a deterministic (and known by all) “Path” based on the HDK implementation. We use Holochain “links” to connect any ProjectMeta to that “Path”, making it findable by anyone on the network who is connected. Call it the “PROJECT_META_PATH”.
- Typically during my #[hdk_extern] functions, like the one where I would call
create_entrywith the new “ProjectMeta” entry, I would also immediately link that entry to its root path, for finding again. - I had a validation function that stated that as long as 0 ProjectMeta entries were found then this new ProjectMeta entry was valid, HOWEVER, when I tried this in practice through the app, it failed validation stating that a “ProjectMeta” entry already exists. This seemed odd, but I eventually realized it was not odd, and part of the Holochain validation design pattern.
Ok, so why did my validation fail when I thought it should have passed.
First, let’s look at my relevant validation code…
#[hdk_extern]
fn validate_create_entry_project_meta(
validate_data: ValidateData,
) -> ExternResult<ValidateCallbackResult> {
Ok(
...
// no project_meta entry should exist at least
// that we can know about
match inner_fetch_project_metas(GetOptions::content())?.0.len() {
0 => ValidateCallbackResult::Valid,
_ => Error::OnlyOneOfEntryType.into(),
}
...
)
}
You can see that it looks to match against the length of the list/vec of ProjectMeta that is returned, which underneath calls “get_links”. If the length is 0, then it’s valid, if the length is any longer than 0, then its invalid. This seemed to make sense to me when I wrote it, and was what surprised me when it failed and I got back the Error::OnlyOneOfEntryType error message.
Let’s look at the code when it didn’t work:
pub fn inner_create_project_meta(entry: ProjectMeta, send_signal: bool) -> ExternResult<ProjectMetaWireEntry> {
let address = create_entry(&entry)?;
let entry_hash = hash_entry(&entry)?;
let path = Path::from(PROJECT_META_PATH);
path.ensure()?;
let path_hash = path.hash()?;
create_link(path_hash, entry_hash.clone(), ())?;
let wire_entry = ProjectMetaWireEntry {
entry,
address: $crate::WrappedHeaderHash(address),
entry_address: $crate::WrappedEntryHash(entry_hash)
};
if (send_signal) {
let signal = convert_to_receiver_signal(ProjectMetaSignal {
entry_type: "project_meta".to_string(),
action: $crate::ActionType::Create,
data: ProjectMetaSignalData::Create(wire_entry.clone()),
});
let _ = $crate::signal_peers(&signal, $get_peers);
}
Ok(wire_entry)
}
#[hdk_extern]
pub fn create_project_meta(entry:ProjectMeta) -> ExternResult<ProjectMetaWireEntry> {
inner_create_project_meta(entry, true)
}
Let’s look at the code after:
#[hdk_extern]
pub fn simple_create_project_meta(entry: ProjectMeta) -> ExternResult<ProjectMetaWireEntry> {
let address = create_entry(&entry)?;
let entry_hash = hash_entry(&entry)?;
let wire_entry = ProjectMetaWireEntry {
entry,
address: WrappedHeaderHash(address),
entry_address: WrappedEntryHash(entry_hash),
};
Ok(wire_entry)
}
#[hdk_extern]
pub fn simple_create_project_meta_link(entry_hash: WrappedEntryHash) -> ExternResult<()> {
let path = Path::from(PROJECT_META_PATH);
path.ensure()?;
let path_hash = path.hash()?;
create_link(path_hash, entry_hash.0, ())?;
Ok(())
}
Why did I fix it by breaking it up into two functions?
To understand this I had to dig back into my knowledge of how validation hooks work… the details and context of when exactly they are called.
My initial mental model told me something like this: a validation hook is called, on the device of the header author (the one committing some action to their source chain) at the point just prior to the committing of the entry to their source chain, which should mean the same moment in the code as, for example, a create_entry method call is invoked
This seemed to be proven false by the code, as their was something cyclical going on… The cycle appeared very strange, and to break a logical model:
- create_entry should only work and a ProjectMeta entry written to my source chain if the call passes
validate_create_entry_project_meta -
validate_create_entry_project_metashould only work if there is no existing ProjectMeta linked off of the PROJECT_META_PATH - there should be no ProjectMeta linked off of the PROJECT_META_PATH because we are still validation the
create_entrycall and thecreate_linkcall happens AFTER the create_entry call - and yet
validate_create_entry_project_metais failing validation
I remembered something about the new “workspaces” implemented in Holochain that underlie the way source chain and DHT writes happen. They are supposed to be “atomic”, but I did not know what that means in practice. I wondered if anything was in the HDK documentation about it. THANKFULLY, there was.
I found this: https://docs.rs/hdk/0.0.100/hdk/#hdk-is-atomic-on-the-source-chain-
All writes to the source chain are atomic within a single extern/callback call.
This means all data will validate and be written together or nothing will .
So, I remembered reading some Holochain source code around validation, and the existence of something called the “scratch space”, which is a temporary workspace in which commits within a “single extern/callback call” all happen and yes, then get validated together.
So this all made sense now. I realized that in order to keep my validation rule “as-is”, which is the logical way to have it written, was to separate the two create_entry and create_link calls into two separate callbacks. Once I did, validation passed.
To restate this:
validate_xcallbacks don’t get called at the exact moment of the invocation ofcreate_entryupdate_entrydelete_entrycreate_linketc. they called just prior to the finalizing of a whole#[hdk_extern]function call, and thus, all validation rules must be written in such a way that assumes possible “multiple actions” to have been taken “at once” or alongside the exact action you thought you were validating. This is what the “atomic” means in practice.
It didn’t create any issue in my front end, I just separated what was previously one call, into two calls:
// we separate this into two calls so that
// validation can happen effectively on the ProjectMeta create_entry
const response = await dispatch(
simpleCreateProjectMeta.create({ cellIdString, payload: projectMeta })
)
// we then proceed to create a link so this entry can be found
await dispatch(
simpleCreateProjectMetaLink.create({
cellIdString,
// we send the entry_address which is the thing that
// needs to be linked to the PROJECT_META_PATH
payload: response.entry_address,
})
)
This is a very important thing to know and understand about writing validation rules, and I hope this post helps you in a similar situation.
