Wanted: A C++ Client

The-A-Man · August 9, 2021, 7:36pm

It is ridiculous that I’m having to write one myself right in the middle of some UI development. It would have been better if there existed a C++ “Holochain-connector” library which, when given the URL&port, and the agent-id you wanna make the call as, and obviously, the fully-resolved function name, take a C++ struct, serialize it with messagepack (using their C++ library), listen the result and have it be deserialized into the supplied template-arg type that the caller expects for the result.

Saying that “We talk WebSocket, so you can connect the UI from any language that talks WebSocket” is rude. It’s equivalent to saying, “If you want to write a (h)app’s UI, you better be ready for drudgery”. You should better provide ready-made client libraries for all languages that people use to write UIs, especially C++ [not everyone loves node].

Connoropolous · August 9, 2021, 9:51pm

so far i’ve helped two folks implement it in C# … @dellams @jakintosh

I’m guessing it’d be really similar, and isn’t too difficult. I’ve suggested that the API needs documentation. Agree that there needs to be more diverse client libraries

At a baseline request needs to look like this:

github.com

holochain/holochain/blob/2d9401a5e0f74934195a0bf02ca198679b53089d/crates/holochain_websocket/src/lib.rs#L150-L158

    
      
          /// A request that requires a response.
          Request {
              /// The id of this request.
              /// Note ids are recycled once they are used.
              id: u64,
              #[serde(with = "serde_bytes")]
              /// Actual bytes of the message serialized as [message pack](https://msgpack.org/).
              data: Vec<u8>,
          },

which serialized is:

    {
        id: u64,
        type: "Request",
        data: Vec<u8>,
    }

The-A-Man · August 9, 2021, 9:53pm

The request::payload has to be binary (msgpack encoded), or rather, the request::data::payload.

But does the “request” that the conductor_api expects have to be binary as well (as you suggest)? Can’t it just be string? The request being: cell_id, fn name, zome name, payload, etc. What’s the point to encode it too with msgpack as “data”?

jakintosh · August 9, 2021, 10:06pm

Yes, the whole thing must also be serialized via msgpack. The inner payload, as well as the outer “Request” struct.

The-A-Man · August 9, 2021, 10:09pm

Also, one more question, @jakintosh:

Did you write your client library separately (so as to be someday pushed to the Github/Holochain organization as a standard C# client library (community developed)), or did you just jam it in your (h)app’s C# UI code itself?

jakintosh · August 9, 2021, 10:33pm

I’m still cleaning it up but it will be easily extractable, though it relies on two (unvetted) packages (one for websockets, and one for msgpack). The holochain interface itself is barely 200 lines of code, and I think after cleaning it up it will be smaller. The most difficult part was making sense of some parts of documentation, and getting help from @Connoropolous about specific corners of the holochain codebase I needed to solve my problem.

For reference, here’s a gist of my C# implementation when it was finally working end-to-end (with absolutely no refactoring or cleanup )

gist.github.com

https://gist.github.com/Jakintosh/8578f39a66e50c4beb24bc34275e6ce5

Holochain.cs

using NativeWebSocket;
using SouthPointe.Serialization.MessagePack;
using System;
using UnityEngine;

namespace Client {

	public class App : MonoBehaviour {
		
		[System.Serializable]

This file has been truncated. show original

lib.rs

use hdk::prelude::*;

#[derive(Serialize, Deserialize, Debug)]
pub struct SomeExternalInput {
    first_name: String,
    last_name: String,
}

#[derive(Serialize, Deserialize, Debug)]
pub struct SomeExternalOutput {

This file has been truncated. show original

Connoropolous · August 9, 2021, 11:41pm

Hey…
I’ve been dancing around the possibility of doing this write up for a while… can y’all check this out? @jakintosh does this read right with your experience. I actually did read your source code again for some degree of confirmation and sanity checking.

Zome Call Serialization API Hackmd

jakintosh · August 9, 2021, 11:45pm

Yep, this covers every gotcha I wrote down in my notes that weren’t Unity/C# specific.

dellams · August 18, 2021, 1:35pm

Now @Connoropolous has finally helped me get a windows binary working again after waiting 2 years when HoloNET was first built (the first C#/Unity client for Holochain), I can now finish the upgrade to RSM as well as the .NET HDK and then STAR ODK, all of which will be released soon…

I was about to create a Nuget package for HoloNET 2 years ago but then NixOS and then RSM threw 2 curved balls at me so I delayed doing that until could get it working properly on Windows again…

Hopefully the very long way will have been worth it! lol

Cheers
D.

The-A-Man · September 7, 2021, 1:31pm

Hey guys,

github.com

holochain/holochain/blob/45e524167615e7751a12ada73c8e35e6f9cf33ae/crates/holo_hash/src/hash_type/primitive.rs#L28

    
      
          // hCkk 5252 <Buffer 84 29 24> * HEADER
          // hCok 5380 <Buffer 84 2a 24> * WASM
          // hCsk 5508 <Buffer 84 2b 24>
          // hCwk 5636 <Buffer 84 2c 24>
          // hC0k 5764 <Buffer 84 2d 24> * DNA
          // hC4k 5892 <Buffer 84 2e 24>
          // hC8k 6020 <Buffer 84 2f 24>
          pub(crate) const AGENT_PREFIX: &[u8] = &[0x84, 0x20, 0x24]; // uhCAk [132, 32, 36]
          pub(crate) const ENTRY_PREFIX: &[u8] = &[0x84, 0x21, 0x24]; // uhCEk [132, 33, 36]
          pub(crate) const DHTOP_PREFIX: &[u8] = &[0x84, 0x24, 0x24]; // uhCQk [132, 36, 36]
          pub(crate) const DNA_PREFIX: &[u8] = &[0x84, 0x2d, 0x24]; // uhC0k [132, 45, 36]
          pub(crate) const NET_ID_PREFIX: &[u8] = &[0x84, 0x22, 0x24]; // uhCIk [132, 34, 36]
          pub(crate) const HEADER_PREFIX: &[u8] = &[0x84, 0x29, 0x24]; // uhCkk [132, 41, 36]
          pub(crate) const WASM_PREFIX: &[u8] = &[0x84, 0x2a, 0x24]; // uhCok [132, 42, 36]
          
          
/// A PrimitiveHashType is one with a multihash prefix.
          /// In contrast, a non-primitive hash type could be one of several primitive
          /// types, e.g. an `AnyDhtHash` can represent one of three primitive types.
          pub trait PrimitiveHashType: HashType {
              /// Constructor
              fn new() -> Self;

For the past 3 days I’ve been banging my head against this error, still no clue… This is the error I get:

??type?Response?id?r?data?z??type?error?data??type?deserialization?data?LBytes(Deserialize(“HoloHash error: BadPrefix(“DnaHash”, [117, 104, 67])”))

My DNA Hash being: uhC0k5PE8a9AjyXj5RDqDPFiQG0zhrY0nghGjyIH5b7pCMOYVvJxP

I convert it to UTF8 then copy the raw bytes on to the buffer. The buffer does represent a valid UTF8 string; totally readable. But then, what is the encoding scheme Holochain expects? From https://github.com/holochain/holochain/blob/45e524167615e7751a12ada73c8e35e6f9cf33ae/crates/holo_hash/src/hash_type/primitive.rs (cc @thedavidmeister) I can not understand how you get the [132, 45, 36] prefix.

@jakintosh’s https://gist.github.com/Jakintosh/8578f39a66e50c4beb24bc34275e6ce5#file-holochain-cs-L80 worked successfully with UTF8, mine doesn’t.

Please help! (cc @Connoropolous).

PS: As @Connoropolous said in the hackmd doc, I’m not serializing the buffers (for DNA Hash and Agent Hash) with messagepack.

jost.schulte · September 7, 2021, 1:57pm

Purely as an aside as I’m still similarly puzzled around the DNA/AgentPubKey hashing conventions: I equally don’t get why the TS/JS conductor api expects the leading u to be trimmed https://github.com/holochain/happ-client-call-tutorial/blob/main/ts/src/app.ts#L16

Some clear documentation for this would be great!

jakintosh · September 7, 2021, 3:15pm

Hey, I ran into this exact same problem: Conductor Library Roadmap - #17 by jakintosh

In my case, I was unnecessarily converting a C# string into a utf8 encoded byte[]; directly passing the string itself solved my problem. I think this is because the hash is not actually encoded in UTF8, but that you do that for readability purposes, but I’m still a little fuzzy on that.

It would help to know what you are doing that results in the error you are getting. From my understanding when I as troubleshooting, you can also run into this problem if you send the wrong kind of hash (like a dna hash when it’s expecting an agent hash). Perhaps you have set up your cell definitions backwards, swapping agent hash and dna hash.

The-A-Man · September 7, 2021, 3:48pm

It worked! Thank you so so so so so much, @jakintosh… You’re my hero! Love you! This saved my life! I was almost on the brink of a mental breakdown (haha!). I now realize that being an early adopter of a really new tech has its downsides. But a community like this one nullifies all those downsides. Thanks for helping really. I guess it was foolish of me to not explore that option myself. I wonder why Connor did not mention that in the hackmd doc. Anyway, right now I’m able to 1) run the app in the sandbox, 2) port-forward the app-ws port from WSL to native Windows, 3) allow it through firewall, 4) connect to the app-ws from my UE4 game/UI, 5) make the zome call without any errors.

The-A-Man · September 7, 2021, 4:01pm

However, the response hasn’t come yet…

For

#[derive(Serialize, Deserialize, SerializedBytes, Debug)]

struct SayHelloRequest {}

#[derive(Serialize, Deserialize, SerializedBytes, Debug)]

struct SayHelloResponse {
pub output: String,
}

#[hdk_extern]

fn say_hello(_: SayHelloRequest) → ExternResult < SayHelloResponse > {
Ok(SayHelloResponse {

    output: "Hello!".to_string(),

})
}

What should my payload be? I’m passing an empty object

github.com

ar90n/msgpack11/blob/e15f1d33f3b06b9552f4e8b24d83f8c89fb31c35/msgpack11.hpp#L57

    
      
              BOOL        = 12 << 2,
              STRING      = 13 << 2,
              BINARY      = 14 << 2,
              ARRAY       = 15 << 2,
              OBJECT      = 16 << 2,
              EXTENSION   = 17 << 2
          };
          
          
// Array and object typedefs
          typedef std::vector<MsgPack> array;
          typedef std::map<MsgPack, MsgPack> object;
          
          
// Binary and extension typedefs
          typedef std::vector<uint8_t> binary;
          typedef std::tuple<int8_t, binary> extension;
          
          
// Constructors for the various types of JSON value.
          MsgPack() noexcept;                // NUL
          MsgPack(std::nullptr_t) noexcept;  // NUL
          MsgPack(float value);              // FLOAT32
          MsgPack(double value);             // FLOAT64

Looks like my app’s init() function panicked or something. 'Cause I just ran hc sandbox call activate-app test-app, and it said

Error: Failed to activate app, got: Error(InternalError(“Conductor returned an error while using a ConductorApi: AppNotInstalled(“test-app”)”))

By the way, calling hc sandbox call install-app yielded

hc sandbox call install-app uhC0k5PE8a9AjyXj5RDqDPFiQG0zhrY0nghGjyIH5b7pCMOYVvJxP

Error: Failed to install app: Expected AdminResponse::AppInstalled but got Error(InternalError(“Conductor returned an error while using a ConductorApi: AppAlreadyInstalled(“test-app”)”))

It sure must be my lengthy init() function that’s failing… Would try calling a simpler app first.

By the way, @Connoropolous, what does activate-app do? Does it call the app’s (the DNA’s) init() function, as I’m assuming?

The-A-Man · September 7, 2021, 5:26pm

LogTemp: Warning: Socket connected.
LogTemp: Warning: CallZomeFunction: assembled the wire-message.
LogTemp: Warning: CallZomeFunction: sent the dumped wire-message. WireMessage: 
LogTemp: data : ??data??cap??cell_id??5uhC0kb8NAI6Ts9fdahNGoz-xttLBSKg1G9v3AgsdcNb70LxmOwzS9?5uhCAk1gCrNfN6cJN6Owloj_lec-eC_6kczLrYjwnSGFt7zCjQh2GA?fn_name?say_hello?payload???Input?Nothing?provenance?5uhCAk1gCrNfN6cJN6Owloj_lec-eC_6kczLrYjwnSGFt7zCjQh2GA?zome_name?map?type?zome_call
LogTemp: id : 6259
LogTemp: type : Request
LogTemp: Warning: Added local function to delegate.
LogTemp: Warning: Test function ends.
LogTemp: Warning: Socket: Plain message received: ??type?Response?id?s?data?-??type?zome_call?data???output?Hello, honey!

Thank you so much guys for all the help. It finally worked!

Connoropolous · September 7, 2021, 7:27pm

Hey!

I’ve been having a week with less-than-average keyboard time so haven’t been able to respond.
Well worked through! I will review my doc to see what exactly is amiss

Connoropolous · September 7, 2021, 7:28pm

Not exactly, ‘init’ gets called at the time of the first zome call…

activate-app is required in order for the conductor to actually allocate memory and resources to the cells in the app. In particular, to perform gossip and make network connections with peers

Connoropolous · September 7, 2021, 7:31pm

No it’s not that because it says ‘uhc0k’ which is what Holochain says it expects as a DnaHash… the problem revolves around the fact that this encoding is a base64 encoded string. utf-8 has nothing to do with it, minus the fact that’s how we read the chars

Oh oops, you said “you can also run into this problem if you send the wrong kind of hash” which is true!

Connoropolous · September 8, 2021, 1:10am

Amaze! This is great to hear. I am so stoked on game dev potentials here

thedavidmeister · September 9, 2021, 11:15am

seems like this is working… hashes are not handled as strings under the hood, encoding is only used for humans to read/debug

if you ever have issues with compatibility i recommend getting rust to print out raw bytes and diff with whatever your code is producing