Strange return Object from DHT Linked Call

Hi all.
Not sure if I’m going mad but I’m trying to make a call into a DHT and now getting a weird return object:

Object(
    {
        "instance_stats": Object(
            {
                "test-instance": Object(
                    {
                        "number_delayed_validations": Number(
                            0,
                        ),
                        "number_held_aspects": Number(
                            0,
                        ),
                        "number_held_entries": Number(
                            0,
                        ),
                        "number_pending_validations": Number(
                            0,
                        ),
                        "number_running_zome_calls": Number(
                            1,
                        ),
                        "offline": Bool(
                            false,
                        ),
                    },
                ),
            },
        ),
        "type": String(
            "InstanceStats",
        ),
    },
)

What I expect and was seeing previously with this call is the following chain data:

"{\"jsonrpc\":\"2.0\",\"result\":\"{\\\"Ok\\\":[{\\\"signal\\\":\\\"Norm\\\",\\\"author_id\\\":\\\"HcSCJjCxUn8Jv4fv7x4BYcqY5TmWvvg3ytrb35FYKMtyc6qwBXsae4w84qhkyjz\\\"},{\\\"signal\\\":\\\"Norm\\\",\\\"author_id\\\":\\\"HcSCJjCxUn8Jv4fv7x4BYcqY5TmWvvg3ytrb35FYKMtyc6qwBXsae4w84qhkyjz\\\"},{\\\"signal\\\":\\\"Norm\\\",\\\"author_id....

I’m able to communicate with Holochain and it’s only when I add links into my code that this error happens.
I’m also getting this same return when I do the Micro Blogging Tutorial so wondering if this is something changed in the backend.

Here’s my function call to the dht which worked previously:

pub fn get(_agent: String) {
        let json = serde_json::json!(
            {"id": "device",
             "jsonrpc": "2.0",
             "method": "call",
             "params": {"instance_id": "test-instance",
             "zome": "txrx",
             "function": "get_signal",
             "args": {"agent_address": _agent,
                }
            }});
            connect("ws://localhost:3402", |out| {
            out.send(json.to_string()).unwrap(); 
            move |msg| {
                println!("Chain Data: {:#?}", msg);
                out.close(CloseCode::Normal)
                }
            }).unwrap(); //change to expect
        }

my call looks like this:
get("<address of agent that committed the entry>".to_string());

Is this some known error/or change to the HDK?

My zome config looks like this:

#[derive(Serialize, Deserialize, Debug, DefaultJson,Clone)]
pub struct Signal {
    signal: String,
    author_id: Address,
}

#[zome]
mod txrx {

    #[init]
    fn init() {
        Ok(())
    }

    #[validate_agent]
    pub fn validate_agent(validation_data: EntryValidationData<AgentId>) {
        Ok(())
    }

    #[entry_def]
    fn signal_entry_def() -> ValidatingEntryType {
        entry!(
            name: "signal",
            description: "this is a single signal value into the dht",
            sharing: Sharing::Public,
            validation_package: || {
                hdk::ValidationPackageDefinition::Entry
            },
            validation: | _validation_data: hdk::EntryValidationData<Signal>| {
                Ok(())
            },
            links: [
                from!(
                   "%agent_id",
                   link_type: "author",
                   validation_package: || {
                       hdk::ValidationPackageDefinition::Entry
                   },
                   validation: |_validation_data: hdk::LinkValidationData| {
                       Ok(())
                   }
                )
            ]
        )
    }

    #[zome_fn("hc_public")]
    pub fn set_signal(signal: String) -> ZomeApiResult<Address> {
        let signal = Signal {signal, author_id: hdk::AGENT_ADDRESS.clone(),};
        let agent_address = hdk::AGENT_ADDRESS.clone().into();
        let entry = Entry::App("signal".into(), signal.into());
        let address = hdk::commit_entry(&entry)?;
        hdk::link_entries(&agent_address, &address, "author", "")?;
        Ok(address)
    }

    #[zome_fn("hc_public")]
    fn get_signal(agent_address: Address) -> ZomeApiResult<Vec<Signal>> {
        hdk::utils::get_links_and_load_type(
            &agent_address,
            LinkMatch::Exactly("author"),
            LinkMatch::Any,
        )
    }

My conductor is pointing to Sim2h:
sim2h_url = 'wss://sim2h.holochain.org:9000'
and have also tried changing that value to:
sim2h_url = 'wss://sim2h.holochain.org:9000'

Hi all. Is anyone able to help with this issue?
Trying to determine if the error I’m seeing is my issue or if it’s something with core.

Hey @simwilso you’re getting conductor stats that are meant for Holoscape! So it looks like your client code is talking directly to the WebSocket port, right? This will mean you have to match up JSON-RPC responses with requests manually. It also looks like conductor stats don’t come as JSON-RPC, I guess because they’re not a response to any function call.

What version of Holochain did you upgrade from / to? The conductor stats have been in for a month or so, so determining what your old version was will help me understand if this is a regression or a new unexpected feature that you’ll have to handle.

thanks @pauldaoust. the version that I was running originally that was fetching this ok was =0.0.34-alpha1.
the current version I’m running is: =0.0.42-alpha5

so sounds like I’d be better off (and the more supported setup) is for me to setup my agents/conductor/websocket via a holoscape client on my machine rather than directly using:

holochain -c agent1_conductor.toml

and pointing to sim2h.

Is this correct? If so I’ll get holoscape going and try that.
thanks Paul.

I wouldn’t worry too much about Holoscape; I’d just grab an off-the-shelf JSON-RPC client like the one from Parity and use that in your Rust cilent code. It should automatically filter out stats signals and pair responses with requests correctly (because JSON-RPC responses over WebSocket are not guaranteed to come back in the same order like they are with over HTTP).

thanks @pauldaoust, have been trying to work this out with Alfredo from EYSS but we’re still having some issues.
Going to try this parity suggestion tomorrow so will let you’ll know how we go but wondering did you find anything on whether this is a regression or unexpected feature in the upgrade.
A couple of Alfredo’s colleagues have reported some similar challenges when calling a chain from a linked search.
I can ask Alfredo to clarify but he mentioned yesterday that one of his colleagues had a similar problem using HTTP as well.

As mentioned though we’ll give the crate you mention above a try tomorrow and report back here if it resolves for us.

Thanks Paul.

Looks like these stats signals broadcast were introduced in 0.0.41 so it looks like it’s a new feature rather than a regression — although other signals have been in Holoscape for a long time, so it surprises me that you haven’t bumped into other things like this. Perhaps it does warrant some deeper digging; I’m curious about the fact that you (and some EYSS devs) seem to get the signals after a certain type of request — and possibly even with HTTP requests too, since I don’t think it even broadcasts any sort of signal over HTTP.

thanks @pauldaoust.
just out of this one that Tom had previously done which is a strongly typed version also returns the same response;

Hi @simwilso — looks like that code is also a roll-your-own JSON-RPC client, so it’d erroneously catch signals as well as JSON-RPC responses. Am I reading that code right?

Hi Paul, feels like we need to go back to first principles at this point.

  • websockets is now taking us in a new direction since 41
  • http is timing out via postman

Can we find a tutorial that works/doesn’t work now?
Are there TDD verification tests that third parties like us can run to re-verify where we are?

Hey @simwilso I got some info from Tom: if you’re using the proper conductor (not hc run) you can turn non-app signals off in the config:

[signals]
trace = false
consistency = false

But I think the more ‘correct’ path would be to use a JSON-RPC client that will filter out signals for you. Of course, if your DNA ever starts emitting signals of its own, you’ll want to maintain two connections to the WebSocket interface — one for the JSON-RPC calls/responses (because it will probably contain its own connection) and one to listen for signals.

@mikeg This HTTP timeout problem, I haven’t seen yet — how long until it times out? I wonder if you’re uncovering a performance issue in Holochain.

thanks Paul.
Just tried making this update in my conductor:

[signals]
trace = false
consistency = false

Then call:
get("HcSCJjCxUn8Jv4fv7x4BYcqY5TmWvvg3ytrb35FYKMtyc6qwBXsae4w84qhkyjz".to_string());

on the function:

pub fn get(_agent: String) {//-> String {
        println!("{:#?}", _agent);
        let json = serde_json::json!(
            {"id": "device",
             "jsonrpc": "2.0",
             "method": "call",
             "params": {"instance_id": "test-instance",
             "zome": "txrx",
             "function": "get_signal",
             "args": {"agent_address": _agent,
                }
            }});
            connect("ws://localhost:3402", |out| {
            out.send(json.to_string()).unwrap(); //change to expect
            move |msg| {
                println!("Chain Data: {:#?}", msg);
                out.close(CloseCode::Normal)
                }
            }).unwrap(); //change to expect
        }

But I still see the stats returned rather than my actual chain data:

Chain Data: Text(
    "{\"type\":\"InstanceStats\",\"instance_stats\":{\"test-instance\":{\"number_held_entries\":0,\"number_held_aspects\":0,\"number_pending_validations\":0,\"number_delayed_validations\":0,\"number_running_zome_calls\":1,\"offline\":false}}}",
)

I’ll give the Parity crate a try to see if I get a different result.

Hey Paul, I tried changing the protocol to HTTP in the conductor file, then I did a request using postman, with this function I got a timeout response after 10 seconds aproximately, with the others functions I got a good response using postman. The get function is the only one with troubles.
I also did a test just like in the Holochain tutorial and I got a correct response from the get function, but just with the test.

@simwilso aw poo, I wondered — those stats signals don’t seem like either tracing signals or consistency signals, so I’d feared that they wouldn’t turn off using that config directive.

@alfred_dev okay, that’s what I suspected re: HTTP; I think that’s performance issues in Holochain itself. What’s happening in that get function — just a get_entry, or sometihng more complex? And I presume you’re using some sort of ‘real’ networking like a public sim2h server?

@pauldaoust It is the same code that @simwilso

Any further word on this?

@mikeg I strongly suspect that the timeout with get_signals was due to performance problems in sim2h. I’m curious if things are better with the new Holochain 0.0.43-alpha3, which comes with a much snappier sim2h (5000 nodes simultaneously chattering, albeit with a very trivial scenario).

@simwilso how did things work out using a JSON-RPC client to filter out signals?

Hey Paul. Great timing as I’m about to try this out again today and will with the new version.
I wasn’t able to get it going with the RPC filter but tbh didn’t try very hard as we had to do a pretty big refactor of our zome code which we’ve been focusing on instead.

But as I say great timing as just today I’m ready to start writing my websocket calls into the zome from Rust and testing interactions across Sim2h.

wish me luck!! I’ll let you know if I find a resolution and upload my code.

1 Like

bummer!

My Cell Data: Text(
    "{\"type\":\"InstanceStats\",\"instance_stats\":{\"test-instance\":{\"number_held_entries\":0,\"number_held_aspects\":0,\"number_pending_validations\":0,\"number_delayed_validations\":0,\"number_running_zome_calls\":1,\"offline\":false}}}",
)

I will try run Sim2h from my local nix-shell and see if that gives a different result.

double bummer!
local Sim2h_server gives same result.

Signal Added: Text(
“{“type”:“InstanceStats”,“instance_stats”:{“test-instance”:{“number_held_entries”:4,“number_held_aspects”:4,“number_pending_validations”:0,“number_delayed_validations”:0,“number_running_zome_calls”:1,“offline”:false}}}”,
)

back to trying the Json RPC client!