This thread is a continuation of a highly detailed discussion started by @pospi regarding GraphQL and where it makes sense to integrate into a Holochain-based stack. Client-side, zome-side, and conductor-side were all considered.
The Holochain core team do not (yet) consider GraphQL as being part of the core framework or officially supported stack.
There is no real performance impact for Holochain users between an in-browser GraphQL query adapter, and a GraphQL query adapter that runs within the conductor.
There is a negligible performance impact for HOLO users when using an in-browser GraphQL query adapter, as it results in more packets going via the wider internet through the hc-web-client connection.
Juniper can be compiled and run within DNA code as WASM, but this makes Holochain apps infeasibly large and is not recommended.
Thus, the recommended way of using Holochain with GraphQL is currently via an in-browser schema adapter. This has been successfully implemented in pure JavaScript in hp-admin; and in TypeScript with compiler-generated types in HoloREA.
So, the Holochain <> GraphQL problem seems to be largely a solved one for now, though ongoing experiments are toying with how to make these integrations more generic and less verbose to implement. There is also work to do in both implementations to make requesting data more efficient by adding something like dataloader.
The next big feature to implement is subscriptions, which looks to be dependent on the PubSubEngine interface (see issue in HoloREA repo). Iām curious as to whether anyone has started work on a Holochain-backed PubSubEngine or has plans to, as this appears to be the ācoreā layer module necessary to get this running end-to-end.
thanks @pospi for that excellent recap, esp the theoretical performance comparisons. I donāt know of anyone whoās started work on a PubSubEngineā¦ what would your needs be? Would this be for DHT-wide messages between agents, or messages between DNA and UI, or something else? (If this thing exists as a separate DNA, I totally we think we should call the individual networks PubSubClubs )
@Flip heh, you really want it in core, eh? There were some experiments embedding Juniper into a DNA (as mentioned by Pospi above) which were a way of prototyping whether itād be worth creating a GraphQL-based SDK/HDK, but yeah ā huge binary sizes.
^this one. And on ācore / not coreāā¦ I donāt think those are semantics we really need to be concerned with. The boundaries between organisations, codebases and parts of this ecosystem are all pretty fuzzyā¦ there is no reason we canāt have an officially supported JavaScript module in the Holochain Github org which provides everything needed to connect Holochain DNAs with JS-based clients built on GraphQL
Ahhhhh I think Iām getting a clearer picture of the vision. An app can have multiple backends (Holochain/SSB/ActivityPub) and multiple frontends, connected by an interface of GraphQL + an app-specific ontology. Then each backend has adapters that map between it and the GraphQL ontology. Is that right? (one wonders if it even makes sense to call it an app at this point )
Because youāre closer to your needs than I am, could you share what a generic, app-agnostic HC-to-GraphQL JS module would look like? The only thing I can imagine being versatile enough is something that accepts functions that map queries and mutations to zome calls, but in my naĆÆve implementation it wouldnāt add much value vs just rolling your own ad-hoc for every appās ontology.
The place Iāve been linking to is pretty much it, and is implemented as you describe.
what a generic, app-agnostic HC-to-GraphQL JS module would look like?
@wollum was playing with something like this, which I think was primarily about mapping Holochainās core entry & link structures onto a generic GraphQL schema that would allow inspecting and following data in a lowest-common-denominator sort of way.
The other cool thing GraphQL offers (which I havenāt played with yet) is āschema delegationā, which essentially allows defining 1 schema in terms of another. For exampe- defining ValueFlows in terms of a low-level Holochain entry/link storage schema.
And the picture gets yet clearer. Thanks! Sounds like the intent, in both your case and @wollumās, is to have as little mismatch between the GraphQL and the zome functions as possible, and thatās what enables this sort of generic-ness. Your strategy is to have the zome understand the appās ontology, but @wollumās strategy is to have an endpoint that doesnāt care about the data and just enables graph traversal.
(BTW your JavaScript reads like Haskell. Very pleasantly declarative and functional.)
WOW, YES! This is my vision too with the OASIS API to connect everything to everythingā¦ To eliminate silios and walled gardensā¦
What you call adapters I am calling Providers in my designā¦
Thanks to the amazing work @pospi and his team are doing over at HoloREA I was made aware of GraphQL & Apollo and have been researching them ever since and have concluded it solves a number of technical design decisions I was going through of how to provide an elegant unified interface to all the back-end providers that the OASIS API supports (Holochain, IPFS, SOLID, Ethereum, ActivityPub & lots more!)
The whole point of the OASIS API is to help expose Holochain to the rest of the world and to act as a bridge or stepping stone for everything to slowly migrate over to Holochainā¦
I look forward to working with @pospi and his amazing team on how we can work together to bridge the gaps and connect everything to everythingā¦ Our World (which is powered by the OASIS API and is like the XR Gamified Interface to it), will also be using HoloREA to power the e-commerce part and in-game purchases, etcā¦
Keep up the good work guys! Love what you are doing!
Forgot to add, the other difference is that I am building a .NET/Unity GraphQL/Apollo interface to Holochain so would be interesting to see where we have overlaps and where parts converge and can be shared/re-used, etcā¦
Can you elaborate on this a little more? Is size a problem because of the WASM runtime? Or a problem for app distribution? App updates? Or does it affect performance in another way?
WASM runtime size, yeah. Juniper is just quite a big package and Iām not sure how many MB of data it requires streaming from the hApp store for initial installation.
Whether itās a problem for app distribution IMO depends on the degree of modularity in which itās deployed. Too tightly bound to other zome code within paired DNAs may cause maintainability problems.
I implemented Juniper for my zome. It added 600KB from 1.9MB to 2.5 to the dist/dna.json. It will probably grow when I implement my types beyond the basics, but it doesnāt seem so bad yet.
yeahā¦ have been banging my head against subscriptions and graphql on the front endā¦
i have got to the understanding that
hdk::signal sets up a callback on the hc-web-client ā¦ onSignal(name:string, callback)
problem is that graphql wants an asynciterator for Subscriptionsā¦ (pubsub)
found a library thats converts a callback into an asynciterator:
We also use onSignal on our project and at first I was trying to make onSignal work with the Subscription of graphql but had the same problem with you. So I instead used it directly in my react component through useEffect here. Itās working fine EXCEPT when a callback uses the hc-web-client ā¦ call() that calls exposed admin functions that alter the confuctor config. I havenāt identified the real cause of this so (as Im attending to something else aotm) Im not sure yet what the problem is. Anyway, thatās what I have so far when it comes to onSignal. The library you found seems really interesting and might be whatās missing for us to use graphql subscription! Will check it out and try to use it as well. If possible, it would be awesome if you could share your feedback as well if ever you used it!