The Future of hApp (or DNA?) Distribution

Technical Discussion App Deployment
, ,
#1

There are some random threads discussing the old pre-RSM Holoscape system of hApp distribution, and I’ve seen some mentions of a “hApp of hApps” or “hApp store” in the future, but couldn’t find a general conversation thread about hApp distribution in general, and especially some of the more future facing opportunities for it. This thread can hopefully serve that role.

To get things started: I’m curious about how the community sees the future. At the present time, it seems like the model most are working toward is to build a self contained “traditional” application, and to then distribute it as part of an executable file for desktop, or to eventually deploy on Holo as an alternative to AWS for webapp/mobile use.

However, this to me still feels rooted in old “siloed app” thinking. Is there interest in achieving a goal of a “unified” ecosystem of hApps? Where DNAs and data structures are layered and highly shared/composable, and UIs are built modularly and separately from the data? I’d be curious to see what that collective vision would look like, where hApp bundles, or DNAs, or even just Zomes are published to that hypothetical hAppstore DHT, and then bundled on the fly, with swappable/composable UI components.

What are your thoughts on this? Who do you know of who is working towards this goal? I’m sure there’s plenty more out there on this kind of thinking.

And finally, are there other visions out there about the future of distribution/deployment of holochain applications?

Related Projects from the Community

Here’s a (somewhat live) list of projects exploring this idea from the community, aggregated from the discussion below:

11 Likes
#2

Note: I didn’t want to clog the first post of this thread with my personal ideas because I didn’t want it to dictate the discussion, but in this post I wanted to outline my thread of thinking in more detail.

Context of my Perspective

I’ve been thinking a lot lately about possibilities of hApp composition and distribution. A lot of the tangible hApp work I’ve seen going around at this stage has been focused on a traditional “app” model, where there is a client that works off of a known (hApp) API. This is also how my current project is structured. However, I recently read through some work (this post and recent programming language by Paul Chiusano, and this entire website from Chris Gebhardt) that has gotten me excited about the potential of alternative ideas.

The gist of these links is the “holy grail” of a global filesystem (similar to IPFS) combined with a more advanced version of the modern web, where data is self describing, UIs are generated on the fly and with user input depending on context, and where the code itself is “content” addressable on the network.

An aside: None of these ideas are really “new” in this community, and much of them have crossover with the core concepts of Ceptr. I believe that seeing these ideas from a perspective “outside of holochain” has given me additional context to better understand them, which is why I bring them up.

This brought up a lot of ideas for me regarding the nature and structure of hApps in general.

  1. With a DNA being represented by a hash, we should be able to easily store the DNA itself on a “hApp of DNAs”, correct?
  • Is there anything preventing us from “assembling” a hApp bundle on the fly?
  • If not, does the .happ file give us anything other than a convenient way to distribute?
  • And if not, does anything prevent us from not worrying about the hApp at all, and only declaring DNA dependencies?

  1. With a DNA being essentially a “definition of data types and their validation rules”, we could anchor type information to the DNA’s hash, on a “DNA metadata/documentation layer”.

  2. Furthermore, this leads to the question: why not separate the DNA out even further, seperating the data structures themselves from the validation rules, so that the data structures are more visible, reusable, and composable?

  • Perhaps this is what a Zome can be seen as in practice? So then a further question might be, can we assemble DNAs on the fly? Can we create a “hApp of Zomes”, and configure a DNA to just be a list of Zome hashes? (do zomes have hashes?)
  • The goal of this idea is to get concrete, pinned versions of data structures (via a content hash) to be accessible from higher levels in a chain, hopefully all the way up to the top (the “app” or “UI”). That would allow UIs to be generated or recomposed on the fly, because they could be built against data type requirements that are validated by those data (or zome) hashes. Perhaps a “Zome metadata/documentation layer” could also provide detailed metadata on data types for given zome hashes.

With all of these pieces, data would be able to be reasoned about and transformed across many different layers through code.

  • Developers could easily import granular data/validation zomes to work from a shared pool of data structures that are verified through their hashes.
  • Simple DNAs would then be definable through a list of zome hashes, and for custom behavior there could be a handwritten zome that ties the data types together for bespoke functionality.
  • If all the data types in a DNA were defined via hashes, then a DNA could easily expose its data contracts to any other DNA, making DNAs easily layerable/augmentable.
  • To deploy an “application”, you would just need to write a UI that declares DNA dependencies. Because the data contracts should be visible all the way up to the UI via their hashes in the DNA definitions, the UI could be highly composable: developers could write UI views that conform to individual Zome specs, higher level DNA specs, all the way up to aggregates of DNAs (a hApp bundle).

This would make it much easier for different “layers” of developers to emerge:

  • Data architects: Those who are defining “building blocks” of useful data schema/validations, and perhaps “conversion” zomes between those schema
  • Micro-service Developers: Those who are defining the self-contained DNA “Networks” using those data definitions to provide useful services: identity, reputation, data storage, link repositories, etc
  • View designers: Those who are writing UI views tied to hash dependencies of a mix of DNAs and Zomes
  • App Developer/End user: Those who tie all of views together into targeted useful apps or applets; with the correct tooling, an end user could also do this work themselves—grabbing microservices and views off the shelf for emergent needs, or even just swapping views in their app with other compatible views to customize their app for their needs.
9 Likes
#3

two things:

and

5 Likes
#4

Totally! I’m excited to see your thoughts.

This kind of thinking is very aligned with what the compository is experimenting with: Compository Demo - YouTube and Compository GrapesJS Demo - YouTube.

I think that right now the flashiest thing there that would interest you is this element, which takes a hash, gets the entry and downloads at runtime the UI components for that zome, and renders the entry. Also see this

My latest thought is to have a compository dna besides the devhub and the happ store. In that DNA, anyone can publish UIs associated with a certain DNA hash, and then the compository infrastructure will do the rest. I’ve come to understand that the rendering of the entry does not depend only on the zome that created it, but also on the DNA in which that zome is embedded. That said, a lot of times the same UI elements will serve entries created in different DNAs. Those UI modules can also be tagged or categorized per themes, or difficulty to use (power users vs technical users).

I also want to implement a flow-composer zome with rete.js, that will let you define high-level workflows to compose different building blocks in your happ.

Nope there is not, I’m intending to do that in some way to meet the requirements for the low code conversations.

I know the core team is thinking about providing something resembling the ABI that smart contracts have in ethereum. I have experimented with rust reflection with promising results.

I think the way to think about this is that a zome is the basic unit of bindings between a set of data structures, their validation rules, and the functions to interact with them. As the content of the data structures depends so much on how you are implementing your validation rules, I don’t know exactly how you would separate them without finding yourself in a lot of trouble. Eg, if I import the ChessGameResults data structure, I basically have to reimplement the same validation rules with the same assumptions, for example, that the entry is countersigned by both players and their signatures validated against deepkey. I would be curious if you can think of a use case in which this is not so much the case.

8 Likes
#5

Oh wow, thanks for this thread. Really glad to see all these ideas and links!

It calls me to share an overview of what I have been working on for the last year since leaving the Holochain core team (that is as an active full-time member).

The idea to get rid of siloed apps, or even the concept of apps in general has been fascinating me for the last 6 years or so, at least in an explicit way - in an unarticulated way even longer. It was what pulled me in with Holochain. I quickly realised that switching your thinking to be agent-centric holds an implicit offer to create an eco-system of “languages” (spoken by those agents) where data can easily be shared and remixed beyond (current) app boundaries (see my article from 3 years ago: Holochain: Reinventing Applications | by Nicolas Luck | Holochain | Medium).

With Holoscape I wanted to show a very rudimentary first step into this direction for hApp developers to start thinking outside of monolithic apps that use Holochain just as a back-end replacement.

Perspect3vism

Working with Junto (and in particular @josh) on realizing ideas described in that article, I came to an interesting insight:

  • agents don’t only speak various languages (=DHTs/DNAs) with each other
  • they also hold different Perspectives when doing so
  • and meaning being contextual is held (even defined) by their very own/personal/subjective perspective, not really by the objective expression of a language (=entry of a DHT)

This led me to think, that the ontology underlying Holochain (1. agents and 2. languages/games/dances/DNAs/DHT) is a great start but incomplete. It leaves open exactly the questions being raised here. And a formal introduction of the concept of perspectives as a root ontological entity next to agents and languages is something worth trying.

So I spent the last year testing that idea :smiley:

The result is a project I’m currently about to birth into its next stage: Perspect3vism · GitHub

Ad4m

The core piece of Perspect3vism is this (meta-) ontology of 3 root entities:

  1. agents
  2. languages
  3. perspectives

Ad4m is the only aspect of Perspect3vism that already has a website (the rest is being worked on right now): https://ad4m.dev.

The Ad4m repository above includes types and interfaces which serve as the smallest common denominator. Apps (here meaning UIs) can be build against the Ad4mClient and thus would be decoupled from languages (DNAs) and sit on top of an abstraction layer that enables mixing of expressions of different languages within perspectives.

Ad4m executor

There is an almost complete, but already well working and tested implementation of Ad4m, called the Ad4m-executor.

Coming from Holochain, you could see this as a super-conductor. Similar to Holoscape, it manages a Holochain conductor for you, such that languages can request to have a DNA installed easily (no need for the language developer to deal with configuring and setting up the conductor).

See for instance the language of languages here: language-persistence/index.ts at master · perspect3vism/language-persistence · GitHub
(which is Ad4m’s version of the hApp of hApps).

But Ad4m Languages are not confined to hApps/DNAs. Nothing in Ad4m/Perspect3vism is actually dependent on Holochain. Ad4m languages can use any kind of distributed or centralized storage technology for their implementation. The Ad4m executor also sets up an IPFS node next to the Holochain conductor. See for instance this simple Ad4m language that implements text notes stored and shared via IPFS: GitHub - perspect3vism/lang-note-ipfs: Ad4m Language using IPFS to store text notes

Ultimately, an Ad4m language is a JS module that exports a create function which returns a Language object - implementing an interface. What happens inside the language (calling zome functions or doing HTTP requests) is completely up to the language)

Ad4m is a spanning layer decoupling semantics represented in perspectives from any kind of storage layer - so you can build apps mixing data stored in Holochain DHTs with transactions in blockchains, IPFS objects or centrally stored Tweets.

Perspectives … and Neighbourhoods

An Ad4m Perspective is just a set of links - that is triplets, i.e. links with predicates. In other words, a perspective is a graph over the expressions of languages.

Perspectives are the building blocks the ad4m implementation provides to sort and index data according to use-cases (not according to the tech used to implement any component of a use-case). UIs can make the ad4m-executor create new, emtpy perspectives like files. So a perspective can serve a specific use-case and encode associations between (objective) expressions (i.e. DHT entries) and thus build a complex graph that contextualizes data - very much akin to Semantic Web approaches.

Perspectives “are agent-centric”. Unless shared, perspectives are local only and fully and only controlled by the agent. They can be regarded as a scope for the user’s 2nd brain.

But perspectives can also be shared. Shared perspectives are called Neighbourhoods. (100% congruent with @sidsthalekar’s definition of a Neighbourhood “generic tech, specific culture” https://neighbourhoods.network/ - Sid and I have been talking periodically about these concepts for more than half a year now)

A Neighbourhood “entangles” several agents’ local perspectives with each other through a LinkLanguage (an ad4m language in which agents share links (the links that make up a perspective) with each other).

When sharing a perspective, a Neighbourhood expression (yes, in a language of Neighbourhoods: GitHub - perspect3vism/neighbourhood-language) gets created. That neighbourhood expression is what a user can share like an invite link (neighbourhood://Qm23hbef23423...) to have other ad4m agents receive the neighbourhood description, which includes the URL/hash of the link language, which in turn can be received via the language of languages and used to entangle a local perspective with other’s perspectives.

Since Perspectives don’t constrain which kind of expressions can be linked, Neighbourhoods are pure social contexts as mentioned by @ViktorZaunders in this low code conversation:

Perspect3ve

There is a general-purpose UI that is still very alpha. It is meant to serve as a generic browser for Perspectives and Neighbourhoods - and as a way to really communicate what Ad4m and Perspect3vism is. (The lack of this app being usable is what kept me from sharing all of this more publicly so far… but we gotta start sometime :wink: )

Status and Outlook

  • Above described workflow of sharing Neighbourhoods and joining them is working and part of ad4m-executor’s integration tests - including a link language built on Holochain
  • Junto is being built on Ad4m which means that their groups and channels are Ad4m Neighbourhoods. Which means any other app/UI (like Perspect3ve) can work with these “social/collaborations contexts” and add other data in the form of links to new/other languages.
    GitHub - juntofoundation/communities
  • Ad4m and the Ad4m-executor have everything that is needed (as currently known) for Junto’s use-case
  • Bootstrapping languages (language of agents, languages, perspectives, neighbourhoods) are implemented and working (using a mix of Holochain and IPFS).
  • we’re in communication with several other projects and there is a common sense that this minimal abstraction of agents, languages and perspectives is powerful enough for all use-cases discovered so far (which is the main claim behind ad4m that needs testing and empirical backing)
  • we just started hiring and assembling a team - looking for at least one more developer and potentially a UI/UX designer

You can talk to us and others looking into building on Ad4m in the Perspect3vism Discord: https://discord.gg/V9MfjpxB

12 Likes
#6

Hey there, thanks for starting this thread! This relates to a few topics on my mind. While I do have ideas for standalone hApps, I also wonder if the siloed design could impede adoption.

For the past few weeks, I’ve been working on an [eventually Holo-hosted] utility packaged as a web component, to be integrated into existing online stores. I had this idea while participating in Square’s recent virtual hackathon, in which they promoted their new Snippets API. I’m learning Svelte/SvelteKit for the UI, due to the lightweight size and included features like routing. (I’ll be happy to share more about the hApp once I’m ready!)

On a note related to distribution, I also wonder if an equivalent to server-side rendering could work on Holo’s architecture. I’d love to know what others think about this.

@guillemcordoba the flow-composer zome reminds me of an idea that I have as well! A while back, I was thinking of how a Holochain workflow tool like Zapier or n8n would work. (I was considering n8n as it is open-source, but haven’t made any progress.) Since then, I’ve used Supabase, an open-source Firebase-like Postgres suite, for my prototype. The Supabase team is currently working on their own open-source low-code workflow tool. I’d love to see whether it’s possible to adapt this tool for the SQLite version of Holochain. I imagine a Holochain version would need capabilities management to be added, as some actions might involve multiple source chains. Let me know if you have any thoughts about this too!

@lucksus thanks for the info on Perspect3vism! It seems like a very useful library that I’ll be looking into more.

2 Likes
#7

Fantastic! I’ve updated the first post with these projects to help direct people towards existing initiatives.

Yeah, this is why I was a bit hesitant to be specific about using Zomes as an example. To rephrase without trying to shoehorn into existing concepts: my goal would be so that data structures can be reused and enforced through content-addressability. So if I have a “blog post” structure, there could be a definition of that data contract that can then be included in a project via hash, and then any other arbitrary code could access and reason about that blog post contract at runtime via that hash. This would mean we don’t have to meet data contracts through “soft” means (the programmer looking up an API and writing code to parse the data structure in question), and would allow apps to reason concretely about the data structures they are using.

So if I’ve got that “blog post” as a part of some baseline data layer, I could write a new layer (DNA) that annotates blog posts, and since it’s pinned against the hash of that data structure, we could also potentially have that new layer work across an arbitrary number of layers, as long as it is consuming that same blog post data structure (confirmed by the hash). You could also version those data contracts, and write conversion functions between different hashes, so at run time an app can see a data structure’s hash, and then query a “data conversion layer” for any available steps to convert a given data structure into the one it requires, and this could all be abstracted away from the code itself. In this system, it would also encourage reuse and collaboration between developers because building your own data structures makes your application less useful, instead of importing pre-existent “standards” that the community has rallied around, and perhaps adding new layers to the existing data.

This is what I envisioned, and I was trying to understand if there might be a way to do it within the existing Holochain patterns. Since Zomes seem to be the smallest unit, I tried to use that as a tool for natively storing content-addressable data structures on the system. (Let me know if this even makes sense.)

Love this, and this is a great article, just read through it and see tons of crossover there. Gotta dig a bit deeper into all of the other links you posted, I’m excited to see all of the work you’ve done on this.

I’m glad this hit a nerve here, clearly there are plenty of people already exploring this.

My next question becomes: how can these parallel initiatives combine forces? Are there areas where they should be talking together or integrated? How do those working on this see these projects as different, and where do you see them as the same?

3 Likes
#8

Finally! We have a good articulation of Perspectivism and AD4M! :slight_smile: This is great.

2 Likes
#9

The link expired. Please could you share a new link (or another admin there)?

Thanks in advance.

#10

Here you go: https://discord.gg/N8ktmJms

1 Like
#11

Thanks Moritz!

#12

thank you so much for this post! what you’re discussing is the crux of the power of these technologies being developed… the word application does not really apply in the traditional sense… a Happ is more akin to the rules of a network. ultimately we are rebuilding the internet from scratch (and in my opinion closer to the original dream) with true sovereignty and customized experiences. imagine if a user’s behavior on Uber could be linked to AirBnB linked to __ Dating App or __ Job App… now all the sudden honest and good participation is highly incentivized. People can decide what’s important to them and not everyone has to use the tools in the same way but can still take advantage of similar functionality. Each “app” (DNA) can maintain specific features with easy and readily customizable UIs, while still allowing for a Single Sign On (SSO). Ad4m provides opportunity for biometrically and/or cryptographically verifiable UUIDs, thus preventing the very serious but often neglected Sybil Attack.

Furthermore, with perspectives, now all the sudden there is an enhanced ability for Agents to “see” whats happening in their respective networks and interact with each other in significant ways. They can agree upon shared meaning of context around various data structures and activity. Without a central server, there are many complexities involved in a distributed architecture that need to be addressed. This offers increased gossip resiliency and an improved ‘immune system’ as well as unprecedented interoperability.

Ultimately what this delivers is a mini-OS. No longer do we need dozens of individual apps but now we can have a bundle of programs all operating within the same network and sharing data collectively without comprising integrity. A single “app” with multi-faceted functionality. This has huge implications for IoT, AI, cryptography and so much more

#13

I’ve been iterating on these thoughts for the past few months and this is sort of where I’ve ended up. I’m slowly working on some documentation and prototypes for what I have tentatively been calling the “Coalescent Computer Protocol”, which isn’t really new; it borrows heavily from projects like Ceptr/Holochain, IPFS, and others, to define a simple self describing binary data format to create a single “conceptual” virtual machine with 256bit content addressable RAM, abstracted by each node’s local file system/database/whatever. All data is self describing and “monotonic”/deterministic, and all code lives as data on the network as well. Technically any piece of software that follows the protocol can interact with the data/network, but I also have a vision of a much lighter weight and performant version of a “web browser” that runs on the protocol as well. All of this could run on a holochain networking layer, but could also easily be run on other similar implementations like Hypercore.

2 Likes
#14

This article offers an alternative to the “Software Application Paradigm” — Paul Chiusano: The future of software, the end of apps, and why UX designers should care about type theory

1 Like
#15

I actually linked to that in my second post on this thread haha, it’s fantastic! If you didn’t see it, I also linked to the infocentral [link] project as well, which has been a big inspiration. Those two projects, in tandem with the vision from Ceptr and Holochain, have all inspired this thread and my work since.

2 Likes