Federated Serving of Machine Learning Models on Holochain

Technical Discussion
#1

If I wanted to serve a machine learning model, ie load weights, receive a prediction query and serve that query using holochain’s toolkit in a distributed, how might such a thing work, and can the execution of the rules of the model be explicitly verified?

My current understanding is Holochain executed “rules” as a payload and those rules could be a model.

1 Like
#2

This is a very good, interesting, and CHALLENGING question. First, I have to confess that I don’t have a very strong grasp of ML — just the basics I learned from a series of articles a few years ago.

Armed with such a weak authority, I’m ready to explore! If I understand correctly, ML algorithms are kinda ‘squishy’ – that is, they’re attempts to find optimal solutions, not perfect solutions, within a problem space that makes solutions really expensive to discover.

Note: to streamline language, I’m going to introduce two terms:

  • worker = node who receives and computes the prediction query
  • validator = node who deterministically verifies the results that a worker produces

After doing a bit of reading, it seems that the work a worker has to do is expensive, but a validator’s work is easy – as long as they have access to the same data set as the worker. That gives me hope that federated ML on Holochain is viable – with the caveat that a data set might be huge, and that might place a big network bandwidth demand on the validator.

One thing I find interesting about this is that a Holochain database – an application DHT – is ‘eventually consistent’, which means that you’re never guaranteed to see an up-to-date replica of the data. This means that, if the data is accumulated as individual data points over time, the worker has to explicitly reference each data point that they used at time of calculation. So different workers will come up with different results because they’re equipped with different views of the whole. On top of that, because of the fuzziness of certain ML algorithms (they intentionally inject some entropy just to shake the algorithm out of stagnation), different workers might come up with answers that are different from each other – both ‘correct’, but one more ‘optimal’ than the other. These are fun problems to think about and design around!

I foresee a few design considerations:

  • There’s a 16MB limit to data chunks in an application’s DHT. If a data set is loaded in as a single chunk, it can’t exceed 16MB.
  • If a data set is loaded in as individual data points, that gets around the 16MB limit, but as I mentioned that’s a lot of network chatter.
  • There’s also a memory cap on the WebAssembly runtime Holochain uses to execute code – I can’t seem to find what it is, probably 2-3 GB, but it’s worth considering for big data sets.
  • In a Holochain app, any random participant might be called upon to verify any piece of data. If some people simply want to run queries and not validate them for reasons of performance constraints, that might not work out too well. You can use some nodes as ‘bridges’ between a public space and the space occupied by workers and validators, as long as you don’t mind a wee bit of centralisation. Or I suppose you could build some verification process in application-space, bypassing the built-in random validator feature.
  • Right now, with the primitive state of end-user-friendly tooling, it would be difficult to distribute and install the ‘rules’ on worker, validator, and user machines. Pretty soon, we’ll have something called the DevHub – a sort of application store, itself a Holochain app, that can automatically install and activate new executable payloads. Apps can use the DevHub’s API to make this process automatic and invisible to the end-user.
4 Likes
#3

why? how could I find out more about this and possible workarounds?

can you elaborate on this please?

many thanks :pray: and thanks John for the question