Release churn, branch stability and changes to binary building

for anyone paying close attention to the release numbers coming out of the holochain-rust repo, you’ve probably noticed that lately we’ve been hitting large numbers like 0.0.x-alpha10 or higher for a single release

you may also have noticed that https://holochain.love is pointing to an older version of binaries (i think 0.0.34)

there’s a few reasons for this:

  • we have been using the same travis jobs to build our binaries since forever… they haven’t even been using nix and as of the last few weeks they started hanging during binary builds - rather than try to fix these scripts i ported as much as i could across to circle using nix-shell as the build environment
  • we started publishing our crates to crates.io which is awesome, but also there’s a lot of fiddly cleanup that needed to happen around getting e.g. README and LICENSE files and versioning scripts all cleaned up and on circle ci
  • the hc cli tool was refactored a few times to both reference the crates publishing changes and also to shift the scaffolding templates off to a separate github repo (so we can maintain the scaffolding without rebuilding all the binaries for every change to the template)
  • the nixpkgs version was moved to a more recent commit from the 19.09 branch upstream which allows for e.g. newer nodejs and electron, and fixes a bug in clang that was causing compilation to fail on mac
  • an urgent binary build was shipped to facilitate the holoscape alpha launch outside the normal release cycle/processes
  • at some point some additional shared/dynamic libs were added that needed to be accommodated downstream in holonix
  • new binary building tasks for sim2h_server and trycp_server were added, and in the process binary building was standardised/paramaterised

there is a new love branch in holonix that https://holochain.love now tracks (it used to track master)

AND

the name of the conductor binary tarballs attached to github releases has changed from conductor-* to holochain-* to be match the name in the Cargo.toml file which is more “standard” with the other binary builds

so the process churn described above is safely quarantined away from anyone using holochain.love, once it all settles down and we feel we have enough green ticks under our belt from the test suites, we can “bless” some commit on master and bump the love branch to match

this will be the process moving forward, we’re not exactly sure what the process of “blessing” looks like (it will likely become more rigorous over time) but it’s safe to assume that love will often be a little behind master

as this becomes clearer it will get documented as per https://github.com/holochain/holonix/issues/107

this allows power users and developers who need to track the bleeding edge to pin against new builds coming in if they need it (instructions at https://docs.holochain.love/docs/configure/)

it also allows for more stability/quality assurance in what holochain.love tracks, which is great for hackathons, other events and the holocurious to mess around with

also as per the above, there are shared libs in some of the binaries now, which makes me a bit concerned that they won’t work outside of the nix shell (i.e. if you just download it and put it on your $PATH manually) which is something i’ll chase up and throw some tests in the mix for (we want everything to be as deterministic and self contained as possible) - atm i think it only affects the hc cli tool, and not holochain itself, so it’s not a huge problem, just something to be aware of

4 Likes

CC: @dhtnetwork @zippy @Wladoo @lucksus @freesig @philipbeadle @jmday

Thank you for all your efforts on this and making this info visible to the community. I’ll also post in the Forum Announcement channel.

CC: @pospi

Thanks heaps @thedavidmeister this is really great work.
One thing I’m worried about is MacOS Catalina as it has some pretty big breaking changes notably:

  • The whole system drive is read-only (even with Sudo).
  • Any non apple store dynamic libraries or binaries have to be explicitly marked safe inside system preferences by the user.

I could try to add a Catalina image to our docs tests and let you know how that goes.

@freesig i’m aware of the catalina issues, i’m not sure that this is the thread for it tho?

Thanks @thedavidmeister!

Should I interpret this as- if I want a stable foundation to build on, I should only use the versions being updated in the love branch?

@pospi probably yeah

1 Like

It seems like crates.io is being a bit weird about the version suffix you’re using? :confused:

I have hdk = "0.0.38-alpha14" in Cargo.toml but when I build it’s giving me v0.0.38-alpha9… pretty whack :confused: but it works for me atm so *shrug*

@pospi to force a specific version you need to do =x.y.z-alphaAA with the = at the start

1 Like