Hi @alx — I moved your post to #technical:app-development.
hdk::query() has pretty limited filtering capabilities, as you say. In order to do the sort of querying you talk about, the underlying storage layer would have to understand something about the structure of the data. But presently the storage layer only understands data as strings.
I suspect that in the future we’ll need to take a look at making it easier to query data in a way that doesn’t require moving all entries into memory. My hunch is that this will look like having the storage layer understand whether the data being handed to it is JSON or a string, and indexing it accordingly. But there aren’t any plans yet.
So for now, you’re doing it exactly the way you should. In these early stages it makes the most sense for our core team to build functionality to a usable state, then focus on efficiency later.