Hacker Newsnew | past | comments | ask | show | jobs | submit | hamilton's commentslogin

Indeed, OLAP cubes keep finding new life. The critical difference here is that there isn't a heavy query engine (just a Parquet reader) and a few http requests to pull from the cube. So it's light on the backend and light on the frontend.


I think it depends on (1) customer expectations for freshness and (2) scale (both for the cubes and for the customer data in toto). There are many types of customer facing dashboards where giving "live data" is a bad idea for them and for you. And recomputation is indeed a problem, but if the volume of data isn't that high to begin with, it's probably easier than setting up an incremental pipeline architecture vs. a grouping set query in DuckDB. But I am not really a data engineer, so perhaps this is naive.


It's a good question. In a sense, the cube is caching, just materialized ahead of time instead of memoized on demand. A result cache still needs a live database behind it for misses; the cube has no misses, since every question the dashboard is designed to answer has data in the cube already. And for this experiment, the goal was to forgo a database to serve the data in the first place.

I provide caveats for when this would work vs. when it doesn't in the post. For a lot of customer-facing dashboards, I think it's probably pretty good.


Agreed, for a public demo like this one, GitHub Pages would work great (or any host that speaks HTTP range requests with CORS). I used R2 partly because I wanted to see how it behaved, and partly because the real use-case doesn't fit Pages. The source data already lives on R2 as Iceberg, the files are per-customer and would probably need auth (signed URLs or a session-checking Worker), and obviously 10k customer cubes on a schedule works better with object PUTs rather than git deploys


> Agreed, for a public demo like this one, GitHub Pages would work great

If you are okay with it being down regularly

Edit: Ironically, that would be the case now: https://www.githubstatus.com/incidents/hcbtzksccj2f


We do have plans. It's a question of effort, not business / philosophy.


It’s good to know it. I live in a heavily regulated workplace and our data usage is constantly monitored.

Good to know a totally offline tool is being considered.

Thanks for the great tool BTW.


That would be nice as it would spare us the effort of replicating the UI, half-baked as we can


Thank you, that's awesome to hear!


it can, but it doesn't format. You can even run the ast!


Obviously one advantage of SQL is everyone knows it. But conceptually, I agree. I think [1]Malloy is also doing some really fantastic work in this area.

This is one of the reasons I'm excited about DuckDB's upcoming [2]PEG parser. If they can pull it off, we could have alternative dialects that run on DuckDB.

[1] https://www.malloydata.dev/ [2] https://duckdb.org/2024/11/22/runtime-extensible-parsers.htm...


Definitely something we want too! (I'm the author / lead for the UI)


Great feedback! Thanks. We agree w/ the red errors. It's not helpful when it feels like your editor is screaming at you.


You should read the post! This is what the feature does.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: