Data science
fromTheregister
1 week agoDuckDB uses RDBMS to tackle lakehouse 'small changes' issue
DuckDB's DuckLake addresses the inefficiencies of small changes in lakehouse implementations using RDBMS for metadata management.
The savings disappear the moment you hit real-world complexity. Disparate data sources and messy inputs, ambiguous situations without clear rule sets, or actually any domain where the rules aren't already obvious. And someone still has to write all those rules.
There's just so much to do. So, the advances that we've gotten over the last five to ten years have been spectacular. We love the tools. We use them every day. But the question is, is this the whole universe of things that needs to happen? And we thought about it very carefully and our answer was no, there's a lot more to do.