AWS is a great home for DuckLabs. They just want people to use more compute and storage, so they have a degree of technology-neutrality. This is the key to allowing DuckDB to continue to grow in whatever directions are natural, rather than being warped by some kind of walled garden data platform strategy.
Fair point, but no one is clearly articulating what AWS gets out of this. They don't get the DuckDB IP, that's owned by the foundation. All the work that the team at DuckLabs is putting in growing DuckDB and DuckLake are already in principle converting into compute/storage demand in AWS. So what does AWS actually want and how will it change DuckDB to accomplish it?
What pgbouncer does is indeed core functionality. Compare Postgres to MySQL and sql server, where analogous standalone connection pools are rarely used. The fundamental reason pgbouncer needs to exist is Postgres’ utterly retrograde design. Other examples: xid wraparound, conflict with recovery, lack of undo space.
Slack is such a simple product, and is so strategic as an interface for AI, Anthropic has to be considering building their own Slack. Hopefully this tag approach is an MVP and it proves the potential of workplace messaging as an interface for AI, but also the limitations of relying on the extension points they salesforce chooses to provide, and it turns into a full fledged slack competitor from anthropic soon.
Nothing, and integration in a chat app would be better served being model agnostic. Anthropic's slack bot here is also at a disadvantage being slack only. 90% of F500 companies are on Teams, not slack. If their only goal is tech companies and startups, fine, but if they intend to grow usage beyond that it needs to be chat app agnostic.
Claude-in-Slack is a big enough feature to overcome the slack-connect network effect. Openness is absolutely key! I wrote this post because I hoped that if Anthropic is already planning to do this I might be able to influence them to make open-data part of the plan. But openness by itself isn't a big enough feature to get users.
It really sounds like you're asking for something else. More like multiple people to be able to talk to the same instance. Which that's a very different thing than Slack
Also true! The most important thing is that the NewSlacks commit to interoperability. I think Anthropic has a special opportunity to lead the way here, because they have a track record of standing by their principles to an extraordinary degree.
Why on earth would Anthropic commit to interoperability?
That is the company that doesn't interoperate with the standard LLM APIs that OpenAI developed, which everyone else in the industry has adopted and uses. Whether OpenAI's APIs are great or perfect or not, they are the standard that the industry has settled on.
You can only access public channel data, you can't even access that at scale, and Claude needs to be more natively integrated in ways that Slack will never allow.
I talked to the timescale CTO at pg conf a few years ago and asked him what timescale does differently than a standard columnar database that makes it better suited for time oriented data. He said a bunch of things and I said “but columnar databases do those things.” Then he got mad at me.
I guess it’s just another columnar dbms after all?
I'd argue we do okay, but of course it's Clickhouses own benchmark it's hard to outperform them there.
It's also not apples to apples. Clickhouse has much less transactional guarantees and isn't postgres SQL compatible. The great thing about Timescale is that you only need one DB for all your analytics and transactional needs. Combined with pgvector postgres also handles search quite well.
In a way Timescale is just postgres on steroids. Sure if you really know your use-case well, are fine with giving up some postgres nicenes, are willing to learn a new query language and are fine with using and syncing multiple data stores you'll outperform timescale. But I think it is still really cool to see how close you can get with essentially just a better postgres.
Depends on your workload? If you don't care about ACID compliance in your use-case, and query speed is all that is relevant to you probably not.
You might still be better of with Timescale/TigerData if your query pattern uses a lot of joins as we do much better there than Clickhouse does. We have our own benchmark too and perform better than Clickhouse on those kind of queries: https://rtabench.com/
But also transactions often make your life as a dev easier in my experience, and being able to use a single DB and stick with 100% postgres compatible SQL without having to change your application is often worth more than squeezing out the last few bit of query performance.
I'm just saying that single-benchmark comparisons rarely tell the full story when evaluating database technologies. ClickHouse is undoubtedly impressive engineering, and it excels in many scenarios. Ultimately the optimal choice depends on your specific use case.
I'm glad you answered as my comment was not very complete. It left out mentioning that transactions should not be expensive on reads unless you are doing something wrong. In general MVCC--which PostgreSQL uses--does not have a lot of performance overhead for this case. ClickHouse also maintains snapshots when reading, so to a certain extent they do the same work. Transactions don't seem like a very strong argument here, since you would be conceding that your implementation is inefficient.
I agree with the other points. ClickHouse is not strong on joins [yet]. It's also nice to have a single database for everything. Yet so far nobody has been able to achieve one that delivers high concurrency, fast updates, and petabyte-level scaling. Mike Stonebraker et al. called this problem out in 2007. [0] It appears they called it right and we'll continue to see 2-3 major categories of databases for the foreseeable future.
"ClickBench evaluates databases using a single table of clickstream data, representative of workloads like web analytics, BI, and log aggregation. It also favors full-table large scans and large-scale aggregations on denormalized data.
Real-time analytics inside applications is different and needs a new benchmark." [0]
This is why we published RTABench. [1]
We believe that it is more representative of real-time analytical workloads.
Do you think all time series databases (like InfluxDB for example) are useless compared to "columnar databases" that "do those things" or just Timescale?
They make a really good criticism of Iceberg: if we have a database anyway, why are we bothering to store metadata in files?
I don’t think DuckLake itself will succeed in getting adopted beyond DuckDB, but I would not be surprised if over time the catalog just absorbs the metadata, and the original Iceberg format fades into history as a transitional form.
This is exactly right. We even went so far as to build a proof of concept internally, and the technical challenges are just very different. The simplest way to explain it is that Fivetran connects a skinny pipe (APIs) to a fat pipe (databases) while Census connects a fat pipe to a skinny pipe.
