https://metastax.com AI-curated links for data engineering, ML/AI, and analytics practitioners. Updated every 6 hours. en-us Fri, 08 May 2026 07:27:17 GMT https://duckdb.org/2026/05/08/announcing-duckcon7.html metastax-5792 Fri, 08 May 2026 00:00:00 GMT The program for DuckCon #7 Amsterdam, a DuckDB user conference, has been announced. The event will be held on June 24, 2026, and will run from 15:00 to 20:00 CEST. DuckDB Blog duckdb analytics https://www.dremio.com/blog/what-are-table-formats-and-why-were-they-needed/ metastax-5759 Thu, 07 May 2026 16:00:00 GMT This is Part 1 of a 15-part Apache Iceberg Masterclass. This article covers the fundamental question: what problem do table formats solve, and why does the choice between them matter? A data lake without a table format is a collection of files. It has no concept of a transaction, no mechanism to pre Dremio Blog iceberg lakehouse https://blog.bytebytego.com/p/container-design-patterns-for-distributed metastax-5735 Thu, 07 May 2026 15:31:08 GMT This article presents container design patterns categorized by their coordination scope, providing a structured overview of common practices for distributed systems. ByteByteGo architecture engineering https://sderosiaux.substack.com/p/from-prompt-engineering-to-data-engineering metastax-5748 Thu, 07 May 2026 15:29:26 GMT The article posits that many agent reliability issues stem from underlying data engineering problems. Hacker News - Quality & Governance data-quality governance https://www.reddit.com/r/apachekafka/comments/1t683y2/using_clickhouse_as_a_kafka_sink_async_inserts/ metastax-5723 Thu, 07 May 2026 11:46:51 GMT The post discusses using ClickHouse as a Kafka sink, focusing on how async insert mode helps with high message rates but has buffering and dedupe behaviors that aren't always obvious. r/apachekafka kafka streaming https://openai.com/index/parloa metastax-5716 Thu, 07 May 2026 11:00:00 GMT Parloa leverages OpenAI models to power scalable, voice-driven AI customer service agents, enabling enterprises to design, simulate, and deploy reliable, real-time interactions. OpenAI Blog llm agents https://anyscale.com/blog/ai-agents-on-ray-serve-single-to-multi-agent-architecture metastax-5787 Thu, 07 May 2026 10:00:00 GMT This Anyscale blog post discusses the architecture of AI agents on Ray Serve, covering both single-agent and multi-agent architectures. It explores how to build and deploy AI agents using Ray. Anyscale Blog agents mlops architecture https://neon.com/blog/turning-off-fpw-for-faster-writes metastax-5734 Thu, 07 May 2026 10:00:00 GMT Neon decoupled storage and compute to deliver up to a 5x performance increase on write-heavy workloads by disabling full-page writes. Neon Blog postgres engineering architecture https://duckdb.org/2026/05/07/delta-uc-updates.html metastax-5696 Thu, 07 May 2026 00:00:00 GMT DuckDB's Delta Lake and Unity Catalog extensions are no longer experimental. The post details the progress of these extensions. DuckDB Blog duckdb delta lakehouse https://www.dremio.com/blog/dremio-iceberg-v3-default-column-values/ metastax-5672 Wed, 06 May 2026 21:13:17 GMT Adding a column to a large production table used to require a plan involving migration scripts, maintenance windows, and backfill jobs that rewrite every data file to include the new column. Iceberg default column values eliminate the need for backfills during schema evolution. Dremio Blog lakehouse iceberg https://huggingface.co/blog/ServiceNow-AI/correctness-before-corrections metastax-5638 Wed, 06 May 2026 19:06:55 GMT The post details improvements to vLLM, focusing on correctness before corrections in reinforcement learning. Hugging Face Blog ml llm mlops https://blog.cloudflare.com/de-tld-outage-dnssec/ metastax-5646 Wed, 06 May 2026 17:00:00 GMT On May 5, 2026, DENIC published broken DNSSEC signatures for the .de TLD, making millions of domains unreachable. Here's what 1.1.1.1 saw, how serve stale cushioned the impact, and how we restored resolution. Cloudflare Blog engineering architecture https://clickhouse.com/blog/odigos-full-fidelity-tracing metastax-5755 Wed, 06 May 2026 15:09:11 GMT How ClickStack and Odigos eliminate observability gaps with zero-code eBPF instrumentation and full-fidelity distributed tracing at scale. ClickHouse Blog clickhouse observability https://www.reddit.com/r/DuckDB/comments/1t58cbz/ducklake_in_action_manage_lakehouses_run_sql/ metastax-5623 Wed, 06 May 2026 09:59:27 GMT This post links to a video demonstrating the DuckLake workflow using Rosetta DBT Studio. The presentation covers creating and importing lakehouse instances, exploring metadata, running SQL queries, and building reusable SQL Notebooks. r/duckdb duckdb lakehouse dbt https://clickhouse.com/blog/write-side-cost-performance-snowflake-clickhouse metastax-5754 Wed, 06 May 2026 06:46:34 GMT Agentic analytics makes query-readiness a write-side cost problem. This post compares Snowflake and ClickHouse under continuous ingest, showing how ClickHouse obtains query-ready data at 22× lower cost and delivers 31× better write-side cost-performance. ClickHouse Blog clickhouse snowflake analytics https://openai.com/index/singular-bank metastax-5670 Wed, 06 May 2026 00:00:00 GMT Singular Bank built Singularity, an internal assistant using ChatGPT and Codex to help bankers save 60–90 minutes daily on meeting prep, portfolio analysis, and follow-up. OpenAI Blog llm agents https://openai.com/index/uber metastax-5671 Wed, 06 May 2026 00:00:00 GMT Uber uses OpenAI to power AI assistants and voice features that help drivers earn smarter and riders book faster across a global real-time marketplace. OpenAI Blog llm agents https://simonwillison.net/2026/May/5/our-ai-started-a-cafe-in-stockholm/#atom-everything metastax-5575 Tue, 05 May 2026 22:14:21 GMT Simon Willison describes how he used AI agents to launch and run a cafe in Stockholm, detailing the architecture and lessons learned. Simon Willison llm agents https://www.confluent.io/blog/ai-kafka-integration-patterns/ metastax-5569 Tue, 05 May 2026 19:40:44 GMT The article compares three patterns for integrating AI inference with Apache Kafka: external RPC, embedded, and sidecar, focusing on avoiding consumer rebalances, cutting costs, and scaling LLM pipelines. Confluent Blog kafka streaming https://www.reddit.com/r/apachekafka/comments/1t4k1ix/jikkou_10_is_out_iceberg_multicluster/ metastax-5563 Tue, 05 May 2026 16:12:13 GMT Jikkou 1.0 is out, featuring Apache Iceberg integration for declarative management of namespaces, tables, and views. It also includes multi-cluster orchestration and Confluent Cloud RBAC. r/apachekafka kafka streaming iceberg https://startree.ai/resources/iceberg-query-benchmark-vs-trino-vs-clickhouse/ metastax-5522 Tue, 05 May 2026 16:07:47 GMT StarTree claims a 5-37x performance improvement for queries on Iceberg tables compared to Trino and ClickHouse, based on their benchmark. Hacker News - Data iceberg trino clickhouse lakehouse benchmarking https://towardsdatascience.com/rag-hallucinates-i-built-a-self-healing-layer-that-fixes-it-in-real-time/ metastax-5532 Tue, 05 May 2026 13:30:00 GMT This article presents a self-healing layer designed to detect and correct hallucinations in RAG systems before they reach users by addressing issues in reasoning. Towards Data Science llm ml https://clickhouse.com/blog/clickhousectl-compare-versions metastax-5758 Tue, 05 May 2026 10:29:49 GMT We use clickhousectl to spin up multiple ClickHouse versions side by side and benchmark two recent performance improvements. ClickHouse Blog clickhouse analytics engineering https://openai.com/index/mrc-supercomputer-networking metastax-5603 Tue, 05 May 2026 10:00:00 GMT OpenAI introduces MRC (Multipath Reliable Connection), a new supercomputer networking protocol released via OCP to improve resilience and performance in large-scale AI training clusters. OpenAI Blog llm agents https://www.redpanda.com/blog/littles-law-in-practice-with-cloud-topics metastax-5644 Tue, 05 May 2026 00:00:00 GMT From spinning disks to CPUs to cloud object storage, shifting bottlenecks have shaped Redpanda's architecture. Here’s what Cloud Topics revealed about today’s demand for high-latency storage. Redpanda Blog streaming kafka https://openai.com/index/openai-pwc-finance-collaboration metastax-5491 Mon, 04 May 2026 21:00:00 GMT OpenAI and PwC are collaborating to help businesses automate finance workflows using AI agents, with the aim of improving forecasting, strengthening controls, and modernizing the CFO function. OpenAI Blog llm agents https://www.figma.com/blog/pgkeeper-building-the-bouncer-we-needed-for-postgres/ metastax-5446 Mon, 04 May 2026 16:49:41 GMT Figma details the architecture and implementation of PGKeeper, their custom Postgres connection pooler, explaining why existing solutions like PgBouncer didn't meet their needs. Hacker News - Data postgres engineering architecture https://clickhouse.com/blog/qonto metastax-5757 Mon, 04 May 2026 12:35:21 GMT How Qonto uses ClickHouse Cloud to power observability at scale — replacing sampling and hour-capped queries with two-week query windows, 99.84% compression on high-cardinality data, and an AI incident companion built on the ClickHouse MCP server. ClickHouse Blog clickhouse analytics observability https://duckdb.org/2026/05/04/ducklake-dataframe.html metastax-5434 Mon, 04 May 2026 00:00:00 GMT This blog post discusses the simplicity of the DuckLake specification for dataframes. DuckDB Blog duckdb analytics https://openai.com/index/delivering-low-latency-voice-ai-at-scale metastax-5460 Mon, 04 May 2026 00:00:00 GMT OpenAI describes how it rebuilt its WebRTC stack to power real-time Voice AI with low latency, global scale, and seamless conversational turn-taking. OpenAI Blog llm agents architecture https://github.com/jordepic/Icestream metastax-5387 Sun, 03 May 2026 20:50:09 GMT Icestream is a project enabling efficient streaming writes in Apache Iceberg. Hacker News - Data iceberg streaming https://www.reddit.com/r/DuckDB/comments/1t2t17p/im_working_on_a_conversational_analytics_agent/ metastax-5385 Sun, 03 May 2026 18:16:46 GMT A developer is building a no-code agent builder that uses DuckDB to create conversational analytics agents. These agents can respond to queries with interactive charts and UI, allowing users to query databases more easily. r/duckdb duckdb agents analytics https://eugeneyan.com//writing/working-with-ai/ metastax-5490 Sun, 03 May 2026 00:00:00 GMT This post proposes a framework for leveraging AI, emphasizing context as infrastructure, taste as configuration, verification for autonomy, scaling through delegation, and closing feedback loops for continuous improvement. Eugene Yan ml llm https://www.reddit.com/r/DuckDB/comments/1t1pctx/extension_ducksmiles_community_extension_for/ metastax-5332 Sat, 02 May 2026 13:04:35 GMT A user shares a community extension for DuckDB called `ducksmiles` that enables processing chemistry data, including SMILES, InChI, and PDB formats, directly within DuckDB. The extension supports functions like `mol_formula` and `mol_weight` and integrates with `read_csv_auto`, `read_text`, and `httpfs` for data ingestion. r/duckdb duckdb engineering https://blog.cloudflare.com/code-orange-fail-small-complete/ metastax-5311 Fri, 01 May 2026 21:07:30 GMT Cloudflare completed an engineering effort to make its infrastructure more resilient using tools like Snapstone and the Engineering Codex. They implemented safer configuration changes and automated best practices to prevent future incidents. Cloudflare Blog engineering architecture https://www.fivetran.com/blog/how-we-accelerated-transpilation-by-compiling-sqlglot-with-mypyc metastax-5272 Fri, 01 May 2026 13:00:02 GMT Fivetran accelerated the transpilation of SQL dialects in SQLGlot by compiling it with mypyc, resulting in faster translation between different SQL dialects for query engines. Fivetran Blog data-engineering etl sql https://arxiv.org/abs/2604.28028 metastax-5261 Fri, 01 May 2026 04:00:00 GMT This arXiv paper introduces a method to improve Text-to-SQL accuracy by using template-constrained decoding, particularly for recurring questions. It addresses challenges in real-world deployment of Text-to-SQL models, especially in complex or unseen schemas. arXiv Databases semantic-layer llm https://www.reddit.com/r/DuckDB/comments/1t03slk/duckdb_infers_nullonly_columns_as_json_type_heres/ metastax-5269 Thu, 30 Apr 2026 17:57:00 GMT DuckDB infers NULL-only columns as the generic JSON type, causing staging issues when real values appear later. The solution involves using a synthetic canonical sample to ensure correct type inference from the outset. r/duckdb duckdb data-quality https://towardsdatascience.com/why-ai-engineers-are-moving-beyond-langchain-to-native-agent-architectures/ metastax-5245 Thu, 30 Apr 2026 12:00:00 GMT This post discusses the shift of AI engineers from using frameworks like LangChain to building native agent architectures for production LLM applications. Towards Data Science agents llm mlops https://openai.com/index/where-the-goblins-came-from metastax-5228 Wed, 29 Apr 2026 20:00:00 GMT The post discusses the timeline, root cause, and fixes behind "goblin outputs," which are personality-driven quirks in GPT-5 behavior. OpenAI Blog llm agents https://github.com/Icemap/duckdb-vector-index metastax-5115 Wed, 29 Apr 2026 19:52:04 GMT This post links to a DuckDB extension for vector search indexes with pluggable quantization. Hacker News - Data duckdb vector-db https://stripe.com/blog/giving-agents-the-ability-to-pay metastax-5138 Wed, 29 Apr 2026 00:00:00 GMT Stripe introduces Link’s wallet for agents, offering programmatic access to generate one-time-use cards or Shared Payment Tokens, built on Stripe’s new Issuing for agents. Stripe Engineering engineering architecture https://www.reddit.com/r/apachekafka/comments/1syg4qb/i_turned_recurring_kafka_production_failures_into/ metastax-5170 Tue, 28 Apr 2026 22:17:35 GMT A user shares a practical troubleshooting guide derived from recurring Kafka production failures, covering issues like consumer lag, producers writing without consumers reading, and duplicate processing after restart due to offset commit problems. r/apachekafka kafka streaming https://blog.bytebytego.com/p/how-stripe-detects-fraudulent-transactions metastax-5085 Tue, 28 Apr 2026 03:03:50 GMT This article explores Stripe's Radar system for detecting fraudulent transactions within 100 ms, detailing the architectural decisions behind its effectiveness. ByteByteGo architecture engineering https://www.dremio.com/blog/dremio-iceberg-v3-deletion-vectors/ metastax-5033 Mon, 27 Apr 2026 13:08:39 GMT The post discusses how Iceberg deletion vectors offer a more efficient way to handle row deletions in data lakehouses, where deleting rows can be an expensive operation due to the immutable nature of Parquet files. Dremio Blog lakehouse iceberg https://openai.com/index/choco metastax-5032 Mon, 27 Apr 2026 00:00:00 GMT Choco used OpenAI APIs to streamline food distribution, boost productivity, and unlock growth, providing a customer story on the real-world impact of AI. OpenAI Blog llm agents https://www.dremio.com/blog/the-journey-from-scattered-data-to-an-apache-iceberg-lakehouse-with-governed-agentic-analytics/ metastax-4967 Sun, 26 Apr 2026 08:11:40 GMT The article outlines a strategy for modernizing data platforms by migrating to an Apache Iceberg lakehouse. It suggests avoiding long ETL pipeline builds and focusing on faster time-to-value for analysts. Dremio Blog lakehouse iceberg https://github.com/pgcentralfoundation/pgrx metastax-4939 Sat, 25 Apr 2026 08:16:19 GMT Pgrx is a framework for building PostgreSQL extensions using Rust, enabling developers to leverage Rust's safety and performance features within the Postgres environment. Hacker News - Data postgres engineering https://clickhouse.com/blog/pg_clickhouse-whats-new-april-2026 metastax-4997 Fri, 24 Apr 2026 17:05:36 GMT Recent pg_clickhouse releases introduce JSONB, date/time, and array function pushdown, plus HTTP result set streaming for lower memory usage. ClickHouse Blog clickhouse postgres https://github.com/tobilg/duckdb-whisper metastax-4888 Fri, 24 Apr 2026 14:55:22 GMT The article discusses using whisper.cpp within DuckDB to translate speech to text. Hacker News - Data duckdb ml