<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Iceberg on vrajat</title><link>https://vrajat.com/tags/iceberg/</link><description>Recent content in Iceberg on vrajat</description><generator>Hugo</generator><language>en</language><lastBuildDate>Tue, 17 Nov 2026 10:00:00 +0530</lastBuildDate><atom:link href="https://vrajat.com/tags/iceberg/index.xml" rel="self" type="application/rss+xml"/><item><title>DuckDB OOM Kills from Untracked Memory</title><link>https://vrajat.com/posts/duckdb-oom-untracked-memory/</link><pubDate>Tue, 17 Nov 2026 10:00:00 +0530</pubDate><guid>https://vrajat.com/posts/duckdb-oom-untracked-memory/</guid><description>&lt;blockquote>
&lt;p>DuckDB&amp;rsquo;s &lt;code>memory_limit&lt;/code> only covers its buffer manager. The engine as well as extensions may allocate memory
outside the buffer manager. We traced an untracked memory allocation in duckdb-avro that caused OOM kills on our
workload. Two factors led to the OOM kills: 1) duckdb-avro reads whole Avro files into memory, and 2) jemalloc
does not return freed pages to the system immediately. Removing either factor prevented the OOM kills.&lt;/p>&lt;/blockquote>
&lt;p>We run a Java service that loads Avro files into Iceberg tables with an in-process DuckDB. Each run opens one
DuckDB instance and runs many table loads on it in parallel.&lt;/p></description></item></channel></rss>