When a Decompression Bomb Meets ZFS: Over 3.700 TB Written to a 15 TB Disk

In our daily work, we often face problems of performance, security, scalability. But every now and then we also come across some decidedly curious cases. Today we want to share with you one of these particular situations, which we encountered on one of the systems managed on behalf of one of our customers.

It all started in a seemingly innocuous way: a backup that lasted too long. Nothing unusual on the surface, except that it was an incremental backup, an operation that usually takes a few minutes. In this case, however, the job had been running for hours, all night. Suspicious behavior.

We then began analyzing the system, discovering a directory populated by hundreds of files from 86 terabyte each. All created on the same day, and all contained in a filesystem hosted on a disk by 15 terabyte.

It may seem absurd to you. We too initially thought it was a display error, some glitch in the system commands. Yet it was all real: those files existed, they were accessible, readable and were processed by the system. But how was it possible?

A 3D model marketplace and a compromised upload

The system in question hosts a marketplace portal where users can upload and sell 3D models in various formats. The files are uploaded, analyzed, and then indexed and put up for sale.

Someone, evidently with malicious intent (or simply to test the limits of the system), thought to upload a file .rar passing it off as a 3D model. The file, once unzipped, generated a huge archive, theoretically 86 terabytes, composed entirely of stares.

This type of file is called decompression bomb or also ZIP BOMB: a very small compressed file (sometimes a few MB), which during extraction expands enormously, saturating resources such as disk, RAM, CPU. In unprotected scenarios, it can bring down entire servers.

How does a small archive generate 86 TB?

At the heart of Decompression Bomb and ZIP Bomb, the answer lies in the nature of the data and the compression algorithms. Files containing only zeros, or repeating patterns, are extremely compressible.. Algorithms like Huffman coding (and derivatives) work by assigning shorter codes to the most frequent patterns. If a file is 100% composed of a single value (e.g. 0), compression can be extremely effective.

To better understand this phenomenon, imagine having to describe in words a long row of 1.000.000 zeros. Instead of writing every single zero, it would be enough to say “write zeros a million times”. This is the principle on which compression algorithms are based: encoding repetitive sequences with compact information.

In this specific case, the file .rar uploaded contained a single file that, in its uncompressed form, must have taken up 86 TB, but its contents consisted only of zero bytes. When compression is applied, the result is a tiny file, even less than 10 MB, because all sequences are encoded extremely compactly.

This type of content is not only highly compressible, but also represents a real danger: it is extremely easy to generate seemingly harmless files that can explode into enormous space consumption once extracted. The effect is all the more devastating the more the system that extracts them lacks controls, quotas, or protections on CPU, RAM or disk space.

Another interesting aspect is that, since the compression algorithm cannot predict the content of the file being decompressed, it must simply follow the instructions: “write zero for 86.000.000.000.000 bytes”. No checks are applied automatically, unless explicitly provided by the system or application. Hence the potential risk: a file of a few MB can become a bomb of hundreds of terabytes.

In an unprepared environment, this easily leads to storage saturation, crashes of active services, and in the worst cases, data corruption and loss of operation.

But in our case, none of this happened. Why?

The Secret Weapon: OpenZFS and LZ4 Compression

The system under attack uses OpenZFS, an advanced file system that natively supports features such as:

• snapshots and rollbacks

• data integrity check

• deduplication

• and, especially in this case, transparent compression

OpenZFS is not just a file system, but a real advanced storage management tool. Among its most powerful features are the ability to enable compression at the dataset level, in a way that is completely transparent to the application and the end user. This means that data is compressed when written and decompressed when read, without manual intervention or application-side software changes.

What is LZ4?

LZ4 is a compression algorithm lossless (lossless) designed to be extremely fast, with a good compromise between compression ratio and speed. Compared to other algorithms such as gzip, LZ4 offers:

• much higher compression and decompression speeds
• extremely low CPU consumption
• Support for binary and structured data

It is particularly effective on redundant or repetitive data, such as our files made up of zeros. Under these conditions, LZ4 is able to compress theoretically huge files (tens of terabytes) into just a few kilobytes.

ZFS dynamically decides whether and how much to compress, based on the achievable gain. If a block of data is not profitable to compress, it is saved in the clear, avoiding unnecessary overhead.

In the specific case of this decompression bomb, the file system interpreted the decompressed files (composed of zeros) and immediately recompressed them with LZ4, going as far as to physically write them to disk occupying only an infinitesimal fraction of logical space.

Why didn't we notice it right away?

Thanks to active compression, the storage did not fill up the available space. The file system correctly showed free space, services were running, CPU load was minimal. Everything seemed normal.

This is precisely the strength – and at the same time the “pitfall” – of ZFS’s transparent compression: on the one hand, it protects us from disastrous scenarios such as decompression bombs, but on the other, it makes it difficult to immediately notice their presence, because everything continues to function apparently without visible anomalies.

Only the analysis of an anomalous scheduling in the backups made us discover the presence of the files. In fact, one of the jobs had been running for many hours, even though it was an incremental backup. It was this detail that made us suspicious and started the investigation.

Using the classic diagnostic commands we immediately noticed an unusual behavior: ls -lh showed huge files of tens of terabytes, while du -sh reported negligible consumption, less than a few megabytes. A clear sign that the files existed logically, but were virtually null at the physical level thanks to the intervention of the file system.

In other words, without a targeted analysis at the ZFS level (or indirect observation of anomalous processes such as slowed backups), there would have been no obvious alert or alarm. This type of attack, if not managed by an advanced file system, would have saturated the storage in a few seconds. With ZFS, however, we found ourselves facing a silent anomaly, which we were able to study calmly, without having to take emergency action.

Compression in ZFS: Practical Benefits

If you haven't considered compression in your ZFS systems yet, here are some concrete reasons to do so. Compression is often thought of only in terms of saving space, but in reality the benefits are much broader and more strategic:

• Reduction of occupied space: Compression allows you to fit significantly more data into the same physical space. This is especially useful in environments where storage is expensive, such as NVMe SSDs or enterprise-grade disks.

• Performance improvement: By reducing the amount of data physically written and read to disk, access times and the load on I/O operations are reduced. In many scenarios, reading less compressed data and decompressing it into RAM is faster than reading uncompressed data directly from disk.

• Positive effect on backups and snapshots: Compressed backups take up less space and are faster to transfer, especially when combined with deduplication mechanisms or ZFS snapshots. Additionally, compressed snapshots can be replicated between hosts more efficiently.

• Transparent functionality: ZFS compression is completely transparent to the software in use. It does not require changes to the applications, does not affect file compatibility, and does not alter the behavior of the read and write processes.

• Mitigating attacks like this: As we saw in the case of the decompression bomb, ZFS compression neutralized a potential disaster by avoiding storage saturation. This type of resilience, enabled simply with an active option, represents a tangible security advantage.

Enabling LZ4 compression on a dataset in ZFS is simple:

zfs set compression=lz4 tank/dataset

You can also check its effectiveness:

A value greater than 1.0 indicates that compression has taken effect.

What we did after the discovery

Once we understood the source of the problem, we blocked the upload of potentially dangerous compressed files by enabling:

• more restrictive controls on the type and structure of uploaded files

• a maximum limit on the expandable size of archives

Additionally, we have suggested to the application development team some best practices to implement on the software side to increase resilience against similar attacks. These include:

• perform a preventive analysis of the internal structure of compressed archives, even in sandbox mode

• calculate, where possible, an estimate of the final size of the files once extracted, to identify anomalous cases before they impact storage

• apply timeouts and memory limits to decompression operations, so as to block any abuse even during extraction

We then deleted the files we created, verified that there were no public references or links to those contents, and returned the system to normal operating conditions.

Lesson learned: File systems make (a lot) of difference

This experience reminded us once again how strategic the choice of file system is in managing an infrastructure. Too often we tend to underestimate this aspect, believing that "one is as good as another", or that the default file system of the Linux distribution is sufficient for every scenario.

Had the system been based on ext4 or XFS, the attack would likely have been successful, saturating the storage within moments and blocking essential services. Without transparent compression, the 86 terabyte files would have quickly consumed all available space, causing cascading failures, downtime, and potentially data loss.

With ZFS, however, thanks to its modern architecture, we contained the impact, identified the problem calmly, and resolved it without downtime. The file system acted intelligently and proactively, automatically compressing highly redundant content and preventing storage saturation.

This story demonstrates how investing in reliable technologies like OpenZFS means not only improving performance, but also providing structural protection against anomalous behavior or unconventional attacks. A file system is not just a technical detail: it can make the difference between a crash and a simple log of a curious event.

OpenZFS: An Overview

For those who do not know him, OpenZFS is the open source implementation of the ZFS file system, originally developed by Sun Microsystems in the early 2000s for Solaris. After the acquisition by Oracle, the community developed an independent fork, which is actively maintained and evolving today.

OpenZFS is one of the most advanced file systems available today for Linux, BSD and derivative environments. It is much more than a traditional file system: it is a complete storage management platform, designed with a focus on security, data integrity and resilience.

Among its key features we find:

• Copy-on-write (COW) architecture: Every change to data generates a new copy, enabling instant snapshots and lossless rollbacks.

• Data integrity via checksum: Each block written is accompanied by a checksum. If data becomes corrupted, ZFS can detect it and – if configured correctly – fix it using redundancy.

• Efficient snapshots and replication: You can take instantly readable snapshots and replicate them incrementally to remote systems.

• Advanced storage managementRAID-Z, vdev, pools, datasets, space limits, deduplication, compression, and multi-level caching are just some of the options available.

ZFS is not “out of the box” on all distributions, but with a minimal learning curve it offers robustness and flexibility superior to any traditional file system. In mission-critical or data-intensive contexts, the difference is truly felt.

Conclusion

Thanks to ZFS's LZ4 compression, we were able to write over 3.700 terabyte on a disk from 15 TB without realizing it. A paradox that is only possible thanks to the advanced features of this file system.

This was not a bug, but a creative and malicious attack that could have compromised an entire server. The combination of constant monitoring, diagnostic tools, and a modern file system allowed us to isolate, understand, and resolve the issue quickly.

But most importantly, this episode reinforced our belief that infrastructure should be designed not only for optimal conditions, but also – and most importantly – to handle the unexpected. In this case, there was no need for emergency manual intervention, no failover, no recovery. The file system just did its job.

ZFS has proven itself to be up to the challenge, silently yet effectively protecting the system. Its ability to react to abnormal situations without compromising data or performance makes it an essential tool for those seeking reliability and resilience.

The bottom line? If you still think ext4 or XFS are “more than enough,” we encourage you to take a closer look at ZFS. You may find that your next ally in performance, integrity, and security is already there.

Good work and… watch out for the decompression bombs!