PrestaShop and inode exhaustion: causes and solutions

Introduction to PrestaShop and the inode problem

PrestaShop is one of the most popular and appreciated e-commerce platforms worldwide, chosen by thousands of online stores for its flexibility, vast ecosystem of modules and its ability to manage large product catalogs. However, this very complexity and the large amount of data it generates can lead to specific problems, including one of the most common: inode exhaustion.

It is important to note that the inode exhaustion problem is not unique to PrestaShop, but is theoretically applicable to any CMS or device that needs to store a large number of files. Any system that generates or manages millions of temporary files, images or structured data can encounter this limitation in traditional file systems. However, over the years, it has become clear that PrestaShop is particularly susceptible to this problem. The platform has earned a reputation as being extremely inode stingy, with configurations that often overload the file systems of the servers where it is hosted.

The combination of large catalogs with tens of thousands of products, dynamic generation of cache files and multiple images, and sometimes lack of optimization in the hosting used, makes PrestaShop worthy of an in-depth analysis on this topic. In this article, we will explore why PrestaShop tends to consume inodes so quickly, analyze what inodes are and their role in modern file systems such as ext4 e XFS, and we will present practical and proven solutions to prevent and manage their exhaustion.

If you run a PrestaShop store or any other platform that requires intensive file management, understanding how inodes work and adopting the right strategies is essential to ensure the continuity and stability of your e-commerce.

What is an inode and what is it used for?

Before analyzing the problem in detail, it is essential to understand what the inodes and what role they play in traditional file systems.

Un inodes (short for “index node”) is a data structure used by file systems to store information about the files and directories on a disk. Each file or directory has an associated inode, which contains metadata fundamentals, such as:

• The location of the file on disk: indicates where the file data is physically stored.
• File size: the amount of bytes occupied.
• Timestamp: when it was created, modified, or last accessed.
• Access permissions: who can read, write, or execute the file.
• Number of links (hard links): how many times the file is referenced from other directories.
• Owner and group information: who owns the file and the group they belong to.

The inodes do not contain the file data itself, but point to the areas of the disk where this data is actually stored. This system separates the metadata from the content, ensuring more efficient management.

Inodes can be explained using the analogy of a filing cabinet, a familiar image that makes the concept more intuitive. Imagine your computer as a large cabinet full of drawers. This cabinet represents the file system, which is the system that organizes and manages data and files.

Each drawer has a label that identifies it, for example “File ABC”. This label corresponds to the name of the file that we see when we navigate to a folder on the computer. However, the name of the file is only a facade. To really understand what the drawer contains, you have to look inside.

Inside the drawer we find an information card, the inode, which contains all the important information about the file. The inode is not the file itself, but a sort of “identity card” for the file. This card reports details such as the size of the file, who owns it, the creation date, access permissions and, most importantly, where the data is physically located on the disk. It is the inode that tells the system how and where to access the actual content.

At this point, imagine that the file data are the objects stored in the drawer. They are the actual documents that we want to read or modify, and the system reaches them thanks to the indications present in the inode.

To summarize with a practical example, if you search for a document called “Relation.doc”, you will first see the file name (the drawer label). When you decide to open it, the system reads the inode to discover all the useful information and, finally, accesses the actual data that represents the content of the file. Inodes, therefore, are fundamental and invisible elements for us users, but without them the computer would not be able to manage the files and information in the “big closet” of its disk.

Just as a library has a maximum number of cards it can hold in its catalog, a file system also has a maximum number of inodes available. This limit is determined when the disk is formatted. If the library catalog is full, no new cards can be added, even if there is still room on the shelves for more books. Similarly, if inodes run out, no new files or directories can be created, even if the disk still has space available.

In the context of complex systems like PrestaShop, this limitation becomes critical because the system generates a large amount of cache files, images and logs, quickly consuming the available inodes. Continuing with the analysis of the problem, we will see how to prevent and manage it to ensure the correct functioning of the server and the e-commerce platform.

Inodes in traditional file systems (now obsolete)

In classic file systems like ext4 (the most common in Linux systems) and XFS, the number of inodes is generally proportional to the size of the file system. For example, in ext4, each gigabyte of disk space can hold about 256 inodes by default. This ratio is sufficient for most standard applications, but can become a problem in scenarios where millions of small files are created, as is frequently the case with PrestaShop-based sites.

An important aspect to consider is that many hosting provider, especially those low cost that use control panels like Plesk o cPanel, they remained faithful for reasons of laziness e standardization to file system as ext4, the default on most Linux distributions, including enterprise ones like CentOS, SoulLinux, Rocky linux, Debian e Ubuntu. Although ext4 is robust and versatile, its default configuration is not always suitable for environments that require handling large amounts of files, such as large-scale e-commerce.

This choice to use ext4 in default configurations is explained by:

• Ease of setup: ext4 is widely supported, stable, and easy to setup, making it ideal for hosting providers looking to automate their installation processes as much as possible.
• Compatibility: being the default file system of many distributions, it does not require any special modifications or configurations to be implemented.
• Reduction of operating costs: standardizing setups allows hosting providers to maintain more efficient processes and avoid complexity in managing technical support.

However, this choice leads to significant limitations in contexts where inode exhaustion is a real possibility. In such cases, file systems like XFS o ZFS can offer substantial benefits.

Main features of the most used (and not) file systems

The choice of file system is a crucial aspect to ensure the efficiency and stability of a server, especially in environments like PrestaShop, where the consumption of resources such as inodes can become a critical issue. File systems differ in how they manage inodes, I/O performance and advanced features, making them more or less suitable for specific scenarios. Below we analyze the main characteristics of the two most commonly used file systems, and one (the best) almost unused by generic Hosting providers, quickly highlighting their strengths and weaknesses in relation to the management of large amounts of files and data.

ext4

• Maximum number of inodes defined when the file system is created: once the limit is reached, new inodes cannot be allocated without reformatting the disk.
• Optimized for variable size files, making it versatile for general use.
• It offers an excellent balance between performance e reliability.
• However, in scenarios with millions of small files, it can quickly run out of inodes, even if there is still space available on the disk.
• It's the file system default in many Linux distributions, including CentOS, AlmaLinux, Ubuntu and Debian, and is widely used by hosting providers who standardize their setups.

XFS

• File System scalable and high performance, particularly suitable for intensive workloads.
• - inodes are allocated dynamically, reducing the risk of premature burnout.
• Optimized to handle large amounts of data and I/O intensive operations, such as databases, log archives, or systems with high concurrent access.
• It is an excellent choice for applications that require efficient management of millions of files, such as e-commerce with large catalogs or big data environments.
• Less suitable for scenarios requiring simplified integration on consumer-ready distributions than ext4.

ZFS

• Integrated file system and volume manager: ZFS combines advanced data management features, including snapshots, native compression, and deduplication.
• The inodes come dynamically allocated, eliminating any hard limit on the maximum amount of files that can be managed.
• Offers a advanced protection against data corruption, thanks to the integration of checksums on data blocks and metadata.
• Suitable for complex systems, such as servers with scalable storage requirements, advanced backups, or environments that handle massive amounts of files.
• Transparent compression: reduces the space occupied by files, increasing data management efficiency and reducing inode consumption.
• It may be more complex to configure than ext4 or XFS, but offers fine-grained control and greater flexibility.
• Perfect for enterprise environments or advanced hosting servers which require reliability and resource optimization.

Causes of inode exhaustion in PrestaShop

PrestaShop, being a complex platform and oriented to manage extensive catalogs (even in the order of millions of products), can cause inode exhaustion for several specific reasons. Here are the main ones:

1. Dynamic cache generated by the system

PrestaShop uses a cache system to optimize performance and improve user experience by reducing page load times. This cache is dynamically generated and stored on disk as temporary files, which are a mapping of frequently requested data such as HTML, CSS, JavaScript, and database query results. In standard configurations, each request generates one or more cache files. If your store has high traffic or a non-optimized cache configuration, the number of temporary files can grow exponentially. For example, stores with thousands of visitors per day can quickly accumulate millions of files in the cache directory, each consuming one inode. This problem is amplified when automatic cleanup rules are not set, or when using disk caching mechanisms instead of in-memory ones (such as Redis or Memcached). Over time, this situation leads to excessive inode consumption, blocking the ability to create new files or update existing ones.

2. High number of product images

PrestaShop is designed to support stores with complex and diverse product catalogs, each of which can include one or more images. For each uploaded image, the system automatically generates versions in different sizes, to adapt them to the different contexts in which they will be used, such as thumbnails in the product list, medium-sized images in product cards, and enlarged images in the gallery. For example, for each original image, up to 10 or more resized versions can be created, depending on the theme and module configurations used. This means that a catalog with 10.000 products could easily generate over 100.000 image files, consuming a significant amount of inodes. Furthermore, if you use modules that offer additional features, such as advanced galleries or dynamic watermarks, the number of files can grow even further. An additional complication occurs when unnecessary images (for example, those of deleted products) are not removed, contributing to the problem.

3. Add-ons that create temporary files

PrestaShop supports a large ecosystem of third-party modules that add advanced functionality to your store, such as analytics reports, marketplace synchronizations, mass imports, or advanced customizations of standard features. Many of these modules generate temporary files to perform their tasks, such as saving intermediate data during an import, creating analytics reports, or processing custom images. In many cases, these temporary files are not automatically deleted after use, and instead accumulate over time. For example, an import module might create a file for each batch of data processed, while a theme customizer module might generate dozens of CSS or JS files each time an update is performed. If you don't follow regular management practices, these unused files can take up thousands of inodes, contributing to the problem and reducing the system's ability to handle new essential files.

4. Logs and debug files

PrestaShop, like many modern platforms, generates detailed logs to record events and track administrative activities, errors, and warnings. These logs are crucial for diagnosing problems and monitoring store activity, but if not managed properly, they can quickly add up. For example, in a high-traffic store, error and transaction log files can grow by several megabytes per day, generating thousands of new files each month. Additionally, enabling debug modes during development or troubleshooting can generate additional detailed log files, which consume significant inodes. In environments where automatic log rotation (e.g. via logrotate) is not configured, obsolete files remain on the system, accumulating over time. In some cases, third-party modules may add additional specific logs, further increasing the load on the inodes. This can lead to inefficient space usage and operational problems if timely cleanup strategies are not implemented.

5. Multiple sites on the same server

In many cases, hosting providers or system administrators set up multiple websites on the same server to optimize resource usage and reduce costs. This practice, while common and often convenient, can become a significant issue when it comes to inodes, especially if the hosted sites are file-intensive, as is the case with platforms like PrestaShop, WordPress, or Magento.

Each site uses its own amount of inodes to manage cache files, images, logs, additional modules, and temporary files. When multiple sites share the same server, the total inode consumption adds up, bringing the system closer to the available limit more quickly. For example, if a server is configured to host three PrestaShop sites, each with large catalogs and poorly optimized caching configurations, the generated files can easily exhaust the inodes, even if there is still physical space on the disk.

How to Identify and Fix an Out of Inodes Problem

If you suspect that your server is running out of inodes, follow these steps to confirm and resolve the issue:

1. Check inode consumption

If you have SSH access you can check the number of inodes used with the command: df -i

This command shows the total number of inodes, used inodes, and available inodes for each file system.

2. Identify problem directories

To find the directories that consume the most inodes, use this command which, although very slow, is suitable for the purpose:

find /path/to/prestashop -xdev -printf '%h\n' | sort | uniq -c | sort -nr | head -n 10

is used to locate the directories that contain the greatest number of files within the specified path (in this case /path/to/prestashop). Here's what it does specifically:

1. find /path/to/prestashop -xdev -printf '%h\n': Recursively searches all directories in the given path, without traversing separately mounted file systems (-xdev), and prints only the paths of directories that contain files (%h).
2. sort: Sort the output of directories alphabetically.
3. uniq -c: Counts how many times each directory appears (equivalent to the number of files in that directory).
4. sort -nr: Sorts the output numerically in descending order, placing directories with the most files at the top.
5. head -n 10: Show only the top 10 directories with the highest number of files.

In practice, this command is useful for diagnosing excessive inode consumption problems by quickly identifying the directories containing the most files.

3. Delete unnecessary files

Identifying and deleting obsolete files, such as cache, unused images, or old logs, is a crucial step to reclaim inodes and improve server efficiency. However, this is a very delicate operation and requires a thorough understanding of PrestaShop's structure and dependencies. It is essential to know which files are safe to remove and which ones could compromise the proper functioning of your store. For example, deleting useless cache files is usually safe, as the system automatically regenerates them, but incorrectly removing images can cause visual errors or break the layout of your site. Similarly, deleting logs without a strategy could deprive you of crucial information to diagnose future problems.

To manage this operation effectively:

• Make a full backup before deleting any files, to ensure that the data can be recovered in case of errors.
• Use tools and commands to analyze inode usage, such as du e find, to identify particularly heavy or problematic directories.
• Remove files from cache directory (/var/cache/) for example with the command rm -rf /path/to/prestashop/var/cache/* only after making sure they are not in use by any running processes.
• Verify images through scripts that compare references in the database with files in the upload directory. Remove only those not associated with active products.
• Set up automatic log rotation and cleanup to prevent excessive log buildup over time.

The approach must be meticulous: errors in deleting files can lead to malfunctions, data loss or even interruption of sales. If you are not fully familiar with the system, it is advisable to rely on an experienced administrator or automated tools for cleaning PrestaShop.

How to Prevent Inode Exhaustion in PrestaShop

Addressing the inode problem requires a combination of good system management practices and PrestaShop-specific optimizations. Here are some effective strategies:

1. Cache optimization

• Using a memory-based cache: Instead of writing cache files to disk, it is recommended to use in-memory caching systems such as Redis or Memcached. These tools not only reduce inode consumption but also improve site performance.
• Configure automatic cache cleaning: PrestaShop allows you to set periodic cache cleaning via back-office settings or cron scripts.

2. Reducing product images

• Using a CDN service: A Content Delivery Network can host and serve images, reducing the need to maintain them on the main server.
• Delete obsolete images: Periodically, it is useful to review and remove images associated with deactivated or no longer used products.
• Compress and optimize images: Plugins like “ImageMagick” can reduce the amount of space and inodes used by images.

3. Temporary File Monitoring

• Temporary File Management: Set up regular cleaning scripts to delete temporary files generated by PrestaShop or its modules.
• Evaluate third-party modules: Use only reliable and well-maintained modules, which efficiently manage temporary files.

4. Use a more suitable file system

The choice of file system is a crucial aspect and absolutely decisive to prevent inode exhaustion, especially in file-intensive environments such as PrestaShop stores. If the previous three points can be considered best practices or half measures, a correct and adequate file system can be considered the definitive solution to the problem. In fact, adopting a file system designed to handle large amounts of files or configuring the default one appropriately can make a big difference. Below, some options and strategies:

Switch to XFS

• XFS It is a scalable and high-performance file system, especially suitable for scenarios with millions of files.
• One of its key features is dynamic inode allocation, which eliminates the need to specify a fixed number of inodes during formatting. This makes it ideal for applications that grow rapidly or require intensive file management.
• Additionally, XFS is optimized for intensive workloads such as databases, log storage, or e-commerce applications with large catalogs. It offers excellent performance by efficiently handling I/O operations.
• Adopting XFS in a PrestaShop server helps prevent inode exhaustion issues without sacrificing overall performance.

Switching to ZFS or OpenZFS

• For those looking for the ne plus ultra in inode management and advanced features, OpenZFS represents an excellent choice.
• OpenZFS does not have a fixed limit on the number of inodes: they are dynamically allocated as needed, completely eliminating the risk of premature exhaustion.
• In addition to inode flexibility, OpenZFS offers advanced security and data integrity features, such as corruption protection thanks to integrated checksums.
• The ability to create instant snapshots and rollback is especially useful in development or production environments, where it is essential to maintain previous versions of data for backup or testing purposes.
• OpenZFS transparent compression reduces disk space consumption, further optimizing server resources.
• While it requires more experience for initial setup than file systems like ext4 or XFS, OpenZFS provides superior stability and advanced features that make it an excellent choice for professional hosting environments.

In the following two images you can see the INode situation of the exact same PrestaShop site, the first one on a classic server with EXT4 filesystem and the second one on ZFS in the OpenZFS variant for Linux.

What is clearly evident when analyzing the /home partition is that in the EXT4 version 14 million inodes occupy 47% of the available inodes, while in the ZFS version, 13 million inodes occupy just 2%. In reality, ZFS has no limit on inodes and therefore that 2% would be the same with 13 million inodes as with 1 billion inodes.

5. Automate log management

Set up a log rotation system, such as logrotate, is a fundamental practice for effectively managing the amount of disk space and the number of inodes used by log files. logrotate allows you to archive older logs, automatically compressing or deleting them, while keeping only a predefined number of recent files. This approach is particularly useful on servers that handle large volumes of traffic, where files such as error_log e access_log web server logs can quickly grow in size and number. While it may seem like a simple stopgap for system log management, its implementation can be extended to include log files generated directly by PrestaShop. The platform generates logs to track administrative activities, errors and warnings, which can quickly accumulate, especially in stores with a high number of visitors or complex configurations. In emergency situations, where inode consumption is already critical, automating the rotation of these logs can provide an immediate solution to avoid a complete system crash. For example, a custom configuration of logrotate could be set to locate PrestaShop log files in specific directories, compressed, and kept for only a limited number of days. This approach not only frees up inodes, but ensures that the most recent logs are always accessible without overloading the system. While not a permanent solution in environments with high inode consumption, automating log management is an effective temporary measure that can prevent more serious problems and buy time to implement more structured solutions, such as reviewing the modules that generate logs or switching to a more suitable file system.