Table of contents of the article:
In the world of web hosting and infrastructure management, AWS (Amazon Web Services) is often considered the benchmark for scalability and reliability. However, like any powerful tool, AWS's success depends on its proper configuration and management. This article analyzes a real-world case study of an astronomy website, passioneastronomia.it, which experienced a 4-hour downtime despite being hosted on AWS . We'll explore the causes of the problem, the use of CloudFront and CloudFlare, and the importance of Full Page Cache for improving performance and reducing resource consumption.
The Case: passionastronomia.it
A few months ago, a valued client of ours with about 50 million monthly page views reported an emerging astronomy site that was periodically experiencing downtime issues due to excessive traffic.
He, who followed the Passione Astronomia page, was informed by Passione Astronomia's Facebook posts. Whenever a post caused a traffic overload and the site went offline, they would announce it, boasting that the server couldn't handle the high traffic . However, for the owner of a site accustomed to handling 50 million page views a month, with peaks of over 3 million a day, it's quite "curious" to see how a site that gets about a tenth of that, or about 6 million a month (according to SimilarWeb estimates), could go down so frequently without anyone doing anything to fix the problem.
This is a real problem, because typically, when a downtime occurs and traffic to the site can't reach it or accumulates numerous 500 errors, browsers using Chromium (the engine behind Chrome), including Chrome of course, report the incident to Google. As a result, Google will send less traffic to that site in the hours that follow, knowing something isn't working properly . This can significantly impact the site's visibility and performance, further penalizing its traffic and reliability in the eyes of users and search engines, as well as obviously damaging its reputation.
Furthermore, even when the site is online, it does not seem to guarantee who knows what performance with a Time To First Byte of around 500ms at 8 in the morning with the site practically little visited and with no posts launched that generate traffic. Below you can, for example, view the TTFB test for Europe with the SpeedVitals.com TTFB test
Below you can view our TTFB of Managedserver.it, a site that we take care of obsessively to obtain the best performance.
We are talking about a TTFB (Time to First Byte) that is 10 times lower and well below the maximum 200 ms that Google considers acceptable before reporting that the server response time is too high, indicating the need to "Improve the server response time”.
Furthermore, the slowness of such a high TTFB, to the point of becoming unreachable at certain times of the day frequently, including prolonged downtime, negatively impacts the Core Web Vitals that we can see from the Google PageSpeed Insight report below.
A high Time to First Byte (TTFB), as shown in the image with a value of 2,3 seconds, can negatively impact other Core Web Vitals metrics . A high TTFB means there is a significant delay before the browser begins receiving data from the server. This delay contributes to a high First Contentful Paint (FCP) (3,4 seconds) and a high Largest Contentful Paint (LCP) (4,7 seconds), as the initial page rendering is delayed. Additionally, a high TTFB can negatively impact Interaction to Next Paint (INP) and Cumulative Layout Shift (CLS), as slow page loading can lead to a less fluid user experience and more frequent layout shifts.
Accustomed to managing significant traffic peaks, even over 2 million visits per hour and 200 thousand per minute, and specialized in the Optimization of Core Web Vitals , we offered our services and our expertise to the owners of passioneastronomia.it by contacting them to propose managed management of their infrastructure or migration to our high-performance hosting, however, our offer was probably misunderstood and underestimated, as well as rejected without too many compliments and mincing words.
Yet our site boasts an optimal TTFB of just 22ms in Italy and a PageSpeed that brilliantly passes the Core Web Vitals test as you can see below.
With a significant publishing client portfolio and the numbers mentioned above, which could be impartially assessed using the tools provided by Google, it seemed incredible that, in a difficult moment characterized by constant downtime, they hadn't seized the opportunity to resolve all the issues immediately. This could have been done at a cost likely half that of their current provider, which frequently saw them offline or with poor performance, as indicated by the tests above.
Passioneastronomia.it and WordPress
Passioneastronomia.it is an editorial site developed in WordPress, based on PHP and MySQL. Although these technologies are now considered dated compared to more modern, asynchronous languages and technologies like Node.js, Go, or NoSQL databases like Cassandra, MongoDB, or REDIS, they continue to be widely used. PHP, for example, was developed in the 90s and has undergone numerous updates over time, but remains less performant under heavy load than modern asynchronous languages. Similarly, MySQL, despite being a robust relational database management system, can struggle with scalability and performance when handling large data volumes and concurrent requests.
Despite these limitations, WordPress remains the most versatile and easy-to-implement content publishing system available today. Its large developer community, myriad plugins available, and intuitive interface make it a popular choice for both small blogs and large businesses. Even major newspapers like Il Fatto Quotidiano use WordPress as their CMS for editorial publishing, demonstrating that, with the right configuration and optimization, excellent performance can be achieved even with technologies considered less modern.
Moreover, even NASA, which has always been the pinnacle of scientific communication in the collective imagination, uses WordPress as a CMS.
Passioneastronomia.it initially (at least since they told us about it about a year ago), was hosted on SiteGround, a hosting service that is not always able to handle traffic of this level. Then in April, the site moved to AWS, the choice of the market and many big names such as Netflix, Airbnb, Spotify, Twitch, LinkedIn, Adobe, Slack and the BBC.
- Netflix: Uses AWS for video streaming, data storage and analytics.
- Airbnb: Leverages AWS to manage its global hospitality platform and online marketplace.
- Spotify: Use AWS for streaming music delivery, data analytics, and machine learning.
- Twitch: The Amazon-owned video streaming platform relies on AWS for live video transmission and data management.
- LinkedIn: Although some of its infrastructure is internal, LinkedIn uses AWS for some of its data storage and analytics services.
- Adobe: Offers its Creative Cloud services and other SaaS applications through AWS.
- Slack: Uses AWS for its messaging and collaboration platform.
- BBC: The BBC uses AWS for on-demand video content delivery and data management.
Despite the power and scalability of AWS, passioneastronomia.it continued to experience downtime, culminating in a 4-hour outage on May 16, from 19:00 PM to 23:13 PM, monitored by our Uptime Kuma tool.
The causes of Downtime
The main problem isn't with AWS as an infrastructure, but with the configuration and management of the service. AWS offers a wide range of services, including EC2, databases, load balancers, distributed file systems, and CDNs like CloudFront . However, if used improperly, AWS can lead to serious performance issues.
The website passioneastronomia.it did not have an adequate Full Page Cache system like CloudFront or the Pro or Business versions of CloudFlare with HTML Cache support, or even a "simple" Varnish. The site's response headers clearly show that HTML caching was not enabled:
HTTP/2 200
date: Thu, 16 May 2024 22:02:18 GMT
content-type: text/html; charset=UTF-8
set-cookie: AWSALBTG=gkrvkTmzuBE7uhKteG6ihiCGQH60BIdF48ki+7cvKP9ia2ltc4cAgn5dVM5l+/WaO8fbb8dzylYF1OYP7PZnmhHdLsauuVLuLntiKviIt8EAxKNbM3yBSyKqrMaGu1SXAQaPGkLnjoHwqz3OkmDHAVqvBB0V3v4d0WOcshbhqixspvTJTic=; Expires=Thu, 23 May 2024 22:02:17 GMT; Path=/
set-cookie: AWSALBTGCORS=gkrvkTmzuBE7uhKteG6ihiCGQH60BIdF48ki+7cvKP9ia2ltc4cAgn5dVM5l+/WaO8fbb8dzylYF1OYP7PZnmhHdLsauuVLuLntiKviIt8EAxKNbM3yBSyKqrMaGu1SXAQaPGkLnjoHwqz3OkmDHAVqvBB0V3v4d0WOcshbhqixspvTJTic=; Expires=Thu, 23 May 2024 22:02:17 GMT; Path=/; SameSite=None; Secure
cf-edge-cache: cache,platform=wordpress
link: <https://www.passioneastronomia.it/wp-json/>; rel="https://api.w.org/"
link: <https://www.passioneastronomia.it/wp-json/wp/v2/pages/49>; rel="alternate"; type="application/json"
link: <https://www.passioneastronomia.it/>; rel=shortlink
vary: Accept-Encoding
cf-cache-status: DYNAMIC
report-to: {"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=dvOaq3CI0sDGSeXsjICpJUT0nF1zmp%2BFj1TKPfqJKvyRd%2FuybtNnkc9FKE6SLu7CldFY7brUo1HZwF1kvPoDyisecYzzZC3aI%2FlrjkKCKcs%2BD4LVRylVZMmSQViSYJRA6fO9JU1sg2ejKUk%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 884ea6b518c20e4d-MXP
alt-svc: h3=":443"; ma=86400
The Free version of CloudFlare does not support HTML Cache in DYNAMIC mode, causing excessive load on AWS servers with continuous HTML requests passing directly through PHP processes and database connections.
In the first case, when the load increases significantly, the origin Web Server reaches saturation, being unable to even respond and causing the reverse proxy of the CloudFlare CDN to time out as in the following image:
In the second case, even if the Web Server manages to respond for a few moments, certainly the load of SQL queries to the MySQL database will return a connection error to the Database as in the screenshot below.
More succinctly, it is correct to say without going too deeply into the world of DBMS, management of tables, queries and indexes that this lack of adequate caching has several negative implications, especially for high traffic sites such aspassionastronomia.it.
Implications of HTML Cache Lack
- Increased load on Origin Servers:Without HTML caching, each HTML page request must be processed directly by the origin servers. This means that each visit to the site activates PHP processes to generate the dynamic content of the page, significantly increasing the load on the servers. As a result, the server must handle a large number of concurrent requests, leading to rapid exhaustion of available resources.
- Database overload:Each HTML request can involve a series of database queries to retrieve the data needed to generate the page. As traffic increases, the number of database connections can exceed the manageable limit, causing the database to slow down or even crash. This database overload is often the root cause of prolonged downtime.
- High Response Times:Without caching, the time required to generate an HTML page can be significantly longer. Each request requires loading and executing PHP scripts, interacting with the database, and dynamically generating content. This process is much slower than serving a cached page, leading to high response times and a suboptimal user experience.
- Risk of Downtime:When origin servers are constantly under pressure due to the high number of requests, the risk of downtime increases. Servers can become unresponsive or even crash if the load exceeds their capacity to handle. This is exactly what happened to passionaastronomia.it, where the site experienced significant downtime due to the inability to handle high traffic without a proper caching system.
Inappropriate Use of AWS EC2
Although AWS EC2 is a powerful solution, using an EC2 instance, even a large Etra, without proper configuration can be ineffective. An EC2 instance is nothing more than a virtualized instance with a certain amount of cores and RAM, which offers significant computational resources. However, without proper traffic management and caching, these resources can quickly become overloaded. Poor configuration can turn a powerful instance into a bottleneck, unable to handle high traffic spikes.
For example, without load balancing mechanisms, a single server can become overwhelmed by an excessive number of concurrent requests, quickly exhausting available CPU and memory. Furthermore, without an effective caching strategy, every user request must be processed entirely by the origin server, involving PHP processes and database queries to generate dynamic pages. This not only increases response times but also puts a strain on the database, which can become a critical point of failure, even though, as we can see, the database is hosted on a managed database service like RDS.
Amazon AWS's MySQL RDS (Relational Database Service) is a managed database service that makes it easy to set up, operate, and scale a MySQL database in the cloud . With MySQL RDS, Amazon handles administrative tasks such as hardware, operating system, database patching, and backups. This allows users to focus more on application development and less on infrastructure management.
Even the best EC2 instances can fail if they aren't supported by a solid backend architecture. To maximize the efficiency of an EC2 instance, it's essential to implement application-level caching systems and use CDNs like CloudFront or CloudFlare to distribute the load. Additionally, configuring Auto Scaling to automatically adapt to traffic changes can prevent sudden overloads. Only with careful management and optimal configuration can an EC2 instance fully exploit its potential, ensuring high performance and reliability.
What are CloudFront and CloudFlare?
CloudFront
CloudFront is a CDN (Content Delivery Network) offered by AWS that distributes content worldwide, reducing latency and improving page loading speed. This service is particularly useful for high-traffic sites like passioneastronomia.it, as it can cache entire HTML pages, reducing the load on origin servers and significantly improving the user experience. Content distribution via a global network of edge nodes brings data closer to end users, ensuring faster response times and greater reliability. Additionally, CloudFront offers advanced features such as DDoS protection and integration with AWS Shield and AWS WAF, which help improve site security. Using CloudFront also allows you to better manage traffic spikes, ensuring that origin server resources are not overloaded during periods of high demand. Its flexibility and scalability make it an ideal solution for improving site performance and offering users a smoother, faster browsing experience.
CloudFlare
CloudFlare is another CDN service that offers a wide range of features to improve website performance and security. Unlike CloudFront, CloudFlare offers several caching options, including HTML Cache for Pro and Business versions.
The advantages of CloudFront and CloudFlare can be very similar, if not even overlapping, the most notable advantage of CloudFlare over CloudFront is that CloudFlare offers a flat pricing plan with a basic plan of $25 per month which would have been more than enough to make the passioneastronomia.it site perfectly compliant with the traffic peaks it receives daily.
CloudFront, on the other hand, offers a pay-as-you-go plan so the cost is directly proportional to the data moved.
In fact, both CloudFlare and CloudFront offer a Full Page Cache service in PaaS (Platform as a Service) mode, distinguishing themselves from self-hosted solutions like Varnish Cache . This difference is significant, as a PaaS service like the one provided by CloudFlare and CloudFront eliminates the need for users to manage and maintain the caching infrastructure. With PaaS solutions, cache deployment and management are automated and integrated into the service, reducing administrative overhead and allowing teams to focus on other critical tasks.
For example, CloudFlare and CloudFront offer user-friendly interfaces for configuring caching rules , managing invalidations, and monitoring site performance, all without requiring in-depth technical server management expertise. Self-hosted solutions like Varnish Cache, on the other hand, require manual configuration and ongoing management of the caching infrastructure. This requires qualified technical staff capable of installing, configuring, monitoring, and updating the caching software, as well as managing the physical or virtual servers on which it operates.
CloudFlare and CloudFront's PaaS solutions also offer significant advantages in scalability and reliability . As cloud services, they can leverage a global network of distributed nodes to deliver high performance and low latency to end users, regardless of their geographic location. Content delivery through a global network of edge servers reduces latency and improves page load times, providing an optimal user experience.
Furthermore, CloudFlare and CloudFront integrate advanced security features, such as DDoS protection and integration with SSL certificates, which can be managed and configured directly from the platform without requiring manual intervention. This level of automation and integration makes it easier to secure web applications and provides greater peace of mind for site owners.
While self-hosted solutions like Varnish Cache can offer more granular and customized control over cache configuration, they require a significant commitment in terms of resources and technical expertise. On the other hand, PaaS solutions from CloudFlare and CloudFront provide a comprehensive, automated, and highly scalable service that simplifies cache management, reduces administrative overhead, and offers superior performance and security.
Full Page Cache what is it?
Full Page Cache (FPC) is a caching technique that stores the entire HTML page generated by a website. This technique is essential for high-traffic sites as it drastically reduces the number of requests to the origin server, reducing the workload of PHP processes and database connections.
How it works
When a user visits a web page, the server generates the HTML page and caches it. Subsequent visits to the same page are served directly from the cache, avoiding page regeneration and reducing response times.
Conclusion
The lack of an HTML cache is a classic example of how the absence of an adequate caching strategy can strain even the most robust infrastructures like AWS. For high-traffic sites, implementing an efficient caching system is crucial to reducing the load on origin servers, improving response times, and preventing significant downtime. In this specific case, using the $25-monthly version of CloudFlare, configured by a skilled system administrator, would have been enough to resolve the downtime issues. Even better, implementing a Varnish Cache system with CloudFlare positioned in front would have allowed for a double layer of caching. This configuration would have offered maximum performance at a significantly lower cost than the current one. Using such solutions, it is possible to achieve significant improvements in site performance and stability, ensuring an optimal user experience even during traffic peaks.
This shows that beyond the provider of the services you decide to use, what really matters is the technical competence and expertise of the technicians who are responsible for implementing the best caching strategy in order to improve performance and content delivery.
Below is the traffic of our client's site that the editorial team of Passionastronomia.it reported to us for problems that we could have solved in 1 hour of work and consultancy, probably saving around 75% of the budget they are spending on AWS to go offline. But as they say in these cases, whoever is the cause of his evil should cry for himself.
It will be funny when the "bills" arrive from AWS and in particular from Amazon RDS for MySQL which is a Pay Per Use service and which will probably increase the costs significantly. Systems engineering is not child's play and as you can see inexperienced technical consultants can not only cause damage, but also drain your budget, not knowing how much more convenient it could be to spend by investing in CDN and Full Page Cache, rather than in Managed Database services. like AWS RDS.
We told him…
Updates two months later.
Following this post and the resulting musings, we felt it was appropriate (though not necessary) to share some technical suggestions with Passione Astronomia. We therefore sent an email containing extremely specific and useful technical advice for the current hosting scenario on Amazon AWS , recommending either installing a server-side Full Page Cache like Amazon's CloudFront, or enabling CloudFlare's HTML Cache, which they are already using in Free mode, perhaps using CloudFlare's APO directly at a low cost of about $5 per month.
We confidently expected that within a few days we would see our precious technical suggestions implemented, however after almost two months, the analysis of the response headers shows no changes taking place, not even the shadow of Cloudfront, of CloudFlare which not even a shadow of HTML cache and a response time exceeding the maximum 200ms tolerated by Google.
The proof of the 9 was given by the downtime of 8 July 2024, that is, yesterday, the evening in which our Uptime Kuma monitoring system reported to us a prolonged downtime of the site in question, with a total and complete offline of 2 hours and 20 minutes , as we can see from the following dashboard.
We were quite shocked by the fact that our advice, completely disinterested and based on years of experience, was neither received nor implemented. It is particularly surprising considering that some of the largest and most important publishing companies on the Italian scene are willing to pay several thousand euros to solve problems of stability, speed and performance that we are able to deal with effectively.
This leads us to a legitimate reflection: are the operators in the sector, as well as our possible "colleagues", really aware of the cornerstones necessary for the management of an important and successful project like this? Perhaps the time has come to think seriously about establishing a professional register with a state exam to qualify for professions such as systems engineering. Such a step would guarantee high standards of competence and responsibility, ensuring that only the most trained professionals can operate in a sector so crucial to the success of modern businesses.
Update almost a year later.
After almost a year, we find ourselves updating this post again, although we should have done it several months ago, approximately one or two months after the last email we sent them, because in the end (hope is the last to die) they decided to deign to follow our advice and our instructions.
In fact, if we limit ourselves to analyzing the HTTP response headers to a curl -i we obtain the following values:
If we focus on the analysis of the HTTP response headers, in particular on the fields cf-cache-status: HIT, cf-apo-via: tcache e x-httpd-modphp, we can get interesting information about what has actually changed after our intervention in which we suggested what to do in the email. These headers, in fact, tell a precise story about the current architecture and the optimizations applied.
-
cf-cache-status: HITindicates that the response was served directly from Cloudflare's edge cache, without involving the originating server. This is a clear sign that the content was properly cached and that the system benefits from a significant reduction in load on the backend and response times for the end user. -
cf-apo-via: tcacherefers to the system Automatic Platform Optimization (APO) from Cloudflare, up and running, which uses aggressive caching at the edge to serve dynamic pages as if they were static, ensuring superior performance especially on CMS like WordPress. The valuetcacheconfirms that the page was served directly from Cloudflare’s even more efficient “Tiered” cache. -
x-httpd-modphpinstead indicates that it is active on the source server mod_php, that is, the PHP interpreter integrated into the Apache web server, a detail that suggests a shared or semi-managed hosting environment, in contrast to more complex and expensive solutions such as the use of PHP-FPM on Dedicated Servers.
It is worth noting that they moved from an expensive infrastructure on Amazon Web Services (AWS) , where they were running a MySQL RDS cluster , to a much cheaper solution like SiteGround , which, thanks to proper setup and Cloudflare integration configuration, is now offering better performance at significantly reduced costs.
Excellent results also on the Google Pagespeed Insight and Core Web Vitals front , if we consider that they improved the TTFB taking it from 2,3 seconds on May 17, 2024 to 200 milliseconds on April 20, 2025.
However, as anyone involved in Web Performance knows, an optimal Time To First Byte (TTFB) is only one of the fundamental elements , and is not sufficient, on its own, to guarantee an overall improvement in the Core Web Vitals metrics , particularly CLS (Cumulative Layout Shift) and LCP (Largest Contentful Paint) . These two metrics, in fact, depend not only on the server's response speed, but also on the quality of frontend rendering and how critical user interface content is structured and loaded.
Our intervention, if it had been fully accepted also at the application level , would certainly have had a positive impact on these aspects, improving the stability of the layout during loading (CLS) and speeding up the display of the main element of the page (LCP), thanks to techniques such as advanced lazy loading, font optimization, asynchronous loading of non-blocking resources and smart management of LCP images.
This type of improvement would have had direct and tangible effects in terms of SEO and organic visibility . It is now consolidated evidence—supported by field data collected through the CrUX reports (Chrome User Experience Report) —that exceeding the "good" threshold in the Core Web Vitals leads to a better positioning in Google's SERPs . And a better ranking almost always corresponds to a direct and proportional increase in organic traffic , with all the implications in terms of advertising monetization and overall brand visibility.
In this context, what remains bitter is the attitude we received: from the first approaches, we were ignored, never engaged, let alone thanked . On the contrary, we were banned from both the group and the fan page of this newspaper , without the possibility of discussion or constructive dialogue.
This inevitably leads us to reflect on whether it was worth offering free consultancy and professional technical support , just to demonstrate—with facts and data—that our proposals were well-founded, concrete, and above all effective.
