How I manage 6+ Million PageView DigitBin.com Server at $6/month?

It’s been a while since I haven’t published anything interesting. Okay, it’s 0440 hours in local time. I woke up too early today and am writing this post on how I manage a server running a blog that garners 6+ million page views every month at only a nominal cost of $6 ($5 server + 1$ backup).

You read it right; it is just a $5/month starter cloud server from the DigitalOcean(DO).

I’ve been managing the DO cloud server since 201 

You can follow this post if you don’t know about DigitalOcean and how happy I have been.

Many things come into play when managing a server; unlike managed cloud hosting, you have to deal with every single issue on your own.

I had many sleepless nights and unlimited make-and-break command-line trials that finally helped create a server that could bear extensive peak load.

Before getting started, here is a sneak peek of DO billing —

and Google Analytics performance graph of DigitBin.com for Nov 2019.

Without further ado, let’s get started —

Note: The post has been updated to support the latest PHP version, however, the screenshot/images still reflect the older version.

Backend Server Configuration

When I say ‘Server’, it’s entirely the backend invisible part that host’s the site. Here are the configuration and specs of the cloud server running behind DigitBin:

Hardware Specs

It’s configured on the same basic standard droplet in DigitalOcean, which is available at $5 per month.

• CPU: virtual shared single-core (1 vCPU)
• Memory: 1 GB RAM
• Disk Space: 25 GB SSD
• Bandwidth: 1 TB transfer
• Location: NYC

I have also enabled the weekly droplet backup that additionally costs 20% of droplet cost, which is $1 for retaining four backups.

Software Specs

After multiple rounds of performance tweaks, I’ve now settled down to the below LEMP Stack configuration:

• Operating System: Ubuntu 22.04 Server
• Web server: Nginx
• PHP version: 7.4
• SQL: MySQL 8.0 
• Mail server: Postfix
• Firewall: UFW

I initially created the server with the Apache2 web server (LAMP); however, we moved to Nginx (LEMP) a few years later.

Now I am not going into detail about how I configured the software and hardware specs cause I already have a series of posts and video demos you can refer to.

Cloud Server Setup

If you follow the guide carefully, I believe you will have the best and most secure server configuration to date.

Backend Performance Configuration

While basic server setup is essential in hosting and running the website, its performance tweaks enhance the server’s capability.

I have tweaked every bit of the server configuration to make it robust and sustainable at peak loads. For example;

Nginx Setup

I consider the Nginx the fastest compared to the Apache webserver. It’s lightweight and has a heavy processing capability.

To further improve its capabilities, I’ve added the snippets to execute browser caching headers, gzip compression, added security snippets, http2, and more. This will eliminate the reliance on third-party extensions in the front end.

PHP Setup

PHP7.4 is the fastest PHP version compared to its predecessors. I have enabled the capability of FPM (FastCGI Process Manager) for processing requests during heavy loads.

This is also used for generating and serving the static html page from dynamic content using FastCGI nginx cache.

MySQL Setup

I use MySQL8.0, considered the fastest among predecessors (MySQL5.7), and likely has the capability of MariaDB.

It also uses the Redis Object Caching that helps cache the complex and most requested queries within the RAM, improving the processing speed and reducing the dependency on MySQL to process every request.

The cloud server setup guide covers many things that help optimize the overall server performance.

WordPress CMS Tweaks

DigitBin.com is running on the WordPress blogging CMS with the rich capability and features to fully control, manage, and improve performance. Though I mostly rely on backend optimization, a few minor tweaks still need to be made in the front end.

WordPress is a lightweight CMS that consumes little memory and CPU for operation. But, things get clumsy when loaded with heavy memory-intensive plugins.

Especially the caching and security plugins like W3 total cache, WP Supercache, Wordfence, etc., are pretty heavy on the server. Hence, I always look up alternatives it can implement in the backend. That’s where the FastCGI cache, nginx browser cache, redis object cache, UFW firewall, etc., is a perfect substitute for the features these plugins offer.

Always choose the plugins wisely. I add plugins if and only if there is no alternative or can’t be managed at the backend, like SEO plugins.

Offloading Static Assets

Few static assets often help to load and visualize the site, like JS, CSS, Images, and HTML. Though HTML is an integral part of the front-end, which cannot be offloaded completely; however can be cached and distributed from the nearest edge caching server (more covered in the Cloudflare CDN section below).

Free Images Offloading:

I am very thankful to WordPress for its generous support that helped offload the images using photon CDN for free (clubbed into JetPack as Site Accelerator). You can read it here if you do not know about photon and its API.

With the Photon CDN, I can save a few TBs of bandwidth against the images. Every image hosted on DigitBin is automatically synced and replaced with a photon CDN URL. Hence, saving the bandwidth cost and serving the image files from the photon’s edge caching server.

Besides saving bandwidth, CDN also converts the image into webp format, which is considered the superior lossless and lossy compression for images on the web.

Eventually, I am planning to migrate the images to Cloudflare CDN under sub-domains.

Offload CSS and JS:

CSS and JS are essential assets that make front-end design attractive and interactive. I have (had) offloaded using plugins named commonWP and NGT jsDelivr. One can use either of the plugins to check which is compatible and working with your installation. However, I have used both interchangeably.

Both plugins will rewrite the site’s static CSS and JS asset URLs to free open source CDN like JSDelivr. Example: https://wpaq.com/staticfiles.js will become https://cdn.jsdelivr.net/staticfiles.js. It further reduces the bandwidth usage as well as server load.

However, there are certain conditions for rewriting the URLs. Like:

1. The active theme and plugins must be from the WordPress repository.
2. The developer should commit the files within /tags/ sub-directory folder (not in our control).

You can refer to the individual plugins details page for more such limitations.

[Note: I recently stopped using the jsdelivr since I wanted to combine JS & CSS assets and serve as a single file. Additional details are in the Cloudflare section]

Optimize Assets

Asset optimization is another important part that helps in improving server performance as well as improving this site loading speed.

Image Optimization:

As mentioned above, the Photon CDN automatically converts the image and compresses it into webp format; hence further optimization is not necessarily required. However, if you think you can improve loading time further, you can compress the image file using smush.it API.

There is a dedicated plugin for smush.it, however, I use WP-Optimize plugins that not only incorporate the smush.it also helps in the database cleanup.

CSS, JS and HTML Optimization:

WP-Optimize also has the feature to minify, cache, compress, and preload HTML, JS, and CSS. However, if you alter or make any changes to JS and CSS files, then free asset offloading using jsdelivr won’t work.

Regarding HTML caching, we already have the Nginx FastCGI caching that works without any additional caching plugin.

I recently started using the Autoptimize plugins for minification and combining assets. Though this has increased the server load a bit since it requires memory for processing, it’s worth it.

Cloudflare CDN Settings

So finally, here is the section I mentioned twice above.

I used Cloudflare CDN a couple of years ago (I do not remember exactly but maybe in 2015), but my experience wasn’t great in those days. The site performance dropped, and it also the loading became slow. Hence, immediately switch back to the original server and stay.

But recently, I heard pretty good reviews regarding improved site speed and fast DNS resolving capability. Hence, I thought of giving it a try again.

I switched from Google DNS to Cloudflare DNS, made it mandatory to set up, and started serving the files from Cloudflare Edge CDN caching.

I currently use the page rules, service workers, Cloudflare DNS for speed, SSL certificate, and minimal WAF security.

Cloudflare Page Rules

The Page Rules in Cloudflare is a helpful feature that helps set up the site performance. It also accepts the wildcard parameter *, making it more robust.

I currently have these two rules which cache everything within the /content/ or /wp-content/ and /wp-includes/ directory and store in edge cache for 7 days.

The files minified and combined using the Autoptimze plugin are automatically served using the below page rules. Hence, there won’t be any additional bandwidth consumption from the cloud server.

Edge caching helps store and sync the cached static files in the nearest edge server. When a user requests the webserver files, the Edge cache serves from the closest CDN pop datacenter—making the site a faster and smoother loading experience.

Cloudflare Service Worker

The Service worker was relatively new when I started using Cloudflare again. The service worker helps deploy the JavasSript worker on the Edge server that can control the website HTTP request.

I use a service worker to Cache Everything on the front-end part. This script helps bust the cache when logged in as a WordPress user.

The cache will help edge cache the dynamic HTML in the Cloudflare server. So when the subsequent web request is sent, Cloudflare automatically serves the HTML copy of the web page from the nearest CDN Edge server datacenter instead of hitting our original DigitalOcean cloud server.

Due to the Edge caching mechanism, this significantly reduces the server load and makes the web serving extremely faster.

The service worker is chargeable. You won’t be charged a penny if you’re within the free tier limit.

Security Settings

You can skip this item using the Let’s Encrypt free SSL certificate service. However, I moved to Cloudflare SSL and discounted the Let’s Encrypt. Cloudflare provides excellent options and controls to improve site security.

Cloudflare also has a bot protection mode known as Web Application Firewall (WAF), which you can customize to any security level. I keep the Low mode since I have multiple crawler bots on the site.

Note: Do not use Cloudflare and Let’s Encrypt together. You may encounter the redirect loop error. Also, remove all the Let’s Encrypt rules you might have added to the Nginx configuration file.

Bandwidth Served from Server

Since most of the static and dynamic assets are served using the Cloudflare and Photon CDN. The total bandwidth usage on the DO droplet is unsurprisingly low. Here are the stats of the DigitalOcean and Cloudflare servers for last month:

Cloudflare:

Last month the Cloudflare CDN helped me save 452 GB of bandwidth by serving all static assets (except Images) and cached HTML pages.

Photon CDN:

The Photon CDN helps to save unlimited image bandwidth. I do not have an exact counter; however, I guess the bandwidth will be near ~20TB. DigitBin is an image-heavy site with the average size of an image being 100KB.

Photon CDN helps save bandwidth and reduces server requests. They are making CPU and RAM available for other critical task-intensive processes.

Eventually, I am planning to move. I’ve moved the images into Cloudflare CDN, making the site even faster.

DigitalOcean:

After cutting off the bandwidth to Cloudflare and Photon CDN, the DigitalOcean has only received limited bandwidth transfer requests — 22 GB per month.

The data transfer size has increased on the DO droplet after installing the Autoptimize plugins and disabling the JSDelivr CDN plugins. Earlier it was less than 10GB (~8GB) per month.

Bottom Line: DigitBin Performance

So this was all about the server and website configuration of DigitBin that I currently manage at a nominal cost.

Though I have been charged additionally for Cloudflare service workers after crossing its graceful free limit, that isn’t very important. We can stop these charges if you do not use the service workers.

Overall I am pretty happy with the DigitalOcean server performance and free Cloudflare CDN and security setup.

I know you might have additional questions in your mind; feel free to drop them in the comments below. Also, please drop any thoughts about my configuration or suggestions.