Determining an optimal value for pm.max_children

How to determine an optimal value for pm.max_children and related php-fpm settings for your server and web applications.

2327 views

By. Jacob

Edited: 2022-11-07 17:55

One of the most confusing things to configure with Nginx and PHP-FPM is the values for pm.max_children and its related options. When this is set too low, you will typically get warnings like the below in your /var/log/php8.1-fpm.log file (note the version in the file name):

WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 10 total children

And also:

WARNING: [pool www] server reached pm.max_children setting (5), consider raising it

You can roughly determine a sensible value for pm.max_children by dividing the available system memory with the average amount of memory consumed by PHP-FPM processes. E.g. If you have a total of 16GB and your processes consume 100 MB each:

16000/100 = 160

This means you should be able to run roughly a little under 160 children before running out of memory. Having a little swap probably does not hurt at this point, but please also monitor your CPU usage, as everything needs to be able to keep up with demand. Some applications require a different resource focus than others, and hence, you will just have to monitor your resource use continually to figure out the optimal settings.

Running multiple services

Since you may be running other services on the system, you should also make sure to leave enough resources for those. E.g. Elasticsearch and MySQL. In total you may only have around 4GB available on a 16GB system, taking other services into account. E.g:

4000/100 = 40

This means you could probably get away with pm.max_children = 40.

Note. Ideally you might want to consider moving these services to their own separate server – not necessarily meaning that each should run on its own server, but nevertheless, keep them separate from your web server so that resources will be dedicated to running Apache/Nginx, PHP, and whatever other required processes you need for stable operation of your website- and scheduled tasks.

But – this is really only needed when it is needed! You will surely notice when services start to compete for resources, as there will be errors!

If you are using a dynamic profile, you will also need to adjust the pm.start_servers, pm.min_spare_servers, and pm.max_spare_servers. As the total number of running processes is determined by pm.max_children, the other settings should be configured with a lower value than max_children.

pm.max_children

Defines the maximum available child processes. There will not be spawned more than this number, not even when other values are relatively high.

pm.start_servers

Defines the number of servers that should be started when php-fpm starts. It can not surpass pm.max_children. According to the documentation the default value is:

min_spare_servers + (max_spare_servers - min_spare_servers) / 2

pm.min_spare_servers

Used on the dynamic setting, and defines the minimum number of idle processes. This is useful for dealing with sudden traffic spikes, because it takes time to start new servers vs just re-using idle processes.

pm.max_spare_servers

Used on the dynamic setting, and defines the maximum number of idle processes.

pm.max_spawn_rate

Defines the number of processes to spawn at a time doing traffic spikes. Default is 32. If you pay attention to your log file you might see a message like this:

WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 10 total children

Here it tells you that the available servers has become busy, typically this means they are still processing or waiting for a response by another service, E.g. Elasticsearch or a Database. Either way, additional php-fpm children will be spawned to deal with the incoming traffic. E.g. spawning 32 children.

pm.max_requests

Finally, pm.max_requests controls how many requests each child process should handle before being restarted. This may help deal with memory leaks caused by dependencies. Under good circumstances, and if you do not experience any problems, you should probably leave this commented out, as restarting processes may affect performance negatively. The default value is "0", meaning that each child process will process requests indefinitely.

A note on using very small servers

It is possible to run multiple services. E.g. MySQL, php-fpm and redis on the same server. Even on super small servers with 1 core and 2GB of RAM – but it might be hard to get the configuration right in terms of figuring out the right configuration for SWAP, and how to best use available memory.

Most importantly, it also depends on the demands of your web application.

If your web application is highly optimized, and has a decent server-sided cache system for HTML and other resources, then you do not really need many resources to run a small webshop or blog. This is further improved by putting Cloudflare in front of your server and allowing it to also cache your HTML.

Stress testing your configuration

Unfortunately, there is no one-size-fits-all setting, and you will benefit more from stress testing your system yourself to discover what works for you, and to get a feel of how these settings work. I am still learning myself; Next-level for me would be to start using load-balancers, but I have, so far, not had the need to do that with the type of websites I am working on.

There is a couple of useful tools for stress testing; one is ab (Aka. Apache Benchmark), and another is called siege; ab seems to have some kind of SSL bug that I have not bothered looking into, so it might not work against https sites.

Typically all you need to test a small server, is another server in the cloud to run test from. To use Siege:

1. Install Siege:

apt install siege

2. Run a Siege with 20-40 concurrent users against your live environment. You best start low, and move higher, and ideally do this at a time when your website is not too busy. E.g:

siege -c 40 -t 1m https://example.com/

Depending on how powerful the target system is, you might even need to run a distributed stress test.

Note. It matters a lot which URL you run the test against. When your server uses caching or you have cloudflare, obviously many requests are not even reaching your server. Ideally you should pick a URL with dynamically generated content so that you are sure the request will reach your server, and if you think it is needed, something that even bypass your own servers cache as well. Running against a search result page or API end point is good for this, as the resource will often be generated on every single request, which places extra stress on your server.

Once your server starts having trouble, you may see the following message in your php-fpm log file:

[06-Nov-2022 12:10:30] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it

This means you need to increase the number of pm.max_children in order to optimally handle the load. It also tells you the number of current children, in this case 5, which is the default for php-fpm after installing on Ubuntu. And, this is actually a fair value for many small blogs and webshop sites.

Even on a small server, you might as well bump it up a bit, just to be sure you can handle traffic spikes.