How output buffering works in PHP

Flushing and output buffering goes hand in hand, and in this article I try to examine the benefits and disadvantages to flushing.

1913 views

By. Jacob

Edited: 2021-11-23 09:20

Instead of sending content instantly to a client as it is echo'ed from a script, PHP uses output buffering to hold content until it is flushed at the end of a script. Sometimes it is turned on by default in PHP's configuration, and sometimes it is not. When you write your scripts, output buffering can be turned on by calling the ob_start function, and this is where it gets confusing, because you can actually start multiple "levels" of buffering — each call to ob_start will begin a new level of buffering.

I personally see no need to have these multiple levels, so I think it is best if you try to keep it at a single level; and as I already mentioned, PHP sometimes enable the first level by default. To check if output buffering is already enabled, you can call ob_get_level before we decide if you should call ob_start.

if (!ob_get_level()) {
  ob_start()
}

While it is best to just disable it on your server, here is a way to close all levels automatically:

while(ob_get_level() > 0) {
  ob_end_clean(); // Close the output buffer and erase its content
}

A developer might sometimes explicitly flush the content of the output buffer. The advantage of doing that is that you, in theory, minimize PHP's memory consumption, decrease the TTFB, which speeds up loading of your web pages by allowing the browser to read parts of the code faster.

Flushing the output buffer is mainly useful when sending large files to users (clients), such as when streaming video files in PHP; it is not very useful for simple HTML pages, because they are generated very fast. In fact, enabling some kind of caching of HTML might be better for such purposes.

This example should work for modern browsers with a buffer of 4096 bytes:

if (!ob_get_level()) {
  ob_start()
}

for ($i = 0; $i < 10; $i++)
{
  $str = "$i\n";

  // Force the browser to flush
  // by padding the $str to 4096 bytes
  // before we flush the output buffer
  echo str_pad($str,4096)."\n";

  ob_flush();
  flush();
  sleep(1);
}
echo "Done.";

// Close output buffer,
// and send the final "done" content to the client
ob_end_flush();
exit();

When to flush

When users request a page, it can take anywhere from 200 to 500ms for the backend server to stitch together the HTML page. During this time, the browser is idle as it waits for the data to arrive. In PHP you have the function flush(). It allows you to send your partially ready HTML response to the browser so that the browser can start fetching components while your backend is busy with the rest of the HTML page. The benefit is mainly seen on busy backends or light frontends.

Flush the Buffer Early - Yahoo

An old advice from Yahoo says to flush early, but in practice, this does not matter much for most websites; and it can also be hard to implement, since you need to carefully configure your server to get it to work when using brotli, deflate or GZIP to compress text/html.

In addition, you might also need to pad the output to force flushing of the client buffer, which is bad because it only makes the page larger; please do not do this unless compression is enabled.

Both your server and the client's browser, may buffer data; even if you manage to configure your server right, if the client's buffer is not filled, nothing will get painted on the client's screen — typically 4096 bytes.

Of course, you should not pad output with spaces, because it increases the size of the page. But, you may still choose to flush early to help loading of large HTML pages. I do not know if browsers read the content in their buffer, or if they have to flush it before they can fetch resources in the content. Do your own testing to figure this out.

The size of HTML

Under normal circumstances, output will be stored in the output buffer, on the server, until a page has finished assembling on the back-end; only then will it be compressed and sent back to the client. Some CMS' might optimize delivery by flushing even HTML output explicitly, but because HTML pages are so small, any gain is likely to be minimal, and, of course, comes with its own disadvantages.

Note. once output has been explicitly flushed, a web developer will no longer be able to modify it, because it has already been sent to the client.

Should you flush HTML?

If you do decide to implement a type of flushing, it is recommended to flush right after central points in your code. Traditionally you might flush right after </head>, since the browser can then fetch resources listed in the head while waiting to download the rest of the page.

If you link external CSS files within the body, flushing right after each link element might also work. I have not tested this personally, and keep in mind it might be hampered by client-buffering.

Flushing will result in the server sending transfer-encoding: chunked header, which is mutually exclusive with content-length: x.

You could also simply use HTTP/2 Server Push to send resources to the client early, so it might not be worth the effort needed to get flushing to work. If you use a MVC pattern, where you try to keep HTML separated from your PHP, then it might also be difficult. You would have to split the HTML template into several files to get it to work. I.e. One file for head and another for the body.

Compression

It can be difficult to get compression to work for HTML while also relying on flush, but it is indeed possible; I managed to get it to work with a setup using deflate and brotli with Apache, PHP8.0, and PHP-FPM. You just need to disable output_buffering, and configure php-fpm with flushpackets (more on this later).

Keep in mind that HTML pages are often very small, so you could also consider to just disable compression for small pages, and instead leverage the benefits of flushing without compressing the page.

The important thing to keep in mind with these things is that there is no one-size-fits-all; you really have to think about your setup, and analyze what works the best, as well as how to implement it. Unfortunately, a CMS is not necessarily optimized fully for content delivery.

Flushing early is not going to yield huge improvements. It does help if you have a lot of slow SQL queries, but then again, you might need to optimize the queries too.

Ajax vs Flushing

If your end-goal is to have a page that loads content dynamically, or rather, displays content as it is loaded, then it sometimes makes more sense to leverage JavaScript to do so-called AJAX requests.

Note. AJAX stands for Asynchronous JavaScript And XML, but it actually has little to do with XML, since it also works for other mime types.

This technique works by sending a HTTP get- or post request to a script located on your back-end. You can even use Beamtic's HTTP client script to do this.

Buffering on the Server-side

While buffering can have multiple levels inside of PHP, your server can also use buffering on multiple levels.

In order for flushing to work, you will need to disable buffering on the server side.

Usually you will only have to disable or change output_buffering in PHP's configuration files, but there might also be buffering going on other places. If you use PHP-FPM instead of mod_php, then you will need to enable flushpackets in the configuration of php-fpm. Follow the steps below:

1. To make deflate work with flushing, you need to first put DeflateBufferSize 100 in your VHOST, outside of the directory block. Brotli does not seem to need this, however.

Typically you also have brotli enabled — which is a better compression than deflate and gzip — and in that case we will need to disable gzip when the client tells us that it supports brotli:

DeflateBufferSize 100

BrotliCompressionQuality 11
<If "%{HTTP:Accept-Encoding} =~ /br/">
  SetEnv no-gzip 1
  AddOutputFilterByType BROTLI_COMPRESS text/html text/plain text/xml text/css text/javascript application/javascript
</If>

Note that Accept-Encoding is a request header sent by the client; in the case of Google Chrome, this will contain: gzip, deflate, br; I am not sure why Chrome sends them in that order, but unfortunately it appears to cause Apache to deliver gzip, even when brotli is supported. The If block above will solve that problem.

Apache automatically loads its own deflate configuration, often located at: /etc/apache2/mods-enabled/deflate.conf, so there should rarely be any need to make changes to this. It will work as the fall-back when brotli is not requested by a client.

2. If you use PHP-FPM, you will also need to enable flushpackets; on Ubuntu this is done inside /etc/apache2/conf-enabled/php8.0-fpm.conf:

<Proxy fcgi://localhost>
    ProxySet flushpackets=on
</Proxy>

If the section is missing, you can go ahead and add it.

A full and working file might look like this:

# Redirect to local php-fpm if mod_php is not available
<IfModule !mod_php8.c>
<IfModule proxy_fcgi_module>
    # Enable http authorization headers
    <IfModule setenvif_module>
    SetEnvIfNoCase ^Authorization$ "(.+)" HTTP_AUTHORIZATION=$1
    </IfModule>

    <FilesMatch ".+\.ph(ar|p|tml)$">
        SetHandler "proxy:unix:/run/php/php8.0-fpm.sock|fcgi://localhost"
    </FilesMatch>
    <Proxy fcgi://localhost>
      ProxySet flushpackets=on
    </Proxy>
# The default configuration works for most of the installation, however it could
# be improved in various ways. One simple improvement is to not pass files that
# doesn't exist to the handler as shown below, for more configuration examples
# see https://wiki.apache.org/httpd/PHP-FPM
#    <FilesMatch ".+\.ph(ar|p|tml)$">
#        <If "-f %{REQUEST_FILENAME}">
#            SetHandler "proxy:unix:/run/php/php8.0-fpm.sock|fcgi://localhost"
#        </If>
#    </FilesMatch>
    <FilesMatch ".+\.phps$">
        # Deny access to raw php sources by default
        # To re-enable it's recommended to enable access to the files
        # only in specific virtual host or directory
        Require all denied
    </FilesMatch>
    # Deny access to files without filename (e.g. '.php')
    <FilesMatch "^\.ph(ar|p|ps|tml)$">
        Require all denied
    </FilesMatch>
</IfModule>
</IfModule>