Custom request headers with cURL in PHP

Setting custom HTTP Headers with cURL is useful when changing User Agent or Cookies. Headers can be changed two ways, both using the curl_setopt function.

75234 views

By. Jacob

Edited: 2021-04-12 12:29

Custom request headers can also be defined when using cURL from PHP; they are useful when you want to change things like the user-agent string, include a referer header, and when you want to support for cookies when performing HTTP requests.

Custom request headers may be defined for all HTTP request types. E.g.:

GET
POST
HEAD
PUT
DELETE
PATCH

In PHP you can set custom request headers with the CURLOPT_HTTPHEADER option — once this is enabled, you may include custom headers by providing the headers as an indexed array.

Example:

$request_headers = []; // Prepare a new array
$request_headers[] = 'user-agent: Just For Fun HTTP Client';
$request_headers[] = 'referer: https://beamtic.com/';

// Prepare the HTTP request
$ch = curl_init("https://beamtic.com/api/request-headers");

// Provide the request headers as an indexed array
curl_setopt($ch, CURLOPT_HTTPHEADER, $request_headers);

// Performs the Request, with specified curl_setopt() options (if any).
$response_body = curl_exec($ch);

Note. In some cases this may also be refereed to as header spoofing — although "spoofing" is a term that should be used only to describe the act of faking headers. PHP can be used both for sending legitimate headers. and for header spoofing. E.g. providing a fake user-agent string.

Adding custom request headers

The CURLOPT_HTTPHEADER option is used in combination with the curl_setopt function to add custom request headers when using the cURL library in PHP. Individual headers can be provided as elements in an indexed array, which is provided as a parameter to the cul_setopt function.

Here is a full example that shows how to set the user-agent and accept request headers:

// URL to fetch
$url = "https://beamtic.com/api/request-headers";

// Setting the HTTP Request Headers
$user_agent = 'Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.43 Safari/537.31';

$request_headers = []; // Prepare a new array
$request_headers[] = 'user-agent: '. $user_agent;
$request_headers[] = 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8';

// Prepare the HTTP request
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HTTPHEADER, $request_headers);

// Performs the Request, with specified curl_setopt() options (if any).
$response_body = curl_exec($ch);

The $response_body variable contains the downloaded content of the resource, also known as the "response body"; you can try to output it directly to the browser with echo — just be sure to match the mime-type of the resource (I.e. text/html).

User agent lists:

Content negotiation

Request headers are used in the content negotiation between the client and the server.

For example, a web browser may use the accept-encoding header to inform a server about which encoding it prefers; this would often be either gzip or brotli compression, where brotli offers the highest compression. When a client has included the header in a HTTP request it might look like this:

Accept-Encoding: gzip, deflate, br

In this case the client is informing the server that it supports gzip, deflate, and brotli (br).

To send the accept-encoding header with cURL you can use the CURLOPT_ENCODING:

curl_setopt($ch, CURLOPT_ENCODING, "br, gzip");

Another request header that is used in content negotiation , that some websites rely on, is the accept-language header; this might be used to request a specific translation of the UI or content of a website:

Accept-Language: en-US,en;q=0.9,da;q=0.8

The q next to each language is called a relative quality factor and is used to indicated the preference by the client, but this is highly inaccurate, since users rarely seem to change their browser defaults. To send the accept-language with cURL, use the following:

$request_headers[] = 'accept-language: en-US,en;q=0.9';
curl_setopt($ch, CURLOPT_HTTPHEADER, $request_headers);

Note. You can not send all headers by using dedicated options, instead you need to use CURLOPT_HTTPHEADER with an array containing the headers that you want to send. It also appears that the ENCODING option does not support the brotli format, so you might need to manually de-compress brotli encoded responses.

Setting cookies with cURL

When you want to use Cookies on a website, you will need to include them in the cURL headers; there is a few ways to add request headers, one is to

The cookie field contains all cookies for a site, so if there is more than one cookie, they should be separated by a semicolon (;).

$cookies = 'CookieName1=Value;CookieName2=Value';
$request_headers[] = 'cookie: '. $cookies;

Note. Cookie Names must be unique.

Using curl_setopt to set headers

Some headers can be set with dedicated options using the curl_setopt function; the main benefit of using options to set headers is that certain things will be handled automatically E.g.: decoding of content that has been compressed using gzip or brotli.

It does not matter much whether you use curl_setopt or CURLOPT_HTTPHEADER, but the former may be easier for some people to use — sometimes you might even use both of them.

Both the User-agent and Cookie headers can be set using the CURLOPT_USERAGENT and CURLOPT_COOKIE options respectively; you may find this easier than adding headers manually.

If you have both a cookie and a user-agent string stored in variables, as shown below:

$User_Agent = 'Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.43 Safari/537.31';
$cookies = 'CookieName1=Value;CookieName2=Value';

We can simply feed them directly via curl_setopt:

curl_setopt($ch, CURLOPT_USERAGENT, $User_Agent);
curl_setopt($ch, CURLOPT_COOKIE, $cookies);

Compared to using an array, this may be easier, and slightly lower the risk of typos.

Tools:

You can use the following API endpoints for testing purposes:

https://beamtic.com/api/user-agent

https://beamtic.com/api/request-headers