How to set a user agent for your web application

Web servers and APIs often block or rate-limit requests that arrive without a user-agent string. This happens because missing user agents are a common trait of poorly configured scripts and automated abuse. Setting a descriptive user-agent string in your application tells the server what is making the request, and stops those requests being flagged.

The steps below show how to override the user agent in curl, wget and PHP. You will also find guidance on formatting your string correctly so it is accepted by servers and proxies.

Format your user-agent string

Your user-agent string does not need to follow a rigid format, but a well-formed string reduces the chance of it being rejected or flagged as suspicious. Use the pattern AppName/Version as a starting point.

Keep these points in mind when writing your string:

• Use only letters, numbers and basic symbols such as hyphens, forward slashes and underscores.
• Include a full version number, for example 1.2.0 rather than just 1.
• Keep the string under 256 characters.
• Do not include HTML tags, encoded characters such as %2F, profanity or personally identifying information.
• If your application is a bot or crawler, include a URL where server administrators can find information about it.

A string like MyApp-Monitor/1.2.0 is clear, concise and accepted by the vast majority of servers.

Override the user agent in curl

By default, curl sends its own user-agent string, but scripts that call curl without any flags may send an empty or stripped value depending on how they are invoked. The -A flag sets the user-agent header explicitly for that request.

Replace the plain request with the version below:

curl -A "YourSiteName-Automation/1.0" https://yoursite.com/endpoint

Your request now includes a user-agent header that identifies your application to the server.

Override the user agent in wget

wget uses the --user-agent flag to set the header. This works the same way as the -A flag in curl and applies to the request made in that command.

wget --user-agent="YourSiteName-Automation/1.0" https://yoursite.com/endpoint

Set a user agent in PHP

PHP provides two common ways to make HTTP requests: file_get_contents with a stream context, and the cURL extension. Both support setting a custom user-agent header.

If you are using file_get_contents, pass the user-agent value through a stream context:

$context = stream_context_create([
    'http' => ['user_agent' => 'YourSiteName-Automation/1.0']
]);
$response = file_get_contents('https://yoursite.com/endpoint', false, $context);

If you are using the PHP cURL extension, set the option on your cURL handle before executing the request:

curl_setopt($ch, CURLOPT_USERAGENT, 'YourSiteName-Automation/1.0');

Once either option is in place, PHP sends the user-agent header with every request made through that context or handle.

Troubleshooting

Requests are still being blocked after setting a user agent

If your requests continue to be rate-limited or rejected, the user-agent string may not be reaching the server as expected. This can happen if the string is being overridden elsewhere in your code, or if the tool wrapping your script strips custom headers.

• Confirm the header is being sent by running curl -v -A "YourSiteName-Automation/1.0" https://yoursite.com/endpoint and checking the output for the User-Agent line in the request headers.
• Check whether another part of your script or a middleware layer is overwriting the header after you set it.
• Verify the string does not contain encoded characters, HTML tags or other patterns that may cause it to be rejected.
• If you are using a third-party plugin or tool, locate the user-agent or request header setting in its configuration rather than setting it at the command level.

PHP cURL handle not defined

The curl_setopt call requires an active cURL handle created with curl_init(). If $ch is not defined before the curl_setopt line, PHP will throw a warning and the option will not be applied.

• Confirm $ch = curl_init('https://yoursite.com/endpoint'); appears before any curl_setopt calls in your script.
• Check that the PHP cURL extension is enabled on your server. You can confirm this by reviewing your phpinfo output.

Wrapping up

You have set a user-agent string for your application using curl, wget or PHP. Requests from your script now identify themselves to the server, which prevents them from being caught by rate limits that target anonymous or unidentified traffic.

If your application makes requests across multiple scripts or services, apply the same user-agent string consistently so all traffic from your application is identifiable. For further guidance on formatting, see the user-agent string best practices guide from WhatIsMyBrowser. If you need to check which PHP version your hosting account is running, see our guide on changing your PHP version in cPanel.