Setting Up a PHP Proxy Client & Server with ProxyTee

When connecting to a network, an IP address serves as a unique identifier, facilitating communication and identification within that network and across the internet. A proxy server acts as an intermediary between a user’s device and the internet, improving security, privacy, and performance by managing requests and concealing the user’s IP address.

Proxy servers ensure anonymity and help filter out threats by masking your IP and controlling website access. When it comes to web scraping specifically, proxy servers play a crucial role, helping users circumvent IP bans and avoid geoblocking measures.

In this tutorial, you’ll explore how to set up a proxy server for web scraping in PHP, leveraging ProxyTee, which provides affordable, reliable, and easy-to-use rotating residential proxies.

ProxyTee offers solutions with unlimited bandwidth, a vast pool of IP addresses, and easy-to-integrate tools. With features like unlimited bandwidth, global IP coverage, multiple protocol support, auto-rotation, and simple API integration, it's a great option for businesses and individuals involved in tasks like web scraping, streaming, or data gathering.

Using a Proxy Server in PHP

If you want to set up a proxy server, there are various ways to do it. You can use server applications like Apache or Nginx as forward proxy servers, or you can opt for dedicated proxy tools like Squid. In this tutorial, you’ll use Apache as the proxy server for demonstration. For an enhanced and more reliable experience, consider using ProxyTee.

To set up a proxy server for web scraping in PHP, you need to make sure PHP is installed on your system. If you haven’t installed it yet, follow the official documentation to do so. This article uses Ubuntu as the operating system.

Set Up a Proxy Server with Apache

In Apache, the mod_proxy, mod_proxy_http, and mod_proxy_connect modules can be used to make Apache act as a proxy server.

To set up the proxy server, you need to start by enabling the modules:

sudo a2enmod proxy
sudo a2enmod proxy_http
sudo a2enmod proxy_connect

Then you need to create a new VirtualHost file where you’ll define the proxy server:

cd /etc/apache2/sites-available/
sudo cp 000-default.conf proxy.conf

In the proxy.conf file, paste the following code:

<VirtualHost *:80>
    ServerName localhost
    ServerAdmin admin@localhost
    <IfModule mod_ssl.c>
            SSLEngine off
    </IfModule>

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

    ProxyRequests On
    ProxyVia On
    <Proxy *>
        Order deny,allow
        Allow from all
    </Proxy>
</VirtualHost>

In this code, ProxyRequests On enables Apache to be a forward proxy server. ProxyVia On adds a Via header that helps inspect the request path in a chain of proxies. Additionally, the <Proxy> control block declares who is allowed to access the proxy. Any host is allowed to access the proxy server.

For more detailed information, check out the official docs.

To finish setting up the proxy server, enable the VirtualHost:

sudo a2ensite proxy.conf
service apache2 reload

Using the Proxy Server

In the following section, you’ll learn how to use the proxy server in PHP code. Typically, any web request or automation framework provides a mechanism to set proxies. In this article, you’ll learn about three methods: curl, file_get_contents, and Symfony BrowserKit. However, it's crucial to remember that managing your own proxy servers can be complex. That's why using ProxyTee's robust Residential Proxy network could greatly simplify your web scraping process.

Set Proxies in curl

The first method you’ll learn about shows you how to integrate proxies into your PHP script using the curl library, a powerful tool for making HTTP requests.

Create a file named curl.php with the following code:

<?php

$proxyUrl = 'http://localhost:80';
$targetUrl = 'https://httpbin.org/get';

$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, $targetUrl);
curl_setopt($ch, CURLOPT_PROXY, $proxyUrl);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);

$response = curl_exec($ch);

if (curl_errno($ch)) {
    echo 'cURL Error: ' . curl_error($ch);
} else {
    echo $response;
}

curl_close($ch);

This script initiates a curl session, sets the target URL and proxy URL using curl_setopt, and executes the session to retrieve the response. The CURLOPT_PROXY option sets the proxy. The proxy URL is set to localhost:80, which leverages the proxy you created in the previous step.

For simplicity, the script also includes options to disable SSL verification. However, make sure you exercise caution in a production environment. Any curl errors encountered during the execution are displayed for debugging purposes.

Run the script with the command php curl.php.

After executing the script, you can inspect the Apache access log file to track the request made to the proxy. The access log file is typically located at /var/log/httpd/access.log.

In this log file, you’ll find an entry using the CONNECT protocol corresponding to the executed script, indicating that the request was successfully processed with an HTTP status code of 200:

127.0.0.1 - - [21/Dec/2024:12:34:56 +0530] "CONNECT httpbin.org:443 HTTP/1.1" 200 -

Set Proxies in file_get_contents

You can also use file_get_contents to set up proxies in PHP and then leverage the Apache proxy to fetch content.

To do so, create a file_get_contents.php file and add the following code:

<?php

$options = [
    'http' => [
        'proxy' => 'tcp://127.0.0.1:80',
        'request_fulluri' => true,
    ],
];
$context = stream_context_create($options);

$response = file_get_contents('https://httpbin.org/get', false, $context);

if ($response === false) {
    echo "Failed to retrieve data from $url";
} else {
    echo $response;
}

This PHP code initiates an HTTP request to retrieve the data from http://wikipedia.org using a specified proxy server. The proxy server details are configured in the $options array, where the ‘proxy’ key defines the server address (‘tcp://127.0.0.1:80’) and request_fulluri is set to true, treating the request URI as a full URI.

A stream context adding these options is created using stream_context_create. The actual HTTP request is made with file_get_contents, adding the created context and resulting in the retrieval of content from HttpBin.

Set Proxies in Symfony BrowserKit

If you’re a Symfony user, integrating proxies is easy with BrowserKit.

If you want to follow along and don’t already have Symfony installed, make sure you install it now.

Once Symfony is set up, you can easily use proxies in your PHP code:

<?php

require './vendor/autoload.php';

use Symfony\Component\BrowserKit\HttpBrowser;
use Symfony\Component\HttpClient\HttpClient;

$proxyServer = 'http://127.0.0.1';
$proxyPort = '80';

$client = new HttpBrowser(HttpClient::create(['proxy' => sprintf('%s:%s', $proxyServer, $proxyPort)]));

$client->request('GET', 'https://httpbin.org/get');

$content = $client->getResponse()->getContent();

echo $content;

This code block uses Symfony BrowserKit and HttpClient to make a GET request to HttpBin through the Apache proxy configured at $proxyUrl. Then the response content is retrieved and displayed. The HttpBrowser instance is set up with the specified proxy settings, making it easy to integrate proxies into Symfony applications.

Limitations of Traditional Proxy Integration Methods

Although configuring proxies in PHP might seem straightforward, the various methods come with certain limitations. For instance, while it’s possible to switch between different proxies manually, this is an inefficient approach when scraping large amounts of data.

Additionally, if the proxy becomes unavailable or the IP becomes restricted or encounters issues, these methods don’t inherently provide a mechanism to switch to an alternative proxy automatically. This results in service interruptions or data retrieval failures requiring manual intervention. Although you could write code to switch proxies automatically, it’s a complicated process, and you’ll have to set up or rent multiple proxy servers across different geographic regions for maximum security.

ProxyTee: A Superior Proxy Solution

The good news is ProxyTee can help. ProxyTee is a provider of rotating residential proxies designed to support a wide range of internet activities. Its features include:

• Unlimited bandwidth, eliminating concerns about data overages.
• Global IP Coverage with over 20 million IP addresses from more than 100 countries.
• Multiple Protocol Support: Supports both HTTP and SOCKS5 protocols.
• User-Friendly Interface: Easy to use platform with minimal setup time.
• Auto Rotation, allowing IP addresses to change automatically, customizable from 3 to 60 minutes.
• Simple API: Seamless integration with various applications.

One of the main advantages of the ProxyTee solution is the automated proxy rotation, which contributes to enhanced anonymity and mitigates detection risks during web scraping activities. ProxyTee offers different types of residential proxies and datacenter proxies that ensure you have the proxy that best suits your use case.

Using ProxyTee with PHP

Using ProxyTee is easy. All you have to do is sign up for an account at ProxyTee.com.

With your ProxyTee details, you can use the ProxyTee proxy to scrape Wikipedia. Start by creating a scrape.php file and adding the following code:

<?php

// ProxyTee proxy details
$proxyUrl = 'Your-proxy-url-from-the access-paramaters';
$proxyUser = 'Your-username:Your-password';

$targetUrl = 'https://en.wikipedia.org/wiki/Proxy_server';

$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, $targetUrl);
curl_setopt($ch, CURLOPT_PROXY, $proxyUrl);
curl_setopt($ch, CURLOPT_PROXYUSERPWD, $proxyUser);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);

$response = curl_exec($ch);

if (curl_errno($ch)) {
    echo 'cURL Error: ' . curl_error($ch);
} else {
    $dom = new DOMDocument();
    @$dom->loadHTML($response);

    $content = $dom->getElementById('mw-content-text')->textContent;

    $headings = [];
    $headingsNodeList = $dom->getElementsByTagName('h2');
    foreach ($headingsNodeList as $heading) {
        $headings[] = $heading->textContent;
    }

    $headingsNodeList = $dom->getElementsByTagName('h3');
    foreach ($headingsNodeList as $heading) {
        $headings[] = $heading->textContent;
    }

    echo "Content:\n";
    echo $content . "\n\n";

    echo "Headings:\n";
    foreach ($headings as $index => $heading) {
        echo ($index + 1) . ". $heading\n";
    }
}

curl_close($ch);

Make sure you replace Your-proxy-url-from-the access-paramaters, Your-username, and Your-password with your specific credentials from your ProxyTee account.

In this code, the proxy details are integrated into the curl request to ensure that the web scraping is done through the ProxyTee network.

Conclusion

In this tutorial, you explored various methods for setting up a proxy server in PHP. While local proxy servers with Apache are an option, leveraging the ProxyTee proxy servers introduces a new level of efficiency and versatility. With ProxyTee, you get access to a diverse range of IPs, including residential proxies and datacenter proxies.

Whether you’re a novice seeking simplicity or an experienced developer requiring advanced capabilities, the ProxyTee proxy servers provide a robust foundation for successful and efficient web scraping endeavors. Check out our pricing today to find the best option for you.