Mastering Web Scraping with Playwright and ProxyTee in 2026

Web scraping has become essential for various industries, from monitoring market trends to extracting product information. The automation it provides goes beyond mere time-saving; it allows for real-time data-driven decisions that would be difficult to obtain through manual collection.

A key aspect of this process involves utilizing reliable tools, and in this article, we'll explore how ProxyTee can enhance the capabilities of Playwright for web scraping. Playwright, known for its efficient browser automation, will be the focal point as we navigate practical examples. ProxyTee's ability to provide secure and fast proxies can enable seamless scraping experiences by allowing for IP rotation and bypassing of anti-scraping mechanisms.

ProxyTee Overview:

ProxyTee provides an affordable, reliable, and easy-to-use solution for anyone needing rotating residential proxies. Its features—such as unlimited bandwidth, a global IP pool, protocol flexibility, auto-rotation, and API integration—make it a great option for businesses and individuals involved in tasks like web scraping, streaming, or data gathering. With a focus on user-friendly design and competitive pricing, ProxyTee delivers strong value for those looking for effective proxy services.

Key Features of ProxyTee

1. Unlimited Bandwidth
   ProxyTee provides proxies with unlimited bandwidth, eliminating concerns about data overages during high-traffic tasks like web scraping or streaming. This feature ensures users can perform data-intensive operations without worrying about additional costs.
2. Global IP Coverage
   With over 20 million IP addresses from more than 100 countries, ProxyTee ensures that users have access to a wide range of geographic locations. This makes it an ideal solution for businesses and individuals targeting specific regions or performing location-based tasks.
3. Multiple Protocol Support
   ProxyTee supports both HTTP and SOCKS5 protocols, ensuring compatibility with a variety of applications and tools. This flexibility allows users to handle tasks like web scraping, bypassing geo-blocks, and more with ease.
4. User-Friendly Interface
   ProxyTee prioritizes simplicity with a clean, intuitive graphical user interface (GUI). This easy-to-use platform allows users to get started quickly, with minimal setup time and no technical expertise required.
5. Auto Rotation
   The auto-rotation feature allows IP addresses to change automatically at intervals of 3 to 60 minutes. This is particularly valuable for web scraping, where frequent IP changes prevent detection and bans from target websites. The rotation interval can be customized to suit different needs.
6. API Integration
   ProxyTee offers a simple API that enables seamless integration with various applications and workflows. This API supports all the service features, making it an ideal choice for developers and businesses looking to automate their proxy-related tasks.

What is Playwright?

Playwright is an automation framework capable of interacting with web browsers. Through code, you can automate tasks such as navigation, text input, button clicks, and data extraction. One of its most powerful features is its ability to manage multiple pages simultaneously without any performance drop. Additionally, by simulating web requests via a browser, Playwright enables the bypassing of CAPTCHAs, enhancing the reliability of automated interactions. Playwright supports major browsers like Google Chrome, Microsoft Edge (Chromium-based), and Firefox and uses WebKit for Safari. This provides a platform for creating and running cross-browser automated testing with the same script. Playwright also supports various languages such as Node.js, Python, Java, and .NET.

Playwright provides comprehensive documentation covering everything from beginner tutorials to intricate explanations of all classes and methods. To grasp Playwright’s full potential, it is suggested to delve deeper into its concepts. Let's get started with Playwright with examples in JavaScript (Node.js) and Python. If you are just getting started with web scraping, our post on how to use proxies might be helpful to get more understanding. If you're new to scraping with JavaScript, you can find great starting points with many comprehensive tutorials.

Basic Web Scraping with Playwright

Let’s begin with how to get Playwright up and running using both Node.js and Python. For Node.js, you’ll need to start a new project and add the Playwright library, using these commands:

npm init -y
npm install playwright

A basic script example in JavaScript to open a dynamic page can look like this:

const playwright = require('playwright');

(async () => {
  for (const browserType of ['chromium', 'firefox', 'webkit']) {
    const browser = await playwright[browserType].launch();
    const context = await browser.newContext();
    const page = await context.newPage();
    await page.goto("https://amazon.com");
    await page.screenshot({ path: `nodejs_${browserType}.png`, fullPage: true });
    await page.waitForTimeout(1000);
    await browser.close();
  };
})();

In the above script, Playwright is imported and multiple browser instances are launched, automating Chromium, Firefox, and Webkit. A new page is opened, navigating to the specified URL using the page.goto() function and a screenshot is saved, with a 1-second delay before the browser closes.

Here’s the same code translated into Python, requiring you to install the library with:

python -m pip install playwright
playwright install

Playwright has synchronous and asynchronous APIs, the following example is asynchronous:

from playwright.async_api import async_playwright
import asyncio

async def main():
  browsers = ['chromium', 'firefox', 'webkit']
  async with async_playwright() as p:
    for browser_type in browsers:
      browser = await p[browser_type].launch()
      page = await browser.new_page()
      await page.goto('https://amazon.com')
      await page.screenshot(path=f'py_{browser_type}.png', full_page=True)
      await page.wait_for_timeout(1000)
      await browser.close()

asyncio.run(main())

The main difference from the Node.js code is using the asyncio library and using snake_case instead of camelCase for function names. In both examples, headless browser instances of the specified browsers are opened. Additionally, if you desire more control, you may handle multiple browser contexts.

Locating Elements

Before any data can be extracted or actions can be taken, you must be able to identify HTML elements on a page. Playwright supports both CSS and XPath selectors. Let’s take a real-world example with an Amazon product page https://www.amazon.com/b?node=17938598011. This page has products under a section with div elements with classes a-section and a-spacing-base.

To target these, we will loop over all div elements. To select div elements by CSS, use:

.s-card-container > .a-spacing-base

For XPath, you could use this alternative:

//*[contains(@class, "s-card-container")]/*[contains(@class, "a-spacing-base")]

Playwright provides these methods to use selectors:

• $eval(selector, function): Selects the first element, sends it to the provided function, and returns the function's result.
• $$eval(selector, function): Does the same as $eval but selects all the elements.
• querySelector(selector): returns the first matching element.
• querySelectorAll(selector): returns a list of matching elements.

These methods work with both CSS and XPath selectors. Playwright locators can be used to further simplify interaction with elements using more reliable actions such as typing, clicking, and waiting for element visibility and state changes.

Scraping Text

Using the example of an Amazon product page from earlier, we'll extract the product names by selecting elements with the $$eval function, which returns an array of elements that you can then loop over:

const products = await page.$$eval('.s-card-container > .a-spacing-base', all_products => {
  // run a loop here
});

all_products.forEach(product => {
    const title = product.querySelector('.a-size-base-plus').innerText;
})

To retrieve data, you can access element properties like innerText, which returns the textual content. The code for a full text scraping implementation in Node.js can look like this:

const playwright = require('playwright');

(async () => {
    const launchOptions = {
        headless: false,
        proxy: {
            server: 'http://us-pr.oxylabs.io:10000',
            username: 'USERNAME',
            password: 'password'
        }
    };

    const browser = await playwright.chromium.launch(launchOptions);
    const page = await browser.newPage();
    await page.goto('https://www.amazon.com/b?node=17938598011');
    await page.waitForTimeout(5000);

    const products = await page.$$eval('.s-card-container > .a-spacing-base', all_products => {
        const data = [];
        all_products.forEach(product => {
            const titleEl = product.querySelector('.a-size-base-plus');
            const title = titleEl ? titleEl.innerText : null;
            const priceEl = product.querySelector('.a-price');
            const price = priceEl ? priceEl.innerText : null;
            const ratingEl = product.querySelector('.a-icon-alt');
            const rating = ratingEl ? ratingEl.innerText : null;
            data.push({ title, price, rating });
        });
        return data;
    });

    console.log(products);
    await browser.close();
})();

The corresponding script in Python slightly differs in that it uses query_selector and query_selector_all. Below is the complete example:

import asyncio
from playwright.async_api import async_playwright


async def main():
    async with async_playwright() as pw:
        browser = await pw.chromium.launch(
            headless=False,
            proxy={
                'server': 'http://us-pr.oxylabs.io:10000',
                'username': 'USERNAME',
                'password': 'password'
            }
        )
        page = await browser.new_page()
        await page.goto('https://www.amazon.com/b?node=17938598011')
        await page.wait_for_timeout(5000)

        all_products = await page.query_selector_all('.s-card-container > .a-spacing-base')

        data = []
        for product in all_products:
            result = dict()
            title_el = await product.query_selector('.a-size-base-plus')
            result['title'] = await title_el.inner_text() if title_el else None
            price_el = await product.query_selector('.a-price')
            result['price'] = await price_el.inner_text() if price_el else None
            rating_el = await product.query_selector('.a-icon-alt')
            result['rating'] = await rating_el.inner_text() if rating_el else None
            data.append(result)

        print(data)
        await browser.close()

if __name__ == '__main__':
    asyncio.run(main())

The data that both Python and Javascript examples outputs will be the same, so choosing one comes down to familiarity. Feel free to explore additional code examples in our GitHub repository

Scraping Images

It is important to be aware of potential legal implications before scraping data. Ensure you consult your legal advisors and respect terms of service and applicable licenses on target websites. When extracting images using Playwright, you will make use of external packages in JavaScript (Node.js) to perform downloads.

Node.js

You'll be needing the 'https' and 'fs' libraries, which allow us to make HTTP requests and interact with the file system.

const playwright = require('playwright');
const https = require('https');
const fs = require('fs');

(async () => {
  const launchOptions = {
    headless: false,
    proxy: {
      server: 'http://pr.oxylabs.io:7777',
      username: 'USERNAME',
      password: 'password'
    }
  };

  const browser = await playwright.chromium.launch(launchOptions);
  const page = await browser.newPage();
  await page.goto('https://sandbox.oxylabs.io/products');
  await page.waitForTimeout(5000);

  const images = await page.$$eval('img', all_images => {
    const image_links = [];
    all_images.forEach((image, index) => {
      if (image.src.startsWith('https://')) {
        image_links.push(image.src);
      }
    });
    return image_links;
  });

  images.forEach((imageUrl, index) => {
    const path = `image_${index}.svg`;
    const file = fs.createWriteStream(path);
    https.get(imageUrl, function (response) {
      response.pipe(file);
    });
  });

  console.log(images);
  await browser.close();
})();

The script navigates to a target webpage, extracts the image elements, filters to secure URL, constructs image name using the index and downloads each by creating file streams, and piping the response to file.

Python

Python's native capabilities for handling file operations and the use of the 'requests' library for http requests simplifies image scraping:

from playwright.async_api import async_playwright
import asyncio
import requests

async def main():
    async with async_playwright() as pw:
        browser = await pw.chromium.launch(
            headless=False,
            proxy={
                'server': 'http://pr.oxylabs.io:7777',
                'username': 'USERNAME',
                'password': 'password'
            }
        )
        page = await browser.new_page()
        await page.goto('https://sandbox.oxylabs.io/products')
        await page.wait_for_timeout(5000)

        all_images = await page.query_selector_all('img')

        images = []
        for i, img in enumerate(all_images):
            image_url = await img.get_attribute('src')
            if not image_url.startswith('data:'):
                content = requests.get('https://sandbox.oxylabs.io/' + image_url).content
                with open(f'py_{i}.svg', 'wb') as f:
                    f.write(content)
                images.append(image_url)

        print(images)
        await browser.close()

if __name__ == '__main__':
    asyncio.run(main())

The code fetches image elements, obtains URLs using getAttribute('src') and skips the base64 encoded ones, makes GET requests for each, and stores them to the directory using index.

Intercepting HTTP Requests with Playwright

With Playwright you can use methods for intercepting HTTP requests that helps customize headers and modify response outputs. You could implement this to skip the loading of images, speed up the web scraping or for debugging, let us take a look at the example implementation:

Python

With this method we use a function handle_route to catch the HTTP requests, retrieve the body content, modify the title and set the header as 'content-type': 'text/html' while blocking all the images from loading.

from playwright.async_api import async_playwright, Route
import asyncio

async def handle_route(route: Route) -> None:
    response = await route.fetch()
    body = await response.body()
    body_decoded = body.decode('latin-1')
    body_decoded = body_decoded.replace('', '<title>Modified Response')
    await route.fulfill(
        response=response,
        body=body_decoded.encode(),
        headers={**response.headers, 'content-type': 'text/html'},
    )

async def main():
    async with async_playwright() as pw:
        browser = await pw.chromium.launch(
            proxy={
                'server': 'http://pr.oxylabs.io:7777',
                'username': 'USERNAME',
                'password': 'password'
            },
            headless=False
        )
        page = await browser.new_page()
        # abort image loading
        await page.route('**/*.{png,jpg,jpeg,svg}', lambda route: route.abort())
        await page.route('**/*', handle_route)
        await page.goto('https://sandbox.oxylabs.io/products')
        await page.wait_for_timeout(5000)
        await browser.close()

if __name__ == '__main__':
    asyncio.run(main())</code></pre><p>The page.route intercepts the requests using a regex to match URL extensions (.png, .jpg, .jpeg and .svg) and calling the abort to stop them from loading. The handle_route intercepts the rest of the traffic, making the needed response updates to title and content type.</p><h3 id="nodejs-1">Node.js</h3><p>Below is the Node.js version to match the python example above:</p><pre><code>const playwright = require('playwright');

(async () => {
    const launchOptions = {
        headless: false,
        proxy: {
            server: 'http://pr.oxylabs.io:7777',
            username: 'USERNAME',
            password: 'password'
        }
    };
    const browser = await playwright.chromium.launch(launchOptions);
    const context = await browser.newContext();
    const page = await context.newPage();

    await page.route(/(png|jpeg|jpg|svg)$/, route => route.abort());
    await page.route('**/*', async route => {
        const response = await route.fetch();
        let body = await response.text();
        body = body.replace('<title>', '<title>Modified Response: ');
        route.fulfill({
            response,
            body,
            headers: {
                ...response.headers(),
                'content-type': 'text/html'
            }
        });
    });

    await page.goto('https://sandbox.oxylabs.io/products');
    await page.waitForTimeout(5000)
    await browser.close();
})();</code></pre><p>The `page.route` matches on the requested URL. The images loading is blocked using `route.abort()`. Then a separate  `route()` modifies the request’s title and headers, providing a very fast page load and an enhanced browsing experience.</p><h2 id="support-for-proxies-in-playwright">Support for Proxies in Playwright</h2><p>Playwright makes it possible to make proxy use seamless. Below are examples to add proxy using different browsers: </p><h3 id="nodejs-2">Node.js</h3><pre><code>const { chromium } = require('playwright');

const browser = await chromium.launch();</code></pre><p>With only small changes can the proxy parameters be provided, `LauchOptions` objects with `proxy` parameter can set `server,` `username`, `password` etc..: </p><pre><code>const playwright = require('playwright');

(async () => {
    for (const browserType of ['chromium', 'firefox', 'webkit']) {
        const launchOptions = {
            headless: false,
            proxy: {
                server: 'http://pr.oxylabs.io:7777',
                username: 'USERNAME',
                password: 'password'
            }
        };
        const browser = await playwright[browserType].launch(launchOptions);
        const page = await browser.newPage();
        await page.goto('https://ip.oxylabs.io/');
        await page.waitForTimeout(1000);
        await browser.close();
    };
})();
</code></pre><h3 id="python-2">Python</h3><p>In Python you only have to specify the parameter as key-value pairs in the launch method, with the below example showing: </p><pre><code>from playwright.async_api import async_playwright
import asyncio

async def main():
    browsers = ['chromium', 'firefox', 'webkit']
    async with async_playwright() as p:
        for browser_type in browsers:
            browser = await p[browser_type].launch(
                headless=False,
                proxy={
                    'server': 'http://pr.oxylabs.io:7777',
                    'username': 'USERNAME',
                    'password': 'password'
                }
            )
            page = await browser.new_page()
            await page.goto('https://ip.oxylabs.io/')
            await page.wait_for_timeout(1000)
            await browser.close()

asyncio.run(main())</code></pre><p>For maximum security and reliability with anti-scraping measures, using <a href="/products/unlimited-residential-proxies/">residential proxies</a> will always be ideal. <a href="/">ProxyTee</a>'s residential proxies offer extensive geographic coverage with high security.</p><h2 id="playwright-vs-puppeteer-and-selenium">Playwright vs Puppeteer and Selenium</h2><p>Other tools that you may see out there to automate tasks is Puppeteer and Selenium, here's a breakdown of their differences and where Playwright stands among them:</p><ul><li>Puppeteer primarily focuses on Chromium and uses Javascript while Playwright provides wider support, with cross-browser compatibility and different languages support, and Selenium that supports all major browsers, has a more cumbersome development experience with performance that lags behind Puppeteer and Playwright. </li></ul><p>Here's a table for a quick comparison:</p><table><thead><tr><th></th><th>PLAYWRIGHT</th><th>PUPPETEER</th><th>SELENIUM</th></tr></thead><tbody><tr><td>SPEED</td><td>Fast</td><td>Fast</td><td>Slower</td></tr><tr><td>DOCUMENTATION</td><td>Excellent</td><td>Excellent</td><td>Fair</td></tr><tr><td>DEVELOPER EXPERIENCE</td><td>Best</td><td>Good</td><td>Fair</td></tr><tr><td>PROGRAMMING LANGUAGES</td><td>JavaScript, Python, C#, Java</td><td>JavaScript</td><td>Java, Python, C#, Ruby, Javascript, Kotlin</td></tr><tr><td>BACKED BY</td><td>Microsoft</td><td>Google</td><td>Community and Sponsors</td></tr><tr><td>COMMUNITY</td><td>Small but active</td><td>Large and active</td><td>Large and active</td></tr><tr><td>BROWSER SUPPORT</td><td>Chromium, Firefox, and WebKit</td><td>Chromium</td><td>Chrome, Firefox, IE, Edge, Opera, Safari, and more</td></tr></tbody></table><p>Puppeteer and Playwright shows similar performance on Chromium when using Javascript. <a href="https://github.com">Detailed analysis</a> between performance of the different tools are out of scope of this article. Puppeteer often performs the best, while Selenium generally the slowest.  </p><p>It's important to remember that Playwright’s strengths include cross-browser support, along with multiple language support. So Playwright becomes your number one choice if cross-browser compatibility is your priority and your desired programming language is anything but Javascript.</p><h2 id="conclusion">Conclusion</h2><p>Playwright is the tool that web scrapers use to scrape dynamic rendered pages thanks to it's asynchronous nature and its cross browser capabilities. In summary, you can use Playwright for text extraction, actions like entering data and clicking buttons, and accessing dynamic elements. Although, there are alternatives like Puppeteer and Selenium. When your use case demands working with multiple browsers, and a preference in language aside from Javascript, then Playwright is your only go to tool.</p><p>Make sure to learn to <a href="/">configure proxies</a> properly with Playwright to prevent being detected and getting banned. </p><p>Other interesting posts are the guides about: <a href="/solutions/">web scraping use-cases</a>, the comparison of  <a href="/compare/">alternatives of proxies providers</a>. There are many exciting fields that web scraping covers with lots of exciting fields waiting for you to dig deeper!</p><h2 id="frequently-asked-questions">Frequently asked questions</h2><h3 id="is-playwright-good-for-web-scraping">Is Playwright good for web scraping?</h3><p>Playwright offers capabilities needed to extract data smoothly. Its deployment, execution is fast and you are enabled to do various actions with little to no manual input from humans. Multiple browsers and programming languages makes it more applicable for different use cases.</p><p>Playwright might be slightly more resource intensive compared to other solutions because of the heavy browser component in the background. On the upside, it excells in handling JavaScript-heavy websites.</p><h3 id="can-playwright-be-detected">Can Playwright be detected?</h3><p>Unfortunately yes, however implementing <a href="/">proxies</a> along with web scraping tools like an API, can greatly reduce the rate you are detected for suspicious behaviour while web scraping.</p><h3 id="does-playwright-have-a-ui">Does Playwright have a UI?</h3><p>Playwright runs in headless browser mode by default. Meaning there’s no GUI. To interact, you must run it through the CLI.</p></body>