Boost Your Web Scraping with ProxyTee and SeleniumBase in 2026

In the dynamic world of web scraping, having the right tools is crucial. This guide explores how to leverage ProxyTee and SeleniumBase to create a robust and efficient scraping setup. We'll delve into why SeleniumBase is a fantastic choice for automation and how ProxyTee’s residential proxies enhance your scraping activities by providing the anonymity and rotation you need.

ProxyTee Overview

ProxyTee is a provider specializing in rotating residential proxies that support a wide array of internet tasks, including web scraping. Known for its budget-friendliness and robust functionality, ProxyTee is perfect for users seeking reliable and cost-effective proxy services. ProxyTee offers solutions with unlimited bandwidth, a vast pool of IP addresses, and user-friendly integration tools.

ProxyTee provides an affordable, reliable, and easy-to-use solution for anyone needing rotating residential proxies. Its features—such as unlimited bandwidth, a global IP pool, protocol flexibility, auto-rotation, and API integration—make it a great option for businesses and individuals involved in tasks like web scraping, streaming, or data gathering. With a focus on user-friendly design and competitive pricing, ProxyTee delivers strong value for those looking for effective proxy services.

Key Features of ProxyTee

Here are some key features that make ProxyTee an excellent choice for your web scraping needs:

1. Unlimited Bandwidth: ProxyTee offers proxies with unlimited bandwidth, eliminating worries about data overages during intensive tasks. You can perform data-heavy operations like web scraping or streaming without incurring extra charges.
2. Global IP Coverage: Access over 20 million IP addresses from more than 100 countries. ProxyTee is excellent for businesses and individuals who need to target specific regions for tasks like localization or gathering localized information.
3. Multiple Protocol Support: ProxyTee supports both HTTP and SOCKS5 protocols, ensuring compatibility with many different applications and tools. This makes it simple to handle tasks like web scraping and bypassing geographic restrictions.
4. User-Friendly Interface: ProxyTee’s intuitive graphical user interface (GUI) ensures a quick start with minimal setup and no technical expertise required.
5. Auto Rotation: IP addresses change automatically at customizable intervals from 3 to 60 minutes. This crucial feature helps prevent detection and bans from target websites during web scraping by making you look more like a natural user.
6. API Integration: ProxyTee’s simple API enables seamless integration with various applications and workflows. Ideal for developers and businesses, the API supports all service features, helping to automate your proxy-related tasks.

What Is SeleniumBase?

ProxyTee’s robust infrastructure combined with SeleniumBase makes web scraping more accessible and effective. SeleniumBase is a powerful Python framework that enhances the capabilities of Selenium/WebDriver. It offers an all-in-one solution for browser automation, making it a reliable option for scraping web pages, automating workflows, and performing many web-related operations.

SeleniumBase vs Selenium: Feature and API Comparison

Let's see a direct comparison of SeleniumBase and vanilla Selenium to see the reasons why it is widely adopted:

SeleniumBase: Features, Methods, and CLI Options

Let's delve into the standout features, practical methods, and customizable command-line interface options that make SeleniumBase a game changer.

Features

Here's a selection of notable SeleniumBase features:

• Includes a Recorder Mode to generate browser tests in Python quickly.
• Supports many browsers, tabs, iframes, and proxies in tests.
• Offers test case management with CasePlans using Markdown.
• Implements a smart waiting mechanism that increases reliability.
• Compatible with pytest, unittest, nose, and behave for test discovery and execution.
• Includes logging tools for dashboards, reports, and screenshots.
• Runs tests in Headless Mode, which hides the browser.
• Supports multithreaded test execution on parallel browsers.
• Allows tests to run with Chromium's mobile device emulator.
• Supports tests run using a proxy server, including authenticated proxies.
• Can modify the browser’s user-agent string for tests.
• Has advanced detection prevention for websites blocking automation.
• Integrates with selenium-wire for inspecting network requests.
• Offers a CLI for executing with specific configurations.
• Provides a global configuration file.
• Supports integration with services like GitHub Actions, Google Cloud, and Docker.
• Can execute JavaScript from Python.
• Can interact with Shadow DOM elements using ::shadow in CSS selectors.

Methods

Some of the most used SeleniumBase methods include:

• driver.open(url): Navigates the browser to the specified URL.
• driver.go_back(): Navigates back to the previous page.
• driver.type(selector, text): Types the specified text into an input.
• driver.click(selector): Clicks the identified element.
• driver.click_link(link_text): Clicks a link based on its visible text.
• driver.select_option_by_text(dropdown_selector, option): Selects a dropdown option using text.
• driver.hover_and_click(hover_selector, click_selector): Hovers and clicks another element.
• driver.drag_and_drop(drag_selector, drop_selector): Drags and drops an element.
• driver.get_text(selector): Gets the text of an element.
• driver.get_attribute(selector, attribute): Gets the specified attribute.
• driver.get_current_url(): Gets the URL of the current page.
• driver.get_page_source(): Gets the HTML source of the current page.
• driver.get_title(): Gets the title of the current page.
• driver.switch_to_frame(frame): Switches the driver focus to an iframe.
• driver.switch_to_default_content(): Moves the driver focus out of the iframe.
• driver.open_new_window(): Opens a new window.
• driver.switch_to_window(window): Changes to the specified window.
• driver.switch_to_default_window(): Returns focus to the main window.
• driver.get_new_driver(OPTIONS): Starts a new driver session with the options.
• driver.switch_to_driver(driver): Switches to the given driver.
• driver.switch_to_default_driver(): Returns to the original browser driver.
• driver.wait_for_element(selector): Waits until an element is visible.
• driver.is_element_visible(selector): Checks if an element is visible.
• driver.is_text_visible(text, selector): Checks if the specified text is visible within an element.
• driver.sleep(seconds): Pauses execution for a given amount of time.
• driver.save_screenshot(name): Saves a screenshot with the given name.
• driver.assert_element(selector): Checks if an element is visible.
• driver.assert_text(text, selector): Checks for specified text in the given element.
• driver.assert_exact_text(text, selector): Check for exact text in the given element.
• driver.assert_title(title): Verifies the page title.
• driver.assert_downloaded_file(file): Checks if a specific file has been downloaded.
• driver.assert_no_404_errors(): Checks that the page has no broken links.
• driver.assert_no_js_errors(): Checks that the page has no JS errors.

CLI Options

SeleniumBase extends pytest, bringing the following command-line options:

• --browser=BROWSER: Chooses the web browser (default: “chrome”).
• --chrome, --edge, --firefox, --safari: Browser shortcuts.
• --settings-file=FILE: Overrides settings.
• --env=ENV, --account=STR, --data=STRING, --var1=STRING,--var2=STRING, --var3=STRING, --variables=DICT: Enables setting extra data during tests.
• --proxy=SERVER:PORT: Uses a proxy server.
• --proxy=USERNAME:password@SERVER:PORT: Sets up an authenticated proxy server.
• --proxy-bypass-list=STRING: Hosts to exclude (e.g., “*.foo.com”).
• --proxy-pac-url=URL: Connect via PAC URL.
• --proxy-pac-url=USERNAME:password@URL: Authenticated proxy with PAC URL.
• --proxy-driver: Uses a proxy for the driver download.
• --multi-proxy: Allows the usage of multiple authenticated proxies in multi-threading scenarios.
• --agent=STRING: Allows modification to the user-agent string.
• --mobile: Enable mobile device emulator.
• --metrics=STRING: Enables you to set mobile metrics (e.g., “CSSWidth,CSSHeight,PixelRatio”).
• --chromium-arg="ARG=N,ARG2" and --firefox-arg="ARG=N,ARG2": Sets browser arguments.
• --firefox-pref=SET: Set Firefox preferences.
• --extension-zip=ZIP and --extension-dir=DIR: Sets the paths for extension to use
• --disable-features="F1,F2": Sets features to be disabled
• --binary-location=PATH: Sets the Chromium binary path.
• --driver-version=VER: Set a specific driver version.
• --headless: Run tests with the browser invisible by default.
• --headless1 and --headless2: Runs tests using different headless modes for Chrome
• --headed: Run the browser visible on Linux
• --xvfb: Run tests with Xvfb on Linux
• --locale=LOCALE_CODE: Set language locale for the browser.
• --reuse-session, --reuse-class-session and --crumbs: These options helps reuse the browser session between tests, and delete cookies
• --disable-cookies: Disables cookies.
• --disable-js: Disables JavaScript.
• --disable-csp: Disable Content Security Policy.
• --disable-ws and --enable-ws: Disables/enables Web Security.
• --log-cdp: Log Chrome DevTools Protocol (CDP) events.
• --remote-debug: Syncs with Chrome Remote Debugger.
• --visual-baseline: Used for visual regression testing.
• --timeout-multiplier=MULTIPLIER: Increases timeout values.

Using SeleniumBase for Web Scraping: Step-By-Step Guide

Here is an example using a target page from Quotes to Scrape to create a web scraper:

Step #1: Project Initialization

1. Ensure Python 3 is installed on your machine. If not, download it here
2. Create the project's directory:

mkdir seleniumbase-scraper

1. Navigate inside:

cd seleniumbase-scraper

1. Initialize the virtual environment:

python -m venv env

1. Create scraper.py in your folder.
2. Activate virtual env on linux:

./env/bin/activate

Or on Windows:

env/Scripts/activate

1. Install seleniumbase:

pip install seleniumbase

Step #2: SeleniumBase Test Setup

Use the SB syntax in the script:

from seleniumbase import SB

with SB() as sb:
    pass
    # Scraping logic...

Execute with:

python3 scraper.py

For headless mode:

python3 scraper.py --headless

Step #3: Connect to the Target Page

Use open() method to connect to target page:

sb.open("https://quotes.toscrape.com/")

Step #4: Select the Quote Elements

Use find_elements() method to select elements

quotes = []
quote_elements = sb.find_elements(".quote")

Step #5: Scrape Quote Data

Select each node and extract data from them with the text attribute:

text_element = quote_element.find_element(By.CSS_SELECTOR, ".text")
text = text_element.text.replace("\“", "").replace("\”", "")

author_element = quote_element.find_element(By.CSS_SELECTOR, ".author")
author = author_element.text

tags = []
tag_elements = quote_element.find_elements(By.CSS_SELECTOR, ".tag")
for tag_element in tag_elements:
  tag = tag_element.text
  tags.append(tag)

Import By at the begining of the script

from selenium.webdriver.common.by import By

Step #6: Populate the Quotes Array

quote = {
    "text": text,
    "author": author,
    "tags": tags
}
quotes.append(quote)

Step #7: Implement Crawling Logic

Use a while loop to scrape each page and click to the next

while sb.is_element_present(".next"):
    # Scraping logic...

    # Visit the next page
    sb.click(".next a")

Step #8: Export the Scraped Data

import csv

with open("quotes.csv", mode="w", newline="", encoding="utf-8") as file:
      writer = csv.DictWriter(file, fieldnames=["text", "author", "tags"])
      writer.writeheader()
      # Flatten the quote objects for CSV writing
      for quote in quotes:
          writer.writerow({
            "text": quote["text"],
            "author": quote["author"],
            "tags": ";".join(quote["tags"])
          })

Step #9: Put It All Together

The complete scraper's code should be like this:

from seleniumbase import SB
from selenium.webdriver.common.by import By
import csv

with SB() as sb:
    # Connect to the target page
    sb.open("https://quotes.toscrape.com/")

    # Where to store the scraped data
    quotes = []

    # Iterate over all quote pages
    while sb.is_element_present(".next"):
        # Select all quote elements on the page
        quote_elements = sb.find_elements(".quote")

        # Iterate over them and scrape data for each quote element
        for quote_element in quote_elements:
            # Data extraction logic
            text_element = quote_element.find_element(By.CSS_SELECTOR, ".text")
            text = text_element.text.replace("\“", "").replace("\”", "")

            author_element = quote_element.find_element(By.CSS_SELECTOR, ".author")
            author = author_element.text

            tags = []
            tag_elements = quote_element.find_elements(By.CSS_SELECTOR, ".tag")
            for tag_element in tag_elements:
              tag = tag_element.text
              tags.append(tag)

            # Populate a new quote object with the scraped data
            quote = {
                "text": text,
                "author": author,
                "tags": tags
            }
            # Add it to the list of scraped quotes
            quotes.append(quote)
        
        # Visit the next page
        sb.click(".next a")

    # Export the scraped data to CSV
    with open("quotes.csv", mode="w", newline="", encoding="utf-8") as file:
          writer = csv.DictWriter(file, fieldnames=["text", "author", "tags"])
          writer.writeheader()
          # Flatten the quote objects for CSV writing
          for quote in quotes:
              writer.writerow({
                "text": quote["text"],
                "author": quote["author"],
                "tags": ";".join(quote["tags"])
              })

Execute the script with:

python3 script.py --headless

Advanced SelenimBase Scraping Use Cases

After creating the simple web scraper using SelenuimBase, explore these more complex situations:

Automate Form Filling and Submission

SeleniumBase allows interactions with elements on a web page, which allows it to work like a user. Let's fill a form on a login page:

# login.py

from seleniumbase import BaseCase
BaseCase.main(__name__, __file__)


class LoginTest(BaseCase):
    def test_submit_login_form(self):
        # Visit the target page
        self.open("https://quotes.toscrape.com/login")

        # Fill out the form
        self.type("#username", "test")
        self.type("#password", "test")

        # Submit the form
        self.click("input[type=\"submit\"]")

        # Verify you are on the right page
        self.assert_text("Top Ten tags")

Launch with pytest login.py and a browser window should appear, automatically fill and submit the form, then it should check for an expected text to be available on the page.

Bypass Simple Anti-Bot Technologies

Anti-scraping measures are common on modern websites. ProxyTee helps you bypass these and SeleniumBase provide some interesting features such as UC Mode(Undetected-Chromedriver Mode). UC Mode makes scraping bots appear human-like, helping evade detection by anti-bot services.

With UC mode, you have:

• Automatic user-agent rotation to avoid detection.
• Automatic configuration of Chromium arguments.
• Specific methods to bypass CAPTCHAs.

Let's access an anti-bot page:

from seleniumbase import SB

with SB(uc=True) as sb:
    # Target page with anti-bot measures
    url = "https://www.scrapingcourse.com/antibot-challenge"

    # Open the URL using UC Mode with a reconnect time of 4 seconds to avoid initial detection
    sb.uc_open_with_reconnect(url, reconnect_time=4)

    # Attempt to bypass the CAPTCHA
    sb.uc_gui_click_captcha()

    # Take a screenshot of the page
    sb.save_screenshot("screenshot.png")

On the first execution PyAutoGUI will be automatically installed and with mouse automation, the challenge will be bypassed

Bypass Complex Anti-Bot Technologies

SeleniumBase provides CDP Mode which operates within UC Mode to work with the CDP-Driver, enabling your bot to appear more human-like and to bypass complex challenges:

from seleniumbase import SB

with SB(uc=True, test=True) as sb:
    # Target page with advanced anti-bot measures
    url = "https://gitlab.com/users/sign_in"
    # Visit the page in CDP Mode
    sb.activate_cdp_mode(url)

    # Handle the CAPTCHA
    sb.uc_gui_click_captcha()

    # Wait for 2 seconds for the page to reload and the driver to retake control
    sb.sleep(2)

    # Take a screenshot of the page
    sb.save_screenshot("screenshot.png")

Conclusion

SeleniumBase with ProxyTee provides an amazing setup for advanced and scalable web scraping needs. You saw the methods available and how to leverage this solution to solve simple and complex challenges in the current modern websites.

If you have very complex tasks for bypassing anti-bot mechanisms consider cloud-based solution like ProxyTee’s rotating residential proxies for even better results, with auto-rotation and global coverage.