How to Build a Zalando Scraper with Python Using ProxyTee

Zalando is a leading online fashion platform in Europe, offering a vast array of products from various brands. For businesses and researchers, extracting data from Zalando can provide invaluable insights into market trends, pricing strategies, and brand popularity. In this guide, we’ll show you how to create a Zalando scraper using Python and ProxyTee's powerful rotating residential proxies to ensure seamless and reliable data collection.

Why Scrape Product Details Data from Zalando?

Web scraping Zalando can provide several key benefits:

• Market Research: Gain a deep understanding of current fashion trends, helping you make data-driven decisions and tailor your product offerings.
• Price Monitoring: Track price fluctuations, identify competitive deals, and monitor market pricing dynamics to optimize your strategies.
• Brand Popularity Analysis: Identify popular products and brands, understand customer preferences, and study successful strategies of market leaders.

With the right tools and approach, scraping Zalando can offer a wealth of information for both businesses and individual users.

Libraries and Tools for Scraping Zalando

Zalando’s dynamic site structure, rendered using JavaScript, requires specialized scraping tools. Here’s what you need:

• Selenium: A browser automation tool capable of running JavaScript. It simulates user interactions with a web page, enabling you to extract dynamically loaded content.
• Python: With its ease of use and vast ecosystem, Python is the perfect language for web scraping.
• ProxyTee's Residential Proxies: To avoid detection and blocks while web scraping, you need a rotating residential proxy service, offering IP rotation and anonymity. ProxyTee provides a pool of 20 million+ IPs across 100+ countries to ensure you get the data you need reliably.

Let's dive into building a Zalando scraper with ProxyTee.

Scraping Product Data from Zalando With Selenium

Follow these steps to build a robust Zalando scraper:

Step 1: Setup a Python Project

Ensure you have the following installed on your system:

• Python 3+: Download the installer from the official Python website.
• Python IDE: Use any IDE of your preference such as PyCharm Community Edition or Visual Studio Code.

Now create a new project and set up your environment:

mkdir zalando-scraper
cd zalando-scraper
python -m venv env

Activate the virtual environment (use either the Linux/macOS or Windows command):

# For Linux or macOS
./env/bin/activate
# For Windows
env\Scripts\activate.ps1

Create a file named scraper.py in your project folder, and test the setup with a simple print statement:

print("Hello, World!")

Execute the script to check your environment:

python scraper.py

If you see "Hello, World!" then you're all set for the next step.

Step 2: Install the Scraping Libraries

Use pip to install Selenium, remember to install or update to latest version with:

pip install selenium -U

Now let's configure a basic Selenium scraper:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service

# set up a controllable Chrome instance
service = Service()
options = webdriver.ChromeOptions()

# browser options
# options.add_argument('--headless=new') # uncomment if you don't want to show up browser, or use `options.add_argument("--headless")` for older Chrome version.
user_agent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36'
options.add_argument(f'user-agent={user_agent}')

driver = webdriver.Chrome(
    service=service,
    options=options
)

# Maximize window to avoid responsive layout.
driver.maximize_window()

# scraping logic here

# close browser and free resources.
driver.quit()

This script initializes a WebDriver instance for Chrome, which you can control programmatically. The headless option will open Chrome without UI if enabled and user-agent can bypass some site limitation for robots.

Step 3: Open the Target Page

For this example, we'll use a Zalando product page URL:

driver.get('https://www.zalando.co.uk/adidas-originals-3mc-trainers-footwear-whitegold-metallic-ad115o0da-a11.html')

The get() function directs the browser to the target URL.

Step 4: Familiarize Yourself with the Page Structure

Inspect the Zalando product page with your browser’s DevTools to understand the structure of the HTML elements. Pay attention to how the DOM changes during page interactions. Zalando often uses dynamically generated CSS classes, making them unreliable for element selection. Instead, look for elements with attributes like data-testid, and use auto-rotation provided by ProxyTee, which can help prevent you from being detected and blocked.

Step 5: Start Extracting the Product Data

Initialize a dictionary to hold the extracted data and begin selecting page elements with Selenium:

from selenium.webdriver.common.by import By

product = {}

brand_element = driver.find_element(By.CSS_SELECTOR, 'h3')
brand = brand_element.text
name_element = driver.find_element(By.CSS_SELECTOR, 'h1')
name = name_element.text

price_elements = name_element \
    .find_element(By.XPATH, 'following-sibling::*[1]') \
    .find_elements(By.TAG_NAME, "p")

discount = None
price = None
original_price = None

if len(price_elements) >= 3:
    discount = price_elements[0].text.replace(' off', '')
    price = price_elements[1].text
    original_price = price_elements[2].text

images = []
image_elements = driver.find_elements(By.CSS_SELECTOR, '[aria-label="Product media gallery"] li')

for image_element in image_elements:
    image = image_element.find_element(By.TAG_NAME, 'img').get_attribute('src')
    images.append(image)
    
colors = []
color_elements = driver.find_elements(By.CSS_SELECTOR, '[aria-label="Available colours"] li')
for color_element in color_elements:
    color = {
        'color': None,
        'image': None,
        'link': None
    }

    link_elements = color_element.find_elements(By.TAG_NAME, 'a')
    if len(link_elements) > 0:
        color['link'] = link_elements[0].get_attribute('href')
    image_elements = color_element.find_elements(By.TAG_NAME, 'img')
    if len(image_elements) > 0:
        color['image'] = image_elements[0].get_attribute('src')
        color['color'] = image_elements[0].get_attribute('alt') \
            .replace('Selected, ', '') \
            .replace('Unselected, ','') \
            .strip()
    colors.append(color)


`

Selenium provides the find_element() and find_elements() to search the DOM. The supported strategies include:

• By.CSS_SELECTOR: Use CSS selectors to select specific elements.
• By.TAG_NAME: Target elements based on their HTML tags.
• By.XPATH: Use XPath expressions to find elements in the DOM tree.

Step 6: Scrape the Product Details Data

Retrieve product details such as delivery information and product info:

delivery = {
    'time': None,
    'type': None,
    'cost': None,
}
delivery_elements = driver \
    .find_element(By.CSS_SELECTOR, '[data-testid="pdp-delivery-info"]') \
    .find_element(By.XPATH, 'parent::*[1]') \
    .find_elements(By.TAG_NAME, 'p')

if len(delivery_elements) == 3:
    delivery['time'] = delivery_elements[0].text
    delivery['type'] = delivery_elements[1].text
    delivery['cost'] = delivery_elements[2].text

info = {}
info_elements = driver.find_elements(By.CSS_SELECTOR, '[data-testid^="pdp-accordion-"]')[:2]

for info_element in info_elements:
    info_element.click()
    info_section_name = info_element.find_element(By.CSS_SELECTOR, 'h5').text
    info[info_section_name] = {}
    for dt_element in info_element.find_elements(By.CSS_SELECTOR, 'dt'):
        info_section_detail_name = dt_element.text.replace(':', '')
        info[info_section_name][info_section_detail_name] = dt_element.find_element(By.XPATH, 'following-sibling::dd').text

Step 7: Populate the Product Object

Add the scraped data into the product dictionary and log it to console to verify:

product['brand'] = brand
product['name'] = name
product['price'] = price
product['original_price'] = original_price
product['discount'] = discount
product['images'] = images
product['colors'] = colors
product['delivery'] = delivery
product['info'] = info

print(product)

Step 8: Export Scraped Data to JSON

To output the results to JSON format, utilize the built-in json module:

import json

with open('product.json', 'w', encoding='utf-8') as file:
    json.dump(product, file, indent=4, ensure_ascii=False)

Step 9: Put It All Together

This code brings all the individual steps together:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
import json

service = Service()

options = webdriver.ChromeOptions()
# options.add_argument('--headless=new')
user_agent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36'
options.add_argument(f'user-agent={user_agent}')

driver = webdriver.Chrome(
    service=service,
    options=options
)
driver.maximize_window()

driver.get('https://www.zalando.co.uk/adidas-originals-3mc-trainers-footwear-whitegold-metallic-ad115o0da-a11.html')

product = {}

brand_element = driver.find_element(By.CSS_SELECTOR, 'h3')
brand = brand_element.text
name_element = driver.find_element(By.CSS_SELECTOR, 'h1')
name = name_element.text

price_elements = name_element \
    .find_element(By.XPATH, 'following-sibling::*[1]') \
    .find_elements(By.TAG_NAME, "p")

discount = None
price = None
original_price = None

if len(price_elements) >= 3:
    discount = price_elements[0].text.replace(' off', '')
    price = price_elements[1].text
    original_price = price_elements[2].text

images = []
image_elements = driver.find_elements(By.CSS_SELECTOR, '[aria-label="Product media gallery"] li')

for image_element in image_elements:
    image = image_element.find_element(By.TAG_NAME, 'img').get_attribute('src')
    images.append(image)

colors = []
color_elements = driver.find_elements(By.CSS_SELECTOR, '[aria-label="Available colours"] li')
for color_element in color_elements:
    color = {
        'color': None,
        'image': None,
        'link': None
    }

    link_elements = color_element.find_elements(By.TAG_NAME, 'a')
    if len(link_elements) > 0:
        color['link'] = link_elements[0].get_attribute('href')
    image_elements = color_element.find_elements(By.TAG_NAME, 'img')
    if len(image_elements) > 0:
        color['image'] = image_elements[0].get_attribute('src')
        color['color'] = image_elements[0].get_attribute('alt') \
            .replace('Selected, ', '') \
            .replace('Unselected, ','') \
            .strip()
    colors.append(color)

delivery = {
    'time': None,
    'type': None,
    'cost': None,
}
delivery_elements = driver \
    .find_element(By.CSS_SELECTOR, '[data-testid="pdp-delivery-info"]') \
    .find_element(By.XPATH, 'parent::*[1]') \
    .find_elements(By.TAG_NAME, 'p')

if len(delivery_elements) == 3:
    delivery['time'] = delivery_elements[0].text
    delivery['type'] = delivery_elements[1].text
    delivery['cost'] = delivery_elements[2].text

info = {}
info_elements = driver.find_elements(By.CSS_SELECTOR, '[data-testid^="pdp-accordion-"]')[:2]

for info_element in info_elements:
    info_element.click()
    info_section_name = info_element.find_element(By.CSS_SELECTOR, 'h5').text
    info[info_section_name] = {}
    for dt_element in info_element.find_elements(By.CSS_SELECTOR, 'dt'):
        info_section_detail_name = dt_element.text.replace(':', '')
        info[info_section_name][info_section_detail_name] = dt_element.find_element(By.XPATH, 'following-sibling::dd').text

driver.quit()

product['brand'] = brand
product['name'] = name
product['price'] = price
product['original_price'] = original_price
product['discount'] = discount
product['images'] = images
product['colors'] = colors
product['delivery'] = delivery
product['info'] = info

print(product)

with open('product.json', 'w', encoding='utf-8') as file:
    json.dump(product, file, indent=4, ensure_ascii=False)

Execute this complete script to output the extracted product data into a JSON file.

Conclusion

In this tutorial, you’ve learned how to scrape product data from Zalando using Python and Selenium with ProxyTee. We highlighted several points:

• Dynamic Content: Zalando's JavaScript-heavy design needs an automated browser, like Selenium, to interact with the DOM.
• Anti-Scraping Measures: Zalando might employ anti-scraping measures. With ProxyTee's residential proxies, you can automatically rotate IPs, enhancing your ability to scrape reliably. Also, unlimited bandwidth will ensure that high-volume data is collected seamlessly.
• Robust Code: Avoid using generated CSS classes and try to select elements using attributes such as data-testid or other HTML attributes as much as possible.
• Product Variation: Each product has different HTML structures, make sure to test your scraper before production or handle more data case.

By following these steps, you can build a scalable and robust scraper with ProxyTee that provides access to high quality unlimited residential proxies with global coverage at an affordable price. ProxyTee offers also Datacenter Proxies, Static Residential Proxies and Mobile Proxies to adapt to your use cases.

Start leveraging the power of web data by making informed, data-driven decisions!