A Step-by-Step Guide to Setting Up a Proxy in Selenium

Proxy servers can be a hassle to set up and manage, but they’re one of the most reliable ways to test localization. By automating these tests with Selenium, you can test hundreds of different locations in just minutes compared to the many hours of manual quality assurance testing that would be required to test every single location for multiple features.

In this guide, we will take a look at how to set up a proxy server in Selenium using Python and the Chrome WebDriver, since Python’s syntax is relatively easy to read and the Chrome WebDriver is the most common option.

Testing with Proxy Servers

Proxy servers are an invaluable tool for localization testing. For example, suppose that you run an ecommerce website and want to ensure that the proper language and currency appears for users from a certain country. Or, you may operate a sports goods retailer and want to show certain promotions to certain states based on their home sports teams.

Rather than assuming that things are showing properly, you can access the website as a user would from a target location. Quality assurance testers can see how the user experience looks, while test engineers can incorporate proxy servers into automated tests to verify functionality over time. Both of these approaches are critical to success.

Selenium is the most popular tool for browser automation. When writing automated tests, developers can use Selenium to test how a browser behaves without opening and running a full browser instance. You ensure that everything is working without having to put in hundreds of hours of manual quality assurance testing!

Setting Up a Basic Proxy

Many free proxy servers are unauthenticated, which means that a username and password are not required. In other cases, you may be able to whitelist your IP address on a premium proxy service, such as WonderProxy, to avoid authentication. This is the best option for most development teams since it simplifies the setup and execution of integration tests.

You can set up an unauthenticated proxy server in Selenium in just five steps:

from selenium import webdriver

PROXY = "12.345.678.910:8080"

chrome_options = WebDriverWait.ChromeOptions()
chrome_options.add_argument('--proxy-server=%s' % PROXY)

chrome = webdriver.Chrome(chrome_options=chrome_options)
chrome.get("https://www.google.com")

1. Import WebDriver from the Selenium package.
2. Define the proxy server (IP:PORT or HOST:PORT).
3. Set ChromeOptions() to a variable.
4. Add the proxy server argument to the options.
5. Add the options to the Chrome() instance.

You can use this Chrome WebDriver instance to execute tests that incorporate the proxy server. For example, the following code snippet tests to ensure that a search field shows the user’s current city as the default location.

def testUserLocationDenver(self):
	self.chrome.get(self.url)
	search = self.chrome.find_element_by_id('user-city')
	self.assertIn('Denver', search.text)

If you want to test multiple locations and/or make this code reusable across separate tests, you can define a method that takes the proxy IP address as an argument.

Using an Authenticated Proxy

Authenticated proxy servers can be challenging to use in automated tests since there’s no built-in way to pass along proxy server credentials in Selenium. As of today, there are two options to handle authenticated proxies and the right choice depends on your individual requirements (e.g. your desired versions of Selenium and the headless browser you prefer to use in tests).

Let’s take a look at these two options in greater detail and explore reasons to consider each approach.

The Easy Way

The easiest way to integrate authenticated proxies with Selenium is by using PhantomJS as a headless browser instead of the Chrome WebDriver. In order to do this, you must use Selenium 3.7.1 since PhantomJS is deprecated and newer versions of Selenium do not support it. The good news is that the features are largely the same and it might be right for your situation.

After setting up Selenium and PhantomJS, you can create a proxied method to create the WebDriver:

def proxied(self, proxy):
    capabilities = DesiredCapabilities.PHANTOMJS.copy()
    capabilities['phantomjs.cli.args'] = [
        '--proxy=' + proxy,
        '--proxy-type=http',
        '--proxy-auth=' + evar.get('PROXY_USER') + ':' + evar.get('PROXY_PASS')
    ]

    return webdriver.Remote(
        command_executor=self.selenium,
        desired_capabilities=capabilities
    )

Note: evar retrieves environmental variables that are set with the username and password for the proxy. This ensures that any tests checked into source control don’t expose sensitive username and password credentials.

You can then run a test using the new method:

def testUserLocationAlbuquerque(self):
    self.driver = self.proxied('albuquerque.wonderproxy.com:11000')
    self.driver.get(self.url)
    search = self.driver.find_element_by_id('user-city')
    self.assertIn('Albuquerque', search.text)

The Hard Way

The second option is adding a browser extension that does the authentication for Selenium, as detailed on this Stack Overflow answer. While this approach is more complex, you can use it with the latest version of Selenium, which may be a requirement for some development teams.

The first step is creating a Chrome extension by including two files in an archive, named proxy.zip:

background.js

var config = {
    mode: "fixed_servers",
    rules: {
      singleProxy: {
        scheme: "http",
        host: "YOUR_PROXY_ADDRESS",
        port: parseInt(YOUR_PROXY_PORT)
      },
      bypassList: ["foobar.com"]
    }
  };

chrome.proxy.settings.set({value: config, scope: "regular"}, function() {});

function callbackFn(details) {
    return {
        authCredentials: {
            username: "YOUR_PROXY_USERNAME",
            password: "YOUR_PROXY_PASSWORD"
        }
    };
}

chrome.webRequest.onAuthRequired.addListener(
        callbackFn,
        {urls: ["<all_urls>"]},
        ['blocking']
);

manifest.js

{
    "version": "1.0.0",
    "manifest_version": 2,
    "name": "Chrome Proxy",
    "permissions": [
        "proxy",
        "tabs",
        "unlimitedStorage",
        "storage",
        "<all_urls>",
        "webRequest",
        "webRequestBlocking"
    ],
    "background": {
        "scripts": ["background.js"]
    },
    "minimum_chrome_version":"22.0.0"
}

The Chrome extension can be added to Selenium using the add_extension method:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

chrome_options = Options()
chrome_options.add_extension("proxy.zip")

driver = webdriver.Chrome(executable_path='chromedriver.exe', chrome_options=chrome_options)
driver.get("http://google.com")
driver.close()

The example above only uses a single proxy server in the extension. If you'd like to add more proxy servers, you would need to make further modifications to the chrome.proxy API. If you're using a CI/CD server, you will also need to be sure that the build machine has Chrome installed and extension added.

The good news is that the setup process only needs to be done once and the tests should be easy to automate in the future. The bad news is that adding an extension to Chrome may add some time to the automated test process. You may also need to keep the extension up-to-date over time.

The Bottom Line

Testing automation with proxy servers can be challenging, but the process is necessary for software that relies on localization. In most cases, the easiest solution is using a premium proxy, such as WonderProxy, with whitelisted IP addresses, but if you’re using authenticated proxies, there are several techniques you can use to get set up.

Another option to consider is Sauce Labs, which enables development teams to test applications on more than 1,000 combinations of browsers, operating systems, and devices. When combined with WonderProxy’s servers around the world, it’s the ultimate way to ensure that your application works properly in nearly every environment imaginable.

Sign up for WonderProxy to see how easy it is to get started with reliable localization testing today. With over 250 proxy servers in the U.S. and around the world, you can ensure that your application is functioning properly everywhere!