Selenium is NOT a test automation tool

Selenium WebDriver (often referred to just as Selenium) is a very popular tool among web testers and developers. Its popularity has led to the misconception that Selenium is the only tool needed in a UI test suite. Selenium is not a stand-alone test automation tool, but it can be a valuable part of an overall UI Automation solution. With that in mind, let’s look at what a full UI Automation solution can look like.

Overview

When we build a UI Automation solution, we’re tackling some problem or set of problems that are important enough to spend valuable resources on. Each problem or set of problems has a set of tools that help us mitigate risks. I like to take these potential risks and make them my testing mission.

A common mission I’ve been tasked with is: does the development team ensure the business can continue to collect our customer’s money with each change we make? Situations in which we can’t collect a customer’s money present an actual monetary loss to the business. Targeting this kind of test at the UI level makes sense because there are many pieces and systems that have to work together in order for a sale or order to be successful. Inevitably once we answer this initial question the next question is one of coverage: can we collect our customers’ money on Safari, Firefox, and Internet Explorer?

Common Components

From this mission we can begin to structure a solution that might solve our problems. I like to start with foundational components such as:

• Test Framework. Sometimes called a “Test Automation Framework”, this is where your tests are written and stored, where you’ll implement your design patterns, use object oriented design principles, and bring in libraries to extend your functionality. There are so many frameworks out there, you’ll have lots of choices depending on your preferred programming language and the style you want to write your tests in (such as BDD or ATDD). Popular frameworks include RSpec, Capybara, and WATIR for Ruby and WATIN and Nunit for .NET, to name a few.
• Test Runners. Many times the test runner is built into the test framework and there is no distinction, while on occasion you might have a separate runner that handles the execution of your tests such as with Karma for JavaScript.

Specific Libraries & APIs

Diving deeper into our mission from above, we know we’ll need to support multiple browsers and interact with our web application like a normal user would, filling out forms for the purchase of services or goods. This means we’ll need some specific libraries as part of our solution:

• Selenium WebDriver. Using the WebDriver protocol, Selenium’s API talks to the individual browser(s) on behalf of our tests through our test framework. We don’t write our tests in Selenium but through this library it can mimic the user interactions we need and help us fill out our forms and purchase goods in real browsers.
• Test Data Generators. Another important part of our solution will be generating useful data that we can input into shipping and billing fields that is correct but also not real.

Environment(s)

Since we’ve decided to use Selenium to run our tests in real browsers, we now have to consider where our automated tests will run. What will that environment look like? Do we plan on using our local desktop? Or run them on a separate service running virtual machines, in a headless AWS instance, in a Docker container, or within a CI platform? Each one of these different environments involves additional configurations and possibly new libraries to solve some of the challenges.

A Real Life Example

Working at BloomNation, I took on a similar mission to the one mentioned above. One of the biggest risks to our platform is being unable to collect customers’ money for a variety of eCommerce transactions. In years past, we had UI automation, but it had grown so old that no one could update it anymore. Once I joined, I took on the task of re-establishing it so we could get back that fast feedback and reduce risk.

I took our existing test framework, MochaJS, and extended it to be able to able to run tests in the UI using a library called WebDriver.io, a JavaScript version (bindings) of Selenium WebDriver. Writing tests in JavaScript was new to me but I knew it had value to the greater organization. Now our tests run regularly in a single AWS EC2 instance using a headless version of Chrome.

Here’s a high level recap of the solution we chose for our UI Automation:

• Test Framework: MochaJS
• Test Runner: WDIO, the built-in test runner for WebDriverIO
• Design Patterns: Page Object Pattern to keep a clean separation between test code and page specific code such as locators.
• APIs & Libraries:
  • WebDriverIO for the Selenium WebDriver bindings
  • Chai for more granular assertions,
  • Faker for generating some data,
  • Moment for doing date-time conversion,
  • npmJS as the package manager to keep version numbers in sync.
• Environment: AWS EC2 (a virtual server)
• Browser Configuration: Chrome in a Docker container

One Part of the Solution

As you can see, Selenium itself is not a test automation tool but it is one part in our overall UI Automation solution: Selenium talks to the browser(s) on behalf of our tests through our selected test framework. Anyone who contributes code to an existing UI framework or is thinking about how to develop their own should be able to speak to this. Knowing that Selenium is one part of the overall solution means you can better plan for things down the road and not make the mistake of thinking it’s a complete test automation tool.