Playwright vs Puppeteer: A tester's perspective

Google and Microsoft, the companies behind the Chrome and Edge browsers, have each launched their own next-generation browser automation initiatives: Puppeteer and Playwright. Here's a quick comparison:

Which is which?

Puppeteer is a Node.js library from the Chrome DevTools team. It is a browser-driver framework that lets programmers interact with a web browser as an object, calling methods such as .goto() or .type(). Puppeteer does not bundle its own testing framework, but it works within generic JavaScript test frameworks like Jest or Mocha. Even though Puppeteer was built by Chrome developers for Chromium, both Firefox and Microsoft Edge have limited support.

Playwright is maintained by Microsoft. It is designed to work with all modern browsers, in both headed (with the browser graphical interface) and headless (without the browser graphical interface). (There are subtle differences in timing and behavior between the two modes, so tests developed for both may be slightly more robust.) Playwright supports the Node.js platform, the .NET platform, Python, and Java. Playwright does come with its own test framework and even has an extension to create videos.

Both Puppeteer and Playwright can run through a cloud-based "grid" for parallel processing. Sauce Labs, the de facto computing grid standard for browser testing, has documentation for both Playwright and Puppeteer. However, if you click on Supported Platforms for Puppeteer, you'll see that Puppeteer support is "coming soon" – the only Sauce Labs support is through your own Docker container. Sauce will provide you the images, but you'll have to run them, either locally or in your own cloud.

The bottom line will be your goals. If you want programming language flexibility and support from third-party services like Sauce Labs, or you find that website bugs vary by browser, then Playwright is the clear choice. Companies that work primarily in JavaScript and focus on Chrome, or find that the bugs that matter occur in all browsers, might choose the lighter-weight Puppeteer.

If things are a little more gray, then it's time to start a proof of concept. To decide that, we'll dive into the details. Specifically, is the code fit for use, and are there any hidden features or gotchas to find earlier rather than later.

Our test case: A login page

Browsers are increasingly trying to figure out if events were done by a real person or a tool to prevent things like screen scraping. Puppeteer and Playwright both simulate your browser's "trusted" event system, so the website you're testing won't register an automated process. Both tools can also read internal browser events, like page or element loads. That means no sleep statements, no waiting 500 milliseconds for a link to exist to click, no guessing if 100 milliseconds will be enough or 500 will be wasteful, and no polling. Here is sample code using Puppeteer, a simple test asserting an error message given bad login credentials:

const should = require('chai').should();
const puppeteer = require('puppeteer');
 
describe('Puppeteer vs Playwright tests - Login', function () {
  this.timeout(20000);  // increase Mocha's default timeout
 
  it('should be present and error message when given bad credentials', async function () {
 
    // Launch browser
    const browser = await puppeteer.launch({headless: false});
 
    // Find the first tab
    const pages = await browser.pages();
    const page = pages[0];
 
    // Navigate to the Wonder Proxy login page
    await page.goto('https://wonderproxy.com/login');
 
    // Fill in fields and click login button
    await page.type('#user-username', 'fake-user');
    await page.type('#user-password', 'wrong-password');
    await page.click('#login-form > button');
 
    // Wait for message
    await page.waitForSelector('#login-form > p');
 
    // Capture message content
    const errorMessageHandle = await page.$('#login-form > p');
    const errorMessageContent = await errorMessageHandle.evaluate(node => node.innerHTML);
 
    await browser.close();
 
    // Assert message content
    errorMessageContent.should.contain('There was a problem with your username and/or password.');
  });
});

Here's the Playwright version:

const { test, expect } = require('@playwright/test');

test('Puppeteer vs Playwright tests - Login', async ({ page }) => {
  // Navigate to the Wonder Proxy login page
  await page.goto('https://wonderproxy.com/login');

  // Fill in fields and click login button
  await page.fill('#user-username', 'fake-user');
  await page.fill('#user-password', 'wrong-password');
  await page.click('#login-form > button');

  // Get locator
  const errorMessageLocator = await page.innerText('#login-form > p');

  // Assert message content
  expect(errorMessageLocator).toEqual(expect.stringContaining('There was a problem with your username and/or password.'));
});

Playwright checks for actionality before interacting with an object, saving the author from having to manually check that the element being tested exists (notice that the Puppeteer example must make a call to waitForSelector() while Playwright does not). Playwright can also deal with pop-ups and pop-on-new-tabs, making it capable of multi-page work in the same browser. Playwright has an emulator that can take X,Y coordinates to emulate a browser at that pixel resolution. It can handle file downloads and uploads, and mock out geolocation.

By now you should have enough information to decide how to get started. We'd love to hear comments about where you end up, and how you got there.