Using the Big List of Naughty Strings to Find Errors
Most quality assurance teams can quickly run through a test plan that involves clicking a series of buttons, but real-world applications often involve dynamic user inputs. From cross-site scripting attacks to ambiguous error messages, it's important to test a wide range of user inputs to ensure applications behave properly in nearly every scenario.
Let's take a look at how the Big List of Naughty Strings can help you cover many edge cases, as well as some best practices to keep in mind when testing user inputs.
What Is the BLNS?
The Big List of Naughty Strings is an evolving list of strings that are likely to cause issues when used as dynamic user inputs. The list is designed to help quality assurance teams ensure that an application properly sanitizes and displays user inputs, as well as displays helpful error messages.
Examples of these inputs include:
-
Emoji characters - Emojis are becoming increasingly popular and should be handled the same as two-byte characters.
-
Numeric strings -
1e+02,1.0/0.0,--1,NaN,0x0, and other numeric strings should be treated as strings, not numbers. -
Special characters - ASCII punctuation should be escaped in some contexts and divided into groups based on keyboards.
-
Script injection - User inputs should be sanitized to remove malicious client-side scripts.
-
Server code injection - User inputs should be sanitized to remove server-side scripts.
-
Not-so-profane strings - Super Bowl XXX, Magna Cum Laude or Shitake Mushrooms shouldn't trigger profanity filters.
It's important to note that these inputs may be malicious in nature (e.g. script injection) or benign (e.g. emoji characters). Malicious inputs should be sanitized and/or blocked, whereas benign user inputs that cause errors should generate helpful error messages.
How to Use It
The Big List of Naughty Strings is designed for both manual and automated quality assurance processes.
Manual quality assurance testers can use the text file, blns.txt, which consists of one naughty string per line and comments that are preceded with the pound symbol (#). The comments divide the strings into easy-to-understand sections for copying and pasting into input forms.
Test engineers can use the JSON file, blns.json, which provides an array with all of the comments stripped out. The file can be easily incorporated into a wide range of test frameworks, such as Selenium, to test user inputs programmatically in a fraction of the time.
If you need to customize the format of the JSON file, the project also contains the Python script that's used to generate the JSON file. You can use these modifications to change the formatting of the JSON file itself or add or remove different parts of it to suit your exact requirements.
You can also add your own strings that could cause issues. For example, applications using geolocation functionality could experience issues with city names, like "Montréal", "Cañon City", "Lindström" or "Llanfairpwllgwyngyllgogerychwyrndrobwllllantysiliogogogoch" (the longest official one-word place name in Europe).
Selenium Example
Selenium makes it easy to automatically loop through the Big List of Naughty Strings to test various user inputs. These tests can be run as part of a continuous integration process with each new code commit to ensuring that there aren't any regressions introduced into the codebase.
Let's take a look at a simple example in Ruby:
require 'json'
require 'selenium-webdriver'
file = File.read('blns.json')
blns = JSON.parse(file)
blns.each do |word|
driver = Selenium::WebDriver.for :chrome
driver.navigate.to 'https://example.com/posts/new'
element = driver.find_element(class: 'body')
element.send_keys word
element.submit
driver.quit
end
Automatic User Inputs
There are some user inputs that aren't explicitly submitted by a user but could still contain malicious data. For instance, a malicious user could modify a cookie or session to gain unauthorized access to resources, or simply cause problems with the server's handling of the request.
Manual QA testers can access cookies in their browsers' Developer Tools to see what happens if the values are manipulated. When it comes to automated tests, Selenium provides easy access to read and write cookies using driver.manage().getCookies() and the FileWriter Class and Cookies.data file.
Another form of automatic user input is their location. Often, applications request a user's location via the browser or approximate it using IP addresses. These tests may require manual testers to use unreliable proxy servers, or need test engineers to create complex Selenium scripts.
WonderProxy simplifies the process for both manual testers and test automation engineers. Its easy-to-use browser plugin makes testing locations as easy as flipping a toggle, and even lets you test a browser-defined location. Meanwhile, its built-in integration with SauceLabs makes Selenium tests a breeze.
WonderProxy’s Browser Plugin
Explore WonderProxy's plans today to learn more.
Addressing the Issues
The quality assurance team has used the Big List to identify potential problems, but how can developers go about fixing them? By understanding how these problems are addressed, quality assurance teams can better diagnose potential issues and write bug reports.
The first line of defense for sanitizing user input is client-side, or browser-based, validation. Using JavaScript, developers can validate and sanitize input before it's even sent to the server where it could cause problems. These validations are best for non-malicious, accidental cases.
The second line of defense for sanitizing user input is server-side validation. Since malicious users can bypass client-side validation, server-side validation is critical to ensuring that no malicious code is executed or reaches the database. That way, it can't be executed to reveal sensitive data.
A final line of defense is sanitizing outputs. For instance, a malicious
The good news is that most client and server frameworks have built-in protections. For example, Ruby on Rails includes the sanitize helper to sanitize any outputs and server-side validations to ensure that various user inputs conform to the application's expectations.
The Bottom Line
Dynamic user inputs can be challenging for manual and automated quality assurance testers. In addition to closing security loopholes, testers must ensure that incidental user inputs don't cause unexpected errors or issues with the application. Fortunately, the BLNS makes these tests easier.
In addition to dynamic user inputs, quality assurance processes must ensure that there are no issues with automatic inputs from a user's browser, such as cookie or location data. WonderProxy and Selenium's built-in tools can help address these issues and cover every base.
