To do basic information security reviews, we first need to get our testing environment set up. This includes preparing our Web browser, any mobile devices we will use to test, and installing some additional software on our local computer.
The tests in this document make extensive use of an intercepting proxy to observe, and in some cases modify, the network requests generated by the application under test. There are many proxies and tools that can be used, and the testing scenarios we describe are universal. However, to make the instructions in this primer as accessible as possible to as many people as possible, our instructions focus on tools that are freely available and -- ideally -- open source.
While the bulk of our tests rely on an intercepting proxy to power more detailed analysis, two other tools -- a cookie reader and a browser extension called Firebug -- are used as well. These two tools can often be used to do a quick, more cursory examination to determine which additional tests might be necessary.
- C1 The Toolkit: A Summary
- C2 Installing and Using Firebug to Observe HTTP and HTTPS Traffic
- C3 Installing and Using Advanced Cookie Manager
- C4 Installing and Using ZAP Proxy to Observe HTTP and HTTPS Traffic
- C4.1 Installation and Initial Setup of OWASP ZAP
- C4.2 Basic Setup, Browser and Proxy on Same Computer
- C4.3 Setup for Testing Mobile Devices and/or Web Browsers on a Different Computer from the Proxy
- C4.4 Installing Proxy SSL Certificate on Browser and Mobile Devices
- C4.5 Observing WebSockets Traffic Using ZAP Proxy
All of the tests documented in the primer are run using:
- the Firefox browser,
- with the Firebug extension,
- the Advanced Cookie Manager plug-in,
- and the OWASP Zed Attack Proxy.
Firefox is a free, open-source browser and can be downloaded here: https://www.mozilla.org/en-US/firefox/new/.
Firebug is a browser extension that enables observation of the network requests and responses for Firefox browsing sessions. It can monitor both http and https traffic. Because of its straightforward setup and integration with the browser, it is a good choice for tests that do not require the extra features of an external proxy such as OWASP ZAP.
More information about Firebug can be found at http://getfirebug.com/.
The Advanced Cookie Manager plug-in is a tool that lets us manipulate, inspect, and delete individual cookies. More information about the Advanced Cookie Manager is available at https://addons.mozilla.org/en-US/firefox/addon/cookie-manager/.
OWASP Zed Attack Proxy, or ZAP, is a freely available, cross-platform, and open-source proxy program developed and maintained by OWASP (Open Web App Security Project). It provides more advanced features than Firebug and can be used to monitor the network traffic of mobile devices, as well as programs or browsers running on any host on the tester's local network. ZAP downloads and documentation are available at the OWASP ZAP project page.
This rest of this section describes setting up the proxy and related testing tools and how to use these tools to perform the tests in this primer.
- Return to Navigation: Getting Started
To install Firebug, go to Tools --> Add-ons, search for "Firebug," and select it for installation to Firefox. A screenshot of the plug-in's description page (to ensure you are getting the correct plug-in) is shown below.
Image Caption: Firebug add-on
As mentioned above, Firebug is a good choice for tests that don't require the advanced functionality of an external proxy such as OWASP ZAP. After Firebug has been installed, it can be viewed by clicking its icon in the Firefox toolbar. In this setup tutorial, we explain how to use some of the tools that come with Firebug alongside screenshots that show them in action.
The image below shows the Firebug icon in the toolbar, with configuration options that can set user preferences.
Image Caption: Firebug configuration options
The Firebug display is shown below. The Net tab shows the network requests and responses; make sure that the Persist and All options are selected.
Image Caption: Set the "Persist" and "All" options.
The Persist option prevents the display from being cleared on each new page load. However, Firebug will collapse prior pages in its display, and the tester may need to expand them back out to view the relevant transactions.
In the example below, the user has logged out of a service, and the browsing session prior to the logout has been collapsed.
Image Caption: Viewing session lists in Firebug
Clicking the highlighted triangle for that portion of the session will expand the display for this part of the session.
Image Caption: Expanding the session information in Firebug
Hovering the mouse over the transaction's entry will show the full URL, enabling a check for https on the request.
Image Caption: Getting URL information in Firebug
The transaction's entry can also be expanded by clicking the triangle next to the transaction's entry.
Image Caption: Using Firebug to get detailed information about a transaction
The descriptions of individual tests in the testing scenarios give more details on how to find the important parts of the request and response in the Firebug display for the test in question. Firebug is a useful tool for initial examinations of Web pages, as it can be used to detect basic issues that merit additional testing. If an application opens new browser tabs or views as the user interacts with it, it may be necessary or more straightforward to observe the traffic in an external proxy (such as ZAP, described below).
- Return to Navigation: Getting Started
Cookies are small text strings sent by servers to browsers in http responses as the user interacts with a Web service. The browser saves the cookie values and includes them with subsequent http requests to the same application or domain. The use of cookies allows Web services to maintain the "state" of a session so that previous interactions can be taken into account when new requests arrive. A cookie editor can be used to examine and modify an application's cookies, and these capabilities are utilized by the test procedures for authentication cookies and cookie handling, covered in Section E5 Authentication Token and Cookie Handling.
The examples in this document use the Advanced Cookie Manager plug-in for Firefox. To find it, go to Tools --> Add-ons, search for "cookie managers," and select Advanced Cookie Manager for installation to Firefox. A screenshot of the plug-in's description page is shown below.
Image Caption: Advanced Cookie Manager plug-in
Once installed, the cookie manager can be opened by clicking its icon in the browser add-ons toolbar or from the add-ons menu. An annotated view of the Advanced Cookie Manager panel is shown below.
Image Caption: Viewing details of cookies in the Advanced Cookie Manager
During several of the tests outlined in the testing scenarios, the cookie manager will be used to view cookies and their flags, delete cookies, save cookies to file, and restore cookies from file. Inspecting cookies is also helpful to learn more about which information is sent to different services on the Internet.
The Advanced Cookie Manager is the only component of our testing toolkit that is not open source, and it has one significant drawback, which we highlight below. We opted to include the Advanced Cookie Manager for use in this primer because after evaluating the other cookie managers available, Advanced Cookie Manager worked better. With that said, we are actively working on an open source option that will replace the Advanced Cookie Manager in future versions.
The main drawback of the Advanced Cookie Manager is that it includes a Facebook "like" button in its user interface. As a result, every time the Cookie Manager is loaded (by clicking on the icon), it calls to Facebook. This drawback is mitigated to an extent by the fact that, as part of preparing to run tests, we clear all cookies from the browser, which limits the information available to Facebook to little more than an IP address. However, even that is less than ideal, and future versions will remove this issue.
- Return to Navigation: Getting Started
As the name implies, an intercepting proxy intercepts and logs the requests and responses between a client (a browser or mobile app) and the network resources with which it interacts. This enables observation of the behavior of the application without interfering with its functionality.
Setting up a proxy on your computer allows the user to review all network traffic on that computer in greater detail. While Firebug allows the user to see some transactions, an intercepting proxy provides a more comprehensive picture. For example, a portion of a proxy log from loading "graphite.org" in a browser is shown below:
Image Caption: Sample log output from ZAP Proxy
Once a browser or mobile device has been configured to send traffic to a proxy, the proxy can observe and capture http (non-encrypted) traffic without additional setup. However, capturing https traffic requires additional steps. A detailed technical description of https (SSL) and TLS are beyond the scope of this document. However, a general description of how a proxy can observe https traffic, along with basic setup steps, is included below.
The normal operation of https creates an encrypted connection between the Web browser/app and the resources with which it communicates. Because the traffic is encrypted, the proxy is not able to decipher the encrypted contents of the messages sent or received. To see the encrypted traffic, we must configure the browser or device to "trust" the proxy and allow it to decode the https communication by installing an SSL certificate -- generated by the proxy -- in the browser or device. This SSL certificate allows the proxy to decrypt the messages as they are received by the user, and then re-encrypt the messages before sending them to the ultimate recipient. The following diagram from the OWASP ZAP documentation (image released under a CC-SA license) illustrates the connections among the user, the proxy, and Web resources.
Image Caption: An overview of OWASP ZAP Proxy
The instructions for setting up OWASP ZAP are broken into five distinct sections:
- C4.1 Installation and Initial Setup of OWASP ZAP
- C4.2 Basic Setup, Browser and Proxy on Same Computer
- C4.3 Setup for Testing Mobile Devices and/or Web Browsers on a Different Computer from the Proxy
- C4.4 Installing Proxy SSL Certificate on Browser and Mobile Devices
- C4.5 Observing WebSockets Traffic Using ZAP Proxy
If you are primarily concerned with testing Web applications, you will not need to use any of the instructions in C4.3 Setup for Testing Mobile Devices and/or Web Browsers on a Different Computer from the Proxy.
If you are going to test mobile apps, then the instructions in C4.3 Setup for Testing Mobile Devices and/or Web Browsers on a Different Computer from the Proxy cover the details of preparing this testing environment.
Using ZAP Proxy requires more setup steps than other elements of the toolkit. Fortunately, these steps only need to be completed once. Additionally, in many cases, Firebug can be used to perform initial reviews that can determine whether a more detailed examination via an intercepting proxy is necessary.
This primer focuses specifically on the features or ZAP we use for the tests covered in the testing scenarios, which is a small subset of all the features offered in ZAP. For a complete overview of the full feature set of ZAP, consult the full ZAP documentation.
ZAP is available for download at the ZAP project page: https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project.
The download packages and installation instructions are platform-specific for Windows, Apple, and Linux. Choose the Standard package and follow the installation instructions for your desired platform.
After installation, modify the Display settings to enable process images in http requests/responses. This configuration causes image loads that may be of interest during testing to be displayed in the history log. In ZAP this setting is managed in the Tools --> Options --> Display menu.
Image Caption: Set ZAP Proxy to display images in http requests and responses.
Select the check box for Process images in HTTP requests/responses. Then, click the OK button.
- Return to Navigation: Getting Started
- Return to Navigation: OWASP ZAP Proxy Setup
The most straightforward setup is when the browser and proxy are on the same computer.
Within ZAP, the local proxy address should be set up as "localhost" and the port set to 8080 (this is the default setting at install). In ZAP, this setting is managed in the Tools --> Options --> Local proxy menu.
Image Caption: Set local proxy address.
Once you have verified that the settings in ZAP are correct, you can proceed to set the correct configuration in the browser you will use for testing.
When the browser and the proxy are on the same computer, the proxy address in the browser should be configured to use a proxy at "localhost" and port 8080.
Firefox manages this in Preferences --> Advanced --> Network --> Connection Settings, as shown below.
Image Caption: Set proxy address in Firefox.
- Select Manual proxy configuration.
- Set HTTP Proxy to "localhost" and Port to "8080."
- Select the option to Use this proxy server for all protocols.
Then, click the OK button.
To complete the setup, skip ahead to the C4.4 Installing Proxy SSL Certificate on Browser and Mobile Devices section below.
- Return to Navigation: Getting Started
- Return to Navigation: OWASP ZAP Proxy Setup
The steps in this section are only relevant in the following scenarios:
- testing traffic coming from a phone or a tablet;
- and testing traffic coming from multiple computers (for example, if the tests will compare traffic from a Chromebook, an Apple computer, and/or a Windows computer).
If you are not testing mobile devices, and/or if you are running ZAP on the same computer as the Web browser you will use for testing, you do not need to follow the steps in this section.
To configure ZAP to support testing with mobile devices, we need to complete two basic steps:
- Get the address of the computer running ZAP;
- and configure all testing devices to use that address.
The steps in C4.3 outline this process.
The first step is to determine the network address of the computer running ZAP Proxy. The procedure is different for Windows, Mac OS, and Linux.
When you find the network address of the computer running the proxy, write it down and store it in an easily accessible place. This network address needs to be added to other devices that will be used for testing.
The network address can be found under:
Control Panel --> Network and Internet --> Network and Sharing Center
From this dialog, select the active network, which will bring up a status menu. Select Details from this menu.
Image Caption: Navigating to "Details" for the active network
The Details dialog will display the network address of the computer. In almost all cases, the IPv4 address (not the IPv6 address) is the one that should be used as the proxy's network address, as shown below.
Image Caption: Finding the correct address in Windows
The network address can be found in System Preferences --> Network, as shown below.
Image Caption: Finding the correct address in OS X
Many distributions include a System Preferences utility that can be used to find the computer's network address. Another option is to use the "ip address" command from the shell and look for the "inet" entry on the appropriate interface, as shown below.
$ ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:20:f6:c1 brd ff:ff:ff:ff:ff:ff
inet 192.168.251.211/24 brd 192.168.251.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe20:f6c1/64 scope link
valid_lft forever preferred_lft forever
In the example above, eth0 is the correct interface, and the address is 192.168.251.211.
The proxy setup procedure is the same as covered in C4.2 Basic Setup, Browser and Proxy on Same Computer, except that the network address of the computer (which we discovered in the prior step) should replace the default addresss setting of localhost.
Image Caption: Updating the local proxy address in ZAP
In Firefox, under Preferences --> Advanced --> Network --> Connection, select Manual proxy configuration and enter the network address found in the steps above.
Once the settings have been updated, click the "OK" button to save the changes.
Image Caption: Updating the proxy address in Firefox
On an iOS (Apple) device, go to Settings --> Wi-Fi, and make sure your device is on the same network as the proxy computer.
Select the info screen for the network the device is connected to.
Image Caption: Verifying network name in iOS
The proxy settings are at the bottom of the network settings screen. Enable manual proxy settings and enter the network address found in the previous steps and the port number as shown below.
Image Caption: Setting the correct address for the proxy in iOS
Open the Settings menu and select Wi-Fi.
Image Caption: Selecting the active wireless network in Android
Press and hold the name of the network to bring up the network settings menu, and select Modify network.
Note: You need to give a longer-than-average "click" on the wireless name, in the range of one to two seconds, to bring up the Modify network option.
Image Caption: Select the "Modify network" option.
Select Show advanced options, and then select Proxy settings --> Manual. Use the network address and port number from step C4.3.1, as illustrated below.
Image Caption: Setting the proxy address in Android
Click the Save link to save the settings.
- Return to Navigation: Getting Started
- Return to Navigation: OWASP ZAP Proxy Setup
Installing the proxy's SSL certificate on browsers and devices will allow the proxy to decrypt the https traffic generated by these clients. In this section, we will cover using ZAP to generate an SSL certificate. Then, we will document how to save this certificate to our testing browser and/or a mobile device.
The proxy's SSL certificate can be saved from the ZAP Preferences --> Options --> Dynamic SSL Certificates menu. Open the dialog and click on Save.
Image Caption: Saving the SSL certificate generated by ZAP
The file will be saved as "owasp_zap_root_ca.cer" in the directory chosen by the user.
In Firefox, certificates can be installed in the Preferences --> Advanced --> Certificates dialog.
Within this dialog, select View Certificates.
Image Caption: Accessing installed certificates in Firefox
In the View Certificates dialog, select Import.
Image Caption: Importing a certificate to Firefox
Then, select the root certificate (owasp_zap_root_ca.cer) downloaded from the ZAP Proxy and open it, as shown in the screenshot below.
Image Caption: Selecting the certificate
Then, select the option to Trust this CA to identify websites, and click the OK button.
Image Caption: Trust the certificate generated by ZAP to identify websites.
Verify that the certificate has been uploaded by browsing to it. All certificate authorities are listed alphabetically. Once you have verified that the certificate has been uploaded, click the OK button.
Image Caption: Verify that the certificate is imported.
Note: When you are not testing, you should remove the certificate from the browser.
To remove the certificate, in Firefox go to the Preferences --> Advanced --> Certificates --> View Certificates menu, select the "OWASP Zed Attack Proxy Root CA" certificate, and click on Delete or Distrust.
Image Caption: Removing the OWASP ZAP Root CA
It is important to take care when transferring the certificate file to the device, as anyone who possesses it could potentially use it to decrypt https communications from devices that install the certificate. While this risk is small and can be mitigated by uninstalling the certificate when you're not testing, handling the certificate safely is highly recommended.
A secure file-sharing service is one good way to transfer the file. For iOS devices, the file must be accessed from the file-sharing service using the browser, not the sharing service's iOS app. Accessing the file from within the browser interface for the sharing service will bring up the Install Profile dialog described in the next step of the setup.
Once the certificate file has been transferred to the device, open it on the device. Opening the file will bring up the Install Profile dialog. (iOS refers to certificates with the broader term "Profile.") Select Install from within this dialog.
Image Caption: Select the "Install" option.
After a device password check, the Install Profile dialog will bring up a second dialog with a warning about installing a certificate to the device. Since the objective is to install the proxy's certificate to allow monitoring of https traffic, it is OK to accept the warning and select Install.
Image Caption: Warning message for OWASP ZAP Root CA
After the certificate is installed, a screen will confirm that the certificate was successfully installed.
Image Caption: Confirmation screen
Note: To remove the certificate after testing is complete, go to the Settings --> General menu and scroll down. The certificate will be listed under Profile.
Image Caption: Finding the installed certificate
Select Profile to enter a dialog that provides the option to delete the certificate.
Image Caption: Deleting the certificate
It is important to take care when transferring the certificate file to the device, as anyone who possesses it could potentially use it to decrypt https communications from devices that install the certificate. While this risk is small and can be mitigated by uninstalling the certificate when you're not testing, handling the certificate safely is highly recommended.
For Android devices, many methods of moving the certificate file to the device's storage (browser, secure file-sharing service, direct transfer over USB or an SD card) can be used.
Copy the proxy certificate file to the Android's SD card or other internal storage. The tutorial below assumes that the certificate has been moved to an SD card.
This document covers installation for Android versions 4.3 and above. The installation is slightly different for Android 4.3, Android 4.4, and higher. Select the installation that corresponds to your device's Android version.
In the Settings --> Security dialog, select Install from SD Card.
Image Caption: Install from SD card.
A dialog titled Name the certificate will pop up. Leave the name unchanged and select OK.
Image Caption: Saving the certificate to an Android 4.3 device
When you are not using the device to test, remove the certificate. To remove the certificate, go to the Settings --> Security --> Trusted credentials --> User dialog.
Image Caption: Navigating to the certificate
Select the OWASP certificate to bring up another menu with an option to remove the certificate.
Image Caption: Finding and selecting the certificate
Tapping the Remove button uninstalls the certificate.
In the Settings --> Security dialog, select Install from Storage.
Image Caption: Install from storage.
A dialog showing the downloads directory will show up. However, the process usually requires the certificate to be accessed through internal storage instead, using the following sequence:
Select the menu tab at the top-left corner.
Image Caption: Access internal storage.
Select Internal storage from the menu.
Image Caption: Select internal storage.
Select the Download directory from the Internal storage choices.
Image Caption: Select the correct directory.
Select the certificate file.
Image Caption: Select the certificate file.
A dialog titled Name the certificate will pop up. Enter a name, leave Credential use set to "VPN and apps," and select OK to install the certificate.
Note: If the device does not have a screen lock PIN or passcode, a window will pop up requiring one to be set.
Image Caption: Name the certificate.
When you are not using the device to test, remove the certificate. To remove the certificate, go to Setup --> Security --> Clear credentials.
Image Caption: Clear credentials.
Click OK to clear credentials. Only the user-installed credentials are cleared.
Image Caption: Provide the OK to clear credentials. Only user-uploaded credentials are cleared.
- Return to Navigation: Getting Started
- Return to Navigation: OWASP ZAP Proxy Setup
WebSockets is a communication link separate from http that browsers and application servers can use to pass information back and forth. It is normally a secondary form of communication that is connected after the Web application loads into the browser. If an application uses WebSockets, the traffic is of interest for security testing because it may contain personal information of the account holder or other users of the system.
ZAP Proxy supports the viewing of WebSockets traffic. (Note: Firebug does not support viewing of WebSockets traffic.) By default the WebSockets pane is not one of the choices shown in the toolbar at the center of the ZAP display.
The WebSockets pane can be added by clicking the green "+" and selecting WebSockets from the drop-down menu.
Image Caption: Enabling the WebSockets pane
The pane will also be added automatically to the toolbar by ZAP if a WebSockets connection is opened through the proxy. A view of the WebSockets traffic pane is shown below. Clicking on an individual message will show the full payload of the message in the upper pane of ZAP (where http requests and responses are normally shown).
From the WebSockets view, it is also possible to determine whether the WebSockets traffic is sent over an encrypted connection. The steps for checking whether the traffic is encrypted are detailed in Section E8 Observation of WebSockets Traffic.
Image Caption: Observing WebSockets traffic in ZAP
- Return to Navigation: Getting Started
- Return to Navigation: OWASP ZAP Proxy Setup
- Proceed to the next chapter: D. Preparing Firefox for Testing