Local WebPagetest Using Docker

https://medium.com/@francis.john/local-webpagetest-using-docker-90441d7c2513

Local WebPagetest Using Docker

Francis John

Francis JohnFollowAug 24, 2017 · 5 min read

(Update 8/25/2017: A day after this was posted, traffic shaping on Mac OS was officially supported, however if you are using Docker it will not work as Docker runs in a linux headless agent.)

Recently at NerdWallet we’ve started using SpeedCurve to have a historical record of our WebPagetest results and shiny graphs to help visualize. It’s been a really great tool in terms of measuring performance for a large production website.

However sometimes it can be useful to have an even tighter feedback loop when you are iterating on performance, to get a sense of what works and by how much.

WebPagetest is awesome but I found the documentation a bit lacking so I thought I’d share into my experience in setting up a WebPageTest private instance for local development.

Docker Setup

WebPageTest has two components that need to be in sync with each other, the server and the agent. The server handles the frontend app for WebPagetest and the agent actually runs the tests. The agent polls the server for tests to run, and posts to an endpoint on the server. To spin these up, the WebPageTest folks have created Docker images for both:

https://hub.docker.com/r/webpagetest/agent/https://hub.docker.com/r/webpagetest/server/

If you aren’t super familiar with Docker (as I wasn’t), I’d recommend going through the tutorials to familiarize yourself. After you’ve gotten setup you’ll want to pull down the WebPageTest server and agent images.

After pulling down the images you can create the containers.

Run the server

Just after running the server, you should be able to browse to http://localhost:4000 and see the webpage test private instance page to confirm it’s up.

NOTE: Feel free to specify ports other than the one(s) I’m using.

Run the agent

The SERVER_URL should be set to <your server>/work/ — it denotes the URL for the agent to poll. The LOCATION should match one of the locations defined in settings/locations.ini in the WebPagetest server. These define the parameters for that agent you are running (e.g. browsers supported, label, and other goodies). These environment variables are used in the startup script of the wptagent python app.

NOTE: The agent will throw errors if it can’t communicate with the server. Try running the above command without detached mode (remove the -d) and it will spit out errors if there are any.

Start it up

Head over to http://localhost:4000 . You should see the frontend for WebPagetest and on the surface things look good. Right? Let’s check. There’s a built in configuration check if you hit http://localhost:4000/install

What you should see at http://localhost:4000/install

If your agent is able to communicate with your server, you should see an agent connected for the Test Location. Easy enough.

However if you’re on OS X like I am, there’s still a bit more work needed.

Traffic Shaping MacOS Workaround (until supported)

Select Test Location from the locations dropdown and then enter a URL to test

Let’s try to run a test. Select the Test Location from the list of locations on your WPT server:

If you attempt to run a test at this point you’re going to see an error Error configuring traffic-shaping:

Error configuring traffic shaping 🙁

This is because traffic shaping has not yet been implemented on MacOS, so
we’ll need to implement a couple workarounds.

You can remove traffic shaping by doing two things — setting a dummy value for connectivity in your settings/locations.ini file on your server, and setting --shaper none when the agent is starting up.

I’d recommend creating a new Docker image based on the original WPT agent/server images but with modifications to the configuration.

Server

Setup a new folder that has two files, a Dockerfile and a locations.ini

Dockerfile

locations.ini

This defines a new docker image that pulls from webpagetest/server , replacing the locations.ini file with one that will work for OS X, particularly setting connectivity to remove traffic shaping.

Agent

Setup a new folder that has two files, a Dockerfile and a script.sh

Dockerfile

script.sh

This defines a new docker image with a copy of the existing WPT entrypoint script that tacks on the --shaper none flag.

Make sure script.sh is executable:

And now we can build our agent image:

Start it up (for real)

Alright now we should be able to start up our local copies. You may need to first stop the existing docker containers.

Stop the parent containers

Start up your local WPT containers

And voila you should now be able hit http://localhost:4000 and run webpage tests against your local environments:

Happy WebPagetesting!

Francis John

WRITTEN BY

Francis John

Software Engineer

Tagged with: ,

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.