On a recent Web application test I encountered a new challenge. The Web application presented a Web API intended to be used by a mobile application, in order for developers to utilise this API the documentation was also served from the Web application.
In order to assess each API function for vulnerabilities I first had to build valid requests from the documentation and then get them into my Web assessment tool of choice Burp Suite Pro.
It would have been possible to accomplish this by reading the documentation and patiently typing the raw HTTP request into Burp repeater. However with over thirty API functions to test and a tight schedule this was not a viable option. I therefore decided to script it.
The first step was to download all of the HTML documentation and parse each page to extract the HTTP method, path, example URL parameters, and, if present, the example body parameters. Using this information I built raw HTTP requests which I stored in text files. (As this first script is quite specific to the client’s application I will not be releasing it at this time).
With a directory full of raw HTTP requests it was time to import them into Burp and start testing proper. However I could not find any method of importing my raw HTTP requests into Burp other than manually copying and pasting them into repeater, an achievable task with the relatively small number of functions I had to test in this instance but a chilling prospect for future, larger tests.
After a coffee I had the idea to simply send the raw HTTP request through Burp by sending them from a Web client with a proxy configured. Since the requests had a variety of HTTP methods and body parameters a Web browser wasn’t an option. I briefly tried using telnet and netcat but these failed for some reason I haven’t identified. I also tried using curl, but this required further processing to issue the request using the curls’s command line options. I therefore turned back to Python and wrote a script to read files from a directory, then for each file: parse them into an object (using BaseHttpRequestHandler), build a request using urllib2 and send this via a proxy.
This resulted in the HTTP request being stored in Burp ready for assessment like any normal request to a Web application – visible in the site map, proxy history and easily sent to Intruder, Repeater, Scanner and Sequencer.
I’ve released this script under the GPLv2 licence in the hope that it will be useful to others, it is available here.
Parse one or more files and send via the default proxy (127.0.0.1:8080):
raw2proxy.py -f FILENAME FILENAME...
Parse a directory of files and send via a proxy running on 192.168.0.1 port 9001:
raw2proxy.py -d DIRECTORY -p 192.168.0.1:9001
Additional options are available:
As always, if you have any comments or suggestions please feel free to get in touch.