Analyse AFL Fuzzing Result

After doing fuzzing, we concentrate on result analysis.

Check Fuzzer Status

afl-whatsup

It helps showing the running status and overall running overview of each fuzzer, where the data is the sum of all fuzzers.
Plus the -s option to display only the overview.

Use:

$ afl-whatsup -s syncdir

To see:

It’s real time status without auto refreshing.
People always give it a loop to have updating feedback.

afl-gotcpu

It helps checking status of every core.

afl-stat

It is one of afl-utils, the AFL auxiliary tools .This tool is similar to the output of afl-whatsup.

Before using it, you need a configuration file to set the output directory of each afl-fuzz instance:

# set file name as afl-stats.conf

{
  "fuzz_dirs": [
      "/root/syncdir/SESSION000",
      "/root/syncdir/SESSION001",
  ...
  "/root/syncdir/SESSION00x"
  ]
}

Then specify the configuration file to run:

$ afl-stats -c afl-stats.conf

[SESSION000 on fuzzer1]
Alive:   1/1
Execs:   64 m
Speed:   0.3 x/s
Pend:    6588/249
Crashes: 101
[SESSION001 on fuzzer1]
Alive:   1/1
Execs:   105 m
Speed:   576.6 x/s
Pend:    417/0
Crashes: 291
...

afl-plot

Besides command line utils, we have afl-plot to produce (plot) more intuitive results.

# install dependency -- gnuplot
$ apt install gnuplot
$ afl-plot afl_state_dir graph_output_dir

# see outputs
$ cd graph_output_dir
$ tree
.
├── exec_speed.png
├── high_freq.png
├── index.html
└── low_freq.png

0 directories, 4 files

Click index.html to see graphs.

When to Stop

Normally you should wait as long as you can, until fuzzer get 100% coverage, but it’s hard to judge.

Look at upper right corner of AFL running panel, you see overall results

With cycles num increasing, it’s color turns from red to yellow, bule and eventually green.
Green means useless to do further fuzzing.

There’re some AFL extensions like pythia.
It shows status and give estimates of crash and path.

• correctness: probability of finding a crash leading input.
• fuzzability: difficulty of finding a new path. The higher, the easier to fuzz.
• current paths: the number of paths.
  currently found.
• path coverag: path coverage.

Once you have waited for several days,

or the path coverage reaches 99%, or correctness decreases to 1e-08, stop the fuzz.

Output Result

Contents of multi-fuzzer instance sync dirs

$ tree -L 3
.
├── fuzzer1
│   ├── crashes
│   │   ├── id:000000,sig:06,src:000019+000074,op:splice,rep:2
│   │   ├── ...
│   │   ├── id:000002,sig:06,src:000038+000125,op:splice,rep:4
│   │   └── README.txt
│   ├── fuzz_bitmap
│   ├── fuzzer_stats
│   ├── hangs
│   │   └── id:000000,src:000007,op:flip1,pos:55595
│   ├── plot_data
│   └── queue
│       ├── id:000000,orig:1.png
│       ├── ....
│       └── id:000101,sync:fuzzer10,src:000102
└── fuzzer2
        ├── crashes
        ├── ...

• queue: stores all testcases with special executive paths.
• crashes: stores cases causing target to receive fatal signal and crash.
• crashes/README.txt： saves command parameters to execute crash cases.
• hangs： cases leading to target timeout.
• fuzzer_stats： afl-fuzz status.
• plot_data： for afl-plot to plot.

Process Result

Crash Exploration Mode

Also called peruvian rabbit mode, making sure the bug is exploitable.

Use -C parameter to turn on crash exploration mode, one crash case as input, afl-fuzz will produce related but different crashes.

$ afl-fuzz -m none -C -i poc -o peruvian-were-rabbit_out -- /path/to/target @@ 

In this way, length of memory address can be messured, and buffer may be detected.

triage_crashes

This is a script in AFL source code dir :
afl-2.52b/experimental/crash_triage/triage_crashes.sh
which helps trigger collected crashes.

$ ./triage_crashes.sh /path/to/afl_output_dir /path/to/tested_binary [...target params...]

  +++ ID 000000, SIGNAL 11 +++
  +++ ID 000001, SIGNAL 06 +++
  +++ ID 000002, SIGNAL 06 +++
  +++ ID 000003, SIGNAL 06 +++
  +++ ID 000004, SIGNAL 11 +++
  +++ ID 000005, SIGNAL 11 +++
  +++ ID 000006, SIGNAL 11 +++

Different SIGNAL means different signals.
For example, 11 means SIGSEGV (indicates buffer overflow), 06 means SIGABRT (indicates double free) …
Linux signal list
signal(7) man page

crashwalk

crashwalk gets you more detailed crashes classification results, and the specific causes of crashes.
It is based on exploitable plugin of gdb, install on ubuntu:

$ apt install gdb golang
$ mkdir ~/tools
$ cd ~/tools
$ git clone https://github.com/jfoote/exploitable.git
$ mkdir go
$ export GOPATH=~/tools/go
$ export CW_EXPLOITABLE=~/tools/exploitable/exploitable/exploitable.py
$ go get -u -v github.com/bnagy/crashwalk/cmd/...

Follow the lead, use correct path. You will find three binary files in $GOPATH.

crashwalk has Manual Mode and AFL Mode.
Manual Mode reads crashes/README.txt to get command.
AFL Mode set parameters manually.

#Manual Mode
$ ~/tools/go/bin/cwtriage -root syncdir/fuzzer1/crashes/ -match id -- /path/to/target @@

#AFL Mode
$ ~/tools/go/bin/cwtriage -root syncdir -afl

afl-collect

One of afl-utils suite, also based on exploitable plugin to check availability of crash.
It also delete useless or duplicated crash sample

usage: afl-collect [-d DATABASE] [-e|-g GDB_EXPL_SCRIPT_FILE] [-f LIST_FILENAME]
 [-h] [-j THREADS] [-m] [-r [-rt TIMEOUT]] [-rr] sync_dir collection_dir -- target_cmd

the following arguments are required: 
sync_dir, collection_dir, target_cmd

# example
$ afl-collect -j 8 -d crashes.db -e gdb_script ./afl_sync_dir ./collection_dir -- /path/to/target --target-opts

Code Coverage

A method to measure coverage of program, like whether a line of source code has been executed, or whether an instruction in the assembly code has been executed.

There’re three levels of coverage detections: function, basic-block, edge.

Basic Block

A set of sequentially executed instructions.
Once the first instruction is executed, following instructions must be executed.

That means, one basic block contains only one entry and only on exit.

For example:

IDA can also divide it into four blocks:

Edge

Edge is used to represent the jump between basic blocks.
Just like the lines between every blocks in IDA view.

Tuple

In the implementation of AFL, a tuple(branch_src, branch_dst) is used to record the information of the current basic block + the previous basic block, to obtain the target execution flow and code coverage.

Do Caculation

GCOV is released with gcc, with same principle of instrumentation by afl-gcc, it generates code coverage info after instrumented by gcc.

LCOV is another tool and also the graph front-end of GCOV.

afl-cov helps call two tools above to process result from afl-fuzz.

Installation

We mainly use afl-cov

$ apt-get install afl-cov
# this version seemes not supporting branch coverage statistics

$ apt-get install lcov
$ git clone https://github.com/mrash/afl-cov.git
$ ./afl-cov/afl-cov -V
$ afl-cov-0.6.2
# this way you get latest version

Rebuild

First use gcov to rebuild source.
Adding -fprofile-arcs and ftest-coverage to CFLAGS.
Better reassign new dir by --prefix so as not to cover files afl-gcc instrumented.

$ make clean 
$ ./configure --prefix=/another/absolute/path CC="gcc" CXX="g++" CFLAGS="-fprofile-arcs -ftest-coverage" --disable-shared
$ make
$ make install

Calculate

Then we execute afl-cov.

# example
$ afl-cov -d /path/to/syncdir --live --enable-branch-coverage -c . -e "cat AFL_FILE | LD_LIBRARY_PATH=/path/to/lib /path/to/target AFL_FILE"

# '-d' to assign afl-fuzz output dir
# '--live' used for one running fuzzer ( updating dir )
# '--enable-branch-coverage' used to enable edge coverage (branch coverage) statistics
# '-c' to assign source code dir
# '-e' to set running command, and in this parameter
# "AFL_FILE" functions same as "@@" for afl-fuzz
# "LD_LIBRARY_PATH" load specific lib

Finally we get a code coverage report in html.