2021
2020
2019
2018
2017
2016

Dutra & Gopinath: Input Languages for Effective and Focused Fuzzing FuzzCon Europe 2021

In this talk, we present several ways in which fuzzers can be enhanced with an input language specification, in order to enable focused fuzzing and reach deeper parts of the code. First, we focus on input languages which are expressed as context-free grammars, as well as refinements of such grammars. Here, we show how those grammars can be mined from the program execution, as well as abstracted to capture particular behaviors, such as a failure-inducing pattern. We also show how the original input grammar can be refined to produce the pattern of interest, or even a boolean combination of such patterns, enabling a full algebra of inputs. Next, we focus on the fuzzing of binary file formats, such as MP4 or ZIP. We show how such formats can be effectively represented using binary templates, which are a format specification used by the 010 Editor. Our new tool FormatFuzzer can turn those binary templates into highly efficient parsers, mutators and generators for the specified format. This can be integrated into existing fuzzers such as AFL++ to boost their efficacy and detect new memory errors.

Rigger: GAP Interview (virtual), 2021

Gopinath: Fuzzing – from Alchemy to a Science Guest Lecture, 2020-2021

Fuzzing is one of the key techniques in the arsenal of a practitioner of cyber-security, and is used for evaluating the robustness of programs against malicious inputs. To fuzz effectively, the input specification of the program under evaluation is practically a requirement. However, such specifications are typically unavailable, obsolete, incomplete, or inaccurate, limiting the reach of fuzzers. This has led to a proliferation of hacky recipes by different fuzzers to get past the input parsing stage, with each recipe working on some but not all programs. That is, fuzzing most resembles alchemy than science at this point.

In this talk, I show how to transform fuzzing to a science. I present an end-to-end framework for recovering precise input specifications of programs. Such mined specifications can immediately be used for effective exploration of the program input space as well as the space of the program behavior, leading to the identification of failure-inducing inputs. Next, given any failure-inducing input, I show how to generalize such inputs into abstract patterns, precisely identifying the failure causing parts of the input. Any number of such abstract patterns can then be combined using the full set of logical connectives — to produce specialized grammars that can be used by any grammar fuzzer for precise control of produced inputs and hence the expected behavior.

Gopinath: The Science of Fuzzing Job Talk, 2020-2021

Fuzzing is a key method for evaluating the robustness of a program against malicious inputs. It involves generating and feeding arbitrary data to a program, and monitoring the program for catastrophic failures. Any failure in the program in processing the data indicates a possible vulnerability in the program. Efficient and effective fuzzing requires the availability of the input specification for the program under test. However, such specifications are typically unavailable, obsolete, incomplete, or inaccurate, limiting the reach of fuzzers. This has led to a proliferation of hacky recipes by different fuzzers to get past the input parsing stage, with each recipe working on some but not all programs. That is, fuzzing most resembles alchemy than science at this point.

In this talk, I show how to transform fuzzing into a science. I present an end-to-end framework for recovering precise input specifications of programs. Such mined specifications can immediately be used for effective exploration of the program input space as well as the space of the program behavior, leading to the identification of failure-inducing inputs. Next, given any failure-inducing input, I show how to generalize such inputs into abstract patterns, precisely identifying the failure-causing parts of the input. Any number of such abstract patterns can then be combined using the full set of logical connectives — to produce specialized grammars that can be used by any grammar fuzzer for precise control of produced inputs and hence the expected behavior.

Gopinath: Sample Free Learning of Input Grammars Singapore University of Technology and Design, 2020

Efficient and Effective fuzzing requires the availability of the input specification for the program under test. However, such specifications are typically unavailable or inaccurate, limiting the reach of fuzzers. In this talk, I present an end-to-end framework for recovering the input specification for given programs without samples, using such mined specifications for isolating and generalizing failure-inducing inputs, as well as for the precise control of fuzzers.

Gopinath: Pygram: Learning input grammars for Python programs TU Darmstadt, Germany (software group retreat), 2017

AUTOGRAM is a method to infer human-readable input grammars from observing program behavior on test runs. The resulting grammars can be used for fuzzing, secondary or parallel validation of new inputs, and identifying the essence of a program in a language agnostic manner. I will present my current work in AUTOGRAM, and discuss my research in taking AUTOGRAM forward.

Gopinath: Who tests our tests: An overview of mutation analysis, its caveats, and pitfalls McGill University, Canada, 2017

A key concern in software engineering is determining the reliability of our software artifacts. Software engineers traditionally rely on extensive testing to ensure that programs are fault free. However, tests are usually also written by human beings, and hence vulnerable to the similar problems as software applications. An inadequate test suite, or one that contains errors, can provide a false sense of security to the programmer or the consumer. Hence a major question in software testing is how to evaluate the quality of test suites.

Following the maxim “you can’t find a bug in what you don’t cover”, code coverage is often used as a measure of test suite quality. However, code coverage can not evaluate the quality of our oracles, and given that our oracles are often insufficient or wrong, code coverage, on its own is insufficient, and mutation testing, a stronger fault-based technique is often recommended. In this talk, I will present the basics of mutation testing, and look at why it is effective. Next, I will examine the limitations to mutation analysis, and how these can be mitigated. I will also examine how insights from mutation testing may be used in fields such as evaluating type annotations, program repair, and program adaptation.

Gopinath: Code Coverage is a Strong Predictor of Test suite Effectiveness in the Real World GTAC, 2016

This talk is about the effectiveness of coverage as a technique for evaluating quality of test suite. We show the utility of coverage by considering its correlation with mutation score, and also show that coverage is a significant defence against bugs. Further, we also critique effectiveness of mutation score as a criteria for test suite quality.