This paper proposes a technique called fuzzing to automatically generate random DRAT refutations and check them against SAT solvers and verified checkers in order to find bugs. The technique found bugs in two existing DRAT checkers, drat-trim and proofcheck, showing they incorrectly reject or accept certain refutations. The paper's contributions are a fuzzing technique to find bugs in DRAT checkers, discovering specific bugs in two checkers, and developing a verified checker in Coq that is complete.

1. Fuzzing and Verifying RAT Refutations with Deletion Information
Walter Forkel, Tobias Philipp, Adri´an Rebola-Pardo, Elias Werner
Introduction
SAT solvers are common tools in the industry
Unfortunately, SAT solvers contain bugs
DRAT format is the de facto standard for emitting unsatisfiable proofs
F SAT solver
SAT
UNSAT, P checker
Single Point of Failure
Are there any unknown bugs in the checker ?
Background
Tautological clause: clause containing A and ¬A for some variable A
Resolvent of C, D with A ∈ C and ¬A ∈ D is (C {A}) ∪ (D {¬A})
Asymmetric Tautology (AT)
Asymmetric Literal Addition (ALA):
alaF (C) = C ∪ {L | L1, ..., Ln, L ∈ F and Li ∈ C for all 1 ≤ i ≤ n}
C is an AT wrt F if there is n ∈ N such that alaF (C) ↑ n is a tautology
Resolution Asymmetric Tautology (RAT)
C is a RAT upon literal L wrt the formula F if
1. C is an AT wrt the formula F, or
2. L ∈ C, and all resolvents of C with any D ∈ F upon L are AT wrt F
DRAT Refutation
Finite sequence of addition and deletion instructions to the empty clause
Each added clause is a RAT wrt the preceding clauses
Our Approach: Fuzzing DRAT Refutations and Check Against a Fully Mechanically Verified Checker
1 2 3 4 5
random formula
generation
SAT solver
checkers
classify P
Verified Checker in Coq
modifier
checkers
classify P
Verified Checker in Coq
F unsatisfiable
proof P
F satisfiable
F all
accept P
inconsistent
P
inconsistent
all reject P
Example
F =
{p, q}
{p, q}
{p, q}
{p, q}
P =
a {¬r} RAT upon ¬r
a {r, p} AT
a {r} AT
d {p, q} deletion
d {¬p, q} deletion
d {p, ¬q} deletion
d {¬p, ¬q} deletion
d {} AT
drat-trim and drat-fc
classify P as accepted
add r to the first clause
P =
a {¬r, r} RAT upon ¬r
a {r, p} AT
a {r} AT
d {p, q} deletion
d {¬p, q} deletion
d {p, ¬q} deletion
d {¬p, ¬q} deletion
d {} AT
drat-trim rejects P
drat-fc accepts P
Our verified checker also
accepts P
Found a bug in drat-trim
Results
Formulas: 2900 clauses, 800 variables on average
Resulting proofs: 2100 steps on average
drat-trim rejects DRAT refutations containing tautological clauses
proofcheck rejects DRAT refutations in which unit clauses are deleted
DRAT emission procedure in the SAT solver minisat constructs
unexpected DRAT refutations of the form (a {})(d C)(a {})
Conclusion
DRAT checkers proofcheck and drat-trim contain bugs
New fuzzing technique that automatically finds bugs in checkers
Our verified DRAT checker is complete
Future Work: adapt the checker and fuzzing procedure to other proof
formats
Code available at github.com/drat-tools
Acknowledgement: This work was supported by RiSE, LogiCS and WWTF grant VRG11-005
FLAIRS 30