Skip to content

Symbolic Execution

Symbolic execution is a powerful technique for analyzing software by treating input values as symbols rather than concrete data. This allows analysts to explore multiple execution paths simultaneously, uncovering conditions that could lead to vulnerabilities, unexpected behavior, or hidden logic in malware. In malware reverse engineering, symbolic execution is particularly valuable for understanding how a binary behaves under varying inputs, identifying potential exploits, or validating hypotheses about its runtime behavior.

Core Concepts of Symbolic Execution

Path Constraints and Branch Coverage

Symbolic execution models program execution by tracking path constraints—logical conditions that must be satisfied for a particular code path to execute. For example, a conditional branch like if (input == 0x42) generates a constraint that the input must equal 0x42 for the branch to be taken. These constraints are solved using Satisfiability Modulo Theories (SMT) solvers, which determine whether a set of constraints can be satisfied.

Concrete vs. Symbolic Values

  • Concrete values are fixed data (e.g., 0x42, "hello").
  • Symbolic values represent unknown inputs (e.g., input_1, input_2), allowing the analysis to explore all possible combinations.

Dynamic Analysis Integration

Symbolic execution bridges static and dynamic analysis by: 1. Exploring code paths without executing the program. 2. Generating test cases that trigger specific behaviors. 3. Detecting runtime conditions that may be exploited (e.g., buffer overflows, privilege escalation).

Ghidra's Symbolic Execution Capabilities

Ghidra (the NSA's reverse engineering framework) provides a Symbolic Execution Engine that enables analysts to: - Define symbolic inputs and their constraints. - Track path conditions through the binary. - Leverage SMT solvers to resolve constraints and identify feasible execution paths.

Key Features

  • Input Variable Management: Users can define symbolic variables (e.g., input_1, input_2) and assign them to specific memory locations or registers.
  • Path Exploration: Ghidra automatically tracks constraints for each code path, allowing analysts to inspect conditions that lead to different outcomes.
  • Integration with SMT Solvers: Ghidra uses external SMT solvers (e.g., Z3) to resolve complex constraints, enabling deeper analysis of conditional logic.

Practical Example: Analyzing a Malicious Binary

Let’s analyze a hypothetical malware binary that checks for a specific input to trigger a payload. Here’s how symbolic execution can uncover its behavior:

Step 1: Define Symbolic Inputs

# In Ghidra, navigate to the Symbolic Execution Engine
# Define symbolic variables for input
input_1 = 0x42  # Example value
input_2 = 0x00  # Another example value

Step 2: Trigger Conditional Logic

# Analyze the binary's conditional branch
if (input_1 == 0x42) {
    // Payload execution logic
}

Step 3: Explore Path Constraints

Ghidra generates constraints like: 

input_1 == 0x42 ∧ input_2 == 0x00
The SMT solver determines if these constraints are satisfiable, revealing whether the payload can be triggered.

Step 4: Generate Test Cases

If constraints are satisfiable, Ghidra can generate concrete inputs (e.g., input_1 = 0x42) to reproduce the behavior during dynamic analysis.

Key Takeaways

  • Symbolic execution enables exploration of all possible code paths by treating inputs as symbols.
  • Ghidra’s Symbolic Execution Engine allows analysts to define inputs, track constraints, and resolve them using SMT solvers.
  • Combining symbolic and dynamic analysis helps uncover hidden logic, validate hypotheses, and generate test cases for malware behavior.