Slides for my talk discussing research on the Evolutionary Neural Network Encoder of Shenanigans (ENNEoS), a proof-of-concept encoder for shellcode. The software uses genetic algorithms to evolve neural networks that contain the shellcode and output it on demand so that it can be executed in memory.
1. Using Neural Networks to Hide
Shellcode
Evolutionary Neural Network Encoder of Shenanigans (ENNEoS)
2. Who am I?
Drew Kirkpatrick
Senior Security Consultant
OSCP, GWAPT
M.S. Computer Science
M.S. Computer Information Systems
B.A. Psychology, Economics
3. What the heck are we going to talk about?
● Hiding shellcode
● Neural networks as black boxes
● The ENNEoS proof-of-concept encoder
● What’s up under the hood
● Fitness functions (the good stuff, trust me!)
● The loader
● Demo
● Heckling session
4. Shellcode
● Shellcode is a small piece of computer code that starts a remote command “shell”
● Obfuscating shellcode is a common way to avoid antivirus detection
● Encrypting the shellcode is one way
● Changing the signature of the shellcode is another way
● Hiding the shellcode in a neural network?
5. So why neural networks?
● Neural networks are inherently opaque
● Recurrent neural networks can have complex sequences of inputs as triggers
● Neural nets are great for storing data like shellcode, right?
6. So what did ya make?
● ENNEoS - Evolutionary Neural Network Encoder of Shenanigans
● Uses genetic algorithms to evolve neural networks that contain and output the
desired shellcode
● Example loader retrieves the shellcode from the neural network and executes it
7. ● As in NeuroEvolution of Augmenting Topologies (NEAT) obviously!
● Genetic algorithms evolve the structure of the
neural network
○ The “hardware” of the neural network
● Genetic algorithms also evolve the weights of
the neural network
○ The “software” of the neural network
● Can create recurrent neural networks
● It has a valid crossover operation!
That sounds NEAT!
8. The important part… Fitness Functions
● Sounds complicated? No sweat!
● The genetic algorithm will solve the problem for you
● Just needs some guidance
● Genetic algorithm requires a scoring algorithm
● Informs the genetic algorithm how good a neural
network is at solving your problem
● The genetic algorithm does the rest
to find a solution (i.e. encoding)
● Yes. Yes that is awesomesauce.
9. Careful what you wish for...
● That sounds so easy!
● 100% guarantee you will screw up your first fitness function
● It will do precisely what you tell it to
● You will get unexpected results
● Those unexpected results will make
complete sense in hindsight
10. Proof of concept fitness function
● Calculates difference between neural network output and desired shellcode
character
● If the character matches, give a bonus
● Square the value
● Proof of concept currently ignores input
11. The encoder
Main Thread
Creates 3,000 bots
Retrieves 3,000 neural
networks from Genetic
Algorithm
Schedules execution
jobs.
Main Thread
Pushes neural network
scores into Genetic
Algorithm
Ends generation
Performs evolution
Execution Threads
Stimulates bots with
inputs
Uses Fitness Function
to score output
03
01 02
12. The loader
DNA
Reads in Genomes
created by the Encoder
Brains!
Creates bots
Inserts neural networks
into bots
Retrieve Shellcode
Pushes inputs into bots
Retrieves outputs from
bots
Pop Shells
Copies bot output into
executable memory
Executes
Doesn’t have to be shellcode.
Super awesome way to hide you Britney Spears mp3’s at work
We’re going to be here a while. Is everyone comfortable?
Dr. Ken Stanley, University of Central Florida, Uber AI Engineering labs
Inputs
Topology/Hardware
Perception
Red, yours vs mine
Inputs and outputs, 0.0 -> 1.0
Random search vs guided search
All the other parts about designing the neural network (topology), the weights, etc, are done automatically for you.
There will be facepalms
Heavily multi-threaded A lot of C++ Almost all complex functionality is abstracted away in the classes Takes shellcode, gives you neural networks
Steps:
Chunks shellcode
Main thread
Initialized CGA
Creates 3000 ShellyBots
Gets 3000 neural networks from CGA for shellybots
Schedules execution jobs
Execution Threads
Takes assigned ShellyBots with brains, stimulates them
Uses fitness function to score the output and push back to main thread
Main thread
Pushes scores into genetic algortihm
Epoch
Rinse and repeat
Creates a neural network per chunk of shellcode
Reads in genome of neural networks, one per chunk
Creates a ShellBot per chunk, inserts brain
Pushes inputs, pulls outputs
Copies neural network outputs into executable memory and calls it
