Hack Like It's 2013 (The Workshop)


Published on

Try to imagine the amount of time and effort it would take you to write a bug-free script or application that will accept a URL, port scan it, and for each HTTP service that it finds, it will create a new thread and perform a black box penetration testing while impersonating a Blackberry 9900 smartphone. While you’re thinking, Here’s how you would have done it in Hackersh:
“http://localhost” \
-> url \
-> nmap \
-> browse(ua=”Mozilla/5.0 (BlackBerry; U; BlackBerry 9900; en) AppleWebKit/534.11+ (KHTML, like Gecko) Version/ Mobile Safari/534.11+”) \
-> w3af
Meet Hackersh (“Hacker Shell”) – A new, free and open source cross-platform shell (command interpreter) with built-in security commands and Pythonect-like syntax.
Aside from being interactive, Hackersh is also scriptable with Pythonect. Pythonect is a new, free, and open source general-purpose dataflow programming language based on Python, written in Python. Hackersh is inspired by Unix pipeline, but takes it a step forward by including built-in features like remote invocation and threads. This 120 minute lab session will introduce Hackersh, the automation gap it fills, and its features. Lots of demonstrations and scripts are included to showcase concepts and ideas.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Hack Like It's 2013 (The Workshop)

  1. 1. Hacking Like Its 2013 /* The Workshop */ #include “Itzik Kotler“
  2. 2. Agenda● Pythonect● Developing Domain-specific Language w/ Pythonect● Hackersh● Q&A
  3. 3. Pythonect● Pythonect is a portmanteau of the words Python and Connect● New, experimental, general-purpose dataflow programming language based on Python● Current “stable“ version (True to Apr 9 2013): 0.4.2● Made available under Modified BSD License● Influenced by: Unix Shell Scripting, Python, Perl● Cross-platform (should run on any Python supported platform)● Website: http://www.pythonect.org/
  4. 4. A few words on the Development● Written purely in Python (2.7) – Works on CPython 2.x, and Jython 2.7 implementations● Tests written in PyUnit● Hosted on GitHub● Commits tested by Travis CI
  5. 5. Installing and Using The Pythonect Interpreter● Install directly from PyPI using easy_install or pip: – easy_install Pythonect OR – pip install Pythonect● Clone the git repository: – git clone git://github.com/ikotler/pythonect.git – cd pythonect – python setup.py install
  6. 6. The Pythonect Interpreter● Written and integrated with the Python environment: % pythonect Python 2.7.3 (default, Aug 1 2012, 05:14:39) [Pythonect 0.4.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>>
  7. 7. Dataflow Programming Programming paradigm that treats data as something originatingfrom a source, flows through a number of components and arrives at a final destination - most suitable when developing applications that are themselves focused on the "flow" of data.
  8. 8. Dataflow Example A video signal processor which may start with video input,modifies it through a number of processing components (i.e. video filters), and finally outputs it to a video display. Video Screen Local B&W Output File Frame Display Reader Procressor
  9. 9. Dataflow ExampleWant to change a feed from a local file to a remote file on a website? No problem! Video Screen URL B&W Output Downloader Frame Display Procressor
  10. 10. Dataflow ExampleWant to write the Video B&W Frame Processor output to both a screen and a local file? No problem! Local File Video Writer URL B&W Downloader Frame Procressor Screen Output Display
  11. 11. Dataflow Programming Advantages● Concurrency and parallelism are natural● Data flow networks are natural for representing process● Data flow programs are more extensible than traditional programs
  12. 12. Dataflow Programming Disadvantages● The mindset of data flow programming is unfamiliar to most programmers● The intervention of the run-time system can be expensive
  13. 13. Dataflow Programming Languages● Spreadsheets are essentially dataflow (e.g. Excel)● VHDL, Verilog and other hardware description languages are essentially dataflow● XProc● Max/Msp● ... Etc.
  14. 14. <Pythonect Examples>
  15. 15. Hello, world -> printString Function
  16. 16. What do we have here?● -> is a Pythonect Control Operator, it means async forward.● Theres also | (i.e. Pipe) which means sync forward.● Hello, world is a literal string● print is a function
  17. 17. "Hello, world" -> [print, print] Function String Function
  18. 18.  ["Hello, world", "Hello, world"] -> print String Function String
  19. 19. range(99, 0, -1) | [ _ % 2 == 0 ] -> str -> _ + " bottle(s) of beer on the wall," -> print -> _.split( on)[0] + . -> print -> print("Take one down, pass it around,")Integer Filter Function Expression Function Function Function Function
  20. 20. Basic Pythonect Syntax Summary● -> is async forward.● | (i.e. Pipe) is sync forward.● _ (i.e. Underscore) is current value in flow
  21. 21. <Pythonect Security Scripts/Examples>
  22. 22. ROT13 Encrypt & Decryptraw_input() -> _.encode(rot13) -> print Function Function Function
  23. 23. Check if FTP Server Supports Anonymous Loginftp.gnu.org -> ftplib.FTP -> _.login() -> print("Allow anonymous") String Class Function Function
  24. 24. (Multi-thread) HTTP Directory Brute-forcesys.argv[1] -> [str(_ + / + x) for x in open(sys.argv[2],r).read().split(n)] -> [(_, urllib.urlopen(_))] -> _[1].getcode() != 404 -> print "%s returns %s" % (_[0], _[1], _[1].getcode()) Function Filter Function String Nested Loop ...
  25. 25. Command line Fuzzer[%s, %n, A, a, 0, !, $, %, *, +, ,, -, ., /, :] | [_ * n for n in [256, 512, 1024, 2048, 4096]] | os.system(/bin/ping + _) Array Nested Loop Function
  26. 26. (Multi-thread) Generic File format Fuzzeropen(dana.jpg, r).read() -> itertools.permutations -> open(output_ + hex(_.__hash__()) + .jpg, w).write(.join(_)) Function String Function ...
  27. 27. Compute MALWARE.EXEs MD5 & SHA1"MALWARE.EXE" -> [os.system("/usr/bin/md5sum " + _), os.system("/usr/bin/sha1sum " + _)] Function String Function
  28. 28. Compute MALWARE.EXEs Entropy● Entropy.py: ● Pythonect: import math "MALWARE.EXE" def entropy(data): entropy = 0 -> open(_, r).read() if data: -> entropy.entropy for x in range(2**8): p_x = float(data.count(chr(x))) / len(data) -> print if p_x > 0: entropy += - p_x * math.log(p_x, 2) return entropy
  29. 29. References / More Examples● My Blog – Scraping LinkedIn Public Profiles for Fun and Profit – Fuzzing Like A Boss with Pythonect – Automated Static Malware Analysis with Pythonect● LightBulbOne (Blog) – Fuzzy iOS Messages!
  30. 30. Pythonect Roadmap● Support Python 3k● Support Stackless Python● Support IronPython● Support GPU Programming● Fix bugs, etc.
  31. 31. Questions?
  32. 32. Moving on!Developing Domain-specific Language (DSL) with Pythonect
  33. 33. Domain-specific Language● Domain-specific language (DSL) is a mini-language aiming at representing constructs for a given domain● DSL is effective if the words and idioms in the language adequately capture what needs to be represented● DSL can also add syntax sugar
  34. 34. Why? Why create a custom tag or an object with methods? Elegant Code ReuseInstead of having to recode algorithms every time you need them, you can justwrite a phrase in your DSL and you will have shorter, more easily maintainable programs
  35. 35. Example for DSLs● Programming Language R● XSLT● Regular Expression● Graphviz● Shell utilities (awk, sed, dc, bc)● Software development tools (make, yacc, lex)● Etc.
  36. 36. <DSL/Examples>
  37. 37. Example #1: XSLT Hello, world<?xml version="1.0"?><xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <xsl:template match="p"> Hello world! - From hello.xsl. </xsl:template></xsl:stylesheet>
  38. 38. Example #2: Graphviz/DOT Hello, worlddigraph G{ Hello → World}
  39. 39. Domain-specific Language with Pythonect● Pythonect provides various features to let you easily develop your own DSLs: – Built-in Python module Autoloader – Concurrency (Threads & Processes) – Abstract Syntax (i.e. Generic Flow Operators)
  40. 40. Built-in Python AutoLoader● The AutoLoader loads Python modules from the file system when needed● In other words, no need to import modules explicitly.● The sacrifice is run-time speed for ease-of-coding and speed of the initial import()ing.
  41. 41. Hello, world -> string.split i.e. import string return string.split
  42. 42. Concurrency (Threads & Processes)● Multi-threading: – Hello, world -> [print, print]● Multi-processing: – Hello, world -> [print, print]● Mix: – Hello, world -> [print, print &]
  43. 43. Abstract Syntax● Brackets for Scope: – []● Arrows and Pipes for Flows: – | and ->● Dict and Logical Keywords for Control Flow: – {} and not/or/and
  44. 44. So, imagine the following is a real script:from_file(malware.exe) -> extract_base64_strings -> to_xml
  45. 45. IT IS!(with Pythonect)
  46. 46. Meet SMALL Simple Malware AnaLysis Language● Toy language for analyzing malware samples● Single Python file (14 functions, 215 lines of text)● Runs on top of Pythonect
  47. 47. SMALL Features● Extract IPv4 Addresses from Binaries● Extract Base64 Strings from Binaries● Calculate MD5/SHA1/CRC32● Determine File Type (via /usr/bin/file)● Create XML Reports
  48. 48. How Does SMALL Work?● SMALL functions are divided into two groups: – Root, these functions start a flow – Normal, these functions continues or closes the flow● Root functions accept String and return dict – e.g. from_file()● Normal functions accept dict and return dict – e.g. extract_base64_strings()
  49. 49. <Pythonect/Security DSL (i.e. SMALL) Examples>
  50. 50. How to Start the SMALL Interpreter pythonect -m SMALL -i● The -m means - run library module as a script● The -i means - inspect interactively after running script● Just like Python :)
  51. 51. Extract Base64 Strings and Save As XML from_file(malware.exe) -> extract_base64_strings -> to_xmlFunction Function Function
  52. 52. Extract IPv4 Addresses and Save As XML from_file(malware.exe) -> extract_ipv4_addresses -> to_xmlFunction Function Function
  53. 53. Compute MD5, SHA1, CRC32, and FileType from_file(malware.exe) -> md5sum -> sha1sum -> crc32 -> file_type -> to_xmlFunction Function Function
  54. 54. Other (Potential) Security Domains:● Reverse Engineering● Malware Analysis● Penetration Testing● Intelligence Gathering● Fuzzing● Etc.
  55. 55. Questions?
  56. 56. Moving on! Hackersh
  57. 57. Hackersh● Hackersh is a portmanteau of the words Hacker and Shell● Shell (command interpreter) written with Pythonect-like syntax, built-in security commands, and out of the box wrappers for various security tools● Current “stable“ version (True to Apr 1 2013): 0.1.0● Made available under GNU General Public License v2 or later● Influenced by: Unix Shell Scripting and Pythonect● Cross-platform (should run on any Python supported platform)● Website: http://www.hackersh.org
  58. 58. A few words on the Development● Written purely in Python (2.7)● Hosted on GitHub
  59. 59. Motivation● Taking over the world● Automating security tasks and reusing code as much as possible
  60. 60. Problems● There are many good security tools out there... – but only a few can take the others output and run on it – but only a few of them give you built-in threads/processes controling for best results● No matter how well you write your shell script, the next time you need to use it - for something slightly different - you will have to re-write it
  61. 61. Hackersh – The Solution● Hackersh provides a “Standard Library“ where you can access your favorite security tools (as Components) and program them as easy as a Lego● Hackersh lets you automagically scale your flows, using multithreading, multiprocessing, and even a Cloud● Hackersh (using Pythonect as its scripting engine) gives you the maximum flexibility to re-use your previous code while working on a new slightly-different version/script
  62. 62. Installing and Using The Hackersh● Install directly from PyPI using easy_install or pip: – easy_install Hackersh OR – pip install Hackersh● Clone the git repository: – git clone git://github.com/ikotler/hackersh.git – cd hackersh – python setup.py install
  63. 63. Implementation● Component-based software engineering – External Components ● Nmap ● W3af ● Etc. – Internal Components ● URL (i.e. Convert String to URL) ● IPv4_Address (i.e. Convert String to IPv4 Adress) ● Etc.
  64. 64. Component as Application● Components accepts command line args: – "localhost" -> hostname -> nmap("-P0")● They also accept internal flags options as: – "localhost" -> hostname -> nmap("-P0", debug=True)
  65. 65. Input/Output: Context● Every Hackersh component (except the Hackersh Root Component) is standardized to accept and return the same data structure – Context.● Context is a dict (i.e. associative array) that can be piped through different components
  66. 66. Same Context, Different Flow● "http://localhost" -> url -> nmap -> ping – Port scan a URL, if *ANY* port is open, ping it● "http://localhost" -> url -> ping -> nmap – Ping the URL, if pingable, scan for *ANY* open ports
  67. 67. Ask The Context● Context stores both Data and Metadata● The Metadata aspect enables potential AI applications to fine- tune their service selection strategy based on service-specific characteristics
  68. 68. Conditional Flow"http://localhost" -> url -> nmap -> [_[PORT] == 8080 and _[SERVICE] == HTTP] -> w3af -> print
  69. 69. Hackersh High-level Diagram Root Literal Component Context Component ...(e.g. String) (e.g. URL)
  70. 70. <Hackersh Scripts/Examples>
  71. 71. TCP & UDP Ports Scanning "localhost" -> hostname -> nmap Built-in ExternalTarget Component Component
  72. 72. Class C (256 Hosts) Ping Sweep -> ipv4_range -> ping Built-in ExternalTarget Component Component
  73. 73. Web Server Vulnerability Scanner -> ipv4_address -> nmap -> nikto Built-in External ExternalTarget Component Component Component
  74. 74. Fork: Target as Hostname + Target as IP"localhost" -> hostname -> [nslookup, pass] -> ... Target ... as Hostname Built-inTarget Component Target ... as IPv4 Addr.
  75. 75. Black-box Web App Pentration Testing"http://localhost" -> url -> nmap -> browse -> w3af -> print Built-in External Built-in External Built-inTarget Component Component Component Component Component
  76. 76. Hackersh Roadmap● Unit Tests● Documention● More Tools – Metasploit – OpenVAS – TheHarvester – Hydra – …● Builtin Commands● Plugins System● <YOUR IDEA HERE>
  77. 77. Hackersh Official TODOhttps://github.com/ikotler/hackersh/blob/master/doc/TODO
  78. 78. Questions?
  79. 79. Thank you! My Twitter: @itzikkotler My Email: ik@ikotler.org My Website: http://www.ikotler.org Pythonect Website: http://www.pythonect.org Hackersh Website: http://www.hackersh.orgFeel free to contact me if you have any questions!