Sf rmr - Servicing Forwarding Remote Multiplexing Relay


Published on

This presentation shows an alternative and new aporoach to using only one port and multiple protocols. The protocols can be defined and will be recognised.
This can be done to use one internet socket for servising multiple applications that use internet sockets.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Sf rmr - Servicing Forwarding Remote Multiplexing Relay

  1. 1. SF-RMR ( http://code.google.com/p/sf-rmr/ ) Port multiplexing Software framework for rule based client-server communication (as opposed to standard port based ). 2010 by Alen Milincevic
  2. 2. <ul><li>SF-RMR is an acronym of: </li></ul><ul><ul><li>Servicing </li></ul></ul><ul><ul><li>Forwarding </li></ul></ul><ul><li>Remote </li></ul><ul><li>Multiplexing </li></ul><ul><li>Relay </li></ul><ul><li>, which is to be read as „sience fiction rumour”, kind of a funny and catchy name.  </li></ul>
  3. 3. <ul><li>Computer ports are generally divided into server ports ( services on the network ), and client ports ( clients , which use the services), i.e. : </li></ul><ul><ul><li>Web browser </li></ul></ul><ul><ul><li>Mail client </li></ul></ul><ul><ul><li>News Client </li></ul></ul><ul><ul><li>FTP client </li></ul></ul><ul><li>etc. </li></ul>
  4. 4. <ul><li>Generally, each service on the internet has it’s own port , i.e standard ports: </li></ul><ul><ul><li>web server: port 80 or 8080 </li></ul></ul><ul><ul><li>Mail server: 25 , 587 , 110 , etc. </li></ul></ul><ul><ul><li>FTP server: 20 , 21 </li></ul></ul><ul><li>etc. </li></ul>
  5. 5. <ul><li>SF-RMR does some multiplexing. This means, one port is shared among multiple services. Each service can be identified by it’s rule. </li></ul><ul><li>Typical scenario: </li></ul><ul><ul><li>client connects (using client port) </li></ul></ul><ul><ul><li>client inits the first line of protocol. This is used to recognise the correct destination. </li></ul></ul><ul><ul><li>SF-RMR recognises the protocol, nased on a rule and reroutes to a local server port (where the service listens to). </li></ul></ul>
  6. 6. <ul><li>Each service has to be started on the server own port. </li></ul><ul><li>Many services can be actually customised in order to change the default server port. </li></ul><ul><li>SF-RMR configuration is set in order to do the rule based rerouting of the SF-RMR port to local service port. </li></ul>
  7. 8. <ul><li>Only one port needs be exposed to the external world, and internally other ports might be, and are used </li></ul><ul><li>Can still serve many clients ( multithreading ) </li></ul><ul><li>Original server ports may still be accessed </li></ul><ul><li>Filters can be applied to the comunication (tunneling), which can be used to circumvent firewalls </li></ul><ul><li>Cryptography and steganography can be applied to communication, in order to hide the content. </li></ul>
  8. 9. # listening port for SF-RMR socket port 80 # just some rules triggers (IP PORT ID TRIGGER) rule 1 28 . 169 . 1 .1 8 080 HTTP HEAD rule 1 28 . 169 . 1 .1 8 080 HTTP GET rule 1 28 . 169 . 1 .1 8 080 HTTP POST rule 1 28 . 169 . 1 .1 8 080 HTTP OPTIONS rule 12 8 . 169 . 1 .1 9001 UP <U> This creates a port 80 server socket to outside world , rerouting HTTP and some other protocol to the real server sockets, 8080 and 9001 respectively. Everything is on the same IP adress in this example.
  9. 10. <ul><li>Filtering and plugin support </li></ul><ul><li>Blacklists and whitelists (allowed and denied hosts) </li></ul><ul><li>Proces output/input redirecting to the socket (similar to netcat) with optional parameters for comunication </li></ul><ul><li>Half-duplex and full-duplex communication mode </li></ul><ul><li>Chaining is possible, so creating inteligent rule based redirections. </li></ul>
  10. 11. <ul><li>rinetd ( http://www.boutell.com/rinetd/ ), redirection through config, not command line </li></ul><ul><li>Netcat ( http://netcat.sourceforge.net/ ), socksifying </li></ul><ul><li>IRC bouncer ( http://en.wikipedia.org/wiki/BNC_%28software%29 ), idea of persistent connect (TODO) </li></ul><ul><li>Possibility to perform filtering, spying and tunneling network traffic. </li></ul><ul><li>Possiblity to do some basic white/blacklisting </li></ul>
  11. 12. <ul><li>None of existing found software could do the rule-based forwarding in the manner as required. </li></ul><ul><li>Chaining with other software (i.e rinetd, netcat, socket bouncer) should be possible (and is). </li></ul>
  12. 13. <ul><li>Besides the normal Java Application, make an adoption as servlet, make native builds (i.e. GCJ ( http://gcc.gnu.org/java/ ) or similar options...) </li></ul><ul><li>Write more filter plugins </li></ul><ul><li>Make more settings, althorough much things are now already configurable, due to Java support (i.e. proxy in some Java versions) </li></ul>
  13. 14. <ul><li>Oracle (Sun) Java ( http://www.java.com/ ), tested on Java 1.6. , older (probably downto 1.4.) might work, if sourcecode is recompiled. </li></ul><ul><li>Anything that can run Java (i.e. Windows, Linux, Solaris, MacOS X, other OS), even some mobile devices.  </li></ul><ul><li>Java settings must be set correctly ! For example, safe modes and permissions must be set correctly, otherwise possible permission errors. Also, any security manager must be set correctly. </li></ul>
  14. 16. advanced work modes
  15. 17. <ul><li>Sometimes, some protocols are ment to work different than most of them. This is usually a justified design decidion. Some server software implementing the protocols also behaves a little different, as should. </li></ul>
  16. 18. <ul><li>first line handling </li></ul><ul><li>prefiltering </li></ul><ul><li>filtering </li></ul><ul><li>reverse TCP mode </li></ul><ul><li>black and whitelists </li></ul><ul><li>dynamic redirection </li></ul>
  17. 19. <ul><li>client connects </li></ul><ul><li>client starts to communicate using the desired protocol (first line) </li></ul><ul><li>SF-RMR waits, until the whole line is entered (CR(/LF)). </li></ul><ul><li>SF-RMR goes through rules and sees if some redirection applies </li></ul><ul><li>SF-RMR redirects the connection and waits until termination </li></ul>
  18. 20. <ul><li>Some protocols are binary , i.e. cannot be handeled like ”first line”. </li></ul><ul><li>Some protocols require a message to be sent back to client, prior to awaiting a message from client. </li></ul><ul><li>This is a global setting, as there is no means to recognise the correct redirection, when first line cannot be handeled. </li></ul>
  19. 21. <ul><li>After the ”first line” has been entered, there is a chance to handle this, do some additional work etc. and then pass back for further processing by SF-RMR or just terminate the connection </li></ul><ul><li>Prefiltering is a plugin class, which has to be written specifically for the wanted usecase. </li></ul>
  20. 22. <ul><li>When a rule has been set up and communication is working, sometimes there is need to filter the upstream and downstream traffic, do some additional work or simply monitor the traffic flow. </li></ul><ul><li>It is possible to define upstream and downstream filter separately. It is also possible, to use only one direction. </li></ul>
  21. 23. <ul><li>Normal mode (in simplest usecase) is: </li></ul><ul><ul><li>client->SF-RMR->server </li></ul></ul><ul><li>scenario: </li></ul><ul><ul><li>client connects to SF-RMR </li></ul></ul><ul><ul><li>SF-RMR connects to server, according to rules </li></ul></ul><ul><ul><li>communciation is established </li></ul></ul><ul><li>Reverse TCP mode: </li></ul><ul><ul><li>client->SF-RMR<-server </li></ul></ul><ul><li>Scenario: </li></ul><ul><ul><li>client connects to SF-RMR </li></ul></ul><ul><ul><li>SF-RMR creates a listening connection, according to rules </li></ul></ul><ul><ul><li>server connects to SF-RMR </li></ul></ul><ul><ul><li>Communcation is established </li></ul></ul>
  22. 24. <ul><li>These control, what clients are allowed and denied to connect. </li></ul><ul><li>By using ”allow” and ”deny” in configuration </li></ul><ul><li>Does not work for reverse TCP mode, works only for the client , not the other side. In reverse TCP mode, timeout must be set accordingly in configuration, and the upstream socket must be set on a less known port. However, it is possible by using filtering classes, authentification shemes etc. to also restrict access. </li></ul>
  23. 25. <ul><li>Configuration must be set </li></ul><ul><li>A java class in following general form must exist and be pluged in: </li></ul><ul><ul><ul><li>import java.io.*; </li></ul></ul></ul><ul><ul><ul><li>public class Firstlinehandler { </li></ul></ul></ul><ul><ul><ul><li>public String streamHandle(BufferedReader br) { </li></ul></ul></ul><ul><ul><ul><li>// class must do something, and return </li></ul></ul></ul><ul><ul><ul><li>// or return null, </li></ul></ul></ul><ul><ul><ul><li>// if wanting to handle all the comunication </li></ul></ul></ul><ul><ul><ul><li>} </li></ul></ul></ul><ul><ul><ul><li>} </li></ul></ul></ul>
  24. 26. <ul><li>prefiltering, upstream and downstream filtering must be set up in config. </li></ul><ul><li>A java class in following general form must exist and be pluged in: </li></ul><ul><ul><ul><li>public class FilteringExample { </li></ul></ul></ul><ul><ul><ul><li>public String streamConvert(String line) { </li></ul></ul></ul><ul><ul><ul><li>// some filterprocessing can be done here </li></ul></ul></ul><ul><ul><ul><li>// i.e. the line adapted </li></ul></ul></ul><ul><ul><ul><li>return line; </li></ul></ul></ul><ul><ul><ul><li>} </li></ul></ul></ul><ul><ul><ul><li>} </li></ul></ul></ul>
  25. 28. Socket chaining, filtering
  26. 29. <ul><li>Socket chaining enables some interesting configurations. </li></ul><ul><li>Can be chained by itself or by the help of other software (i.e. rinetd, netcat) </li></ul><ul><li>Can be used for protocol conversion (i.e. TCP to UDP by netcat, when using as proces) </li></ul><ul><li>Can be used as an simple intermediate firewall (SF-RMR allow and deny hosts options) </li></ul>
  27. 30. <ul><li>By using different configuration for every instance of SF-RMR in the part of the chain. </li></ul><ul><li>Forwarding rules must be written accordingly, in order to work. </li></ul>
  28. 31. <ul><li>i.e.: </li></ul><ul><li>In order to convert one protocol into another </li></ul><ul><li>To encrypt/decrypt the protocol </li></ul><ul><li>To conseal information or intermix socket communication (i.e. steganography) </li></ul>
  29. 32. <ul><li>prefilter – filters the criteria itself </li></ul><ul><li>upstream filter – filters the upstream traffic </li></ul><ul><li>downstream filter – filters the downstream traffic </li></ul>
  30. 33. <ul><li>Filters are Java classes , with only one method, which accepts a stream of bytes (strings) and returns a stream of (modified) bytes. </li></ul><ul><li>i.e. : </li></ul><ul><ul><ul><li>public class FilterExample { </li></ul></ul></ul><ul><ul><ul><li>public String streamConvert(String line) { </li></ul></ul></ul><ul><ul><ul><li>// Here should be the part, where the filter is applied </li></ul></ul></ul><ul><ul><ul><li>return l ine; </li></ul></ul></ul><ul><ul><ul><li>} </li></ul></ul></ul><ul><ul><ul><li>} </li></ul></ul></ul>
  31. 34. <ul><li>Java calls the filter class by reflection, the single method: streamConvert </li></ul><ul><li>The method may do whatever it wishes, whatever Java can do </li></ul><ul><li>The method returns a result as converted stream </li></ul>
  32. 35. <ul><li>Prefilter is a global filter for rules, and is defined in config by: </li></ul><ul><ul><ul><li>prefilter Somefilter </li></ul></ul></ul><ul><ul><li>In this example, the filter Java class is called: Somefilter.class </li></ul></ul><ul><li>Upstream and downstream filters are rule specific filters , and are defined in config i.e. : </li></ul><ul><ul><li>option SOMEID upstream Somefilter1 </li></ul></ul><ul><ul><li>o ption SOMEID downstream Somefilter2 </li></ul></ul>
  33. 36. <ul><li>Upstream and downstream filters can be used as coder/decoder for some protocol. </li></ul><ul><li>i.e. : </li></ul><ul><ul><li>option SOMEID upstream HEXENCODE </li></ul></ul><ul><ul><li>o ption SOMEID downstream HEXDECODE </li></ul></ul><ul><li>could be used as hex encoding/decoding pair for some data relay. </li></ul>
  34. 37. <ul><li>The possiblity of prefilter is to do some work before some rule is applied, i.e. communication is redirected to another port. </li></ul><ul><li>This could be, but not limited to: </li></ul><ul><ul><li>SOCKS handler </li></ul></ul><ul><ul><li>HTTP CONNECT prehandler </li></ul></ul><ul><ul><li>Etc. </li></ul></ul>
  35. 39. <ul><li>The master filter (first in chain) is defined in config. It is his responsibility to call other filters in the chain. </li></ul>
  36. 40. <ul><li>Sometime, it is not possible, to use straight mode (i.e. Some restrictions). SF-RMR can also work in the ”reverse TCP mode” on a per rule basis, if neccessary. In this mode, SF-RMR acts as a mere repeater, binding two client sockets. Chaining works as usual, only some additional configuration must be performed. </li></ul>
  37. 42. choosing correct protocol rules
  38. 43. <ul><li>Rule format is, in general: </li></ul><ul><ul><li>rule HOSTNAME PORT RULE_ID EXPRESSION </li></ul></ul><ul><li>HOSTNAME – host to redirect to </li></ul><ul><li>PORT – port to redirect to </li></ul><ul><li>RULE_ID – unique ID, for individual options </li></ul><ul><li>EXPRESSION – first line to use for evaluation </li></ul><ul><li>EXPRESSION is the whole remainder of the line, and can be anything. </li></ul>
  39. 44. <ul><li>Rule format for proceses is, in general: </li></ul><ul><ul><li>rule PROCES 0 RULE_ID EXPRESSION </li></ul></ul><ul><li>PROCES – proces name redirect input/output </li></ul><ul><li>0 – not used </li></ul><ul><li>RULE_ID – unique ID, for individual options </li></ul><ul><li>EXPRESSION – first line to use for evaluation </li></ul><ul><li>EXPRESSION is the whole remainder of the line, and can be anything. </li></ul>
  40. 45. <ul><li>Usually, most protocols use some keywords, by which they can be identified. </li></ul><ul><li>This can be used as the expression, to use for the relay redirection. </li></ul><ul><li>If some rule expresion is not on the begining of the line (default behaviour), some of the other modes are to be used (contains, ends with or regexp, the most powerfull one.) </li></ul>
  41. 46. <ul><li>More comon protocols are: </li></ul><ul><ul><li>Telnet - the ”grandfather” of all TCP protocols </li></ul></ul><ul><ul><li>Echo – simply reply, what is sent </li></ul></ul><ul><ul><li>HTTP – hypertext transfer protocol, web is based on it </li></ul></ul><ul><ul><li>FTP – file transfer protocol (this one is a bit tricky) </li></ul></ul><ul><ul><li>NNTP – network news transfer protocol (used for newsgroups) </li></ul></ul><ul><ul><li>SMTP – simple mail transfer protocol (sending mail) </li></ul></ul><ul><ul><li>POP3 – post office protocol (used for mail manipulation) </li></ul></ul><ul><ul><li>I MAP – internet message access protocol (used for mail manipulation) </li></ul></ul><ul><ul><li>Finger - for the exchange of human-oriented status and user information </li></ul></ul><ul><ul><li>Whois - query and response protocol used for querying database for registered users or assignees of an internet resource </li></ul></ul><ul><ul><li>LDAP - Lightweight Directory Access Protocol ( for querying and modifying data of directory services) </li></ul></ul><ul><li>etc. </li></ul>
  42. 47. <ul><li>http://en.wikipedia.org/wiki/HTTP </li></ul><ul><li>This would be an example, to redirect HTTP to localhost, port 8080: </li></ul><ul><ul><li>rule 8 080 RULE_ID1 HEAD </li></ul></ul><ul><ul><li>rule 8 080 RULE_ID2 GET </li></ul></ul><ul><ul><li>rule 8 080 RULE_ID3 POST </li></ul></ul><ul><ul><li>rule 8 080 RULE_ID3 OPTIONS </li></ul></ul>
  43. 48. <ul><li>http://en.wikipedia.org/wiki/SMTP </li></ul><ul><li>This is an example, to redirect SMTP to port 2525, localhost: </li></ul><ul><ul><li>rule 2525 RULE_ID1 HELO </li></ul></ul>
  44. 49. <ul><li>http://en.wikipedia.org/wiki/POP3 </li></ul><ul><li>This is an example to redirect POP3 to port 110110: </li></ul><ul><ul><li>rule 110110 RULE_ID_N APOP </li></ul></ul>
  45. 50. <ul><li>http://en.wikipedia.org/wiki/IMAP </li></ul><ul><li>This is an example to redirect IMAP to port 14314: </li></ul><ul><ul><li>rule 14314 RULE_ID_N openssl </li></ul></ul><ul><ul><li>rule 14314 RULE_ID_N2 +OK </li></ul></ul><ul><li>IMAP is tricky, in that the ”+” sign could be anything, so it is better to use the regexp funionality. </li></ul>
  46. 51. <ul><li>http://en.wikipedia.org/wiki/NNTP </li></ul><ul><li>This is an example for NNTP: </li></ul><ul><ul><li>rule 11919 RULE_ID_N LIST ACTIVE </li></ul></ul>
  47. 52. <ul><li>Rule must be something unique, specific to the certain protocol and occuring only once. In other words, the first command. As first command line may be various, anything allowed by protocol, rules must be chosen carefully. </li></ul><ul><li>Distinction must be made between every wanted protocol in the configuration. </li></ul>
  48. 53. <ul><li>Some specific protocols (like i.e. FTP) use more than one port for comunication. As this is not possible to achieve with SF-RMR (yet), such cases will not work. Only one port per rule. </li></ul><ul><li>FINGER and WHOIS protocols are tricky to configurem due to heavy predictive nature of the first line. </li></ul><ul><li>On the other hand, some servers use messages, which are to be sent before the first line . This case is not handeled by SF-RMR (as of yet?!) as well. The calling application must take care, to tolerate not receiving a response and try ”blindly” to send the first line, in order to initiate comunication. </li></ul>
  49. 54. <ul><li>telnet is actually the master protocol . It can contain any of the lines, which are required by server. Therefore it can be used to test many TCP protocols (i.e. HTTP, NNTP, SMTP, POP3, IMAP, and partially FTP (command port) ). </li></ul><ul><li>Rules for this depend on the specific case, and must be tailored to the needs. </li></ul>
  50. 55. <ul><li>Some person, which understands the general expressions needed, in order to uniquely and umambigously identify a redirection rule. </li></ul><ul><li>Eventually, reading of the needed protocol specification will help clearify the correct order of comunication for a given protocol (rule). </li></ul><ul><li>First possible line is the important one. </li></ul>
  51. 56. <ul><li>There are some protocols, i.e. SOCKS, which work by first communicating over a binary bytes sequence. </li></ul><ul><li>Such protocols are not fully supported as of yet. </li></ul><ul><li>However, it is possible, by using a custom firstlinehandler, to handle also such cases </li></ul>
  52. 57. <ul><li>Depending on the configuration of the machine, on which SF-RMR is running, it might be neccessary to use something like this: </li></ul><ul><ul><li>rule 0:0:0:0:0:0:0:1 8080 GET </li></ul></ul><ul><li>would redirect to localhost, port 8080, if IPV6 is used. Sometimes, it is possible to use both IPV4 and IPV6 adreses, which depends on the machine TCP/IP stack configuration. </li></ul>
  53. 59. config, rules and limits
  54. 60. <ul><li>Generally, there are: </li></ul><ul><ul><ul><li>relaying rules </li></ul></ul></ul><ul><ul><ul><li>global options </li></ul></ul></ul><ul><ul><ul><li>local options </li></ul></ul></ul><ul><ul><ul><li>comments, empty lines and everything else </li></ul></ul></ul><ul><li>The configuration is parsed at start. Global options are set and local ones are only stored for later. Comments, empty lines, etc. are ignored. Options must be specified at the beginning of the line. </li></ul>
  55. 61. <ul><li>If some option is specified multiple times, either the last occurence overwrites every previous occurence, or the occurences are stacked. For white/blacklists, relay options by ID and the rules itself are stacked. </li></ul>
  56. 62. <ul><li>General redirection rules format is as follows: </li></ul><ul><ul><li>rule IP PORT ID TRIGGER </li></ul></ul><ul><li>Meaning in concrete that: </li></ul><ul><ul><li>rule 85 HTTP1 GET </li></ul></ul><ul><li>Will redirect to port 85 on localhost (IPv4), when a GET is encountered . By default, it is at the beginning of a line . If regular expression is used: </li></ul><ul><ul><li>regexp 3 </li></ul></ul><ul><li>then the regular expression is evaluated, and first which matches (top to bottom) is used. </li></ul>
  57. 63. <ul><li>These specify certain IP’s, which are allowed/denied. Others have no access and will be immediatelly disconnected. </li></ul><ul><li>example: </li></ul><ul><ul><ul><li>deny 0:0:0:0:0:0:0:1 </li></ul></ul></ul><ul><ul><ul><li>deny </li></ul></ul></ul><ul><ul><ul><li>allow 0:0:0:0:0:0:0: 2 </li></ul></ul></ul><ul><ul><ul><li>allow </li></ul></ul></ul><ul><li>Will deny access to two hosts and allow access to only two hosts. Again, either IPv4 or IPV6, depending on Java system properties. </li></ul>
  58. 64. <ul><li>Unlike TCP socket redirection, proceses are specified in this format: </li></ul><ul><ul><ul><li>rule PROCESNAME N/A ID TRIGGER </li></ul></ul></ul><ul><li>Procesname could also be a full path, but withouth blank signs. Example: </li></ul><ul><ul><ul><li>rule ftp.exe 0 EXE ftp </li></ul></ul></ul><ul><li>Would start the proces called ftp.exe with the ID EXE. ID can be used in order to specify more options for the ID. </li></ul>
  59. 65. <ul><li>In order to limit the load on server, following settings exist: </li></ul><ul><ul><li>limit_threads </li></ul></ul><ul><ul><li>limit_unique_thread_counter </li></ul></ul><ul><ul><li>limit_unique_relay_counter </li></ul></ul><ul><li>Limit threads limits the maximum concurent threads, which server clients. Keep in mind, that full-duplex sockets consume 2 threads per client (upstream+dowstream in paralel). </li></ul><ul><li>Unique thread counter and unique relay counter are used only for debuging purposes, and normaly need not be set. </li></ul>
  60. 66. <ul><li>Comments can be specified by using the # sign, i.e. : </li></ul><ul><ul><li># This is a comment </li></ul></ul><ul><li>also, they can be used for temporarly commenting/uncommenting settings, i.e. : </li></ul><ul><ul><li>#socketmode 3 </li></ul></ul><ul><li>would (temporarly) disable forced half-duplex mode. </li></ul><ul><li>Everything else, not defined as an rule/option/comment is ignored when reading configuration. </li></ul>
  61. 68. Dynamic rule handling
  62. 69. Static rule handling <ul><li>Handling done in configuration file </li></ul><ul><li>Each rule must be unique identifable </li></ul><ul><li>Only non-binary protocols supported (telnet based) </li></ul>
  63. 70. Dynamic rule handling <ul><li>Advanced </li></ul><ul><li>More flexible </li></ul><ul><li>More complicated to configure </li></ul><ul><li>Programatic knowledge required (Java) </li></ul>
  64. 71. How is it accomplished? <ul><li>By using one or more of the following features: </li></ul><ul><li>first line filter class </li></ul><ul><li>prefilter class </li></ul><ul><li>upstream filter class </li></ul><ul><li>downstream filter class </li></ul><ul><li>dynamic redirection class </li></ul><ul><li>Invalid rule fallback class </li></ul>
  65. 72. First line filter class Checks the first protocol line and allows for either prefiltering of data, data validation or sending a generated pre-response, if neccessary. The general format is: import java.io.*; public class Firstlinehandler { public String streamHandle(BufferedReader br) { // The actual handling is done here // result should be either something handeled or null } }
  66. 73. Upstream and downstream filtering This can be used for filtering or more complicated purposes (i.e. logging, monitoring, etc.). General form is: public class FilteringExample { public String streamConvert(String line) { // some conversion or other stuff is done // returns the filtered line } } Prefiler class is used in the same manner.
  67. 74. Dynamic redirection and fallback These can be used in order to handle diynamic redirection (not from the config). General form is: public class RuleHandler { public MultiplexingRelayStream HandleRule(String line) { // Returns either a valid formated redirection destination // or null if invalid (or wished so) } }
  68. 75. Static redirection order 1. first line is waited for 2. first line is evaluated from configuration 3. based on the rule, redirection is performed
  69. 76. Dynamic redirection order 1. first line is waited for ( firstline filter can be applied) 2. prefiltering is applied ( prefilter can be applied) 2. first line is evaluated from configuration ( dynamic redirection filter can be applied) 3. based on the rule, redirection is performed (either static configuration redirecting, or fallback filter can be used)
  70. 77. Redirection handling itself 1. dynamic redirection is used (if configured) 2. static configuration is used, if 1. unsuccessfull 3. fallback redirection is used, if 2. unsuccessfull If neither dynamic redirection, nor fallback redirection are configured, then only the default static redirection from configuration is used (default behaviour).
  71. 78. Connector <ul><li>Used to connect: </li></ul><ul><li>first line filter class </li></ul><ul><li>prefilter class </li></ul><ul><li>dynamic redirection class </li></ul><ul><li>Invalid rule fallback class </li></ul><ul><li>The configuration is read from static configuration properties. </li></ul>
  72. 79. Implementing connector <ul><li>Must extend the SFRMR_Connector class! </li></ul><ul><li>first line filter class must extend the SFRMR_FirstLineHandler class! </li></ul><ul><li>prefilter class must extend the SFRMR_Filtering class! </li></ul><ul><li>dynamic redirection class must extend the SFRMR_RuleHandler class! </li></ul><ul><li>Invalid rule fallback class must extend the SFRMR_RuleHandler class! </li></ul><ul><li>These are combined inside the connector class, and can interact to each other (through Java mechanisms), in order to create a more dynamic redirection. </li></ul>
  73. 81. Rules,options
  74. 82. <ul><li>By the native inspiration for this software by comparison to: </li></ul><ul><ul><li>rinetd ( http://www.boutell.com/rinetd/ ) </li></ul></ul><ul><ul><li>netcat ( http://netcat.sourceforge.net/ ) </li></ul></ul><ul><li>, it was decided, that a file based rule system will be used. </li></ul><ul><li>SF-RMR is started by invoking the program and giving the config filename as a parameter. </li></ul>
  75. 83. <ul><li>When writing rules, it is possible to write rules, which match multiple criteria. In this case, only the first matching occurence is used. It is advised, to check rules very good before applying. </li></ul><ul><li>Not yet implemented!: </li></ul><ul><li>Situation without rule (default state). Currently, this throws an error and disconnects. </li></ul>
  76. 84. <ul><li>Global options apply to SF-RMR as whole </li></ul><ul><li>Per rule options apply to a specific relay rule </li></ul><ul><ul><li>” option” must be used, with the correct rule ID, in order to apply an option to a specific rule. </li></ul></ul><ul><li>There are some stacking options, and also some overwriting options, meaning that the last defined (from top to bottom) is valid. </li></ul>
  77. 85. <ul><li>i.e. : </li></ul><ul><li>rule 8 080 HTTP GET </li></ul><ul><li>, would redirect to the host, port 8080, when a GET is found (HTTP protocol, web server). </li></ul><ul><li>The parts are: </li></ul><ul><li>- hostname, port, rule ID, criteria </li></ul>
  78. 86. <ul><li>Is used in order to identifiy a specific redirection rule. </li></ul><ul><li>Specific options might be set on a per rule basis </li></ul>
  79. 87. <ul><li>port – the port to listen on for clients </li></ul><ul><li>allow - IP to allow (might be IPV4 or IPV6) </li></ul><ul><li>deny – IP to deny (might be IPV4 or IPV6) </li></ul><ul><li>socketmode – half-duplex, full duplex etc. </li></ul><ul><li>threadbuffersize – size of the buffer </li></ul><ul><li>regexp – starts with, ends with, contains, regular expression </li></ul><ul><li>prefilter – rule handling prefilter (if any) </li></ul><ul><li>option – used for the specific rule ID </li></ul><ul><li>loggerfile – specifies a log file </li></ul><ul><li>s ystemproperty – allows setting a system property </li></ul><ul><li>limit_threads – limits the maximum concurrent threads (load-balancing) </li></ul><ul><li>limit_unique_thread_counter , limit_unique_relay_counter – should normally not be used </li></ul><ul><li>logger_monitor_traffic – if set to 1, traffic flow is monitored </li></ul><ul><li>firstlinefilterclass – class used for the filtering of the first line </li></ul><ul><li>dynamicredirection – used for dynamic redirection </li></ul><ul><li>staticfallback – whether to use static rules or not (works in conjunction withdynamicredirection) </li></ul><ul><li>Invalidrulefallback – if static configuration rule does not apply, redirect to this class </li></ul>
  80. 88. <ul><li>isproces – a proces, not socket redirection is used </li></ul><ul><li>waitingfor – half-duplex, wait for expression </li></ul><ul><li>nofirstlinerelay – the rule expression itself relay </li></ul><ul><li>timeout – timeout of a connection </li></ul><ul><li>charmode – character mode </li></ul><ul><li>unidirectional – half-duplex per socket basis </li></ul><ul><li>upstream – upstream filter class name </li></ul><ul><li>downstream – downstream filter class name </li></ul><ul><li>timeout – timeout to use </li></ul><ul><li>ReverseTCP – uses the reverse TCP and specifies the timeout </li></ul>
  81. 89. <ul><li>When invoked i.e. : </li></ul><ul><ul><li>java -jar SF-RMR.jar Protumulti.properties </li></ul></ul><ul><ul><li>, and Protumulti.properties content is defined so: </li></ul></ul><ul><ul><li>port 80 </li></ul></ul><ul><ul><li>r ule 1 28 . 169 . 1 .1 8 080 HTTP HEAD </li></ul></ul><ul><ul><li>rule 1 28 . 169 . 1 .1 8 080 HTTP GET </li></ul></ul><ul><ul><li>rule 1 28 . 169 . 1 .1 8 080 HTTP POST </li></ul></ul><ul><ul><li>rule 1 28 . 169 . 1 .1 8 080 HTTP OPTIONS </li></ul></ul><ul><ul><li>rule 12 8 . 169 . 1 .1 9001 UP <U> </li></ul></ul><ul><ul><li>Would listen on port 80. For any rule which is HEAD, GET, POST, OPTIONS and <U> the corresponding rule is invoked. </li></ul></ul>
  82. 90. <ul><li>For any Rule ID, specific options could be defined, i.e. : </li></ul><ul><ul><li>option HTTP upstream UpstreamFilter </li></ul></ul><ul><ul><li>option HTTP downstream DownstreamFilter </li></ul></ul><ul><ul><li>option HTTP timeout 30000 </li></ul></ul><ul><ul><li>Would define extra options for rule ID ”HTTP”. In this example, there would be UpstreamFilter.class and DownstreamFilter.class applied to communication, and if no communication occurs in the time of 30000 msec (30 sec), the connection is dropped. </li></ul></ul>
  83. 91. <ul><li>Because Java is very flexible, some options can be defined outside SF-RMR without it’s intervention, i.e. If needed: </li></ul><ul><li>proxy server </li></ul><ul><li>java class locations (classpath) </li></ul><ul><li>etc. </li></ul>
  84. 92. <ul><li>Filtering rule can be also a java regular expression. </li></ul><ul><li>More on Java regular expression can be found on: </li></ul><ul><li>http://download.oracle.com/javase/tutorial/essential/regex/ </li></ul><ul><li>If using regular expressions, the global option regexp must be set! : </li></ul><ul><ul><ul><li>regexp 3 </li></ul></ul></ul>
  85. 94. executing proceses (socksifying)
  86. 95. <ul><li>Executing proceses enables any console aplication to be ”socksified”, meaning the input and output are redirected over the network (remote shell). </li></ul>
  87. 96. Any proces input/output can be redirected over the network, instead being localy bound. This makes it a remote proces, which can be used localy. This brings also an added value, that the proces could be run on a more powerfull machine and/or in a cloud, while the actual input/output data could be presented in a remote location.
  88. 97. <ul><li>The communication is unsecured by default. Anyone could easly spy on the comunication transfered over the network, unlike the physical input/output. This can be changed if a secure shell is used. </li></ul>
  89. 98. <ul><li>example: </li></ul><ul><ul><ul><ul><li>rule ftp.exe 0 EXE ftp </li></ul></ul></ul></ul><ul><ul><ul><ul><li>option EXE isproces 1 </li></ul></ul></ul></ul><ul><ul><ul><ul><li>option EXE waitingFor ftp> </li></ul></ul></ul></ul><ul><ul><ul><ul><li>option EXE nofirstlinerelay 1 </li></ul></ul></ul></ul><ul><ul><ul><ul><li>option EXE timeout 6000 </li></ul></ul></ul></ul><ul><li>Creates a redirection rule with the ID ”EXE”, which starts, if ”ftp” is encountered. This rule has options: </li></ul><ul><ul><ul><li>is a proces </li></ul></ul></ul><ul><ul><ul><li>until prompt ”ftp>”, whole data is echoed </li></ul></ul></ul><ul><ul><ul><li>the rule itself is not relayed to proces </li></ul></ul></ul><ul><ul><ul><li>timeout is 6 seconds, if no activity occurs </li></ul></ul></ul><ul><li>The proces, once started, is run until termination from either side. Only the rule and ”isproces” are mandatory. </li></ul>
  90. 100. Java system properties
  91. 101. <ul><li>These control, how Java, and consequentely, SF-RMR will behave. </li></ul><ul><li>They apply on a per-instance-basis, menaing only are valid for the current SF-RMR (in case multiple configurations are run on a machine). </li></ul><ul><li>They can be set either when invoking SF-RMR or through configuration. </li></ul><ul><li>They rely heavy on Java version and flavour used! </li></ul>
  92. 102. <ul><li>For SF-RMR, these might be, if required: </li></ul><ul><ul><li>proxy </li></ul></ul><ul><ul><li>IPV4 against IPV6 protocol </li></ul></ul><ul><ul><li>others </li></ul></ul>
  93. 103. <ul><li>If SF-RMR is invoked directly in Java, then for example: </li></ul><ul><ul><ul><li>java -jar –D java.net.preferIPv4Stack=true PM.jar Protumulti.properties </li></ul></ul></ul><ul><li>or, through the environment variable (Linux): </li></ul><ul><ul><ul><li>export _JAVA_OPTIONS=‘ java.net.preferIPv4Stack=true ’ </li></ul></ul></ul><ul><li>or, in i.e. Windows: </li></ul><ul><ul><ul><ul><ul><li>set _JAVA_OPTIONS= java.net.preferIPv4Stack=true </li></ul></ul></ul></ul></ul>
  94. 104. <ul><li>This can also be done in the SF-RMR configuration file, for example: </li></ul><ul><ul><ul><li>systemproperty java.net.preferIPv4Stack=true </li></ul></ul></ul><ul><ul><li>Which would set the corresponding property (left) to the desired value (right). This is valid until SF-RMR is terminated. More than one system property can be set, by using this option more than once in configuration, each time changing the name and value of the system property. </li></ul></ul>
  95. 105. <ul><li>Depending on the Java flavour and version, the available system properties might vary. </li></ul><ul><li>A list of available system properties is generally accessible somewhere in the documentation of the Java flavour and version. </li></ul><ul><li>For original Oracle (Sun) Java, the list of the essential ones can be seen on the internet, i.e. : </li></ul><ul><ul><li>http://download.oracle.com/javase/tutorial/essential/environment/sysprop.html </li></ul></ul>
  96. 106. <ul><li>Java itself has the ability to list the available system properties, i.e. thorugh the following code snippet: </li></ul><ul><ul><ul><ul><li>   Enumeration list = System.getProperties().propertyNames();      while(list.hasMoreElements()){     System.out.println((String) list.nextElement());   } </li></ul></ul></ul></ul>
  97. 107. These examples apply to original Java, some versions: System property Meaning java.net.preferIPv4Stack Preferered protocol stack, IPv4 instead of IPv6 java.net.preferIPv6Addresses This controls whether IPv6 (true) or IPv4 (false) addresses are used. http.proxyHost the host name of the proxy server http.proxyPort the port number, the default value being 80 http.nonProxyHosts a list of hosts that should be reached directly, bypassing the proxy. socksProxyHost the host name of the SOCKS proxy server socksProxyPort the port number, the default value being 1080
  98. 108. http://download.oracle.com/javase/1.5.0/docs/guide/net/ipv6_guide/index.html http://download.oracle.com/javase/1.4.2/docs/guide/net/properties.html http://download.oracle.com/javase/6/docs/technotes/guides/net/proxies.html
  99. 109. <ul><li>Java has some secuirity features, and will not allways allow anyone to change just any system property. Security manager must be configured to allow change in the particular rule. </li></ul><ul><li>More reading: </li></ul><ul><li>http://download.oracle.com/javase/tutorial/essential/environment/security.html </li></ul>
  100. 111. UDP sockets
  101. 112. What are TCP sockets? <ul><li>transfer control protocol sockets </li></ul><ul><li>more secure </li></ul><ul><li>more robust </li></ul><ul><li>statefull </li></ul><ul><li>Connection-oriented (the info is contained in the socket) </li></ul>
  102. 113. What are UDP sockets? <ul><li>user datagram sockets </li></ul><ul><li>do not guarantee that all data is transmitted </li></ul><ul><li>stateless </li></ul><ul><li>connection data is contained in the datagram </li></ul>
  103. 114. UDP versus TCP <ul><li>lighter and less bandwidth hungry </li></ul><ul><li>less secure </li></ul>
  104. 115. Why use UDP? Some protocols heavy depend on it (i.e. video and audio transmissions, time transmissions etc.). Always, when there is no need to transfer 100% of data, but to transfer on time.
  105. 116. How to use UDP? In configuration file, there are local rule options: option Rule_ID isUDP 1 -> this uses UDP instead of TCP on relay side option Rule_ID streamedUDP 0 -> This creates an “UDP stream” client option Rule_ID streamedUDP 8080 -> This would create an “UDP stream” server, which would be listening to UDP port 8080. “ UDP stream” server UDP mode is similar as “reverse TCP” mode, albeit UDP itself is a connection-less protocol, therefore these modes work similar. It only metters towards the relay side, if other applications need one or another.
  106. 118. usage scenarios
  107. 119. <ul><li>client(s) and server(s) communicate directly </li></ul>
  108. 120. <ul><li>SF-RMR is inserted inside the chain, acting as an intermediate integrated part, a gateway. </li></ul>
  109. 121. <ul><li>When inserting SF-RMR, either: </li></ul><ul><ul><li>existing servers need to be reconfigured to use different ports </li></ul></ul><ul><li>or </li></ul><ul><ul><li>SF-RMR has to use an unused port and then be bound to the server(s) real ports. </li></ul></ul>
  110. 122. <ul><li>SF-RMR can be invoked as a standard Java application, by using: </li></ul><ul><ul><ul><li>java -jar SF-RMR.jar Protumulti.properties </li></ul></ul></ul><ul><li>In this example, it is assumed, that SF-RMR is in jar form and the configuration file to be used is called Protumulti.properties. Some for of Java environment must be installed, in order for this to work! </li></ul>
  111. 123. <ul><li>Albeit it is possible to run SF-RMR as an applet or servlet, this is not yet officially implemented. </li></ul><ul><li>In order to use this functionality, a Java wrapper class must be used, either a servelt or an applet class, which instances SF-RMR. </li></ul>
  112. 124. <ul><li>Depending on the operating system used, it is possible to run SF-RMR in the background, have it started automatically every time the system start etc. This is realised differently, depending on the OS used. </li></ul><ul><li>Some OS’es use the & sign for background operation, some use special commands, i.e. nice, renice and start commands to set priorities etc. How exatly a demon/service is started, depends on the OS itself. </li></ul>
  113. 125. <ul><li>As Java can be recompiled to native application using some of already existing tools, SF-RMR can also be started natively. </li></ul><ul><li>The big advantage is greater speed (claimed 30% faster by the Java native compiler software), which could be important when large amount of clients is connected. </li></ul><ul><li>In this case, java classes need to be compiled to native code, either by straight compiling, or crosscompiling. </li></ul><ul><li>This working mode is heawly architecture dependable. </li></ul>
  114. 126. <ul><li>This is the most complex feature, as the output of one SF-RMR instance can be used as input for another instance, or even as an instance for something else (i.e. other socket software , i.e. rinetd or netcat etc.). </li></ul>
  115. 127. <ul><li>SF-RMR can also be chained by other proceses. Everything, that can redirect standard input/output (stdin & stdout). On some Oses some special pipe symbol combinations need to be used... </li></ul><ul><li>Usually, this would mean the special signs < > | >> << etc. </li></ul>
  116. 128. Where can it be downloaded? <ul><li>http://code.google.com/p/sf-rmr/ </li></ul>