WebTalkImplementing WebServices with a dedicatedJava daemonGeert Van Pamel – HP-Interex BeluxMannheim, Germany 10 November...
Introduction• A Java application behaving as a web server• Java & the HTTP protocol is the base• and IP TCP + UDP programm...
Why this Project?• Interoperability of networked clients & servers• Using Open standards• HTTP protocol• Network socket pr...
Why Java?• Using standard Java• Extensible• Can run on any platform – CORE (code once run everywhere)• Easy testing• Easy ...
ArchitectureWebTalkClientWebTalkServerWebTalkClientLegacyserverLegacyserverManagementStationAlert
Useful? Let me know!• Home use• Enterprise use• You manage everything• No dependencies with other packages• It is free, no...
The Basic Algorithm• Initialise the program• Verify that it runs on one of the registered servers• Verify the command para...
Program Headerpackage webtalk;import java.io.*;import java.net.*;import java.util.*;public class WebTalk {// Creates a new...
Program initialization (1)private static int tcpPort = 2006; // P1 = listen port: 2006private static String logDirName = "...
Program initialization (2)// P3: Open the log file at startupif (args.length >= 3 && args[2].indexOf("log") >= 0) try {log...
Blocking Unauthorized Server Implementationtry {serverAddress = InetAddress.getLocalHost();remoteHost = serverAddress.getH...
Open main Server Sockettry {server = new ServerSocket(tcpPort);} catch (IOException e) {sendPanic(e.toString());System.exi...
Waiting the next Clientwhile (true) {try {client = server.accept();} catch (IOException e) {sendPanic(e.toString());System...
Client Access Controlprivate final static int MAXIP = 50;private static int clientCount[] = new int[1+MAXIP];private stati...
Getting Browser InputString webTalkIn;webTalkIn = "n-- Start request";// Get the client requestwhile (webTalkIn != null &&...
Parsing the Client URLString webRequest[]; // GET /urlPath/...?webParam=value&... HTTP/1.1String mainPath[];String urlPath...
Parsing HTTP Headers• User-Agent• Via• Accept-Language• HTTP/1.x• Authentication• Connection: keep-alive
Identifying the Browser/* set default userAgent */if (userAgent == unkownUserAgent&& webRequest.length >= 2&& webRequest[0...
The main Server LogicSome examples we have implemented with success in production are:• Validating Web forms by calling We...
Returning Server Resultsstatic void writeBody (String bodyText) {int bodyLength;String headerStr;if (userAgent > unknownUs...
GMT Time in HTTP Format01234567890123456789012345678Fri Nov 03 13:51:56 GMT 2006 standard GMT format (gmTime)Fri, 03 Nov 2...
Disabling Client Browser CacheOur application is data drivenWe must ensure that browsers, or proxies nevercache requests.T...
Close the Doortry {in.close();out.close();client.close();} catch (IOException e) {sendPanic(e.toString());System.exit(6);}...
Client Examples in practice• Normal clients can be browsers like Firefox [8], Opera [8], or InternetExplorer:http://server...
Other Clients• Perl• PHP• Unix Socket programs• .Net clients• Database systems like Oracle [6]• Telecom servers like IVRAn...
Example: Perluse IO::Socket::INET;$Server = "servername:2006";my $sock = IO::Socket::INET->new(PeerAddr => $Server, Proto ...
Example: Oracle stored ProcedureCREATE FUNCTION web_request (request in string)return string isc utl_tcp.connection; len P...
Caveat Firefox and Opera[server loop]if (urlPath.length >= 2 &&urlPath[1].equals("favicon.ico")) {runMode = "ignore";}[gen...
Backend Systems• Oracle with permanent connection• MySQL database gateway• Middleware• Other SOA systems
Server Management – Starting the Server• On Unix systems:su - apache -c "java -jar /html/java/bin/webtalk.jar 2006 normal ...
Stopping the Server[Unix]• ps –ef |grep webtalk• kill $pid[Windows]• tasklist /fi “imagename eq java.exe”• taskkill /pid P...
Temporarily Disable – Panic Button• Uncouple the system to avoid downstream catastropheshttp://servername:2006/disablehttp...
Resume normal functioninghttp://servername:2006/normalhttp://servername:2006/[no]tracehttp://servername:2006/[no]debugInte...
Recycling the Log Filedaynum=$(date +%u)for port in 2005 2006 ;dolynx -dump http://servername:$port/nolog > /dev/nullmv /t...
Open and close the Log Fileif (urlPath.length >= 2 && urlPath[1].equals("log") && log == null) {clientResult = "Append log...
Local Server MonitoringTo verify that the process runs:[Unix]ps -ef |grep "java -jar"apache 114167 1 0.0 14:01:20 pts/0 0:...
Remote Server Monitoringfor tn in 2005 2006 ;dotelnet servername $tn <<EOF 2>&1 >/dev/null |grep -v Connection closed by f...
Status of the Serverhttp://servername:2006/status[server loop]if (urlPath.length >= 2 && urlPath[1].equals("status")) {run...
Usage Statisticshttp://servername:2006/countersUsage counters per IP address since Mon Jun 04 20:56:39 GMT 2007:192.168.86...
Resetting Countershttp://servername:2006/resetif (urlPath.length >= 2 && urlPath[1].equals("reset")) {resetTime = gmTime;c...
Building the Application• java -versionjava version "1.4.2"Java(TM) 2 Runtime Environment, Standard EditionFast VM (build ...
Run Interactively• [Unix]export CLASSPATH=lib1.jar:lib2.jar:.• [Windows]set CLASSPATH=lib1.jar;lib2.jar;.• To start the pr...
Build the JAR File• The manifest file can contain a list of all other required jar files andthe name of the main module to...
Simulate a Server (1)vi netcat.txtHTTP/1.1 200 OKDate: Tue, 29 May 2007 11:47:08 GMTServer: WebTalk/20070528 (Java/HTTP)La...
Simulate a Server (2)while true ;do nc –lp 2005 < netcat.txt ;donelynx –dump http://localhost:2005/testlynx –source http:/...
Error HandlingDaemon process!Alarm notificationAllow for easy supportError loggingThe program stopsFatal errorsLogged, sta...
General Exception Handlingtry {// put main program logic here} catch (Exception e) {e.printStackTrace ();sendPanic(e.toStr...
Enable Debugging• To enable & disable debugginghttp://servername:2006/[no]debugif (urlPath.length >= 2 && urlPath[1].equal...
Enable Tracing• To enable & disable tracing:http://servername:2006/[no]traceif (urlPath.length >= 2 && urlPath[1].equals("...
Enable Quiet Mode• To set quiet mode(when you would have too many transactions)http://servername:2006/quietif (urlPath.len...
Error Loggingstatic void sendPanic (String panicString) {writeLog(panicString);if (smsRelayAddress != null)sendSMS (server...
Sending SMS Alarmstatic void sendSMS (String smsString) { byte sms[];DatagramPacket smsPacket;DatagramSocket smsSocket;try...
UDP to TCP Relay Server (1)import java.io.*;import java.net.*;public class RelaySmsUdp {public static void main (String ar...
try {smsSocket = new DatagramSocket(535); // listen to portwhile (true) try {smsPacket = new DatagramPacket (sms, sms.leng...
Disable SMS at Startup• Can be very handy while developingprivate static InetAddress smsRelayAddressDefault;private static...
Disable or Enable SMS at runtimelynx -dump http://servername:2006/[no]sms• The following code gets executed:if (urlPath.le...
Supported Server PlatformsJava virtually runs on all platforms:– You only have to distribute the byte code in a Jar file!•...
Implemented Standards• DNS• HTTP, see [2]• Internet date & timeformat• Java, see [3] [4] [5]• NFS during development• NTP•...
What I have learned• Think in objects• Keep it simple (divide and conquer)• Client /server programming is fun... debugging...
Design Principles• Separate code and data• Flexible data structures• Fully table & data driven• Keep the code simple• Code...
Future Extensions and Migration• Advanced Java debugging• Build and use appliances• Application setup via config fileinste...
References[1] Full program source and additional material: http://www.hp-interex.be/pub/project/webtalk[2] The HTTP protoc...
Thanks• My colleague Danny De Thaye to encourage me in myendeavors• Connect Management for accepting my lecture• To you, t...
Questions
Abstract• The speaker presents a Java application that behaves like a dedicated web serverand provides Web services. It ru...
About the Author• The author works at Belgacom since 1997 in the Customer division as anICT Project Manager.• He is Chairm...
Useful Utilities• Any web browser [8] To test the server (Firefox, Internet Explorer, Opera, lynx)• crontab To reset the d...
Technical Terms• API Application Programming Interface• CRLF Carriage Return and Line Feed• GMT Greenwich Mean Time, almos...
nslookupstatic String nslookup (String hostName) {String ipAddress;try {ipAddress =InetAddress.getByName(hostName).getHost...
rnslookupstatic String rnslookup (String ipAddress) {String hostName;try {hostName =InetAddress.getByName(ipAddress).getHo...
Upcoming SlideShare
Loading in …5
×

WebTalk - Implementing Web Services with a dedicated Java daemon

6,606 views

Published on

The speaker presents a Java application that behaves like a dedicated web server and provides Web services. It runs as a standalone network daemon and only calls system and network services that are directly executed by the JVM. No other software is necessary. The URL path referenced by the clients is not referring to files. Instead it is calling other middleware services and serves as a Web service gateway. The application only requires TCP/IP and Java library programming. Because it behaves as a web server, the browser can be any client capable of interfacing with the standard HTTP protocol.

Learn about Java network programming, specifically about security, reliability, supportability, monitoring, browser cache disabling, kit building, starting the Java server process, troubleshooting and monitoring, implementing on different operating systems and platforms, and integrating with other services. Debugging server processes is difficult because there is no direct user interface. Every interaction goes via network sockets, or process signals. Remotely control the server by using the HTTP protocol, enable and disable logging, tracing, and debugging info.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
6,606
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

WebTalk - Implementing Web Services with a dedicated Java daemon

  1. 1. WebTalkImplementing WebServices with a dedicatedJava daemonGeert Van Pamel – HP-Interex BeluxMannheim, Germany 10 November 2008Talking tothe web
  2. 2. Introduction• A Java application behaving as a web server• Java & the HTTP protocol is the base• and IP TCP + UDP programming• KISS – Keep it simple, but fully functional• Virtual file system– Not about Applets, Servlets or JSP– Not about Jakarta or Tomcat– No SOAP, no BPEL, no WSDL
  3. 3. Why this Project?• Interoperability of networked clients & servers• Using Open standards• HTTP protocol• Network socket programming (the network is the system)• Interfacing with middleware (building blocks)• Any client supporting the HTTP protocol• Reliable and performance• 24/7 daemon written in Java
  4. 4. Why Java?• Using standard Java• Extensible• Can run on any platform – CORE (code once run everywhere)• Easy testing• Easy implementation (distribute byte code)• Can easily interface with other Java classes• Use with any backend supporting Java• Stable, resilient & very good exception handling• Well documented, tools available• Modern language – many programmers available
  5. 5. ArchitectureWebTalkClientWebTalkServerWebTalkClientLegacyserverLegacyserverManagementStationAlert
  6. 6. Useful? Let me know!• Home use• Enterprise use• You manage everything• No dependencies with other packages• It is free, no licenses• Just invest your own few mandays• Download the template!
  7. 7. The Basic Algorithm• Initialise the program• Verify that it runs on one of the registered servers• Verify the command parameters, overruling default parameters• Open the log file if requested• Create the main server listening socket• Perform the main server loop (endless)– Listen to the network port and wait for the next client request– Get the client identification– Check access authorisation– Get the client capabilities– Get the client input parameters– Parse input parameters– Validate input parameters– Perform a server transaction– Convert the status and results to the client capabilities– Log the request and the results in the error log– Return status and results to the client program– Close the client network connectionA daemonnever dies
  8. 8. Program Headerpackage webtalk;import java.io.*;import java.net.*;import java.util.*;public class WebTalk {// Creates a new instance of WebTalkpublic WebTalk() {}// main program}
  9. 9. Program initialization (1)private static int tcpPort = 2006; // P1 = listen port: 2006private static String logDirName = "/tmp";private static FileOutputStream logFile; // P3 = logging mode: log,smsprivate static PrintWriter log = null;public static void main(String[] args) {String crlfStr = System.getProperty("line.separator");String runModeDefault = "normal"; // P2 = run mode: normalif (crlfStr.equals("rn")) logDirName = "c:temp";if (args.length >= 1 && !args[0].equals("")) tcpPort = Integer.parseInt(args[0]); // P1: listen portlogFileName = "webtalk-" + tcpPort + ".log";if (args.length < 3 || args[2].indexOf("nolog") < 0) try { // P3: Open the log file at startuplogFile = new FileOutputStream(new File(logDirName, logFileName), true);log = new PrintWriter (logFile, true);} catch (FileNotFoundException e) {writeLog(e.toString()); // But we continue processing, since not fatal}if (args.length >= 2 && !args[1].equals("")) runModeDefault = args[1]; // P2: Set trace modeif (!runModeDefault.equals("normal")) writeLog("Enable " + runModeDefault);}Firstimplemented in2006
  10. 10. Program initialization (2)// P3: Open the log file at startupif (args.length >= 3 && args[2].indexOf("log") >= 0) try {logFile = new FileOutputStream(new File(logDirName,logFileName), true);log = new PrintWriter (logFile, true);} catch (FileNotFoundException e) {writeLog(e.toString());// But we continue processing, since not fatal}defaultnolog
  11. 11. Blocking Unauthorized Server Implementationtry {serverAddress = InetAddress.getLocalHost();remoteHost = serverAddress.getHostAddress();serverHost = remoteHost;serverName = serverAddress.getCanonicalHostName();} catch (UnknownHostException e) {sendPanic(e.toString());System.exit(8);}if (!allowRemoteHost (remoteHost)) {sendPanic("Reject host"); // Unauthorized server (address)System.exit(2);}
  12. 12. Open main Server Sockettry {server = new ServerSocket(tcpPort);} catch (IOException e) {sendPanic(e.toString());System.exit(1);}private final static String versionString ="WebTalk/20081013 (Java/HTTP)";sendPanic("Starting " + versionString);Server upand running
  13. 13. Waiting the next Clientwhile (true) {try {client = server.accept();} catch (IOException e) {sendPanic(e.toString());System.exit(4);}remoteAddress = client.getRemoteSocketAddress().toString();remoteHost = remoteAddress.substring(1, remoteAddress.indexOf(:, 1));try{in = new BufferedReader(new InputStreamReader(client.getInputStream(), webCharset));out = new PrintStream(client.getOutputStream(), false, webCharset);} catch (IOException e) {sendPanic(e.toString());System.exit(5);}// main processing logic here}A daemonnever dies
  14. 14. Client Access Controlprivate final static int MAXIP = 50;private static int clientCount[] = new int[1+MAXIP];private static String allowAddress[] = newString [1+MAXIP];…allowAddress[14] =nslookup("client1.mshome.net");allowAddress[15] = "192.168.210.34";…if (!allowRemoteHost (remoteHost)) {sendPanic("Reject host: " +remoteHost);}static boolean allowRemoteHost (StringremoteAddress) {boolean allowIP;allowIP = false;for (int i = 1; i < allowAddress.length; i++){if(remoteAddress.equals(allowAddress[i])) {allowIP = true;clientCount[i]++;break;}}return allowIP;}
  15. 15. Getting Browser InputString webTalkIn;webTalkIn = "n-- Start request";// Get the client requestwhile (webTalkIn != null && !webTalkIn.equals("")) {// Here comes the parsing of the browser request// Get the next HTTP header until empty line encounteredwebTalkIn = in.readLine();}TCP doesnot haveEOF
  16. 16. Parsing the Client URLString webRequest[]; // GET /urlPath/...?webParam=value&... HTTP/1.1String mainPath[];String urlPath[];String webParam[];String paramPair[];writeLog(remoteHost + " " + gmTime + " " + webTalkIn);webRequest = webTalkIn.split(" ");mainPath = webRequest[1].split("[?]", 2);urlPath = mainPath[0].split("/");webParam = mainPath[1].split("&");/* Here we should map the key/value pairs into the application logic */for (int c = 0; c < webParam.length; c++) {paramPair = webParam[c].split("=", 2);paramPair[0] = paramPair[0].toUpperCase();//System.out.println(paramPair[0]);//System.out.println(paramPair[1]);}
  17. 17. Parsing HTTP Headers• User-Agent• Via• Accept-Language• HTTP/1.x• Authentication• Connection: keep-alive
  18. 18. Identifying the Browser/* set default userAgent */if (userAgent == unkownUserAgent&& webRequest.length >= 2&& webRequest[0].equals("User-Agent:")) {if (runMode.equals("trace")) writeLog(webTalkIn);if (webRequest[1].startsWith("Mozilla")) {// Firefox, Internet ExploreruserAgent = mozillaUserAgent;} else {// Lynx, OperauserAgent = otherUserAgent;}}
  19. 19. The main Server LogicSome examples we have implemented with success in production are:• Validating Web forms by calling WebTalk via TCP socketprogramming• .Net calling the WebTalk server to validate customer orders• Validating customer orders via a DTMF and a text to speechsystem• Oracle PL/SQL calling the WebTalk server to validate customerorders via the UTL_HTTP and UTL_TCP package [6]• Storing database transactions via HTTP protocol (avoid locking)• Apache Embedded Perl calling WebTalk for order validation
  20. 20. Returning Server Resultsstatic void writeBody (String bodyText) {int bodyLength;String headerStr;if (userAgent > unknownUserAgent) {bodyLength = 2 + bodyText.length();if (userAgent >= otherUserAgent) bodyLength += 6; // <PRE> tagheaderStr = "HTTP/1.1 200 OKnDate: " + httpTime+ "nServer: " + versionString+ "nExpires: -1nCache-Control: no-cache, no-store,max-age=0, must-revalidatenPragma: no-cache"+ "nContent-Length: " + bodyLength+ "nConnection: closenContent-Type: text/htmln“;if (userAgent >= otherUserAgent) headerStr += "n<pre>";if (runMode.equals("debug")) writeLog(headerStr);headerStr = headerStr.replaceAll("n","rn") + "r";out.println(headerStr);}if (runMode.equals("debug")) writeLog(bodyText);if (userAgent > unknownUserAgent)bodyText = bodyText.replaceAll("n","rn") + "r"; // Avoid protocol errorout.println(bodyText);}HTTPneeds rn
  21. 21. GMT Time in HTTP Format01234567890123456789012345678Fri Nov 03 13:51:56 GMT 2006 standard GMT format (gmTime)Fri, 03 Nov 2006 14:51:56 GMT internet GMT format (httpTime)private static String gmTime;private static String httpTime;Date now;TimeZone tz = TimeZone.getTimeZone("GMT");tz.setDefault(tz);now = new java.util.Date();gmTime = now.toString();httpTime = gmTime.substring(0,3) + ,+ gmTime.substring(7,10)+ gmTime.substring(3,7)+ gmTime.substring(23,28)+ gmTime.substring(10,23);HTTP =WWW thusGMT time
  22. 22. Disabling Client Browser CacheOur application is data drivenWe must ensure that browsers, or proxies nevercache requests.Therefore we generate the following headers:Expires: -1Cache-Control: no-cache, no-store, max-age=0, must-revalidatePragma: no-cache
  23. 23. Close the Doortry {in.close();out.close();client.close();} catch (IOException e) {sendPanic(e.toString());System.exit(6);}• One client at a time• Avoid locking usingsingle process databaseinserts• Let the TCP stack bufferincoming requests• Latent DoS problem(if connection not properly closed)
  24. 24. Client Examples in practice• Normal clients can be browsers like Firefox [8], Opera [8], or InternetExplorer:http://servername:2006/hub/2186/ivr?p1=1234&p2=5678• With telnet:telnet servername 2006GET /hub/2186/ivr?p1=1234&p2=5678(enter)• With Lynx [8]:lynx "http://servername:2006/hub/2186/ivr?p1=1234&p2=5678"• With netcat:echo "GET /hub/2186/ivr?p1=1234&p2=5678n" |nc servername 2006
  25. 25. Other Clients• Perl• PHP• Unix Socket programs• .Net clients• Database systems like Oracle [6]• Telecom servers like IVRAny programcapable oftalking HTTP
  26. 26. Example: Perluse IO::Socket::INET;$Server = "servername:2006";my $sock = IO::Socket::INET->new(PeerAddr => $Server, Proto => tcp);$webStat = "";if ($sock) {$sock->write("GET /statusnn");$sock->recv($webStat, 2000);close($sock);}print "$webStatn";
  27. 27. Example: Oracle stored ProcedureCREATE FUNCTION web_request (request in string)return string isc utl_tcp.connection; len PLS_INTEGER;result string(2000);BEGINc := utl_tcp.open_connection (servername,2006);len := utl_tcp.write_line(c, GET || request);len := utl_tcp.write_line(c);result := utl_tcp.get_line(c, TRUE);utl_tcp.close_connection(c);return result;END;selectweb_request(‘/status’)from dual;
  28. 28. Caveat Firefox and Opera[server loop]if (urlPath.length >= 2 &&urlPath[1].equals("favicon.ico")) {runMode = "ignore";}[generating output]if (runMode.equals("ignore")) {writeBody(clientResult);}
  29. 29. Backend Systems• Oracle with permanent connection• MySQL database gateway• Middleware• Other SOA systems
  30. 30. Server Management – Starting the Server• On Unix systems:su - apache -c "java -jar /html/java/bin/webtalk.jar 2006 normal log,sms> /dev/null 2> /dev/null &"• On Windows platforms you can start the server from a scheduled tasks (at system startup)java -jar C:htmljavabinwebtalk.jar 2006 trace > nul 2> nul• General format of startup command:java -jar webtalk.jar [port] [runmode] [log,sms] [> nul] [2> nul]Parameters:port: TCP port number (default 2006; you could use 2005 for testing)runmode: debug | test | trace | quiet | disable | enable | normal (default)log: "log" to trigger the logging to disk, with nolog logging is disabledsms: "nosms" to disable sending SMS messages
  31. 31. Stopping the Server[Unix]• ps –ef |grep webtalk• kill $pid[Windows]• tasklist /fi “imagename eq java.exe”• taskkill /pid ProcessIDKilling thedaemon
  32. 32. Temporarily Disable – Panic Button• Uncouple the system to avoid downstream catastropheshttp://servername:2006/disablehttp://servername:2006/enableInternally the following code gets executed:if (urlPath.length >= 2&& (urlPath[1].equals("disable")|| urlPath[1].equals("enable") ) ) {runModeDefault = urlPath[1];clientResult = "Server: " + runModeDefault;sendPanic(clientResult);}
  33. 33. Resume normal functioninghttp://servername:2006/normalhttp://servername:2006/[no]tracehttp://servername:2006/[no]debugInternally the following code gets executed:if (urlPath.length >= 2 && urlPath[1].equals("normal")) {runModeDefault = urlPath[1];clientResult = "Enable " + urlPath[1];}
  34. 34. Recycling the Log Filedaynum=$(date +%u)for port in 2005 2006 ;dolynx -dump http://servername:$port/nolog > /dev/nullmv /tmp/webtalk-$port.log /scratch/webtalk/log/webtalk-$port-$daynum.loglynx -dump http://servername:$port/log > /dev/nulldoneAvoiddisk fullerrors
  35. 35. Open and close the Log Fileif (urlPath.length >= 2 && urlPath[1].equals("log") && log == null) {clientResult = "Append logfile";logFile = new FileOutputStream(new File(logDirName,logFileName), true);log = new PrintWriter (logFile, true);}else if (urlPath.length >= 2 && urlPath[1].equals("nolog")&& log != null) {clientResult = "Closing logfile";logFile.close();log = null;}
  36. 36. Local Server MonitoringTo verify that the process runs:[Unix]ps -ef |grep "java -jar"apache 114167 1 0.0 14:01:20 pts/0 0:01.63 java -jar /html/java/bin/webtalk.jar2006[Windows]tasklist /fi "imagename eq java.exe" /vImage Name PID Session Name Session# Mem Usage Statusjava.exe 716 Console 0 12.676 K UnknownNetwork monitoringnetstat [-a] [-n] |grep webtalk |wc
  37. 37. Remote Server Monitoringfor tn in 2005 2006 ;dotelnet servername $tn <<EOF 2>&1 >/dev/null |grep -v Connection closed by foreign host. |/usr/local/bin/sendpanic "" "servername:$tn"GET /status HTTP/1.0EOFdone
  38. 38. Status of the Serverhttp://servername:2006/status[server loop]if (urlPath.length >= 2 && urlPath[1].equals("status")) {runMode = "status";}[generating output]if (runMode.equals("status")) { runMode = runModeDefault;clientResult = "Server: " + serverName + "(" + serverHost + ")" +"nPort: " + tcpPort + "nClient: " + remoteHost + "nStartTime: " + startTime + "nReset Time: " + resetTime + "nTime:" + gmTime + "nVersion: " + versionString + "nMode: " +runMode + "nLanguage: " + acceptLanguage + "nUser Agent:" + userAgent + "nTop Client: " + allowAddress[1] + "(" +clientCount[1] + ")" ; writeBody(clientResult);}
  39. 39. Usage Statisticshttp://servername:2006/countersUsage counters per IP address since Mon Jun 04 20:56:39 GMT 2007:192.168.86.109 2110 client1.mshome.net192.168.219.113 1198 client2.mshome.net192.168.112.50 916 client3.mshome.net• This is handled by the following code:if (urlPath.length >= 2 && urlPath[1].equals("counters")) {clientResult = "Usage counters per client since " + resetTime + ":n";for (int i = 1; i < allowAddress.length; i++)if (!allowAddress[i].equals(""))clientResult += "n" + allowAddress[i]+ "t" + clientCount[i]+ "t" + rnslookup (allowAddress[i]);
  40. 40. Resetting Countershttp://servername:2006/resetif (urlPath.length >= 2 && urlPath[1].equals("reset")) {resetTime = gmTime;clientResult = "Reset counters";for (int i = 1; i < allowAddress.length; i++)clientCount[i] = 0;}
  41. 41. Building the Application• java -versionjava version "1.4.2"Java(TM) 2 Runtime Environment, Standard EditionFast VM (build 1.4.2-6, native threads, mixed mode, 01/09/2007-23:17)• Store the source code in a subdirectory, andcompile with javac.javac webtalk/WebTalk.javamkdir webtalk/RCSci –l webtalk/WebTalk.java
  42. 42. Run Interactively• [Unix]export CLASSPATH=lib1.jar:lib2.jar:.• [Windows]set CLASSPATH=lib1.jar;lib2.jar;.• To start the program interactively, immediately aftercompiling:java webtalk/WebTalk 2005 debug nolog,nosms
  43. 43. Build the JAR File• The manifest file can contain a list of all other required jar files andthe name of the main module to run. If your application iscomplex, multiple jar files will be needed.jar -cv0mf webtalk.jar webtalk/manifest.mf webtalk/*.classvi webtalk/manifest.mfManifest-Version: 1.0Created-By: 1.3.0_05 (Sun Microsystems Inc.)Class-Path: lib1.jar lib2.jarMain-Class: webtalk/WebTalk
  44. 44. Simulate a Server (1)vi netcat.txtHTTP/1.1 200 OKDate: Tue, 29 May 2007 11:47:08 GMTServer: WebTalk/20070528 (Java/HTTP)Last-Modified: Tue, 29 May 2007 11:47:08 GMTExpires: -1Cache-Control: no-cache, no-store, max-age=0, must-revalidatePragma: no-cacheContent-Length: 18Connection: closeContent-Type: text/htmlP1:0003|P2:42108Coding,compiling,testing takes alot of time
  45. 45. Simulate a Server (2)while true ;do nc –lp 2005 < netcat.txt ;donelynx –dump http://localhost:2005/testlynx –source http://localhost:2005/testnc –p 2005 localhosttelnet localhost 2005
  46. 46. Error HandlingDaemon process!Alarm notificationAllow for easy supportError loggingThe program stopsFatal errorsLogged, status to clientUnexpected errorsReturned to clientapplication, not loggedExpected errorsHow to handleError Type
  47. 47. General Exception Handlingtry {// put main program logic here} catch (Exception e) {e.printStackTrace ();sendPanic(e.toString()); // could also use e.getMessage()System.exit(7);} finally {sendPanic("Server closing down");}
  48. 48. Enable Debugging• To enable & disable debugginghttp://servername:2006/[no]debugif (urlPath.length >= 2 && urlPath[1].equals("debug")) {runModeDefault = urlPath[1];clientResult = "Enable debug";}
  49. 49. Enable Tracing• To enable & disable tracing:http://servername:2006/[no]traceif (urlPath.length >= 2 && urlPath[1].equals("trace")) {runModeDefault = urlPath[1];clientResult = "Enable trace";}
  50. 50. Enable Quiet Mode• To set quiet mode(when you would have too many transactions)http://servername:2006/quietif (urlPath.length >= 2 && urlPath[1].equals("quiet")) {runModeDefault = urlPath[1];clientResult = "Enable quiet mode";}
  51. 51. Error Loggingstatic void sendPanic (String panicString) {writeLog(panicString);if (smsRelayAddress != null)sendSMS (serverName + " " + serverHost + ":"+ tcpPort + " " + panicString+ " from " + remoteHost + " " + gmTime);}static void writeLog (String logText) {system.err.println(logText);if (log != null) {log.println(logText);}}
  52. 52. Sending SMS Alarmstatic void sendSMS (String smsString) { byte sms[];DatagramPacket smsPacket;DatagramSocket smsSocket;try {sms = smsString.getBytes();smsPacket = new DatagramPacket (sms,sms.length, smsRelayAddressDefault, 535);smsSocket = new DatagramSocket();smsSocket.send (smsPacket);smsSocket.close();} catch (Exception e) { // ignore}}UDP
  53. 53. UDP to TCP Relay Server (1)import java.io.*;import java.net.*;public class RelaySmsUdp {public static void main (String args[]) {DatagramSocket smsSocket;DatagramPacket smsPacket;InetAddress clientAddress;String clientHost;String panicString;Socket outSocket;PrintWriter out;byte sms[] = new byte[165]; // byte arrayString panicDest;String smsHost;panicDest = "0123456789"; // your Mobile alarm numberif (args.length >= 1 && !args[0].equals("")) panicDest = args[0];smsHost = "sms-tcp-gateway";if (args.length >= 2 && !args[1].equals("")) smsHost = args[1];
  54. 54. try {smsSocket = new DatagramSocket(535); // listen to portwhile (true) try {smsPacket = new DatagramPacket (sms, sms.length);smsSocket.receive (smsPacket);clientAddress = smsPacket.getAddress();clientHost = clientAddress.getHostAddress();panicString = new String (smsPacket.getData(), 0, smsPacket.getLength());if (!panicString.startsWith("0")) panicString = panicDest + " " + panicString;// prefix with Mobile number if neededif (clientHost.startsWith("192.168.")) { // Filter non-registered IPaddressesoutSocket = new Socket (smsHost, 535);out = new PrintWriter(outSocket.getOutputStream());if (panicString.length() > 165)panicString = panicString.substring(0, 165);out.println (panicString);out.close();outSocket.close();}} catch (UnknownHostException e) {writeLog(e.toString());} catch (IOException e) {writeLog(e.toString());}} catch (SocketException e) {writeLog(e.toString());}Relay to TCPfor messagetransmission
  55. 55. Disable SMS at Startup• Can be very handy while developingprivate static InetAddress smsRelayAddressDefault;private static InetAddress smsRelayAddress;try {smsRelayAddressDefault = InetAddress.getByName("sms-udp-gateway");} catch (Exception e) {smsRelayAddressDefault = null;}smsRelayAddress = smsRelayAddressDefault;if (args.length >= 3 && args[2].indexOf("nosms") >= 0)smsRelayAddress = null;
  56. 56. Disable or Enable SMS at runtimelynx -dump http://servername:2006/[no]sms• The following code gets executed:if (urlPath.length >= 2 && urlPath[1].equals("sms")) {smsRelayAddress = smsRelayAddressDefault;clientResult = "Enable sms notification";sendPanic(clientResult);}else if (urlPath.length >= 2 && urlPath[1].equals("nosms")) {smsRelayAddress = null;clientResult = "Disable sms notification";sendPanic(clientResult);}If you aredrown withalarms
  57. 57. Supported Server PlatformsJava virtually runs on all platforms:– You only have to distribute the byte code in a Jar file!• Any Intel Linux• Any Intel Windows XP or Vista desktop• Alpha Tru64 UNIX V5.1B-3 with Java 1.4.2-7, see [11]• Itanium• Windows server 2003 with Java 1.5.0_07
  58. 58. Implemented Standards• DNS• HTTP, see [2]• Internet date & timeformat• Java, see [3] [4] [5]• NFS during development• NTP• TCP/IP• UDP for error relay• Web Cache control• ZIP for jar filemanipulation
  59. 59. What I have learned• Think in objects• Keep it simple (divide and conquer)• Client /server programming is fun... debugging and fault finding even more• It boosts the team spirit• It stimulates your own brains andcreativity• Java (network) programming is notreally exotic, nor complicated• Web applications must disable proxyand browser cache• TCP sockets do not have an end-of-file...– a server socket cannot know when theclient sends the last packet– this is why a final empty line has beenbuilt into the HTTP and SMTP protocol• Java initialises all variables to 0, "", ornull• Java always start with element 0• In Java you do not use pointers andlinked lists to create dynamic datastructures• Sparse matrices are natural in Java• Error handling and planning for theunforeseen is the most important –try / catch should be a habit• The HTTP protocol terminates everyline by CRLF. Originally I terminatedHTTP server output only by LF usingprintln. None of the Unix systems didcomplain. .Net servers replied with anerror “The server committed aprotocol violation”.• Keep the history of your changes. Usee.g. RCS, Revision Control System.
  60. 60. Design Principles• Separate code and data• Flexible data structures• Fully table & data driven• Keep the code simple• Code only once (no duplicateinstructions)• Do normal things straight• Use industry standards• Use as much as possible standardsystem functions• Do care about and beconservative with systemresources• Give the power to the users• Be flexible, and extensible• Be future proof• Be generous on what you accept,be strict on what you return• Do the unforeseen via exceptionhandling; be robust• Be fool proof• Avoid catastrophes at all times• Use what you know• Have maximum security andaccess control• Allow for logging, tracing, anddebugging• Care about supportability• Be platform independent• Write a complete documentation
  61. 61. Future Extensions and Migration• Advanced Java debugging• Build and use appliances• Application setup via config fileinstead of hardcoded• DNS caching• IPv6, Internet protocol V6• Java runtime optimisation(tuning, memory usage)• kill –HUP, to change debug level,or restart the server• Legacy applications• Load balancing• Multithreaded server• UDP monitoring• NTLM, Active Directory userauthentication• POST or other HTTP protocols• SIF, Service InvocationFramework = SOA services forJava• Timeout, drop a bad behavingclient (currently we suffer from aDoS problem)• SNMP monitoring• SOA, Service OrientedArchitecture• System parameters• XML, Input / Output in XMLformat
  62. 62. References[1] Full program source and additional material: http://www.hp-interex.be/pub/project/webtalk[2] The HTTP protocol and related standards: http://www.w3.org[3] Documentation and kits for Java: http://www.sun.com[4] Java technical documentation: http://java.sun.com/j2se/1.4.2/docs[5] Java search page: http://java.sun.com/j2se/1.4.2/search.html[6] Oracle UTL_HTTP:http://www.oracle.com/technology/sample_code/tech/pl_sql/htdocs/x/Utl_Http_Package_Enhancements/Cr_Using_Utl_Http.htm[7] Iptables: http://www.netfilter.org[8] Browsers:The Lynx browser: http://lynx.browser.orgFirefox: http://www.firefox.comOpera: http://www.opera.com[9] Perl libraries: http://search.cpan.org[10] NetBeans: http://www.netbeans.org[11] HP Java implementations: http://www.hp.com/java[12] Bruce Eckel, Thinking in Java, The definitive introduction to object-oriented programming inthe language of the World Wide Web, Fourth Edition, ISBN 0-13-187248-6, 2006, Prentice Hall,Pearson Education, Code & Supplements at http://www.mindview.net
  63. 63. Thanks• My colleague Danny De Thaye to encourage me in myendeavors• Connect Management for accepting my lecture• To you, the audiencegeert.van.pamel@belgacom.nethttp://www.hp-interex.be/wiki/index.php/User:Geertivp
  64. 64. Questions
  65. 65. Abstract• The speaker presents a Java application that behaves like a dedicated web serverand provides Web services. It runs as a standalone network daemon and only callssystem and network services that are directly executed by the JVM. No othersoftware is necessary. The URL path referenced by the clients is not referring tofiles. Instead it is calling other middleware services and serves as a Web servicegateway. The application only requires TCP/IP and Java library programming.Because it behaves as a web server, the browser can be any client capable ofinterfacing with the standard HTTP protocol.• Learn about Java network programming, specifically about security, reliability,supportability, monitoring, browser cache disabling, kit building, starting the Javaserver process, troubleshooting and monitoring, implementing on differentoperating systems and platforms, and integrating with other services. Debuggingserver processes is difficult because there is no direct user interface. Everyinteraction goes via network sockets, or process signals. Remotely control theserver by using the HTTP protocol, enable and disable logging, tracing, anddebugging info.
  66. 66. About the Author• The author works at Belgacom since 1997 in the Customer division as anICT Project Manager.• He is Chairman of HP-Interex Belux since 2008.• He is Board member of HP-Interex Belux since 2002. Member of DECUSsince 1984, and CUO since 1998.• In his previous life he worked as Project manager at Digital EquipmentCorporation (DEC) [company merged with Compaq in 1998 and with HP,Hewlett Packard in 2002].• His early activities in the world of IT were biometrical statistics, teachingcomputer courses, real time car manufacturing automation, and medicalimage processing using nuclear isotopes computer tomography.• He is an engineer in Agriculture and Applied Biological Sciences, andpostgraduate in Computer Science. He is eager to implement technologyas a business advantage.
  67. 67. Useful Utilities• Any web browser [8] To test the server (Firefox, Internet Explorer, Opera, lynx)• crontab To reset the daily log files, to monitor• Google For help with Java, use e.g. Google: Java ServerSocket• host To lookup the IP address of a machine• init To start the Web Service• jar Java archive builder• java Java runtime interpreter• javac Java Compiler• nc, netcat To test, debug, or simulate• NetBeans [10] Java IDE Integrated Development Environment Unix & Windows• NFS The server is developed, build & tested cross-platform• nmblookup, nbtstat To lookup the name of a Windows client, if not registered in DNS• NTP, ntpq To synchronise system time• Perl [9] Practical Extraction and Reporting Language• ps, grep, kill To monitor or change run status• telnet To monitor and test special requests (does not work in batch)• vi Terminal based editor - on linux, you might have colours• zip, Winzip To verify (or update) the contents of jar files.
  68. 68. Technical Terms• API Application Programming Interface• CRLF Carriage Return and Line Feed• GMT Greenwich Mean Time, almost the same as UTS (Universal Time Coordinated)• HW Hardware• IVR Interactive Voice Response• Jar Java archive• JDK Java Development Kit• JRE Java Runtime Environment• JSP Java Server Pages• JVM Java Virtual Machine• OS Operating System• Platform A combination of hardware and software• SMS Mobile text message (do not confuse with Microsoft System Management Server)• SW Software (operating system and layered products)• Unix Any type of Unix, or Linux OS, in general• UNIX™ UNIX Trademark
  69. 69. nslookupstatic String nslookup (String hostName) {String ipAddress;try {ipAddress =InetAddress.getByName(hostName).getHostAddress();} catch (UnknownHostException e) {ipAddress = "";}return ipAddress;}
  70. 70. rnslookupstatic String rnslookup (String ipAddress) {String hostName;try {hostName =InetAddress.getByName(ipAddress).getHostName();} catch (UnknownHostException e) {hostName = "";}return hostName;}

×