Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Usage Note of Apache Thrift for C++ Java PHP Languages

3,397 views

Published on

Note that how to use Apache/Facebook Thrift IDL for C++, Java and PHP programming languages.

Published in: Technology
  • Be the first to comment

Usage Note of Apache Thrift for C++ Java PHP Languages

  1. 1. Usage Note of Apache Thrift for C++ / Java / PHP Languages William.L wiliwe@gmail.com 2015-06-02
  2. 2. Index What is Apache Thrift? .................................................................................................................................. 3 Install Tool & Library .................................................................................................................................... 4 GCC 4.9............................................................................................................................................................ 4 Boost C++ Library.................................................................................................................................. 5 Oracle/Sun Java 8 ................................................................................................................................... 5 Configure Ant.................................................................................................................................. 9 Upgrade Autoconf / Automake / Bison.................................................................................................. 9 LibEvent................................................................................................................................................. 10 Apache Thrift ........................................................................................................................................ 10 Add Server Listen Port Number into Firewall/iptables .................................................................... 12 Using Thrift ................................................................................................................................................... 14 C++ Server/Client ......................................................................................................................................... 15 Java Server/Client......................................................................................................................................... 16 PHP Client ..................................................................................................................................................... 25
  3. 3. What is Apache Thrift? Apache Thrift is an interface definition/description language (IDL) and binary communication protocol that is used to define and create scalable services for numerous programming languages and combines a software stack with a code generation engine to build services that work efficiently. It is used as a remote procedure call (RPC) software framework. Thrift was developed at Facebook to expedite development and implementation of efficient and scalable cross-language (Web)services. Its primary goal is to enable efficient and reliable communication across programming languages by abstracting the portions of each language that tend to require the most customization into a common library that is implemented in each language. Specifically, Thrift allows developers to define datatypes and service interfaces in a single language-neutral file and generate all the necessary code to build RPC (remote procedure call) clients and servers. The Apache Thrift official site and tutorial: https://thrift.apache.org/ https://thrift.apache.org/tutorial/ The Thrift white paper from Facebook could be downloaded from: https://thrift.apache.org/static/files/thrift-20070401.pdf Example codes and Apache Thirft source archive for this documentation could be downloaded from the GitHub: https://github.com/wiliwe/apache-thrift-example
  4. 4. Install Tool & Library The environment used in this document: * Linux CentOS 6.5 64-bit (could be updated to 6.6 or 6.7) * GCC C/C++ compiler v4.9.0 (which support C++11 and C++14 standards) * Apache Thrift v1.0.0 Dev (through Git) * Boost C++ library v1.58.0 * Java 8 update45 * PHP v5.3.3 * LibEvent v2.0.22 Stable * Qt Creator v3.3.1 IDE * Eclipse v4.4 Luna (with JDT installed for Java) Here using v1.0.0 (a development version) for example and the target programming languages are C++, Java and PHP. Apache Thrift install guide: * http://thrift.apache.org/docs/install/ * http://thrift.apache.org/docs/install/centos Update & Install Linux System Library 1) Update system package repository. $ su (change to root) # yum update 2) Install development tools, such GCC compilers. # yum -y groupinstall "Development Tools" 3) Install required libraries. # yum install php php-devel php-common pcre-devel php-pecl-memcache automake libtool flex bison pkgconfig gcc-c++ zlib-devel python-devel openssl-devel glib2-devel.x86_64 libev-devel.x86_64 bzip2-devel.x86_64 libicu-devel.x86_64 GCC 4.9 You could use other version of GCC that supports C++11 standard. GCC v4.7 or above supports C++11, while v4.9 or above supports C++14. The built-in GCC in CentOS 6.5 is v4.4.7, this is too old to build most libraries! If you want to upgrade GCC, it could refer to the documentation: http://www.slideshare.net/wiliwe/upgrade-gcc-install-qt-54-on-centos-65
  5. 5. Boost C++ Library The version MUST be v1.53 or above. Here using v1.58.0. 1) First of all, remove Boost library installed defaultly by Linux $ su (change to root) # yum erase boost* 2) Download Boost v1.58.0 source archive, boost_1_58_0.tar.gz, from http://sourceforge.net/projects/boost/files/boost/1.58.0/ 3) Unpack source archive and it will generate a folder name boost_1_58_0. 4) Enter folder boost_1_58_0 folder and run below commands: $ ./bootstrap.sh --prefix=/usr --libdir=/usr/lib64 $ su (change to root) # ./b2 install Oracle/Sun Java 8 For enabling Java language generation, it needs to install JDK and Ant into the system. Current Version http://www.oracle.com/technetwork/java/javase/downloads/index.html Older Version http://www.oracle.com/technetwork/java/javase/archive-139210.html Here using JDK 1.8 update 45. http://www.oracle.com/technetwork/java/javase/downloads/java-archive-javase8-2177648.html#jdk-8u45-oth-JPR * for RHEL/CentOS/Fedora 32-bit, download the archive jdk-8u45-linux-i586.tar.gz. * for RHEL/CentOS/Fedora 64-bit, download the archive jdk-8u45-linux-x64.tar.gz. , here using the archive jdk-8u45-linux-x64.tar.gz.
  6. 6. 1) First of all, to check the already installed Java version, enter the following command: $ java -version $ which java $ ll /usr/bin/java $ ll /etc/alternatives/java If Java 1.6, 1.7 or 1.8 have been installed already, you can uninstall them using the following commands: $ su (change to root) # yum remove java-1.6.0-openjdk or # yum remove java-1.7.0-openjdk or # yum remove java-1.8.0-openjdk One of version of the above OpenJDK may be used for LiberOffice tool, so you might not want to remove it. For this case, it could use alternatives utility to change current used Java tools; this will be mentioned later. 2) Upack the downloaded archive jdk-8u45-linux-x64.tar.gz and it will generate a folder named jdk1.8.0_45. 3) Next, to configure the absolute path to folder jdk1.8.0_45 that the Linux system could find Java tools (e.g. java (interpreter), javac (compiler), javaws (Web Start), Java standard librariese (class files), etc) when
  7. 7. you type these tool/command under console or use a Java IDE (Integrated Development Environment) tool. There have two ways to configure the path to Java tools: <I> Using alternatives Tool If there have more than one versions of JDK in your Linux system, it might need to use alternatives utility to install Java tools (to add symbolic links to individual executable files to the /usr/bin directory). alternatives utility is available in "chkconfig" package, you could install it if it is absent: $ su (change to root) # yum install chkconfig.x86_64 To setup javac, jar and javaws commands path using alternatives: $ su (change to root) # alternatives --install /usr/bin/java java /opt/jdk1.8.0_45/bin/java 2 # alternatives --set java /opt/jdk1.8.0_45/bin/java # alternatives --install /usr/bin/javac javac /opt/jdk1.8.0_45/bin/javac 2 # alternatives --set javac /opt/jdk1.8.0_45/bin/javac # alternatives --install /usr/bin/jar jar /opt/jdk1.8.0_45/bin/jar 2 # alternatives --set jar /opt/jdk1.8.0_45/bin/jar # alternatives --install /usr/bin/javaws javaws /opt/jdk1.8.0_45/bin/javaws 2 # alternatives --set javaws /opt/jdk1.8.0_45/bin/javaws To verify configurations: # alternatives --config java # alternatives --config javac
  8. 8. # alternatives --config jar # alternatives --config javaws # java -version # javac -version # javaws -version <II> Set Environment Variables Setup JAVA_HOME variable. $ export JAVA_HOME=/ home/william/jdk1.8.0_45 Setup JRE_HOME variable. $ export JRE_HOME=/ home/william/jdk1.8.0_45/jre Setup CLASSPATH variable. $ export CLASSPATH= $JAVA_HOME/lib: :$JAVA_HOME/jre/lib:. (the last path is “current directory”, being presented as a dot ".") Add path to PATH variable for searching Java tools. $ export PATH=$PATH:$JAVA_HOME/bin:$JAVA_HOME/jre/bin
  9. 9. Configure Ant The version of Ant must be v1.7 or above. Download current release of Ant from the site: https://www.apache.org/dist/ant/binaries/ Here using v1.9.6. 1) Download the archive apache-ant-1.9.6-bin.tar.gz. 2) Unpack the archive and it will generate a folder named apache-ant1.9.6. 3) Set ANT_HOME environment variable with the path to the unpacked Ant folder. $ export ANT_HOME=/home/william/apache-ant-1.9.4 4) Add the path to Ant’s tool to PATH variable for searching ant tools. $ export PATH=$PATH:$ANT_HOME/bin It could write the above exporting commands(for Java and ANt) in user’s bashrc file, /home/UserName/.bashrc, this lets the system could find Java tools each time a user login or open a virtual terminal (shown as below snapshot). Upgrade Autoconf / Automake / Bison 1) Download source archives of these three tool $ wget http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz $ wget http://ftp.gnu.org/gnu/automake/automake-1.14.tar.gz $ wget http://ftp.gnu.org/gnu/bison/bison-2.5.1.tar.gz 2) Unpack each archives, enter separate folders and run commands: $ ./configure $ make $ su (change to root) # make install
  10. 10. LibEvent 1) Download the archive libevent-2.0.22-stable.tar.gz from http://sourceforge.net/projects/levent/files/libevent/libevent-2.0/ 2) Unpack the archive to generate a source folder named libevent-2.0.22-stable. 3) Enter the folder libevent-2.0.22-stable and run commands: $ ./configure --prefix=/usr --libdir=/usr/lib64 $ make $ su (change to root) # make install If this library does not exist in the system or the version of the installed one is too old, it will show below error messages: <Case-I> In file included from src/thrift/server/TNonblockingServer.cpp:24:0: ./src/thrift/server/TNonblockingServer.h:41:33: fatal error: event2/event_compat.h: No such file or directory #include <event2/event_compat.h> libthriftnb.so error: undeclared reference evbuffer_pullup <Case-II> Apache Thrift 1) Download source archive from https://thrift.apache.org/download , or using Git tool: $ git clone https://git-wip-us.apache.org/repos/asf/thrift.git $ cd thrift $ ./bootstrap.sh $ ./configure --prefix=/usr --libdir=/usr/lib64 --with-python=no --with-perl=no --with-haskell=no --with-go=no --with-haxe=no --with-d=no --with-lua=no --with-ruby=no --with-csharp=no --with-erlang=no --with-nodejs=no --with-boost=yes --with-boost-libdir=/usr/lib64 Note that if it uses “” parameter, it will lead Thrift configuration fail to find Boost library. During configuring compilation environment, it may issue some problems: <Solution> To resolve this, you'll need to find your pkg.m4 (installed by the pkg-config package) file and copy it to the Thrift-Src /aclocal directory. [Thrift syntax error in ./configure] "syntax error near unexpected token"
  11. 11. In Thrift source folder, copy pkg.m4 to the sub-directory aclocal: $ cp /usr/share/aclocal/pkg.m4 Thrift-Src-Folder/aclocal Then, re-run ./bootstrap.sh and ./configure. (Note that pkg.m4 is created by the pkg-config tool. If your /usr/share/aclocal directory doesn't contain the pkg.m4 file, you may not have pkg-config installed.)” More information about this problem could be found in the sites: * https://thrift.apache.org/docs/install/windows * http://wiki.apache.org/thrift/FAQ 2) If everything is okay, the result of successful configuration will be as below snapshot: Now, start to build and install Thrift library and tool: $ make
  12. 12. $ su (change to root) # make install After installing successfully, verify Thrift’s thrift tool as this: Note * If it show error messages that it could not find .plo files for building files under "thrift/lib/cpp/test" , I tried to run "./bootstrap", "./configure" again and it succeeded to complie. * If you want to use Thrift T_LOG or T_LOG_T macro to print debugging messages, it must set the below macro to 1 in "ThriftSrc/lib/cpp/src/thrift/TLogging.h" : #define T_GLOBAL_DEBUGGING_LEVEL 1. Add Server Listen Port Number into Firewall/iptables To allow Apache Thrift server could be connected from the clients locating on other host, it needs to add listen port number to the firewall/iptables configuration file. 1) Open the configuratoin file, /etc/sysconfig/iptables, using a text editor. $ su (change to root)
  13. 13. # vi /etc/sysconfig/iptables 2) Add this lines: -A INPUT -m state --state NEW -m tcp -p tcp --dport 8989 -j ACCEPT Here using port number 8989 for example, it needs to be replaced with you wanted number. 3) Restart iptables: # service iptables restart
  14. 14. Using Thrift Basic steps of using Thrift are as below: 1) Write a Thrift interface file describing the interface name and its parameter. The interface files usually end with “.thrift” extension name. Below snapshots are examples for Thrift interface writing. It could declare all interfaces name in the interface file or use a header to contain it and include this header file into the interface file. 2) Use Thrift tool, thrift, with the written interface definition file as input to generate source files for target programming language (such as C++, Java, PHP, etc), run: thrift --gen <language> <Thrift filename> To recursivly generate source code from a Thrift file and all other Thrift files included by it, run thrift -r --gen <language> <Thrift filename> The Thrift interface file used for this documentation is named Person.thrift whose content is as below:
  15. 15. C++ Server/Client The guide for Thrift C++ language programming is: https://thrift.apache.org/tutorial/cpp To generate C++ source from Thrift interface fiel, run: $ thrift --gen cpp Person.thrift Here using Qt Creator IDE and Qmake to maintain C++ client and server codes. In Qt project file (.pro), it needs to add below lines for building codes:
  16. 16. Java Server/Client The guide for Thrift Java language programming is: https://thrift.apache.org/tutorial/java To generate Java source from Thrift interface fiel, run: $ thrift --gen java Person.thrift After building Thrift library and tool successfully, it generates JAR files for Thrift Java. These JARs locate under folders Thrift-Source/lib/java/build/ and Thrift-Source/lib/java/build/lib (as below snapshot). When installing, these JAR files will put to /usr/local/lib defaultly. If you want to change the installation path for JARs, you could use JAVA_PREFIX option when doing configuration. You could copy these JARs into a folder for later usage. Here copying them into a folder named lib together with a folder named src containing server and client codes (as below snapshot). Here showing how to use Eclipse IDE to maintain Java client and server codes. 1) Click “File -> New -> Java Project”
  17. 17. 2) In “New Java Project” dialogue, fill “Project name” field, navigate “Location” to your Java source folder.
  18. 18. The source folder for Thrift Java is as Click Next button. 3) It could view source code and libraries Eclipse found. For Source tab, check “Allow output folders for source folders” Finally, click Finish button. You could see the project content in Project view as below snapshot:
  19. 19. In the Thrift Java source folder, it could find two files (.classpath, .project) are added for Eclipse project management. 4) Now, invoke server application by righ click server source file, PersonServer.java, and click menu item “Run As -> Java Application”
  20. 20. Then, you could see the message coded in server source code in the Console view (as below snapshot). 5) Next, invoke client application by righ click client source file, PersonClient.java, and click menu item “Run As -> Java Application”
  21. 21. Then, you could see the response message after calling RPC interface in the Console view (as below snapshot). 6) To terminate the running server, switch to Debug perspective by clicking “Window -> Open Perspective -> Debug” In Debug perspective, you could see the running server application. Right click the running server application, and click “Terminate” item to terminate it.
  22. 22. After terminating the server application, right clicking Debug perspective and click “Remove All Terminated.”
  23. 23. If you want to load an existent project in Eclipse, just follow below steps. 1) Clicke “File -> Import…” 2) In “Import” dialogue, select “General -> Existing Projects into Workspace” and click Next.
  24. 24. 3) In “Select root directory” field, nevigate to the existent Eclipse project folder and then click Finish.
  25. 25. PHP Client The guide for Thrift PHP language programming is: https://thrift.apache.org/tutorial/php . For testing PHP client, it should have a Web server with PHP interpreter installed in your testing host. Here having a documentation for setting up LAMP (Linux, Apache, MySQL DB and PHP), you could refer to how to setup Apache Web server and install/configure PHP module into Apache server. http://www.slideshare.net/wiliwe/lamp-installation-note-centos-65 To generate PHP source from Thrift interface file, run: $ thrift --gen php Person.thrift In Thrift source folder, there have PHP library files for Thrift communication. They locate under folders Thrift-Source/lib/php/lib/Thrift (as below snapshot). You could copy the Thrift folder under Thrift-Source/lib/php/lib/ for later usage. To use Thrift PHP client, follow below steps: 1) Put generated Thrift PHP code and Thrift provided PHP library into Web server’s document root folder. Here using /var/www/html/ for example (as below snapshot) and using a folder named “thrift” to contain generated PHP source folder “IDV” and Thrift PHP library folder “Thrift”. 2) Follow Thrift tutorial Web site, write a PHP Thrift client and put it into /var/www/html/ . The PHP Thrift client here is the personClient.php. 3) Change mode of “thrift” folder to executable recursively. $ su (change to root) # chmod –R 755 ./thrift/ If it does not be set to executabe, all Thrift PHP libraries and generated codes would not be loaded when
  26. 26. executing personClient.php and the Web server will log error messages showing that it has no permission to access those files (as below snapshot). The output of successfully executing personClient.php would be as below snapshots when using “php” tool and Web browser:

×