Thrift is a software framework that allows for efficient cross-language communication. It provides features such as RPC, code generation, and serialization to make it easy to define and develop services that can be used across multiple languages. Supported languages include C++, Java, Python, PHP and more. Thrift handles low-level details like data serialization while providing an interface definition language to define services and data structures.
Apache thrift-RPC service cross languagesJimmy Lai
This slides illustrate how to use Apache Thrift for building RPC service and provide demo example code in Python. The example scenario is: we have a prepared machine learning model, and we'd like to load the model in advance as a server for providing prediction service.
RESTLess Design with Apache Thrift: Experiences from Apache Airavatasmarru
Apache Airavata is software for providing services to manage scientific applications on a wide range of remote computing resources. Airavata can be used by both individual scientists to run scientific workflows as well as communities of scientists through Web browser interfaces. It is a challenge to bring all of Airavata’s capabilities together in the single API layer that is our prerequisite for a 1.0 release. To support our diverse use cases, we have developed a rich data model and messaging format that we need to expose to client developers using many programming languages. We do not believe this is a good match for REST style services. In this presentation, we present our use and evaluation of Apache Thrift as an interface and data model definition tool, its use internally in Airavata, and its use to deliver and distribute client development kits.
Apache thrift-RPC service cross languagesJimmy Lai
This slides illustrate how to use Apache Thrift for building RPC service and provide demo example code in Python. The example scenario is: we have a prepared machine learning model, and we'd like to load the model in advance as a server for providing prediction service.
RESTLess Design with Apache Thrift: Experiences from Apache Airavatasmarru
Apache Airavata is software for providing services to manage scientific applications on a wide range of remote computing resources. Airavata can be used by both individual scientists to run scientific workflows as well as communities of scientists through Web browser interfaces. It is a challenge to bring all of Airavata’s capabilities together in the single API layer that is our prerequisite for a 1.0 release. To support our diverse use cases, we have developed a rich data model and messaging format that we need to expose to client developers using many programming languages. We do not believe this is a good match for REST style services. In this presentation, we present our use and evaluation of Apache Thrift as an interface and data model definition tool, its use internally in Airavata, and its use to deliver and distribute client development kits.
Fluentd meetup dive into fluent plugin (outdated)N Masahiro
Fluentd meetup in Japan. I talked about "Dive into Fluent plugin".
Some contents are outdated. See this slide: http://www.slideshare.net/repeatedly/dive-into-fluentd-plugin-v012
PHPCR e API Platform: cosa significa davvero sviluppare un CMF con SymfonyInnoteam Srl
Scrivere un CMF da zero: uno degli strumenti che il mondo PHP/Symfony mette a disposizione per la gestione dei contenuti è PHPCR. Nelle slide di questo talk presentato al SymfonyDay 2018 spieghiamo come abbiamo utilizzato Doctrine PHPCR ODM e API Platform per poter costruire un CMF ed esporre le sue funzionalità in maniera REST. Inoltre mostriamo gli imprevisti che ci ha lasciato e come li abbiamo superati.
Developing high-performance network servers in LispVladimir Sedach
Overview of current high-performance Common Lisp web servers and implementation techniques, and description of a new hybrid approach to asynchronous I/O based on separate racing accept() and epoll() thread pools.
Fluentd meetup dive into fluent plugin (outdated)N Masahiro
Fluentd meetup in Japan. I talked about "Dive into Fluent plugin".
Some contents are outdated. See this slide: http://www.slideshare.net/repeatedly/dive-into-fluentd-plugin-v012
PHPCR e API Platform: cosa significa davvero sviluppare un CMF con SymfonyInnoteam Srl
Scrivere un CMF da zero: uno degli strumenti che il mondo PHP/Symfony mette a disposizione per la gestione dei contenuti è PHPCR. Nelle slide di questo talk presentato al SymfonyDay 2018 spieghiamo come abbiamo utilizzato Doctrine PHPCR ODM e API Platform per poter costruire un CMF ed esporre le sue funzionalità in maniera REST. Inoltre mostriamo gli imprevisti che ci ha lasciato e come li abbiamo superati.
Developing high-performance network servers in LispVladimir Sedach
Overview of current high-performance Common Lisp web servers and implementation techniques, and description of a new hybrid approach to asynchronous I/O based on separate racing accept() and epoll() thread pools.
Building scalable and language independent java services using apache thriftTalentica Software
This presentation is about the key challenges of cross language interactions and how they can be overcome. We discuss the Apache Thrift as a solution and understand its principle of Operation with code snippets and examples.
As a follow up to my illustration of the Axolotl Protocol as implemented by the TextSecure Protocol V2 https://github.com/WhisperSystems/TextSecure/wiki/ProtocolV2 here is TextSecure's Protocol Buffer usage https://code.google.com/p/protobuf/ illustrated
AWS re:Invent 2016: Deep Dive: Amazon EMR Best Practices & Design Patterns (B...Amazon Web Services
Amazon EMR is one of the largest Hadoop operators in the world. In this session, we introduce you to Amazon EMR design patterns such as using Amazon S3 instead of HDFS, taking advantage of both long and short-lived clusters, and other Amazon EMR architectural best practices. We talk about how to scale your cluster up or down dynamically and introduce you to ways you can fine-tune your cluster. We also share best practices to keep your Amazon EMR cluster cost-efficient. Finally, we dive into some of our recent launches to keep you current on our latest features. This session will feature Asurion, a provider of device protection and support services for over 280 million smartphones and other consumer electronics devices. Asurion will share how they architected their petabyte-scale data platform using Apache Hive, Apache Spark, and Presto on Amazon EMR.
Data Science & Best Practices for Apache Spark on Amazon EMRAmazon Web Services
Organizations need to perform increasingly complex analysis on their data — streaming analytics, ad-hoc querying and predictive analytics — in order to get better customer insights and actionable business intelligence. However, the growing data volume, speed, and complexity of diverse data formats make current tools inadequate or difficult to use. Apache Spark has recently emerged as the framework of choice to address these challenges. Spark is a general-purpose processing framework that follows a DAG model and also provides high-level APIs, making it more flexible and easier to use than MapReduce. Thanks to its use of in-memory datasets (RDDs), embedded libraries, fault-tolerance, and support for a variety of programming languages, Apache Spark enables developers to implement and scale far more complex big data use cases, including real-time data processing, interactive querying, graph computations and predictive analytics. In this session, we present a technical deep dive on Spark running on Amazon EMR. You learn why Spark is great for ad-hoc interactive analysis and real-time stream processing, how to deploy and tune scalable clusters running Spark on Amazon EMR, how to use EMRFS with Spark to query data directly in Amazon S3, and best practices and patterns for Spark on Amazon EMR.
Thrift vs Protocol Buffers vs Avro - Biased ComparisonIgor Anishchenko
Igor Anishchenko
Odessa Java TechTalks
Lohika - May, 2012
Let's take a step back and compare data serialization formats, of which there are plenty. What are the key differences between Apache Thrift, Google Protocol Buffers and Apache Avro. Which is "The Best"? Truth of the matter is, they are all very good and each has its own strong points. Hence, the answer is as much of a personal choice, as well as understanding of the historical context for each, and correctly identifying your own, individual requirements.
Organizations need to perform increasingly complex analysis on data — streaming analytics, ad-hoc querying, and predictive analytics — in order to get better customer insights and actionable business intelligence. Apache Spark has recently emerged as the framework of choice to address many of these challenges. In this session, we show you how to use Apache Spark on AWS to implement and scale common big data use cases such as real-time data processing, interactive data science, predictive analytics, and more. We will talk about common architectures, best practices to quickly create Spark clusters using Amazon EMR, and ways to integrate Spark with other big data services in AWS.
Learning Objectives:
• Learn why Spark is great for ad-hoc interactive analysis and real-time stream processing.
• How to deploy and tune scalable clusters running Spark on Amazon EMR.
• How to use EMR File System (EMRFS) with Spark to query data directly in Amazon S3.
• Common architectures to leverage Spark with Amazon DynamoDB, Amazon Redshift, Amazon Kinesis, and more.
AWS re:Invent 2016: Big Data Architectural Patterns and Best Practices on AWS...Amazon Web Services
The world is producing an ever increasing volume, velocity, and variety of big data. Consumers and businesses are demanding up-to-the-second (or even millisecond) analytics on their fast-moving data, in addition to classic batch processing. AWS delivers many technologies for solving big data problems. But what services should you use, why, when, and how? In this session, we simplify big data processing as a data bus comprising various stages: ingest, store, process, and visualize. Next, we discuss how to choose the right technology in each stage based on criteria such as data structure, query latency, cost, request rate, item size, data volume, durability, and so on. Finally, we provide reference architecture, design patterns, and best practices for assembling these technologies to solve your big data problems at the right cost.
From common errors seen in running Spark applications, e.g., OutOfMemory, NoClassFound, disk IO bottlenecks, History Server crash, cluster under-utilization to advanced settings used to resolve large-scale Spark SQL workloads such as HDFS blocksize vs Parquet blocksize, how best to run HDFS Balancer to re-distribute file blocks, etc. you will get all the scoop in this information-packed presentation.
Apache big data 2016 - Speaking the language of Big Datatechmaddy
With the advent of feature based teams, software architecture styles like Microservices and deployment patterns like Devops are taking over. Each team takes autonomous decisions on technologies used, but there is always a need to define a common language for the services to communicate with each other. This way there will be a common wire format and avoid lot of mappers across the application. The other common scenario is in big data projects where the cluster of nodes need to communicate efficiently and effectively, with ease of API.
This talk highlights on Apache Avro and Apache Thrift which are used in Big data solutions -- which act as common language across different services/nodes in big data applications. These technologies act as language and platform neutral way of serializing structured data. This talk also shows examples and demos -- highlighting the pain points they solve.
Secured (Kerberos-based) Spark Notebook for Data Science: Spark Summit East t...Spark Summit
This presentation will provide technical design and development insights in order to set up a Kerberosied (secured) JupyterHub notebook using Spark. Joy will show how Bloomberg set up the Kerberos-based Spark-notebook-integrating JupyterHub, Sparkmagic, and Levy. Sparkmagic provides the Spark kernel for Scala and Python. Livy is one of the most promising open source software to allow to submit Spark jobs over http-based REST interfaces. In this presentation, Joy will highlight the capabilities of Sparkmagic and Livy, along with the gap or development required in order to integrate the software seamlessly to work with your secured Spark cluster. The Kerberos integration techniques that he’ll discuss can be applied to other types of authenticators, such as OAuth. No prior knowledge of any of these technologies is requied in order to understand this presentation.
Big data never stops and neither should your Spark jobs. They should not stop when they see invalid input data. They should not stop when there are bugs in your code. They should not stop because of I/O-related problems. They should not stop because the data is too big. Bulletproof jobs not only keep working but they make it easy to identify and address the common problems encountered in large-scale production Spark processing: from data quality to code quality to operational issues to rising data volumes over time.
In this session you will learn three key principles for bulletproofing your Spark jobs, together with the architecture and system patterns that enable them. The first principle is idempotence. Exemplified by Spark 2.0 Idempotent Append operations, it enables 10x easier failure management. The second principle is row-level structured logging. Exemplified by Spark Records, it enables 100x (yes, one hundred times) faster root cause analysis. The third principle is invariant query structure. It is exemplified by Resilient Partitioned Tables, which allow for flexible management of large scale data over long periods of time, including late arrival handling, reprocessing of existing data to deal with bugs or data quality issues, repartitioning already written data, etc.
These patterns have been successfully used in production at Swoop in the demanding world of petabyte-scale online advertising.
What No One Tells You About Writing a Streaming App: Spark Summit East talk b...Spark Summit
So you know you want to write a streaming app but any non-trivial streaming app developer would have to think about these questions:
How do I manage offsets?
How do I manage state?
How do I make my spark streaming job resilient to failures? Can I avoid some failures?
How do I gracefully shutdown my streaming job?
How do I monitor and manage (e.g. re-try logic) streaming job?
How can I better manage the DAG in my streaming job?
When to use checkpointing and for what? When not to use checkpointing?
Do I need a WAL when using streaming data source? Why? When don’t I need one?
In this talk, we’ll share practices that no one talks about when you start writing your streaming app, but you’ll inevitably need to learn along the way.
Introduction to Amazon EMR design patterns such as using Amazon S3 instead of HDFS, taking advantage of Spot EC2 instances to reduce costs, and other Amazon EMR architectural best practices.
Azul Virtual Machine Engineer Douglas Hawkins describes how decisions made by the JVM affect how your code is compiled and run. Learn how this affects application performance and what steps you can take to optimize how the JVM acts on your code.
Dr archana dhawan bajaj's Sales JetView.Dr archana dhawan bajaj is a renowned practising doctor of non-commercial of India and Dr archana various Hospitals and Meternity.Dr archana dhawan bajaj has a PH.d master's degree in both human & International client Management along with other fellow.Dr archana is known to be proficient in Hindi,English,Sanskrit and Urdu.Dr archana dhawan bajaj has acquired extensive practice experience in maternity primarily in relation to corporate and international matter delivery.Dr archana dhawan bajaj's Sales JetView.Dr archana dhawan bajaj is a renowned practising doctor of non-commercial of India and Dr archana various Hospitals and Meternity.Dr archana dhawan bajaj has a PH.d master's degree in both human & International client Management along with other fellow.Dr archana is known to be proficient in Hindi,English,Sanskrit and Urdu.Dr archana dhawan bajaj has acquired extensive practice experience in maternity primarily in relation to corporate and international matter delivery.Dr archana dhawan bajaj's Sales JetView.Dr archana dhawan bajaj is a renowned practising doctor of non-commercial of India and Dr archana various Hospitals and Meternity.Dr archana dhawan bajaj has a PH.d master's degree in both human & International client Management along with other fellow.Dr archana is known to be proficient in Hindi,English,Sanskrit and Urdu.Dr archana dhawan bajaj has acquired extensive practice experience in maternity primarily in relation to corporate and international matter delivery.Dr archana dhawan bajaj's Sales JetView.Dr archana dhawan bajaj is a renowned practising doctor of non-commercial of India and Dr archana various Hospitals and Meternity.Dr archana dhawan bajaj has a PH.d master's degree in both human & International client Management along with other fellow.Dr archana is known to be proficient in Hindi,English,Sanskrit and Urdu.Dr archana dhawan bajaj has acquired extensive practice experience in maternity primarily in relation to corporate and international matter delivery.Dr archana dhawan bajaj's Sales JetView.Dr archana dhawan bajaj is a renowned practising doctor of non-commercial of India and Dr archana various Hospitals and Meternity.Dr archana dhawan bajaj has a PH.d master's degree in both human & International client Management along with other fellow.Dr archana is known to be proficient in Hindi,English,Sanskrit and Urdu.Dr archana dhawan bajaj has acquired extensive practice experience in maternity primarily in relation to corporate and international matter delivery.Dr archana dhawan bajaj's Sales JetView.Dr archana dhawan bajaj is a renowned practising doctor of non-commercial of India and Dr archana various Hospitals and Meternity.Dr archana dhawan bajaj has a PH.d master's degree in both human & International client Management along with other fellow.Dr archana is known to be proficient in Hindi,English,Sanskrit and Urdu.Dr archana dhawan bajaj has acquired extensive practice exp
JVM Mechanics: When Does the JVM JIT & Deoptimize?Doug Hawkins
HotSpot promises to do the "right" thing for us by identifying our hot code and compiling "just-in-time", but how does HotSpot make those decisions?
This presentation aims to detail how HotSpot makes those decisions and how it corrects its mistakes through a series of demos that you run yourself.
Talk given at DomCode meetup in Utrecht (August 2014) on different frameworks to do asynchronous I/O y Python, with a strong focus on asyncio (PEP-3156).
Sharding and Load Balancing in Scala - Twitter's FinagleGeoff Ballinger
My presentation at Mostly Functional (http://mostlyfunctional.com), part of this year's Turing Festival Fringe (http://turingfestival.com) in Edinburgh. The example source code is up on Github at https://github.com/geoffballinger/simple-sharder
Original slides from Ryan Dahl's NodeJs intro talkAarti Parikh
These are the original slides from the nodejs talk. I was surprised not find them on slideshare so adding them. The video link is here https://www.youtube.com/watch?v=ztspvPYybIY
containerd is an industry-standard container runtime with an emphasis on simplicity, robustness and portability. It is available as a daemon for Linux and Windows, which can manage the complete container lifecycle of its host system: image transfer and storage, container execution and supervision, low-level storage and network attachments, etc.
containerd is designed to be embedded into a larger system, rather than being used directly by developers or end-users.
Swift is a powerful new language, with a rapidly growing user base—which now includes Linux server-side developers. This presentation is a briief overview of the current Swift web server frameworks, as well as a highlight of the best features that Swift has to offer.
Rust is a system programming language focused on safety, speed, and concurrency. It's standard library provides API for dealing with I/O, but for now in a synchronous way. In this talk we'll dive into the ecosystem of asynchronous libraries published so far on crates.io and how to use them in order to build robust, scalable, and production ready network clients and servers.
Lambda Chops - Recipes for Simpler, More Expressive CodeIan Robertson
While the new Streams API has been a great showcase for lambda methods, there are many other ways this new language feature can be used to make friendlier APIs and more expressive code. Lambdas can be used for a number of tasks which historically required significant boilerplate, type-unsafe constructs, or both. From new ways to express metedata, to emulating Groovy's null-safe navigation operator, we'll take a look at a myriad of ways, big and small, that you can use lambdas to improve APIs and streamline your code. We'll also look at some of the limitations of lambdas, and some techniques for overcoming them.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
3. Thrift
• a software framework for scalable cross language
service development. It combines software stack
with a code generation engine to build services that
work efficiently and seamlessly
• Provides cross-language RPC and serialization library
• 지원 언어 :
C++ ,C# ,Cocoa ,Erlang ,Haskell ,Java ,Ocaml ,Perl ,P
HP ,Python ,Ruby ,Smalltalk
• Champion
– Doug Cutting (하둡)
• Apache license
6. 장점
• 버전닝 지원
• 여러 언어에서 사용 가능하며 언어에 맞도록 소스가 생성되고, 언어
간의 Serialization 가능
• Sync, Async API 제공
• XML 설정이 필요 없음
• Layer에 맞는 Interface를 사용 및 Customizing 구축 가능
• RPC 기능 제공 (Google Protocol Buffer에는 없는 기능)
– 서버 기능 좋음
• 서블릿 제공(org.apache.thrift.server.TServlet)
• 멀티쓰레드 지원 (org.apache.thrift.server.ThreadPoolServer : worker thread 지정)
• Async 지원 (org.apache.thrift.server. TNonblockingServer : single threaded)
• Multi-thread Half-Sync/Half-Async지원 : org.apache.thrift.server. THsHaServer
• Exception을 제공 (Google Protocol Buffer에는 없는 기능)
• Set, Map 지원 (Google Protocol Buffer에는 없는 기능)
7. 단점 ?
• 자바로 코드가 생성될 때, Slf4j를 기본적으로 가지고
가며, 내부 코드는 모두 thrift lib가 필요함
• C++의 Server threading part는 Boost에 의존을 가짐
• 자바 쪽 이클립스 플러그인 없음
• Thrift lib의 api가 자주 바뀌어서, 버전 업마다 소스를
좀 보고 코딩해야 함. (인터넷 예제가 거의 없음)
• XML Serialization/Deserialization 기능 없음
• 문서가 확실히 적음
• 생성된 코드가 Google Protocol Buffer에 비해서 보기
는 편하지는 않음 (특히 C++)
9. 설치 및 사용
• 원래는 configure/make를 필요 (리눅스/윈도우)
• 윈도우는 그냥 실행하면 됨 (여기서는 윈도우 버전으로
만 테스트)
10. (소스 컴파일시)
Basic requirements
• A relatively POSIX-compliant *NIX system
– Cygwin or MinGW can be used on Windows
• g++ 3.3.5+
• boost 1.33.1+ (1.34.0 for building all
tests)
• Runtime libraries for lex and yacc might
be needed for the compiler.
11. (소스 컴파일시)
Language requirements
• C++
– Boost 1.33.1+
– libevent (optional, to build the nonblocking server)
– zlib (optional)
• Java
– Java 1.5+
– Apache Ant
– Apache Ivy (recommended)
– Apache Commons Lang (recommended)
– SLF4J
• C#: Mono 1.2.4+ (and pkg-config to detect it) or Visual Studio 2005+
• Python 2.4+ (including header files for extension modules)
• PHP 5.0+ (optionally including header files for extension modules)
• Ruby 1.8+ (including header files for extension modules)
• Erlang R12 (R11 works but not recommended)
• Perl 5
– Bit::Vector
– Class::Accessor
15. Thrift파일
샘플 #1
service HelloWorld {
oneway void hello(1:string name)
}
C:thrift>thrift-0.7.0.exe --gen java HelloWorld.thrift
Generated Code
public class HelloWorld {
public interface Iface {
public void hello(String name) throws org.apache.thrift.TException;
}
public interface AsyncIface {
public void hello(String name, org.apache.thrift.async.AsyncMethodCallback<AsyncClient.hello_call> resultHandler) throws
org.apache.thrift.TException;
}
public static class Client extends org.apache.thrift.TServiceClient implements Iface {
…
}
public static class AsyncClient extends org.apache.thrift.async.TAsyncClient implements AsyncIface {
…
}
public static class Processor<I extends Iface> extends org.apache.thrift.TBaseProcessor implements org.apache.thrift.TProcessor {
}
public static class hello_args implements org.apache.thrift.TBase<hello_args, hello_args._Fields>, java.io.Serializable, Cloneable {
…
}
16. Server Code : 사용자는 generated code를 상속하고.Thrift API를 이용해서 포트매핑
public class HelloWorldServer {
public static void main(String[] argss) {
try {
Factory portFactory = new TBinaryProtocol.Factory(true, true);
TServerSocket serverTransport = new TServerSocket(8000);
HelloWorld.Processor<HelloWorld.Iface> processor =
new HelloWorld.Processor<HelloWorld.Iface>(new
HelloWorldImpl());
Args args = new Args(serverTransport);
args.processor(processor);
args.protocolFactory(portFactory);
TServer server = new TThreadPoolServer(args);
System.out.println("Starting server on port 8000 ...");
server.serve();
} catch (TTransportException e) {
e.printStackTrace();
}
}
}
class HelloWorldImpl implements HelloWorld.Iface {
@Override
public void hello(String name) throws TException {
long time = System.currentTimeMillis();
System.out.println("Current time : " + time);
System.out.println("Hello " + name);
}
}
17. Client Code
public class HelloWorldClient {
public static void main(String[] args) {
TTransport transport;
try {
transport = new TSocket("localhost", 8000);
TProtocol protocol = new TBinaryProtocol(transport);
HelloWorld.Client client = new HelloWorld.Client(protocol);
transport.open();
client.hello("World!!!");
transport.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}
실행결과
Starting server on port 8000 ...
Current time : 1317014309989
Hello World!!!
27. 기타
• comments
– /* */
– //
• namespace
namespace java com.example.project
• Include
include "tweet.thrift”
...
struct TweetSearchResult {
1: list<tweet.Tweet> tweets;
}
28. 기타
• Constants
const i32 INT_CONST = 1234;
const map<string,string> MAP_CONST = {"hello": "world",
"goodnight": "moon"}
• 안되는 것
– cyclic structs
– Polymorphism
– Overloading
– null return
– heterogeneous containers : items in a container
must be of the same type (자바 generic을 생각)
29. IDL>Java
• bool: boolean
• byte: byte
• i16: short
• i32: int
• i64: long
• double: double
• string: String
• list<type>: List<type>
• set<type>: Set<type>
• Map<key, value>: Map<key, value>
• Typedef a b : just B
37. Transport Layer
• TFileTransport – This transport writes to a file.
• TFramedTransport – This transport is required
when using a non-blocking server. It sends data
in frames, where each frame is preceded by a
length information.
• TMemoryTransport – Uses memory for I/O. The
Java implementation uses a simple
ByteArrayOutputStream internally.
• TSocket – Uses blocking socket I/O for transport.
• TZlibTransport – Performs compression using zlib.
Used in conjunction with another transport. Not
available in the Java implementation.
41. Protocol layer
• TBinaryProtocol – A straight-forward binary format
encoding numeric values as binary. It is faster than
the text protocol but more difficult to debug.
• TCompactProtocol – Very efficient, dense encoding
of data.
• TDebugProtocol – Uses a human-readable text
format to aid in debugging.
• TDenseProtocol – Similar to TCompactProtocol,
striping off the meta information from what is
transmitted.
• TJSONProtocol – Uses JSON for encoding of data.
• TSimpleJSONProtocol – A write-only protocol using
JSON. Suitable for parsing by scripting languages.
46. processor
• Thrift 컴파일러에 의해서 interface에 맞는
processor가 생성됨
• Processor는 input/ouput stream을 관장하
는 클래스(encapsulates the ability to read
data from input streams and write to
output streams.)
interface TProcessor {
bool process(TProtocol in, TProtocol out) throws TException
}
48. Server Layer
• TNonblockingServer – A multi-threaded
server using non-blocking io (Java
implementation uses NIO channels).
TFramedTransport must be used with
this server.
• TSimpleServer – A single-threaded server
using std blocking io. Useful for testing.
• TThreadPoolServer – A multi-threaded
server using std blocking io.
50. 기존 IDL를 변경할 때..
• 기존 IDL를 바꿀때마다 고생하지 않아도 되는
방법
– Don’t change the numeric tags for any existing
fields.
– Any new fields that you add should be optional.
– Non-required fields can be removed, as long as
the tag number is not used again in your
updated message type
"OBSOLETE_“으로 이름 변경하기
– Changing a default value is generally OK, as
long as you remember that default values are
never sent over the wire.