SlideShare a Scribd company logo
1 of 26
1
ROMA --
User-Customizable NoSQL
Database in Ruby
Rakuten, Inc., Rakuten Institute of Technology, Tokyo | Muga Nishizawa | Nov. 1st, 2010
2
Agenda
• Background
• Features of ROMA
• Overall architecture of ROMA
• Plug-in architecture and its domain specific language
• Conclusion
3
Spread of NoSQL databases
• Needs for NoSQL databases are rising in order to easily store rapidly increasing
data
• Basic features of several NoSQL databases are
– High scalability
– High availability
– High throughput
– Other features vary by databases
• ROMA
– Started development with Matz in 2007
– Open sourced in Oct. 2009
• See http://github.com/roma/roma/
4
ROMA Usage in Rakuten
• ROMA is used in various Rakuten services
– ROMA runs on dozens of servers
– Various types of data are stored
• E.g. session data, personal page view history, etc
• Rakuten, Inc.
– It provides many e-commerce platforms
• E.g. Rakuten Ichiba, Rakuten Travel, Rakuten Books, etc
– Rakuten has tens of millions of users
5
Increase of Application-Specific Needs
• Many specific needs from application-side
– These specific needs come from actual service development sites
• For example, users say
– “How can we store structured data?”
– “How can we easily process stored data on DWH or Hadoop?”
– “How can we delete null character that was stored somewhere by error?”
– “How can we delete duplicate data that were accidentally stored somewhere?”
• We want to respond to all of these needs
– We are trying to solve these problems, one by one
6
Case Study: List Operation in Rakuten Travel
• “How can we access to list data that were stored in ROMA easily?”
– Request from Rakuten Travel to apply page view history using ROMA
– Page view history function is useful function for users
7
Case Study: List Operation in Rakuten Travel
• The application stores list data for users in database
– Key: user ID, Value: a list of pages that the user viewed
• To update list data that were stored in NoSQL database, application
1. Gets binary data of specified key
2. Deserializes it as list data
3. Updates the list data according to user requirement
4. Serializes the list data to binary data
5. Set it to database
“Insert 2nd
elm into list” req
① get binary data
② deserialize binary as list data
③ insert 2nd
elm into list
④ serialize list data to binary
⑤ put binary data
Memcached cluster
8
ROMA
• User-customizable NoSQL database in Ruby
• Features
– High scalability
– High availability
– Fault-tolerance
– Better throughput
– And…
• To address application-specific needs, ROMA provides
– Plug-in architecture
– Domain specific language (DSL)
• ROMA enables responding to the above need from Rakuten Travel
9
Overall Architecture of ROMA
• ROMA integrates several well-known techniques to achieve scalability, availability and
fault-tolerance
– For example, consistent hashing, virtual nodes, chain replication-like mechanism, lamport clocks,
etc
• ROMA node consists of 4 modules
– Network IO module: Receiving data from clients and other ROMA nodes
– Command exec module: Creating and executing commands
– Routing module: Maintaining ring information
– Storage module: Storing data
Command Execution
module
Storage module
Network IO module
Routing module
10
Data Partitioning
• Consistent hashing and virtual nodes
– ROMA consists of several nodes that run on servers
– Many virtual nodes are allocated on 1-dimensional hash space of 32-bits
• Each virtual node has a 32-bits ID
– To determine ROMA nodes that are responsible for serving key-value
– SHA-1 hash
ROMA
E.g. ROMA consists of three nodes
11
Data Accessing
• Getting value of specified key from ROMA
1. User accesses to ROMA nodes
2. The node determines others that are responsible for value of specified key
3. The node gets the value from the other node
4. The node returns it to user
ROMA
E.g. ROMA consists of three nodes
①
②
12
Ring Membership Model
• Each node maintains and periodically shares routing table with others
– If several versions exist, node updates latest version of routing table
– Lamport clocks and Merkle hash tree
• ROMA node multicasts with others to share routing tables
– Current version of ROMA doesn’t use gossip-based protocol
ROMA
E.g. ROMA consists of three nodes
Sharing ring information
13
Ring Cache on Client-side
• ROMA client enables direct accessing to data
– Cache of routing table
– It checks that routing table is updated or not every 3 sec.
• Getting value of specified key, ROMA client
1. Determines nodes
2. Gets the value from the node directly
ROMA
E.g. ROMA consists of three nodes
14
Data Replication
• Automatic data replication
– Client waits until data replication succeeds
– If data replication failed, the data is push to asynchronous queue in node
– Eventual consistency
• Each data is replicated at N nodes
– N: this parameter is configured before runtime
ROMA
E.g. ROMA consists of three nodes
15
Message Protocol
• Extended Memcached protocol over TCP
– Used between clients and ROMA nodes
• Memcached client libraries are available
– Without distribution concern
– Users can use telnet
$ telnet localhost 11211
Trying 127.0.0.1...
Connected to localhost.
Escapecharacter is ‘^]’.
set foo 0 0 3
bar
STORED
get foo
VALUE foo 0 3
bar
END
User can set/get data in ROMA
with telnet like memcached
16
Failure Detection
• Heartbeat detection
– Each node multicasts periodic heartbeat with others
– Heartbeat is flooded every 1 sec.
• If heartbeat is missed continuously, the node is declared as failed
– Failover
– Removal of the node
ROMA
E.g. ROMA consists of three nodes
Sending heartbeat
17
Plug-in Architecture
• Plug-ins allow users to extend behavior of ROMA
• For example,
– Command plug-ins enable to change behavior of command module
– Users can append user-defined commands to ROMA
– Current version provides plug-ins for command module only
• Plug-ins for other modules coming soon
Command Execution
module
Routing moduleCommand Plug-ins
Storage module
Network IO module
18
Case Study: Commands for List Operations
• ROMA allows defining commands for list operations as plug-ins
– Users can atomically access to list data stored in ROMA as value
“Insert 2nd
elm into list” req
Insert 2nd
elm into list data
Memcached cluster
ROMA
“Insert 2nd
elm into list” req
① get binary data
② de-serialize binary data
③ insert 2nd
elm into list
④ serialize list data
⑤ put binary data
19
Case Study: Commands for List Operations
# alist_insert <key> <index> <bytes> [forward]rn
# <data block>rn
#
# (STORED|NOT_STORED|SERVER_ERROR <error message>)rn
def ev_alist_insert(s)
hname, k, d, vn, nodes = calc_hash(s[1])
data = read_bytes(s[3].to_i); read_bytes(2)
return forward2(nodes[0], s, data) if nodes[0] != @nid
ddata = @storages[hname].get(vn, k, d)
v = [[], []] unless ddata
v = Marshal.load(ddata) if ddata
v[0].insert(s[2].to_i, data)
v[1].insert(s[2].to_i, Time.now.to_i)
expt = 0x7fffffff
ret = @storages[hname].set(vn, k, d, expt, Marshal.dump(v))
@stats.write_count += 1
if ret
redundant(nodes[1..-1], hname, k, d, ret[2], expt, ret[4])
send_data("STOREDrn")
end
send_data("NOT_STOREDrn") unless ret
end
Method declaration
for list command
named alist_insert”“
20
Command Plug-in Mechanism
• Command exec module
– Loads plug-in and registers plug-in method at startup
• Plug-in method has method name starting with ev_
• For example, ev_alist_insert is plug-in method
• Ruby allows adding new methods to classes dynamically
– Calls plug-in method
1. Receives data from network IO module and creates new command
2. Finds registered plug-in method responding to the command
3. Calls plug-in method
• Uses send method provided by Ruby
Command Execution
module
Routing module
Storage module
Command Plug-ins
Network IO module
21
DSL for Command Plug-in
• DSL enables users to simply declare commands
– Without distribution concern (data replication, data partitioning)
– def_write_command_with_key_value
• Allows easily defining commands for storing structured data in ROMA
• For example,
– User can declare a alist_insert command with DSL
# alist_insert <key> <index> <bytes> [forward]rn
# <data block>rn
#
# (STORED|NOT_STORED|SERVER_ERROR <error message>)rn
def_write_command_with_key_value :alist_insert, 3 do |ctx|
v = [[], []]
v = Marshal.load(ctx.stored.value) if ctx.stored
v[0].insert(ctx.argv[2].to_i, ctx.params.value
v[1].insert(ctx.argv[2].to_i, Time.now.to_i)
expt = 0x7fffffff
[0, expt, Marshal.dump(v), :write, 'STORED']
end
22
Commands for Map Operations with DSL
• Another example: User can declare map_set command with DSL
– User can store map data in ROMA as value of specified key
# map_set <key> <mapkey> <flags> <expt> <bytes>rn
# <data block>rn
#
# (STORED|NOT_STORED|SERVER_ERROR <error message>)rn
def_write_command_with_key_value :map_set, 5 do |ctx|
v = {}
v = Marshal.load(ctx.stored.value) if ctx.stored
v[ctx.argv[2]] = ctx.params.value
expt = ctx.argv[4].to_i
if expt == 0
expt = 0x7fffffff
elsif expt < 2592000
expt += Time.now.to_i
end
[0, expt, Marshal.dump(v), :write, 'STORED']
end
23
DSL Mechanism
• def_write_command_with_key_value is declared as method in Ruby
def def_write_command_with_key_value(cmd, idx_of_val_len, forward = :one_line, &block)
define_method "ev_#{cmd}" do |s|
params = CommandParams.new
params.key, params.hash_name = s[1].split("e")
params.digest = Digest::SHA1.hexdigest(params.key).hex % @rttable.hbits
params.vn = @rttable.get_vnode_id(params.digest)
params.nodes = @rttable.search_nodes_for_write(params.vn)
params.value = read_bytes(s[idx_of_val_len].to_i)
read_bytes(2)
stored = StoredData.new
stored.vn, stored.last, stored.clk, stored.expt, stored.value =
@storages[params.hash_name].get_raw(params.vn, params.key, params.digest)
stored = nil if stored.vn == nil || Time.now.to_i > stored.expt
ctx = CommandContext.new(s, params, stored)
ret = instance_exec(ctx, &block)
if ret
redundant(ctx.params.nodes[1..-1], ctx.params.hash_name,
ctx.params.key, ctx.params.digest, ret[2], expt, ret[4])
send_data("#{msg}rn")
end
send_data("NOT_#{msg}rn") unless ret
end
24
Conclusion
• Background
– Spread of NoSQL databases
– Increase of application-specific needs
• Features of ROMA
– To respond to specific needs, ROMA provides user-customizable interfaces
– Plug-in architecture and its domain specific language
• Overall architecture of ROMA
– It integrates several well-known techniques
• Consistent hashing, chain replication-like mechanism, lamport clocks, etc
• Plug-in architecture and DSL
– Plug-ins allow enhancing behavior of ROMA easily
25
Fin
• Thanks for your listening
26

More Related Content

What's hot

What's hot (20)

Cisco chapter
Cisco chapter Cisco chapter
Cisco chapter
 
Ftp: a slideshow on File transfer protocol
Ftp: a slideshow on File transfer protocol Ftp: a slideshow on File transfer protocol
Ftp: a slideshow on File transfer protocol
 
File Transfer Protocol(ftp)
File Transfer Protocol(ftp)File Transfer Protocol(ftp)
File Transfer Protocol(ftp)
 
Vancouver Uses FME to Open Data to the World
Vancouver Uses FME to Open Data to the WorldVancouver Uses FME to Open Data to the World
Vancouver Uses FME to Open Data to the World
 
12 processor structure and function
12 processor structure and function12 processor structure and function
12 processor structure and function
 
Link state routing protocol
Link state routing protocolLink state routing protocol
Link state routing protocol
 
MPLS Traffic Engineering
MPLS Traffic EngineeringMPLS Traffic Engineering
MPLS Traffic Engineering
 
Ftp
FtpFtp
Ftp
 
Ftp.75 to 76
Ftp.75 to 76Ftp.75 to 76
Ftp.75 to 76
 
Computer network coe351- part2- final
Computer network coe351- part2- finalComputer network coe351- part2- final
Computer network coe351- part2- final
 
Application Layer and Protocols
Application Layer and ProtocolsApplication Layer and Protocols
Application Layer and Protocols
 
file transfer and access utilities
file transfer and access utilitiesfile transfer and access utilities
file transfer and access utilities
 
Mule TCP Component
Mule TCP ComponentMule TCP Component
Mule TCP Component
 
MPLS Traffic Engineering
MPLS Traffic EngineeringMPLS Traffic Engineering
MPLS Traffic Engineering
 
Data Center Network Multipathing
Data Center Network MultipathingData Center Network Multipathing
Data Center Network Multipathing
 
Application layer protocols
Application layer protocolsApplication layer protocols
Application layer protocols
 
Lec 8(FTP Protocol)
Lec 8(FTP Protocol)Lec 8(FTP Protocol)
Lec 8(FTP Protocol)
 
Exchange Server 2013 : les mécanismes de haute disponibilité et la redondance...
Exchange Server 2013 : les mécanismes de haute disponibilité et la redondance...Exchange Server 2013 : les mécanismes de haute disponibilité et la redondance...
Exchange Server 2013 : les mécanismes de haute disponibilité et la redondance...
 
Routing Presentation
Routing PresentationRouting Presentation
Routing Presentation
 
BASICS OF ROUTING IN NETWORKS
BASICS OF ROUTING IN NETWORKSBASICS OF ROUTING IN NETWORKS
BASICS OF ROUTING IN NETWORKS
 

Viewers also liked

Devsumi2012フィードバック オープニング
Devsumi2012フィードバック オープニングDevsumi2012フィードバック オープニング
Devsumi2012フィードバック オープニング
Rakuten Group, Inc.
 

Viewers also liked (20)

[Rakuten TechConf2014] [Sendai] Sense of SENS
[Rakuten TechConf2014] [Sendai] Sense of SENS[Rakuten TechConf2014] [Sendai] Sense of SENS
[Rakuten TechConf2014] [Sendai] Sense of SENS
 
インターネットガバナンス徒然2010-2
インターネットガバナンス徒然2010-2インターネットガバナンス徒然2010-2
インターネットガバナンス徒然2010-2
 
買収案件のセキュリティテストを円滑に進める手段
買収案件のセキュリティテストを円滑に進める手段買収案件のセキュリティテストを円滑に進める手段
買収案件のセキュリティテストを円滑に進める手段
 
Hadoop Conference Japan 2009 #2
Hadoop Conference Japan 2009 #2Hadoop Conference Japan 2009 #2
Hadoop Conference Japan 2009 #2
 
Hadoop Conference Japan 2009 #1
Hadoop Conference Japan 2009 #1Hadoop Conference Japan 2009 #1
Hadoop Conference Japan 2009 #1
 
第4回楽天研究開発シンポジウム.開会挨拶
第4回楽天研究開発シンポジウム.開会挨拶第4回楽天研究開発シンポジウム.開会挨拶
第4回楽天研究開発シンポジウム.開会挨拶
 
プレゼン勉強会 「イイタイコト 革命」Study session on presentation "What you want to say" Revol...
プレゼン勉強会 「イイタイコト 革命」Study session on presentation "What you want to say" Revol...プレゼン勉強会 「イイタイコト 革命」Study session on presentation "What you want to say" Revol...
プレゼン勉強会 「イイタイコト 革命」Study session on presentation "What you want to say" Revol...
 
[RakutenTechConf2013] [C-0] 日本にいたまま英語ペラペラ化!来年は海外カンファレンスでしゃべろう!
[RakutenTechConf2013] [C-0] 日本にいたまま英語ペラペラ化!来年は海外カンファレンスでしゃべろう![RakutenTechConf2013] [C-0] 日本にいたまま英語ペラペラ化!来年は海外カンファレンスでしゃべろう!
[RakutenTechConf2013] [C-0] 日本にいたまま英語ペラペラ化!来年は海外カンファレンスでしゃべろう!
 
ROMA on JRuby at JRubyKaigi 2010
ROMA on JRuby at JRubyKaigi 2010ROMA on JRuby at JRubyKaigi 2010
ROMA on JRuby at JRubyKaigi 2010
 
ネットワーク分散型フレームワークConView
ネットワーク分散型フレームワークConViewネットワーク分散型フレームワークConView
ネットワーク分散型フレームワークConView
 
Devsumi2012フィードバック オープニング
Devsumi2012フィードバック オープニングDevsumi2012フィードバック オープニング
Devsumi2012フィードバック オープニング
 
楽天テクノロジーカンファレン2009 基調講演
楽天テクノロジーカンファレン2009 基調講演楽天テクノロジーカンファレン2009 基調講演
楽天テクノロジーカンファレン2009 基調講演
 
Afthon rakuten api_introduce_20130406
Afthon rakuten api_introduce_20130406Afthon rakuten api_introduce_20130406
Afthon rakuten api_introduce_20130406
 
楽天WEB APIワークショップ@東工大
楽天WEB APIワークショップ@東工大楽天WEB APIワークショップ@東工大
楽天WEB APIワークショップ@東工大
 
楽天のアクセス解析とスマートフォンへの取組み マルチデバイス時代に向け、アクセス解析の可能性と課題を探る
楽天のアクセス解析とスマートフォンへの取組み マルチデバイス時代に向け、アクセス解析の可能性と課題を探る楽天のアクセス解析とスマートフォンへの取組み マルチデバイス時代に向け、アクセス解析の可能性と課題を探る
楽天のアクセス解析とスマートフォンへの取組み マルチデバイス時代に向け、アクセス解析の可能性と課題を探る
 
スクラムマスター1年生⇒2年生
スクラムマスター1年生⇒2年生スクラムマスター1年生⇒2年生
スクラムマスター1年生⇒2年生
 
第22回 #TFSUG ちぇんわ祭り - ALM Essentials
第22回 #TFSUG ちぇんわ祭り - ALM Essentials第22回 #TFSUG ちぇんわ祭り - ALM Essentials
第22回 #TFSUG ちぇんわ祭り - ALM Essentials
 
楽天プロジェクトX:基幹DB移設 編
楽天プロジェクトX:基幹DB移設 編楽天プロジェクトX:基幹DB移設 編
楽天プロジェクトX:基幹DB移設 編
 
iPhoneとAndroidのアプリ開発最新潮流
iPhoneとAndroidのアプリ開発最新潮流iPhoneとAndroidのアプリ開発最新潮流
iPhoneとAndroidのアプリ開発最新潮流
 
楽天エンジニアライフ
楽天エンジニアライフ楽天エンジニアライフ
楽天エンジニアライフ
 

Similar to ROMA -- An Customizable-NoSQL Database in Ruby at NoSQL Afternoon in Japan

Packet Analysis - Course Technology Computing Conference
Packet Analysis - Course Technology Computing ConferencePacket Analysis - Course Technology Computing Conference
Packet Analysis - Course Technology Computing Conference
Cengage Learning
 

Similar to ROMA -- An Customizable-NoSQL Database in Ruby at NoSQL Afternoon in Japan (20)

Network architecure (3).pptx
Network architecure (3).pptxNetwork architecure (3).pptx
Network architecure (3).pptx
 
ROMA User-Customizable NoSQL Database in Ruby
ROMA User-Customizable NoSQL Database in RubyROMA User-Customizable NoSQL Database in Ruby
ROMA User-Customizable NoSQL Database in Ruby
 
Unit 2 oracle9i
Unit 2  oracle9i Unit 2  oracle9i
Unit 2 oracle9i
 
Hadoop
HadoopHadoop
Hadoop
 
Viloria osi layer4-7
Viloria osi layer4-7Viloria osi layer4-7
Viloria osi layer4-7
 
Presentation3 Multi-User Architecture.pdf
Presentation3 Multi-User Architecture.pdfPresentation3 Multi-User Architecture.pdf
Presentation3 Multi-User Architecture.pdf
 
Alternative metrics
Alternative metricsAlternative metrics
Alternative metrics
 
Exchange 2013 Haute disponibilité et tolérance aux sinistres (Session 1/2 pre...
Exchange 2013 Haute disponibilité et tolérance aux sinistres (Session 1/2 pre...Exchange 2013 Haute disponibilité et tolérance aux sinistres (Session 1/2 pre...
Exchange 2013 Haute disponibilité et tolérance aux sinistres (Session 1/2 pre...
 
02 protocols and tcp-ip
02 protocols and tcp-ip02 protocols and tcp-ip
02 protocols and tcp-ip
 
Iso model
Iso modelIso model
Iso model
 
Chap1
Chap1Chap1
Chap1
 
LISA_Sol_Linux_Perf.ppt
LISA_Sol_Linux_Perf.pptLISA_Sol_Linux_Perf.ppt
LISA_Sol_Linux_Perf.ppt
 
chapter 1 &2 RIPv1&2.ppt
chapter 1 &2 RIPv1&2.pptchapter 1 &2 RIPv1&2.ppt
chapter 1 &2 RIPv1&2.ppt
 
Packet Analysis - Course Technology Computing Conference
Packet Analysis - Course Technology Computing ConferencePacket Analysis - Course Technology Computing Conference
Packet Analysis - Course Technology Computing Conference
 
UNIT II DIS.pptx
UNIT II DIS.pptxUNIT II DIS.pptx
UNIT II DIS.pptx
 
ElasticSearch as (only) datastore
ElasticSearch as (only) datastoreElasticSearch as (only) datastore
ElasticSearch as (only) datastore
 
06 - OSI Model.ppt
06 - OSI Model.ppt06 - OSI Model.ppt
06 - OSI Model.ppt
 
06 - OSI Model.ppt
06 - OSI Model.ppt06 - OSI Model.ppt
06 - OSI Model.ppt
 
Apache Thrift, a brief introduction
Apache Thrift, a brief introductionApache Thrift, a brief introduction
Apache Thrift, a brief introduction
 
Sonali Bank Network Design Project Report
Sonali Bank Network Design Project ReportSonali Bank Network Design Project Report
Sonali Bank Network Design Project Report
 

More from Rakuten Group, Inc.

More from Rakuten Group, Inc. (20)

コードレビュー改善のためにJenkinsとIntelliJ IDEAのプラグインを自作してみた話
コードレビュー改善のためにJenkinsとIntelliJ IDEAのプラグインを自作してみた話コードレビュー改善のためにJenkinsとIntelliJ IDEAのプラグインを自作してみた話
コードレビュー改善のためにJenkinsとIntelliJ IDEAのプラグインを自作してみた話
 
楽天における安全な秘匿情報管理への道のり
楽天における安全な秘匿情報管理への道のり楽天における安全な秘匿情報管理への道のり
楽天における安全な秘匿情報管理への道のり
 
What Makes Software Green?
What Makes Software Green?What Makes Software Green?
What Makes Software Green?
 
Simple and Effective Knowledge-Driven Query Expansion for QA-Based Product At...
Simple and Effective Knowledge-Driven Query Expansion for QA-Based Product At...Simple and Effective Knowledge-Driven Query Expansion for QA-Based Product At...
Simple and Effective Knowledge-Driven Query Expansion for QA-Based Product At...
 
DataSkillCultureを浸透させる楽天の取り組み
DataSkillCultureを浸透させる楽天の取り組みDataSkillCultureを浸透させる楽天の取り組み
DataSkillCultureを浸透させる楽天の取り組み
 
大規模なリアルタイム監視の導入と展開
大規模なリアルタイム監視の導入と展開大規模なリアルタイム監視の導入と展開
大規模なリアルタイム監視の導入と展開
 
楽天における大規模データベースの運用
楽天における大規模データベースの運用楽天における大規模データベースの運用
楽天における大規模データベースの運用
 
楽天サービスを支えるネットワークインフラストラクチャー
楽天サービスを支えるネットワークインフラストラクチャー楽天サービスを支えるネットワークインフラストラクチャー
楽天サービスを支えるネットワークインフラストラクチャー
 
楽天の規模とクラウドプラットフォーム統括部の役割
楽天の規模とクラウドプラットフォーム統括部の役割楽天の規模とクラウドプラットフォーム統括部の役割
楽天の規模とクラウドプラットフォーム統括部の役割
 
Rakuten Services and Infrastructure Team.pdf
Rakuten Services and Infrastructure Team.pdfRakuten Services and Infrastructure Team.pdf
Rakuten Services and Infrastructure Team.pdf
 
The Data Platform Administration Handling the 100 PB.pdf
The Data Platform Administration Handling the 100 PB.pdfThe Data Platform Administration Handling the 100 PB.pdf
The Data Platform Administration Handling the 100 PB.pdf
 
Supporting Internal Customers as Technical Account Managers.pdf
Supporting Internal Customers as Technical Account Managers.pdfSupporting Internal Customers as Technical Account Managers.pdf
Supporting Internal Customers as Technical Account Managers.pdf
 
Making Cloud Native CI_CD Services.pdf
Making Cloud Native CI_CD Services.pdfMaking Cloud Native CI_CD Services.pdf
Making Cloud Native CI_CD Services.pdf
 
How We Defined Our Own Cloud.pdf
How We Defined Our Own Cloud.pdfHow We Defined Our Own Cloud.pdf
How We Defined Our Own Cloud.pdf
 
Travel & Leisure Platform Department's tech info
Travel & Leisure Platform Department's tech infoTravel & Leisure Platform Department's tech info
Travel & Leisure Platform Department's tech info
 
Travel & Leisure Platform Department's tech info
Travel & Leisure Platform Department's tech infoTravel & Leisure Platform Department's tech info
Travel & Leisure Platform Department's tech info
 
OWASPTop10_Introduction
OWASPTop10_IntroductionOWASPTop10_Introduction
OWASPTop10_Introduction
 
Introduction of GORA API Group technology
Introduction of GORA API Group technologyIntroduction of GORA API Group technology
Introduction of GORA API Group technology
 
100PBを越えるデータプラットフォームの実情
100PBを越えるデータプラットフォームの実情100PBを越えるデータプラットフォームの実情
100PBを越えるデータプラットフォームの実情
 
社内エンジニアを支えるテクニカルアカウントマネージャー
社内エンジニアを支えるテクニカルアカウントマネージャー社内エンジニアを支えるテクニカルアカウントマネージャー
社内エンジニアを支えるテクニカルアカウントマネージャー
 

Recently uploaded

Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
FIDO Alliance
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
FIDO Alliance
 

Recently uploaded (20)

UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overview
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!
 
Introduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxIntroduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptx
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentation
 
How we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfHow we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdf
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development Companies
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
 
Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - Questionnaire
 
2024 May Patch Tuesday
2024 May Patch Tuesday2024 May Patch Tuesday
2024 May Patch Tuesday
 
ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage Intacct
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps Productivity
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cf
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 

ROMA -- An Customizable-NoSQL Database in Ruby at NoSQL Afternoon in Japan

  • 1. 1 ROMA -- User-Customizable NoSQL Database in Ruby Rakuten, Inc., Rakuten Institute of Technology, Tokyo | Muga Nishizawa | Nov. 1st, 2010
  • 2. 2 Agenda • Background • Features of ROMA • Overall architecture of ROMA • Plug-in architecture and its domain specific language • Conclusion
  • 3. 3 Spread of NoSQL databases • Needs for NoSQL databases are rising in order to easily store rapidly increasing data • Basic features of several NoSQL databases are – High scalability – High availability – High throughput – Other features vary by databases • ROMA – Started development with Matz in 2007 – Open sourced in Oct. 2009 • See http://github.com/roma/roma/
  • 4. 4 ROMA Usage in Rakuten • ROMA is used in various Rakuten services – ROMA runs on dozens of servers – Various types of data are stored • E.g. session data, personal page view history, etc • Rakuten, Inc. – It provides many e-commerce platforms • E.g. Rakuten Ichiba, Rakuten Travel, Rakuten Books, etc – Rakuten has tens of millions of users
  • 5. 5 Increase of Application-Specific Needs • Many specific needs from application-side – These specific needs come from actual service development sites • For example, users say – “How can we store structured data?” – “How can we easily process stored data on DWH or Hadoop?” – “How can we delete null character that was stored somewhere by error?” – “How can we delete duplicate data that were accidentally stored somewhere?” • We want to respond to all of these needs – We are trying to solve these problems, one by one
  • 6. 6 Case Study: List Operation in Rakuten Travel • “How can we access to list data that were stored in ROMA easily?” – Request from Rakuten Travel to apply page view history using ROMA – Page view history function is useful function for users
  • 7. 7 Case Study: List Operation in Rakuten Travel • The application stores list data for users in database – Key: user ID, Value: a list of pages that the user viewed • To update list data that were stored in NoSQL database, application 1. Gets binary data of specified key 2. Deserializes it as list data 3. Updates the list data according to user requirement 4. Serializes the list data to binary data 5. Set it to database “Insert 2nd elm into list” req ① get binary data ② deserialize binary as list data ③ insert 2nd elm into list ④ serialize list data to binary ⑤ put binary data Memcached cluster
  • 8. 8 ROMA • User-customizable NoSQL database in Ruby • Features – High scalability – High availability – Fault-tolerance – Better throughput – And… • To address application-specific needs, ROMA provides – Plug-in architecture – Domain specific language (DSL) • ROMA enables responding to the above need from Rakuten Travel
  • 9. 9 Overall Architecture of ROMA • ROMA integrates several well-known techniques to achieve scalability, availability and fault-tolerance – For example, consistent hashing, virtual nodes, chain replication-like mechanism, lamport clocks, etc • ROMA node consists of 4 modules – Network IO module: Receiving data from clients and other ROMA nodes – Command exec module: Creating and executing commands – Routing module: Maintaining ring information – Storage module: Storing data Command Execution module Storage module Network IO module Routing module
  • 10. 10 Data Partitioning • Consistent hashing and virtual nodes – ROMA consists of several nodes that run on servers – Many virtual nodes are allocated on 1-dimensional hash space of 32-bits • Each virtual node has a 32-bits ID – To determine ROMA nodes that are responsible for serving key-value – SHA-1 hash ROMA E.g. ROMA consists of three nodes
  • 11. 11 Data Accessing • Getting value of specified key from ROMA 1. User accesses to ROMA nodes 2. The node determines others that are responsible for value of specified key 3. The node gets the value from the other node 4. The node returns it to user ROMA E.g. ROMA consists of three nodes ① ②
  • 12. 12 Ring Membership Model • Each node maintains and periodically shares routing table with others – If several versions exist, node updates latest version of routing table – Lamport clocks and Merkle hash tree • ROMA node multicasts with others to share routing tables – Current version of ROMA doesn’t use gossip-based protocol ROMA E.g. ROMA consists of three nodes Sharing ring information
  • 13. 13 Ring Cache on Client-side • ROMA client enables direct accessing to data – Cache of routing table – It checks that routing table is updated or not every 3 sec. • Getting value of specified key, ROMA client 1. Determines nodes 2. Gets the value from the node directly ROMA E.g. ROMA consists of three nodes
  • 14. 14 Data Replication • Automatic data replication – Client waits until data replication succeeds – If data replication failed, the data is push to asynchronous queue in node – Eventual consistency • Each data is replicated at N nodes – N: this parameter is configured before runtime ROMA E.g. ROMA consists of three nodes
  • 15. 15 Message Protocol • Extended Memcached protocol over TCP – Used between clients and ROMA nodes • Memcached client libraries are available – Without distribution concern – Users can use telnet $ telnet localhost 11211 Trying 127.0.0.1... Connected to localhost. Escapecharacter is ‘^]’. set foo 0 0 3 bar STORED get foo VALUE foo 0 3 bar END User can set/get data in ROMA with telnet like memcached
  • 16. 16 Failure Detection • Heartbeat detection – Each node multicasts periodic heartbeat with others – Heartbeat is flooded every 1 sec. • If heartbeat is missed continuously, the node is declared as failed – Failover – Removal of the node ROMA E.g. ROMA consists of three nodes Sending heartbeat
  • 17. 17 Plug-in Architecture • Plug-ins allow users to extend behavior of ROMA • For example, – Command plug-ins enable to change behavior of command module – Users can append user-defined commands to ROMA – Current version provides plug-ins for command module only • Plug-ins for other modules coming soon Command Execution module Routing moduleCommand Plug-ins Storage module Network IO module
  • 18. 18 Case Study: Commands for List Operations • ROMA allows defining commands for list operations as plug-ins – Users can atomically access to list data stored in ROMA as value “Insert 2nd elm into list” req Insert 2nd elm into list data Memcached cluster ROMA “Insert 2nd elm into list” req ① get binary data ② de-serialize binary data ③ insert 2nd elm into list ④ serialize list data ⑤ put binary data
  • 19. 19 Case Study: Commands for List Operations # alist_insert <key> <index> <bytes> [forward]rn # <data block>rn # # (STORED|NOT_STORED|SERVER_ERROR <error message>)rn def ev_alist_insert(s) hname, k, d, vn, nodes = calc_hash(s[1]) data = read_bytes(s[3].to_i); read_bytes(2) return forward2(nodes[0], s, data) if nodes[0] != @nid ddata = @storages[hname].get(vn, k, d) v = [[], []] unless ddata v = Marshal.load(ddata) if ddata v[0].insert(s[2].to_i, data) v[1].insert(s[2].to_i, Time.now.to_i) expt = 0x7fffffff ret = @storages[hname].set(vn, k, d, expt, Marshal.dump(v)) @stats.write_count += 1 if ret redundant(nodes[1..-1], hname, k, d, ret[2], expt, ret[4]) send_data("STOREDrn") end send_data("NOT_STOREDrn") unless ret end Method declaration for list command named alist_insert”“
  • 20. 20 Command Plug-in Mechanism • Command exec module – Loads plug-in and registers plug-in method at startup • Plug-in method has method name starting with ev_ • For example, ev_alist_insert is plug-in method • Ruby allows adding new methods to classes dynamically – Calls plug-in method 1. Receives data from network IO module and creates new command 2. Finds registered plug-in method responding to the command 3. Calls plug-in method • Uses send method provided by Ruby Command Execution module Routing module Storage module Command Plug-ins Network IO module
  • 21. 21 DSL for Command Plug-in • DSL enables users to simply declare commands – Without distribution concern (data replication, data partitioning) – def_write_command_with_key_value • Allows easily defining commands for storing structured data in ROMA • For example, – User can declare a alist_insert command with DSL # alist_insert <key> <index> <bytes> [forward]rn # <data block>rn # # (STORED|NOT_STORED|SERVER_ERROR <error message>)rn def_write_command_with_key_value :alist_insert, 3 do |ctx| v = [[], []] v = Marshal.load(ctx.stored.value) if ctx.stored v[0].insert(ctx.argv[2].to_i, ctx.params.value v[1].insert(ctx.argv[2].to_i, Time.now.to_i) expt = 0x7fffffff [0, expt, Marshal.dump(v), :write, 'STORED'] end
  • 22. 22 Commands for Map Operations with DSL • Another example: User can declare map_set command with DSL – User can store map data in ROMA as value of specified key # map_set <key> <mapkey> <flags> <expt> <bytes>rn # <data block>rn # # (STORED|NOT_STORED|SERVER_ERROR <error message>)rn def_write_command_with_key_value :map_set, 5 do |ctx| v = {} v = Marshal.load(ctx.stored.value) if ctx.stored v[ctx.argv[2]] = ctx.params.value expt = ctx.argv[4].to_i if expt == 0 expt = 0x7fffffff elsif expt < 2592000 expt += Time.now.to_i end [0, expt, Marshal.dump(v), :write, 'STORED'] end
  • 23. 23 DSL Mechanism • def_write_command_with_key_value is declared as method in Ruby def def_write_command_with_key_value(cmd, idx_of_val_len, forward = :one_line, &block) define_method "ev_#{cmd}" do |s| params = CommandParams.new params.key, params.hash_name = s[1].split("e") params.digest = Digest::SHA1.hexdigest(params.key).hex % @rttable.hbits params.vn = @rttable.get_vnode_id(params.digest) params.nodes = @rttable.search_nodes_for_write(params.vn) params.value = read_bytes(s[idx_of_val_len].to_i) read_bytes(2) stored = StoredData.new stored.vn, stored.last, stored.clk, stored.expt, stored.value = @storages[params.hash_name].get_raw(params.vn, params.key, params.digest) stored = nil if stored.vn == nil || Time.now.to_i > stored.expt ctx = CommandContext.new(s, params, stored) ret = instance_exec(ctx, &block) if ret redundant(ctx.params.nodes[1..-1], ctx.params.hash_name, ctx.params.key, ctx.params.digest, ret[2], expt, ret[4]) send_data("#{msg}rn") end send_data("NOT_#{msg}rn") unless ret end
  • 24. 24 Conclusion • Background – Spread of NoSQL databases – Increase of application-specific needs • Features of ROMA – To respond to specific needs, ROMA provides user-customizable interfaces – Plug-in architecture and its domain specific language • Overall architecture of ROMA – It integrates several well-known techniques • Consistent hashing, chain replication-like mechanism, lamport clocks, etc • Plug-in architecture and DSL – Plug-ins allow enhancing behavior of ROMA easily
  • 25. 25 Fin • Thanks for your listening
  • 26. 26