- Kerberos is used to authenticate Hadoop services and clients running on different nodes communicating over a non-secure network. It uses tickets for authentication.
- Key configuration changes are required to enable Kerberos authentication in Hadoop including setting hadoop.security.authentication to kerberos and generating keytabs containing principal keys for HDFS services.
- Services are associated with Kerberos principles using keytabs which are then configured for use by the relevant Hadoop processes and services.
Data Lake, Virtual Database, or Data Hub - How to Choose?DATAVERSITY
Data integration is just plain hard and there is no magic bullet. That said, three new data integration techniques do ameliorate the misery, making silo-busting possible, if not trivial. The three approaches – data lakes, virtual databases (aka federated databases), and data hubs – are a boon to organizations big enough to have separate systems, separate lines of business, and redundant acquired or COTS data stores. Each approach has its place, but how do you make the right decision about which data silo integration approach to choose and when?
This webinar describes how you can use the key concepts of data Movement, Harmonization, and Indexing to determine what you are giving up or investing in, and make the best decision for your project.
- Kerberos is used to authenticate Hadoop services and clients running on different nodes communicating over a non-secure network. It uses tickets for authentication.
- Key configuration changes are required to enable Kerberos authentication in Hadoop including setting hadoop.security.authentication to kerberos and generating keytabs containing principal keys for HDFS services.
- Services are associated with Kerberos principles using keytabs which are then configured for use by the relevant Hadoop processes and services.
Data Lake, Virtual Database, or Data Hub - How to Choose?DATAVERSITY
Data integration is just plain hard and there is no magic bullet. That said, three new data integration techniques do ameliorate the misery, making silo-busting possible, if not trivial. The three approaches – data lakes, virtual databases (aka federated databases), and data hubs – are a boon to organizations big enough to have separate systems, separate lines of business, and redundant acquired or COTS data stores. Each approach has its place, but how do you make the right decision about which data silo integration approach to choose and when?
This webinar describes how you can use the key concepts of data Movement, Harmonization, and Indexing to determine what you are giving up or investing in, and make the best decision for your project.
UI tests have supported re-engineering of the Cookpad iOS app, which has over 35 million users. The app is developed continuously, releasing every 2-4 weeks. UI tests were implemented in 2014 to check external and internal changes as the app is re-written and refactored. Automated UI tests help reduce fear for developers during re-engineering. They also help ensure quality and stability as the mobile environment and services change frequently. UI tests are important for legacy re-engineering projects to enable continuous development.
The document discusses DevOps practices at Flickr for deploying code 10 times per day. It links to a slideshare presentation about Flickr's cooperation between development and operations teams. Additional links are provided about codebases, infrastructure as code, and other DevOps topics.
UI tests have supported re-engineering of the Cookpad iOS app, which has over 35 million users. The app is developed continuously, releasing every 2-4 weeks. UI tests were implemented in 2014 to check external and internal changes as the app is re-written and refactored. Automated UI tests help reduce fear for developers during re-engineering. They also help ensure quality and stability as the mobile environment and services change frequently. UI tests are important for legacy re-engineering projects to enable continuous development.
The document discusses DevOps practices at Flickr for deploying code 10 times per day. It links to a slideshare presentation about Flickr's cooperation between development and operations teams. Additional links are provided about codebases, infrastructure as code, and other DevOps topics.
This document discusses how to make software more green and environmentally friendly. It defines green software as software that is carbon efficient, energy efficient, hardware efficient, and carbon aware. It provides recommendations for various roles within an organization on driving green initiatives, including focusing on efficiency for CxOs, architects, infrastructure engineers, and developers. Examples include optimizing resource usage, using public clouds effectively, prioritizing equipment standardization, and developing applications that can run more efficiently.
Simple and Effective Knowledge-Driven Query Expansion for QA-Based Product At...Rakuten Group, Inc.
The document proposes a knowledge-driven query expansion approach for question answering (QA)-based product attribute extraction. It trains QA models using attribute-value pairs from training data as knowledge, while mimicking imperfect knowledge at test time through techniques like knowledge dropout and token mixing. This helps induce better query representations, especially for rare and ambiguous attributes. Experiments on a cleaned product attribute dataset show the proposed approach with all techniques outperforms baseline methods in both macro and micro F1 scores.
This document summarizes Andrew Hajinikitas' work developing Rakuten's private cloud infrastructure. It describes the key components of Rakuten's infrastructure including metal instances, microservers, and GPU servers. It provides details on Rakuten's software stack and their goals to expand managed services. Currently, Rakuten operates 9 data centers in Japan and overseas providing around 30,000 servers to support their ecosystem. Their future plans include extending network self-service, making GPU resources available as a platform service, and improving efficiency through optimized hardware selection.
The document discusses the Travel & Leisure Platform Dept and its responsibilities related to data and platform management. It provides an overview of the technical stack including private/public clouds, databases, containers, and automation/monitoring tools. It then discusses recent projects involving business continuity, containerization, alert integration, and automation. Finally, it describes open roles for a DBA and DevOps position and their responsibilities related to database provisioning, backup/recovery, infrastructure as code, and providing platforms and tools for developers.
This presentation introduces the OWASP Top 10:2021.
It explains how to look at the data related to OWASP Top 10:2021, and provides detailed explanations of items with distinctive data. It also introduces the OWASP Project related to each item.
Gora API Group technology provides a microservices architecture and APIs for Rakuten's golf course reservation system, improving the user experience and increasing customer loyalty and annual golf rounds. The architecture migrates the monolithic reservation system to microservices using Kotlin, Spring Boot, and other technologies, exposing APIs for the frontend and new products while sustaining the legacy system through services, queues, continuous delivery, and operations monitoring.
5. Company Profile
• Founded: February 7, 1997
• IPO: April 19, 2000 (Jasdaq Stock Exchange)
• Chairman & CEO: Hiroshi Mikitani
The First Office: Atago in Tokyo, Japan
Employees : 5 (at the establishment)
Business: Rakuten Ichiba (online mall)
5
6. Company Profile (Continued)
New Office: Rakuten Tower (Tokyo, Japan)
Employees: 7,615 (as of Dec. 2011)
Market Cap: 1.2 Trillion yen or US$15.6 Billion
(JASDAQ, as of September 20, 2011)
Businesses: Rakuten Ichiba, Rakuten Travel, Rakuten Securities,
Rakuten KC, Rakuten Bank, etc.
6
14. インターネットの時代
ネットワークの向こう側の誰かとコラボレー
ション – Mass Collaboration
Wikipedia
OSS – Open Source Software
勉強会
Facebook, twitter, ...
2ch
ニコ動
14
15. Free Software
GNU Project – GNU’s Not Unix、1983
GNU General Public License
http://www.gnu.org/licenses/gpl.html
Linux、1991
Open Source Software、1998
15
16. GNU project
テープで配布
送付手数料をFSFに送金
テープで配布(CDROMですらなかった)
ソフトウェア開発はバザールモデルでなかった
Internet 商用化以前
16
17. Unix & Hacker Culture
• 1969, AT&T
• Ken Thompson, Dennis Ritchie, Brian
Kernighan, Douglas McIlroy, Michael
Lesk and Joe Ossanna.
• by 1973, C
• 1983, BSD 4.2
17
18. Hacker Ethic
• Access to computers—and anything which might teach
you something about the way the world works—should
be unlimited and total. Always yield to the Hands-On
Imperative
• All information should be free
• Mistrust authority — promote decentralization
• Hackers should be judged by their hacking, not criteria
such as degrees, age, race, sex, or position
• You can create art and beauty on a computer
• Computers can change your life for the better
http://en.wikipedia.org/wiki/Hacker_ethic
http://www.amazon.co.jp/exec/obidos/ASIN/487593100X/
18
45. Linux
• Kernel Development
• Who is doing the work
• 2.6.11, 389 developers, 68 companies
• 2.6.20, 673 developers, 138 companies
• 2.6.30, 1150 developers, 249 companies
• 2.6.39, 1258 developers, 239 companies
• 3.0, 1131 developers, 331 companies
• 3.2, 1316 developers, 226 companies
• All 7,944 developers, 855 companies
• Linux Kernel Development: How Fast it is Going, Who
is Doing It, What They are Doing, and Who is
Sponsoring It, By Linux foundation
• http://go.linuxfoundation.org/who-writes-linux-2012
45
60. ZONE DEFINITION
RED ZONE: More than 200 points away from target
YELLOW ZONE: Between 100-199 points away from target
ORANGE ZONE: Between 1 – 99 points away from target
GREEN ZONE: Score meets or exceeds target
Employee Not Reached Not Reached Not Reached Reached Target
Grade (RED) (YELLOW) (ORANGE) (GREEN)
AAA -550 551-650 651-749 750-
AA -500 501-600 601-699 700-
A -450 451-550 551-649 650-
BBB -400 401-500 501-599 600-
BB -400 401-500 501-599 600-
B -400 401-500 501-599 600-
60
61. ZONE STATUS
As of June 30th, 2012
29% GREEN 87%
42% 45% 48% 49% 51% 53% 54% 56% 58% 60% 63% 66% 72% 80%
9%
11%
14%
36%
16% ORANGE 8%
19%
YELLOW
No Score 20% 19% 17% 15% 13% 10% 7% 6% 6%
4%
5% 4% 2%
M A M J J A S O N D J F M RED 1%
2011 2012 A M J
Data: Ranten, Inc (Total may not equal 100% due to rounding) 61