SlideShare a Scribd company logo
SmartNews's
journey into microservices
Takumi Sakamoto
Livesense TechNight
2015.09.29
Takumi Sakamoto
@takus
SmartNews's journey into microservices
News Delivery Pipelines
36 months ago
The first system is developed
Crawler
Analyzer
Indexer
On memory DB
API
Monolithic App
Social
Media
Article
Lists
12 months ago
Split crawlers
Crawler
Queue
Monolithic App
Analyzer
Indexer
On memory DB
API
https://medium.com/@takus/velocity-conference-2015-wrap-up-velocityconf-f3811e05f1e8
4 months ago
At Velocity Conference
2 months ago
Dive into the microservices world
Crawler
Kinesis
IndexerAnalyzer
Kinesis CloudSearch
Why microservices?
• A lot of different types of services are required for us
• crawler, image analyzer, content extractor ...
• Adapt to technical progress / market change
• every system should be replaceable
• Make developers happy :)
• avoid to maintain "legacy" "large" "monolithic"
• enable us to develop any languages
A part of microservices
Source A
Analyzer Indexer Search API
A
PhotoqualCaffe
Reporting
API
Gateway
Realtime
feedback
Image
Analysis
Extractor
Content
extraction
Source B
Search API
B
Ranking
algorithm
A/B test
Language
detecter
Crawling
Language
detection
Spark
Streaming
KPI
Dashboard
Admin
Operation
References (Japanese only)
Stream 処理と Offline 処理の統合
http://bit.ly/1VnQQao
スマートニュースの世界進出を支えるログ解析基盤
http://bit.ly/1MCOyBX
SmartNews の Webmining を支えるプラットフォーム
http://bit.ly/1RcVGqk
Journey into microservices
When you use microservices you have to work on
automated deployment,
monitoring,
dealing with failure,
eventual consistency,
and other factors that a distributed system introduces.
http://martinfowler.com/bliki/MicroservicePremium.html
Automated Deployment
with AWS CodeDeploy
Today's topic
Deployment is not easy
• Manage complexity
• multiple roles
• multiple environments
• auto scaling
• Avoid downtime
• JVM based applications doesn't support hot deploy
• Monitoring
• stop deployment if something wrong
• which instances are succeeded or failed
AWS CodeDeploy
http://www.slideshare.net/AmazonWebServices/aws-codedeploy-manage-deployment-complexity
2W1H configurations
http://docs.aws.amazon.com/codedeploy/latest/userguide/deployment-steps.html
Where?How?What?
Deployment Flow
http://docs.aws.amazon.com/codedeploy/latest/userguide/deployment-steps.html
What?
How?
Where?
What to Deploy?
• Prepare your "revision" archive
• appspec.yml
• application (WAR/JAR etc...)
• (optional) scripts
• register to ELB / deregister from ELB
• pre-compile assets
• waiting cache warmup
App Spec
How to Deploy?
Where to Deploy?
EC2 Tag / Auto Scaling Group
My favorites
• Simple, Fast, Flexible, Free
• Require small changes for current system
• Integration w/ AWS Auto Scaling
• deploy last successful revision to a new instance
• wait for registering ELB until finishing deployment
Pitfall
Exclusive lock is required :P
Deployment is skipped during scale-out
Automated Deployment
at SmartNews
Deployment pipelines
Slack notification
Role / Env / Version
Deployment status
Commits
Datadog monitoring
Datadog alerts
Manage alerts by DSL
Most of alerts are created automatically
Powered by airbnb/interferon
Docker for C++ apps
Towards future...
We’re hiring!!!
Join our journey into microservices
Site reliability engineer
Productivity engineer
Web application engineer
iOS/Android engineer
ML/NLP engineer
Ad engineer
http://about.smartnews.com/en/careers/
Wait for you @ SmartKitchen

More Related Content

What's hot

Batch Processing with Amazon EC2 Container Service
Batch Processing with Amazon EC2 Container ServiceBatch Processing with Amazon EC2 Container Service
Batch Processing with Amazon EC2 Container Service
Amazon Web Services
 
Architecting on The Cloud
Architecting on The CloudArchitecting on The Cloud
Architecting on The Cloud
Amazon Web Services
 
AWS re:Invent 2016: Serverless Computing Patterns at Expedia (SVR306) )
AWS re:Invent 2016: Serverless Computing Patterns at Expedia (SVR306) )AWS re:Invent 2016: Serverless Computing Patterns at Expedia (SVR306) )
AWS re:Invent 2016: Serverless Computing Patterns at Expedia (SVR306) )
Amazon Web Services
 
A 60-mn tour of AWS compute (March 2016)
A 60-mn tour of AWS compute (March 2016)A 60-mn tour of AWS compute (March 2016)
A 60-mn tour of AWS compute (March 2016)
Julien SIMON
 
Moving Viadeo to AWS (2015)
Moving Viadeo to AWS (2015)Moving Viadeo to AWS (2015)
Moving Viadeo to AWS (2015)
Julien SIMON
 
AWS & Cloud competition from Azure, openstack
AWS & Cloud competition from Azure, openstack AWS & Cloud competition from Azure, openstack
AWS & Cloud competition from Azure, openstack
AWSKRUG - AWS한국사용자모임
 
An Overview of AWS IoT (November 2016)
An Overview of AWS IoT (November 2016)An Overview of AWS IoT (November 2016)
An Overview of AWS IoT (November 2016)
Julien SIMON
 
The Lost Tales of Platform Design (February 2017)
The Lost Tales of Platform Design (February 2017)The Lost Tales of Platform Design (February 2017)
The Lost Tales of Platform Design (February 2017)
Julien SIMON
 
AWS re:Invent 2016: Get Technically Inspired by Container-Powered Migrations ...
AWS re:Invent 2016: Get Technically Inspired by Container-Powered Migrations ...AWS re:Invent 2016: Get Technically Inspired by Container-Powered Migrations ...
AWS re:Invent 2016: Get Technically Inspired by Container-Powered Migrations ...
Amazon Web Services
 
Building a PaaS with Docker and AWS
Building a PaaS with Docker and AWSBuilding a PaaS with Docker and AWS
Building a PaaS with Docker and AWS
Amazon Web Services
 
A 60-minute tour of AWS Compute (November 2016)
A 60-minute tour of AWS Compute (November 2016)A 60-minute tour of AWS Compute (November 2016)
A 60-minute tour of AWS Compute (November 2016)
Julien SIMON
 
AWSKRUG 콘퍼런스 - re:Invent 신규 서비스 (박상욱) - 보안, 인프라 관련 서비스 소개
 AWSKRUG 콘퍼런스 - re:Invent 신규 서비스 (박상욱) - 보안, 인프라 관련 서비스 소개 AWSKRUG 콘퍼런스 - re:Invent 신규 서비스 (박상욱) - 보안, 인프라 관련 서비스 소개
AWSKRUG 콘퍼런스 - re:Invent 신규 서비스 (박상욱) - 보안, 인프라 관련 서비스 소개
AWSKRUG - AWS한국사용자모임
 
"How to optimize the architecture of your platform" by Julien Simon
"How to optimize the architecture of your platform" by Julien Simon"How to optimize the architecture of your platform" by Julien Simon
"How to optimize the architecture of your platform" by Julien Simon
TheFamily
 
PaaSing Your Code Around
PaaSing Your Code AroundPaaSing Your Code Around
PaaSing Your Code Around
Chris Tankersley
 
Deep Learning with AWS (November 2016)
Deep Learning with AWS (November 2016)Deep Learning with AWS (November 2016)
Deep Learning with AWS (November 2016)
Julien SIMON
 
An introduction to serverless architectures (February 2017)
An introduction to serverless architectures (February 2017)An introduction to serverless architectures (February 2017)
An introduction to serverless architectures (February 2017)
Julien SIMON
 
Building Serverless APIs on AWS
Building Serverless APIs on AWSBuilding Serverless APIs on AWS
Building Serverless APIs on AWS
Julien SIMON
 
Continuous Deployment with Amazon Web Services
Continuous Deployment with Amazon Web ServicesContinuous Deployment with Amazon Web Services
Continuous Deployment with Amazon Web Services
Julien SIMON
 
DevOps for the Enterprise: Virtual Office Hours
DevOps for the Enterprise: Virtual Office HoursDevOps for the Enterprise: Virtual Office Hours
DevOps for the Enterprise: Virtual Office Hours
Amazon Web Services
 
AWS Webcast - AWS OpsWorks Continuous Integration Demo
AWS Webcast - AWS OpsWorks Continuous Integration Demo  AWS Webcast - AWS OpsWorks Continuous Integration Demo
AWS Webcast - AWS OpsWorks Continuous Integration Demo
Amazon Web Services
 

What's hot (20)

Batch Processing with Amazon EC2 Container Service
Batch Processing with Amazon EC2 Container ServiceBatch Processing with Amazon EC2 Container Service
Batch Processing with Amazon EC2 Container Service
 
Architecting on The Cloud
Architecting on The CloudArchitecting on The Cloud
Architecting on The Cloud
 
AWS re:Invent 2016: Serverless Computing Patterns at Expedia (SVR306) )
AWS re:Invent 2016: Serverless Computing Patterns at Expedia (SVR306) )AWS re:Invent 2016: Serverless Computing Patterns at Expedia (SVR306) )
AWS re:Invent 2016: Serverless Computing Patterns at Expedia (SVR306) )
 
A 60-mn tour of AWS compute (March 2016)
A 60-mn tour of AWS compute (March 2016)A 60-mn tour of AWS compute (March 2016)
A 60-mn tour of AWS compute (March 2016)
 
Moving Viadeo to AWS (2015)
Moving Viadeo to AWS (2015)Moving Viadeo to AWS (2015)
Moving Viadeo to AWS (2015)
 
AWS & Cloud competition from Azure, openstack
AWS & Cloud competition from Azure, openstack AWS & Cloud competition from Azure, openstack
AWS & Cloud competition from Azure, openstack
 
An Overview of AWS IoT (November 2016)
An Overview of AWS IoT (November 2016)An Overview of AWS IoT (November 2016)
An Overview of AWS IoT (November 2016)
 
The Lost Tales of Platform Design (February 2017)
The Lost Tales of Platform Design (February 2017)The Lost Tales of Platform Design (February 2017)
The Lost Tales of Platform Design (February 2017)
 
AWS re:Invent 2016: Get Technically Inspired by Container-Powered Migrations ...
AWS re:Invent 2016: Get Technically Inspired by Container-Powered Migrations ...AWS re:Invent 2016: Get Technically Inspired by Container-Powered Migrations ...
AWS re:Invent 2016: Get Technically Inspired by Container-Powered Migrations ...
 
Building a PaaS with Docker and AWS
Building a PaaS with Docker and AWSBuilding a PaaS with Docker and AWS
Building a PaaS with Docker and AWS
 
A 60-minute tour of AWS Compute (November 2016)
A 60-minute tour of AWS Compute (November 2016)A 60-minute tour of AWS Compute (November 2016)
A 60-minute tour of AWS Compute (November 2016)
 
AWSKRUG 콘퍼런스 - re:Invent 신규 서비스 (박상욱) - 보안, 인프라 관련 서비스 소개
 AWSKRUG 콘퍼런스 - re:Invent 신규 서비스 (박상욱) - 보안, 인프라 관련 서비스 소개 AWSKRUG 콘퍼런스 - re:Invent 신규 서비스 (박상욱) - 보안, 인프라 관련 서비스 소개
AWSKRUG 콘퍼런스 - re:Invent 신규 서비스 (박상욱) - 보안, 인프라 관련 서비스 소개
 
"How to optimize the architecture of your platform" by Julien Simon
"How to optimize the architecture of your platform" by Julien Simon"How to optimize the architecture of your platform" by Julien Simon
"How to optimize the architecture of your platform" by Julien Simon
 
PaaSing Your Code Around
PaaSing Your Code AroundPaaSing Your Code Around
PaaSing Your Code Around
 
Deep Learning with AWS (November 2016)
Deep Learning with AWS (November 2016)Deep Learning with AWS (November 2016)
Deep Learning with AWS (November 2016)
 
An introduction to serverless architectures (February 2017)
An introduction to serverless architectures (February 2017)An introduction to serverless architectures (February 2017)
An introduction to serverless architectures (February 2017)
 
Building Serverless APIs on AWS
Building Serverless APIs on AWSBuilding Serverless APIs on AWS
Building Serverless APIs on AWS
 
Continuous Deployment with Amazon Web Services
Continuous Deployment with Amazon Web ServicesContinuous Deployment with Amazon Web Services
Continuous Deployment with Amazon Web Services
 
DevOps for the Enterprise: Virtual Office Hours
DevOps for the Enterprise: Virtual Office HoursDevOps for the Enterprise: Virtual Office Hours
DevOps for the Enterprise: Virtual Office Hours
 
AWS Webcast - AWS OpsWorks Continuous Integration Demo
AWS Webcast - AWS OpsWorks Continuous Integration Demo  AWS Webcast - AWS OpsWorks Continuous Integration Demo
AWS Webcast - AWS OpsWorks Continuous Integration Demo
 

Similar to SmartNews's journey into microservices

Microservices and Serverless for Mega Startups - DevOps IL Meetup
Microservices and Serverless for Mega Startups - DevOps IL MeetupMicroservices and Serverless for Mega Startups - DevOps IL Meetup
Microservices and Serverless for Mega Startups - DevOps IL Meetup
Boaz Ziniman
 
DevOps on AWS - Building Systems to Deliver Faster
DevOps on AWS - Building Systems to Deliver FasterDevOps on AWS - Building Systems to Deliver Faster
DevOps on AWS - Building Systems to Deliver Faster
Amazon Web Services
 
Easy Docker Deployments with Mesosphere DCOS on Azure
Easy Docker Deployments with Mesosphere DCOS on AzureEasy Docker Deployments with Mesosphere DCOS on Azure
Easy Docker Deployments with Mesosphere DCOS on Azure
Mesosphere Inc.
 
20140708 - Jeremy Edberg: How Netflix Delivers Software
20140708 - Jeremy Edberg: How Netflix Delivers Software20140708 - Jeremy Edberg: How Netflix Delivers Software
20140708 - Jeremy Edberg: How Netflix Delivers Software
DevOps Chicago
 
AWS re:Invent 2016: DevOps on AWS: Accelerating Software Delivery with the AW...
AWS re:Invent 2016: DevOps on AWS: Accelerating Software Delivery with the AW...AWS re:Invent 2016: DevOps on AWS: Accelerating Software Delivery with the AW...
AWS re:Invent 2016: DevOps on AWS: Accelerating Software Delivery with the AW...
Amazon Web Services
 
Serverless brewbox
Serverless   brewboxServerless   brewbox
Serverless brewbox
Lino Telera
 
DevOps in Amazon.com
DevOps in Amazon.com DevOps in Amazon.com
DevOps in Amazon.com
Amazon Web Services
 
Why real integration developers ride Camels
Why real integration developers ride CamelsWhy real integration developers ride Camels
Why real integration developers ride Camels
Christian Posta
 
Logic apps and PowerApps - Integrate across your APIs
Logic apps and PowerApps - Integrate across your APIsLogic apps and PowerApps - Integrate across your APIs
Logic apps and PowerApps - Integrate across your APIs
Sriram Hariharan
 
Deep Dive Into Elasticsearch: Establish A Powerful Log Analysis System With E...
Deep Dive Into Elasticsearch: Establish A Powerful Log Analysis System With E...Deep Dive Into Elasticsearch: Establish A Powerful Log Analysis System With E...
Deep Dive Into Elasticsearch: Establish A Powerful Log Analysis System With E...
Tyler Nguyen
 
First Look at Azure Logic Apps (BAUG)
First Look at Azure Logic Apps (BAUG)First Look at Azure Logic Apps (BAUG)
First Look at Azure Logic Apps (BAUG)
Daniel Toomey
 
From Monoliths to Services: Grafually paying your Technical Debt
From Monoliths to Services: Grafually paying your Technical DebtFrom Monoliths to Services: Grafually paying your Technical Debt
From Monoliths to Services: Grafually paying your Technical Debt
David Litvak Bruno
 
DevOps at Amazon: A Look at Our Tools and Processes
DevOps at Amazon: A Look at Our Tools and ProcessesDevOps at Amazon: A Look at Our Tools and Processes
DevOps at Amazon: A Look at Our Tools and Processes
Amazon Web Services
 
A Tale of Two Pizzas: Developer Tools at AWS - DevDay Los Angeles 2017
A Tale of Two Pizzas: Developer Tools at AWS - DevDay Los Angeles 2017A Tale of Two Pizzas: Developer Tools at AWS - DevDay Los Angeles 2017
A Tale of Two Pizzas: Developer Tools at AWS - DevDay Los Angeles 2017
Amazon Web Services
 
SRV312 DevOps on AWS: Building Systems to Deliver Faster
SRV312 DevOps on AWS: Building Systems to Deliver FasterSRV312 DevOps on AWS: Building Systems to Deliver Faster
SRV312 DevOps on AWS: Building Systems to Deliver Faster
Amazon Web Services
 
Optimus XPages: An Explosion of Techniques and Best Practices
Optimus XPages: An Explosion of Techniques and Best PracticesOptimus XPages: An Explosion of Techniques and Best Practices
Optimus XPages: An Explosion of Techniques and Best Practices
Teamstudio
 
Dev ops on aws deep dive on continuous delivery - Toronto
Dev ops on aws deep dive on continuous delivery - TorontoDev ops on aws deep dive on continuous delivery - Toronto
Dev ops on aws deep dive on continuous delivery - Toronto
Amazon Web Services
 
DevOps On AWS - Deep Dive on Continuous Delivery
DevOps On AWS - Deep Dive on Continuous DeliveryDevOps On AWS - Deep Dive on Continuous Delivery
DevOps On AWS - Deep Dive on Continuous Delivery
Mikhail Prudnikov
 
DevCon 2018 - 5 ways to use AWS with Alfresco
DevCon 2018 - 5 ways to use AWS with AlfrescoDevCon 2018 - 5 ways to use AWS with Alfresco
DevCon 2018 - 5 ways to use AWS with Alfresco
Gavin Cornwell
 
Netflix MSA and Pivotal
Netflix MSA and PivotalNetflix MSA and Pivotal
Netflix MSA and Pivotal
VMware Tanzu Korea
 

Similar to SmartNews's journey into microservices (20)

Microservices and Serverless for Mega Startups - DevOps IL Meetup
Microservices and Serverless for Mega Startups - DevOps IL MeetupMicroservices and Serverless for Mega Startups - DevOps IL Meetup
Microservices and Serverless for Mega Startups - DevOps IL Meetup
 
DevOps on AWS - Building Systems to Deliver Faster
DevOps on AWS - Building Systems to Deliver FasterDevOps on AWS - Building Systems to Deliver Faster
DevOps on AWS - Building Systems to Deliver Faster
 
Easy Docker Deployments with Mesosphere DCOS on Azure
Easy Docker Deployments with Mesosphere DCOS on AzureEasy Docker Deployments with Mesosphere DCOS on Azure
Easy Docker Deployments with Mesosphere DCOS on Azure
 
20140708 - Jeremy Edberg: How Netflix Delivers Software
20140708 - Jeremy Edberg: How Netflix Delivers Software20140708 - Jeremy Edberg: How Netflix Delivers Software
20140708 - Jeremy Edberg: How Netflix Delivers Software
 
AWS re:Invent 2016: DevOps on AWS: Accelerating Software Delivery with the AW...
AWS re:Invent 2016: DevOps on AWS: Accelerating Software Delivery with the AW...AWS re:Invent 2016: DevOps on AWS: Accelerating Software Delivery with the AW...
AWS re:Invent 2016: DevOps on AWS: Accelerating Software Delivery with the AW...
 
Serverless brewbox
Serverless   brewboxServerless   brewbox
Serverless brewbox
 
DevOps in Amazon.com
DevOps in Amazon.com DevOps in Amazon.com
DevOps in Amazon.com
 
Why real integration developers ride Camels
Why real integration developers ride CamelsWhy real integration developers ride Camels
Why real integration developers ride Camels
 
Logic apps and PowerApps - Integrate across your APIs
Logic apps and PowerApps - Integrate across your APIsLogic apps and PowerApps - Integrate across your APIs
Logic apps and PowerApps - Integrate across your APIs
 
Deep Dive Into Elasticsearch: Establish A Powerful Log Analysis System With E...
Deep Dive Into Elasticsearch: Establish A Powerful Log Analysis System With E...Deep Dive Into Elasticsearch: Establish A Powerful Log Analysis System With E...
Deep Dive Into Elasticsearch: Establish A Powerful Log Analysis System With E...
 
First Look at Azure Logic Apps (BAUG)
First Look at Azure Logic Apps (BAUG)First Look at Azure Logic Apps (BAUG)
First Look at Azure Logic Apps (BAUG)
 
From Monoliths to Services: Grafually paying your Technical Debt
From Monoliths to Services: Grafually paying your Technical DebtFrom Monoliths to Services: Grafually paying your Technical Debt
From Monoliths to Services: Grafually paying your Technical Debt
 
DevOps at Amazon: A Look at Our Tools and Processes
DevOps at Amazon: A Look at Our Tools and ProcessesDevOps at Amazon: A Look at Our Tools and Processes
DevOps at Amazon: A Look at Our Tools and Processes
 
A Tale of Two Pizzas: Developer Tools at AWS - DevDay Los Angeles 2017
A Tale of Two Pizzas: Developer Tools at AWS - DevDay Los Angeles 2017A Tale of Two Pizzas: Developer Tools at AWS - DevDay Los Angeles 2017
A Tale of Two Pizzas: Developer Tools at AWS - DevDay Los Angeles 2017
 
SRV312 DevOps on AWS: Building Systems to Deliver Faster
SRV312 DevOps on AWS: Building Systems to Deliver FasterSRV312 DevOps on AWS: Building Systems to Deliver Faster
SRV312 DevOps on AWS: Building Systems to Deliver Faster
 
Optimus XPages: An Explosion of Techniques and Best Practices
Optimus XPages: An Explosion of Techniques and Best PracticesOptimus XPages: An Explosion of Techniques and Best Practices
Optimus XPages: An Explosion of Techniques and Best Practices
 
Dev ops on aws deep dive on continuous delivery - Toronto
Dev ops on aws deep dive on continuous delivery - TorontoDev ops on aws deep dive on continuous delivery - Toronto
Dev ops on aws deep dive on continuous delivery - Toronto
 
DevOps On AWS - Deep Dive on Continuous Delivery
DevOps On AWS - Deep Dive on Continuous DeliveryDevOps On AWS - Deep Dive on Continuous Delivery
DevOps On AWS - Deep Dive on Continuous Delivery
 
DevCon 2018 - 5 ways to use AWS with Alfresco
DevCon 2018 - 5 ways to use AWS with AlfrescoDevCon 2018 - 5 ways to use AWS with Alfresco
DevCon 2018 - 5 ways to use AWS with Alfresco
 
Netflix MSA and Pivotal
Netflix MSA and PivotalNetflix MSA and Pivotal
Netflix MSA and Pivotal
 

More from SmartNews, Inc.

SmartNewsを支えるデータパイプラインとその運用
SmartNewsを支えるデータパイプラインとその運用SmartNewsを支えるデータパイプラインとその運用
SmartNewsを支えるデータパイプラインとその運用
SmartNews, Inc.
 
Spring で実現する SmartNews のニュース配信基盤
Spring で実現する SmartNews のニュース配信基盤Spring で実現する SmartNews のニュース配信基盤
Spring で実現する SmartNews のニュース配信基盤
SmartNews, Inc.
 
エンジニアからプロダクトマネージャーへ
エンジニアからプロダクトマネージャーへエンジニアからプロダクトマネージャーへ
エンジニアからプロダクトマネージャーへ
SmartNews, Inc.
 
SpringOne Platform 2016 報告会「A Lite Rx API for the JVM」/ 井口 貝 @ SmartNews, Inc.
SpringOne Platform 2016 報告会「A Lite Rx API for the JVM」/ 井口 貝 @ SmartNews, Inc.SpringOne Platform 2016 報告会「A Lite Rx API for the JVM」/ 井口 貝 @ SmartNews, Inc.
SpringOne Platform 2016 報告会「A Lite Rx API for the JVM」/ 井口 貝 @ SmartNews, Inc.
SmartNews, Inc.
 
SmartNewsのニュース配信を支えるサーバ技術 / Kazhiro Sera @ SmartNews,Inc. #jjug_ccc
SmartNewsのニュース配信を支えるサーバ技術 / Kazhiro Sera @ SmartNews,Inc. #jjug_cccSmartNewsのニュース配信を支えるサーバ技術 / Kazhiro Sera @ SmartNews,Inc. #jjug_ccc
SmartNewsのニュース配信を支えるサーバ技術 / Kazhiro Sera @ SmartNews,Inc. #jjug_ccc
SmartNews, Inc.
 
Stream Processing in SmartNews #jawsdays
Stream Processing in SmartNews #jawsdaysStream Processing in SmartNews #jawsdays
Stream Processing in SmartNews #jawsdays
SmartNews, Inc.
 
Building a Sustainable Data Platform on AWS
Building a Sustainable Data Platform on AWSBuilding a Sustainable Data Platform on AWS
Building a Sustainable Data Platform on AWS
SmartNews, Inc.
 
SmartNews TechNight Vol.5 : AD Data Engineering in practice: SmartNews Ads裏のデ...
SmartNews TechNight Vol.5 : AD Data Engineering in practice: SmartNews Ads裏のデ...SmartNews TechNight Vol.5 : AD Data Engineering in practice: SmartNews Ads裏のデ...
SmartNews TechNight Vol.5 : AD Data Engineering in practice: SmartNews Ads裏のデ...
SmartNews, Inc.
 
SmartNews TechNight Vol.5 : SmartNews Ads の配信最適化の仕組みはどうなってるの? (エンジニア / SmartN...
SmartNews TechNight Vol.5 : SmartNews Ads の配信最適化の仕組みはどうなってるの? (エンジニア / SmartN...SmartNews TechNight Vol.5 : SmartNews Ads の配信最適化の仕組みはどうなってるの? (エンジニア / SmartN...
SmartNews TechNight Vol.5 : SmartNews Ads の配信最適化の仕組みはどうなってるの? (エンジニア / SmartN...
SmartNews, Inc.
 
SmartNews TechNight Vol5 : SmartNews AdServer 解体新書 / ポストモーテム
SmartNews TechNight Vol5 : SmartNews AdServer 解体新書 / ポストモーテムSmartNews TechNight Vol5 : SmartNews AdServer 解体新書 / ポストモーテム
SmartNews TechNight Vol5 : SmartNews AdServer 解体新書 / ポストモーテム
SmartNews, Inc.
 
SmartNews TechNight vol5 SmartNews Ads大図解
SmartNews TechNight vol5 SmartNews Ads大図解SmartNews TechNight vol5 SmartNews Ads大図解
SmartNews TechNight vol5 SmartNews Ads大図解
SmartNews, Inc.
 
NLP in SmartNews
NLP in SmartNewsNLP in SmartNews
NLP in SmartNews
SmartNews, Inc.
 
Strem処理(Spark Streaming + Kinesis)とOffline処理(Hive)の統合
Strem処理(Spark Streaming + Kinesis)とOffline処理(Hive)の統合Strem処理(Spark Streaming + Kinesis)とOffline処理(Hive)の統合
Strem処理(Spark Streaming + Kinesis)とOffline処理(Hive)の統合
SmartNews, Inc.
 
SmartNews の Webmining を支えるプラットフォーム
SmartNews の Webmining を支えるプラットフォームSmartNews の Webmining を支えるプラットフォーム
SmartNews の Webmining を支えるプラットフォーム
SmartNews, Inc.
 
AWS meetup「Apache Spark on EMR」
AWS meetup「Apache Spark on EMR」AWS meetup「Apache Spark on EMR」
AWS meetup「Apache Spark on EMR」
SmartNews, Inc.
 
Smartnews Product Manager Night
Smartnews Product Manager NightSmartnews Product Manager Night
Smartnews Product Manager Night
SmartNews, Inc.
 
SmartNews Ads System - AWS Summit Tokyo 2015
SmartNews Ads System - AWS Summit Tokyo 2015SmartNews Ads System - AWS Summit Tokyo 2015
SmartNews Ads System - AWS Summit Tokyo 2015
SmartNews, Inc.
 
インフラ専任エンジニアが一人もいないSmartNewsにおけるクラウド活用法
インフラ専任エンジニアが一人もいないSmartNewsにおけるクラウド活用法インフラ専任エンジニアが一人もいないSmartNewsにおけるクラウド活用法
インフラ専任エンジニアが一人もいないSmartNewsにおけるクラウド活用法
SmartNews, Inc.
 

More from SmartNews, Inc. (18)

SmartNewsを支えるデータパイプラインとその運用
SmartNewsを支えるデータパイプラインとその運用SmartNewsを支えるデータパイプラインとその運用
SmartNewsを支えるデータパイプラインとその運用
 
Spring で実現する SmartNews のニュース配信基盤
Spring で実現する SmartNews のニュース配信基盤Spring で実現する SmartNews のニュース配信基盤
Spring で実現する SmartNews のニュース配信基盤
 
エンジニアからプロダクトマネージャーへ
エンジニアからプロダクトマネージャーへエンジニアからプロダクトマネージャーへ
エンジニアからプロダクトマネージャーへ
 
SpringOne Platform 2016 報告会「A Lite Rx API for the JVM」/ 井口 貝 @ SmartNews, Inc.
SpringOne Platform 2016 報告会「A Lite Rx API for the JVM」/ 井口 貝 @ SmartNews, Inc.SpringOne Platform 2016 報告会「A Lite Rx API for the JVM」/ 井口 貝 @ SmartNews, Inc.
SpringOne Platform 2016 報告会「A Lite Rx API for the JVM」/ 井口 貝 @ SmartNews, Inc.
 
SmartNewsのニュース配信を支えるサーバ技術 / Kazhiro Sera @ SmartNews,Inc. #jjug_ccc
SmartNewsのニュース配信を支えるサーバ技術 / Kazhiro Sera @ SmartNews,Inc. #jjug_cccSmartNewsのニュース配信を支えるサーバ技術 / Kazhiro Sera @ SmartNews,Inc. #jjug_ccc
SmartNewsのニュース配信を支えるサーバ技術 / Kazhiro Sera @ SmartNews,Inc. #jjug_ccc
 
Stream Processing in SmartNews #jawsdays
Stream Processing in SmartNews #jawsdaysStream Processing in SmartNews #jawsdays
Stream Processing in SmartNews #jawsdays
 
Building a Sustainable Data Platform on AWS
Building a Sustainable Data Platform on AWSBuilding a Sustainable Data Platform on AWS
Building a Sustainable Data Platform on AWS
 
SmartNews TechNight Vol.5 : AD Data Engineering in practice: SmartNews Ads裏のデ...
SmartNews TechNight Vol.5 : AD Data Engineering in practice: SmartNews Ads裏のデ...SmartNews TechNight Vol.5 : AD Data Engineering in practice: SmartNews Ads裏のデ...
SmartNews TechNight Vol.5 : AD Data Engineering in practice: SmartNews Ads裏のデ...
 
SmartNews TechNight Vol.5 : SmartNews Ads の配信最適化の仕組みはどうなってるの? (エンジニア / SmartN...
SmartNews TechNight Vol.5 : SmartNews Ads の配信最適化の仕組みはどうなってるの? (エンジニア / SmartN...SmartNews TechNight Vol.5 : SmartNews Ads の配信最適化の仕組みはどうなってるの? (エンジニア / SmartN...
SmartNews TechNight Vol.5 : SmartNews Ads の配信最適化の仕組みはどうなってるの? (エンジニア / SmartN...
 
SmartNews TechNight Vol5 : SmartNews AdServer 解体新書 / ポストモーテム
SmartNews TechNight Vol5 : SmartNews AdServer 解体新書 / ポストモーテムSmartNews TechNight Vol5 : SmartNews AdServer 解体新書 / ポストモーテム
SmartNews TechNight Vol5 : SmartNews AdServer 解体新書 / ポストモーテム
 
SmartNews TechNight vol5 SmartNews Ads大図解
SmartNews TechNight vol5 SmartNews Ads大図解SmartNews TechNight vol5 SmartNews Ads大図解
SmartNews TechNight vol5 SmartNews Ads大図解
 
NLP in SmartNews
NLP in SmartNewsNLP in SmartNews
NLP in SmartNews
 
Strem処理(Spark Streaming + Kinesis)とOffline処理(Hive)の統合
Strem処理(Spark Streaming + Kinesis)とOffline処理(Hive)の統合Strem処理(Spark Streaming + Kinesis)とOffline処理(Hive)の統合
Strem処理(Spark Streaming + Kinesis)とOffline処理(Hive)の統合
 
SmartNews の Webmining を支えるプラットフォーム
SmartNews の Webmining を支えるプラットフォームSmartNews の Webmining を支えるプラットフォーム
SmartNews の Webmining を支えるプラットフォーム
 
AWS meetup「Apache Spark on EMR」
AWS meetup「Apache Spark on EMR」AWS meetup「Apache Spark on EMR」
AWS meetup「Apache Spark on EMR」
 
Smartnews Product Manager Night
Smartnews Product Manager NightSmartnews Product Manager Night
Smartnews Product Manager Night
 
SmartNews Ads System - AWS Summit Tokyo 2015
SmartNews Ads System - AWS Summit Tokyo 2015SmartNews Ads System - AWS Summit Tokyo 2015
SmartNews Ads System - AWS Summit Tokyo 2015
 
インフラ専任エンジニアが一人もいないSmartNewsにおけるクラウド活用法
インフラ専任エンジニアが一人もいないSmartNewsにおけるクラウド活用法インフラ専任エンジニアが一人もいないSmartNewsにおけるクラウド活用法
インフラ専任エンジニアが一人もいないSmartNewsにおけるクラウド活用法
 

Recently uploaded

Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
Safe Software
 
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
Priyanka Aash
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
SynapseIndia
 
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...
Bert Blevins
 
Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
Torry Harris
 
ARTIFICIAL INTELLIGENCE (AI) IN MUSIC.pdf
ARTIFICIAL INTELLIGENCE (AI) IN MUSIC.pdfARTIFICIAL INTELLIGENCE (AI) IN MUSIC.pdf
ARTIFICIAL INTELLIGENCE (AI) IN MUSIC.pdf
Inglês no Mundo Digital
 
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptxRPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
SynapseIndia
 
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfBT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
Neo4j
 
IPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite SolutionIPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite Solution
IPLOOK Networks
 
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and OllamaTirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Zilliz
 
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
Edge AI and Vision Alliance
 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
maigasapphire
 
Implementations of Fused Deposition Modeling in real world
Implementations of Fused Deposition Modeling  in real worldImplementations of Fused Deposition Modeling  in real world
Implementations of Fused Deposition Modeling in real world
Emerging Tech
 
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
Kief Morris
 
How to build a generative AI solution A step-by-step guide (2).pdf
How to build a generative AI solution A step-by-step guide (2).pdfHow to build a generative AI solution A step-by-step guide (2).pdf
How to build a generative AI solution A step-by-step guide (2).pdf
ChristopherTHyatt
 
The Role of Technology in Payroll Statutory Compliance (1).pdf
The Role of Technology in Payroll Statutory Compliance (1).pdfThe Role of Technology in Payroll Statutory Compliance (1).pdf
The Role of Technology in Payroll Statutory Compliance (1).pdf
paysquare consultancy
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
BrainSell Technologies
 
Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
Zilliz
 
How Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdfHow Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdf
HackersList
 
Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024
aakash malhotra
 

Recently uploaded (20)

Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
 
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
 
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...
 
Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
 
ARTIFICIAL INTELLIGENCE (AI) IN MUSIC.pdf
ARTIFICIAL INTELLIGENCE (AI) IN MUSIC.pdfARTIFICIAL INTELLIGENCE (AI) IN MUSIC.pdf
ARTIFICIAL INTELLIGENCE (AI) IN MUSIC.pdf
 
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptxRPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
 
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfBT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
 
IPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite SolutionIPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite Solution
 
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and OllamaTirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
 
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
 
Implementations of Fused Deposition Modeling in real world
Implementations of Fused Deposition Modeling  in real worldImplementations of Fused Deposition Modeling  in real world
Implementations of Fused Deposition Modeling in real world
 
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
 
How to build a generative AI solution A step-by-step guide (2).pdf
How to build a generative AI solution A step-by-step guide (2).pdfHow to build a generative AI solution A step-by-step guide (2).pdf
How to build a generative AI solution A step-by-step guide (2).pdf
 
The Role of Technology in Payroll Statutory Compliance (1).pdf
The Role of Technology in Payroll Statutory Compliance (1).pdfThe Role of Technology in Payroll Statutory Compliance (1).pdf
The Role of Technology in Payroll Statutory Compliance (1).pdf
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
 
Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
 
How Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdfHow Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdf
 
Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024
 

SmartNews's journey into microservices