SlideShare a Scribd company logo
MECAB IN DOCKERACTION!!!
USING DOCKERACTION OF IBM
CLOUD FUNCTIONS ( OPENWHISK )
2017.10.19 Bluemix User Group Tsukiji,Japan#1
ABOUT MYSELF
 (KUNITO Atsunori)
 Atsunori_kunito@niandc.co.jp
 I’m a Java programmer at NI and C Soft.inc.
 We are Nippon Information and Communication
Corporation Group.
1.MOTIVATION
MY PRIVATE PROJECT
 I was creating a personal tool which monitoring google
search ranking everyday. It works on Bluemix -
OpenWhisk.
 Next , I’d like to extract frequently appeared keyword
from the top 50 web pages.
 I wish I used Watson - Natural Language
Understanding(NLU). But the NLU doesn’t support
Japanese!!!
SOLUTION – WHAT IS MECAB?
 MeCab: Yet Another Part-of-Speech and Morphological
Analyzer(http://taku910.github.io/mecab/)
 MeCab is OSS which is created by Taku Kudo.
 Japanese has no separator(space) between word each
other.
 So I can count frequency of the words with the MeCab.
DOCKER ACTION
 MeCab is written in C++. How do I run this tool on
Cloud?
Docker Action !!!
wsk action update
wsk action invoke
/init Init process
Run MeCab/run
2.DOCKERACTION STEP BY STEP
Local
Bluemix - OpenWhisk
OVERVIEW
MeCab
MeCab
MeCabMeCab
exec
exec.zip
exec
STEP 1) Make Dockerfile
STEP 2) Push Docker image
STEP 3) Write entry point
STEP 4) Deploy exec
invoke
Docker Pull
STEP 1) MAKE DOCKERFILE
Key Learnings:
 Use ‘openwhisk/dockerskeleton’ as base image.
 Inject MeCab , dependencys and other tools(if needed) to
your docker image.
 Your docker image will be uploaded to
DockerHub(https://hub.docker.com/) and will be
PUBLIC. So You SHOULD NOT include private
informations. Be aware to password , API-key etc…
 Lunch actionproxy.py. It’s contained in
‘openwhisk/dockerskeleton’ .
DOCKERFILE
FROM openwhisk/dockerskeleton
RUN apk add --no-cache gcc libc-dev g++
RUN apk add --no-cache --virtual .build-deps git make ¥
&& apk add --no-cache libxml2-dev libxslt-dev
# install mecab
RUN cd /tmp && ¥
git clone https://github.com/taku910/mecab.git
RUN cd /tmp/mecab/mecab && ¥
./configure --enable-utf8-only && ¥
make && ¥
make install
RUN echo 'export LD_LIBRARY_PATH="/usr/local/lib:$LD_LIBRARY_PATH"' >> ~/.bashrc && ¥
. ~/.bashrc && ¥
cd /tmp/mecab/mecab-ipadic && ¥
./configure --with-charset=utf8 && ¥
make && ¥
make install
RUN pip install --no-cache-dir lxml
RUN pip install --no-cache-dir requests
RUN pip install --no-cache-dir beautifulsoup4
RUN apk del .build-deps
CMD ["/bin/bash", "-c", "cd actionProxy && python -u actionproxy.py"]
STEP 2) PUSH YOUR DOCKER IMAGE AND
CREATE ACTION
 Create an account in DockerHub(if you didn’t have it).
docker build -t openwhiskdatest/mecabda .
docker login
docker push openwhiskdatest/mecabda
bx wsk action create wskMeCabDH --docker openwhiskdatest/mecabda
STEP 3) WRITE AN ENTRY POINT OF YOUR
ACTION
 I wrote web crawler with Beautiful Soup and lxml in
python.
 Execute the MeCab.
 Make MeCab’s outputs into single-lined-JSON format. I
wrote this process in python too.
 Finally , I wrote BASH script which organize those
programs described above. This script must be named
‘exec’.
STEP 4) CREATE ZIP FILE AND DEPLOY IT
 Package programs described in step 3 into a zip file.
 Upload it to your action.
 Actionproxy.py receive the zip file. Then It extracts and
deploys contained programs.
bx wsk action update wskMeCabDH exec.zip --docker openwhiskdatest/mecabda
NOW WE CAN INVOKE DOCKER ACTION !!!
bx wsk action invoke wskMeCabDH --blocking --result -p "url" https://bmxug.connpass.com/event/66961/
IT WORKS ON MY PROJECT
 Google search word is ‘OpenWhisk’.
 Crawling top 50 web pages.
 Extract frequently appeared keyword.
 It’s represented with wordcloud2 in R.
 Can you read ‘Lambda’?
It shows many sites about Openwhisk
compare with Lambda , I guess.
3.CONCLUSION
CONCLUSION
Key Learnings:
 Docker Action is great way to execute some binaries on
Bluemix.
 Every Openwhisk actions are implemented on Docker
Action. In other words , Action Runtime is a pre-defined
Docker Action for specific programming language.
USECASE OF DOCKER ACTION
 Execute some binaries or shell scripts on Bluemix, of
course.
 You can create your runtime which supports your
favorite programing language.
 If you wanted to use some dependencies which are
unsupported in runtime , You can create a dockerfile to
inject those dependencies.
REFERENCES
 https://hub.docker.com/r/openwhisk/spellcheck/
 https://www.ibm.com/blogs/bluemix/2017/01/dock
er-bluemix-openwhisk/
 http://jamesthom.as/blog/2017/01/16/openwhisk-
docker-actions/
 DockerHub
 https://hub.docker.com/r/openwhisk/dockerskeleton/
 https://www.niandc.co.jp/sol/tech/date20170626_6
98.php
Thank you !!!

More Related Content

What's hot

Workshop - Golang language
Workshop - Golang languageWorkshop - Golang language
Workshop - Golang language
Vincent Composieux
 
OpenShift State of the Union, brought to you by JBoss
OpenShift State of the Union, brought to you by JBossOpenShift State of the Union, brought to you by JBoss
OpenShift State of the Union, brought to you by JBoss
Eric D. Schabell
 
Introduction to Express and Grunt
Introduction to Express and GruntIntroduction to Express and Grunt
Introduction to Express and Grunt
Peter deHaan
 
Plone and docker
Plone and dockerPlone and docker
Plone and docker
Alin Voinea
 
Bringing Javascript to the Desktop with Electron
Bringing Javascript to the Desktop with ElectronBringing Javascript to the Desktop with Electron
Bringing Javascript to the Desktop with Electron
Nir Noy
 
Development with Vagrant
Development with VagrantDevelopment with Vagrant
Development with Vagrant
John Coggeshall
 
Docker & Kubernetes
Docker & KubernetesDocker & Kubernetes
Docker & Kubernetes
Troy Harvey
 
Docker and plone
Docker and ploneDocker and plone
Docker and plone
Alin Voinea
 
Grunt - The JavaScript Task Runner
Grunt - The JavaScript Task RunnerGrunt - The JavaScript Task Runner
Grunt - The JavaScript Task Runner
Mohammed Arif
 
Docker, developers take power
Docker, developers take powerDocker, developers take power
Docker, developers take power
Enalean
 
Docker: from zero to nonzero
Docker: from zero to nonzeroDocker: from zero to nonzero
Docker: from zero to nonzero
Francesco Lo Franco
 
2016 may-31 dockercon2016–cool-hackssubmission
2016 may-31 dockercon2016–cool-hackssubmission2016 may-31 dockercon2016–cool-hackssubmission
2016 may-31 dockercon2016–cool-hackssubmission
Michael Bright
 
Production Ready Javascript With Grunt
Production Ready Javascript With GruntProduction Ready Javascript With Grunt
Production Ready Javascript With Grunt
XB Software, Ltd.
 
What grunt?
What grunt?What grunt?
What grunt?
Lucio Martinez
 
CoffeeScript in 5mins
CoffeeScript in 5minsCoffeeScript in 5mins
CoffeeScript in 5minsMasakuni Kato
 
Ratpack On the Docks
Ratpack On the DocksRatpack On the Docks
Ratpack On the Docks
GR8Conf
 
Groovy Powered Clean Code
Groovy Powered Clean CodeGroovy Powered Clean Code
Groovy Powered Clean Code
GR8Conf
 
Bower & Grunt - A practical workflow
Bower & Grunt - A practical workflowBower & Grunt - A practical workflow
Bower & Grunt - A practical workflow
Riccardo Coppola
 
Grunt JS - Getting Started With Grunt
Grunt JS - Getting Started With GruntGrunt JS - Getting Started With Grunt
Grunt JS - Getting Started With Grunt
Douglas Reynolds
 
Docker & FieldAware
Docker & FieldAwareDocker & FieldAware
Docker & FieldAwareJakub Jarosz
 

What's hot (20)

Workshop - Golang language
Workshop - Golang languageWorkshop - Golang language
Workshop - Golang language
 
OpenShift State of the Union, brought to you by JBoss
OpenShift State of the Union, brought to you by JBossOpenShift State of the Union, brought to you by JBoss
OpenShift State of the Union, brought to you by JBoss
 
Introduction to Express and Grunt
Introduction to Express and GruntIntroduction to Express and Grunt
Introduction to Express and Grunt
 
Plone and docker
Plone and dockerPlone and docker
Plone and docker
 
Bringing Javascript to the Desktop with Electron
Bringing Javascript to the Desktop with ElectronBringing Javascript to the Desktop with Electron
Bringing Javascript to the Desktop with Electron
 
Development with Vagrant
Development with VagrantDevelopment with Vagrant
Development with Vagrant
 
Docker & Kubernetes
Docker & KubernetesDocker & Kubernetes
Docker & Kubernetes
 
Docker and plone
Docker and ploneDocker and plone
Docker and plone
 
Grunt - The JavaScript Task Runner
Grunt - The JavaScript Task RunnerGrunt - The JavaScript Task Runner
Grunt - The JavaScript Task Runner
 
Docker, developers take power
Docker, developers take powerDocker, developers take power
Docker, developers take power
 
Docker: from zero to nonzero
Docker: from zero to nonzeroDocker: from zero to nonzero
Docker: from zero to nonzero
 
2016 may-31 dockercon2016–cool-hackssubmission
2016 may-31 dockercon2016–cool-hackssubmission2016 may-31 dockercon2016–cool-hackssubmission
2016 may-31 dockercon2016–cool-hackssubmission
 
Production Ready Javascript With Grunt
Production Ready Javascript With GruntProduction Ready Javascript With Grunt
Production Ready Javascript With Grunt
 
What grunt?
What grunt?What grunt?
What grunt?
 
CoffeeScript in 5mins
CoffeeScript in 5minsCoffeeScript in 5mins
CoffeeScript in 5mins
 
Ratpack On the Docks
Ratpack On the DocksRatpack On the Docks
Ratpack On the Docks
 
Groovy Powered Clean Code
Groovy Powered Clean CodeGroovy Powered Clean Code
Groovy Powered Clean Code
 
Bower & Grunt - A practical workflow
Bower & Grunt - A practical workflowBower & Grunt - A practical workflow
Bower & Grunt - A practical workflow
 
Grunt JS - Getting Started With Grunt
Grunt JS - Getting Started With GruntGrunt JS - Getting Started With Grunt
Grunt JS - Getting Started With Grunt
 
Docker & FieldAware
Docker & FieldAwareDocker & FieldAware
Docker & FieldAware
 

Similar to MeCab in docker action(OpenWhisk)

IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...
IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...
IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...
Eric Smalling
 
Microservices, la risposta che (forse) cercavi!
Microservices, la risposta che (forse) cercavi!Microservices, la risposta che (forse) cercavi!
Microservices, la risposta che (forse) cercavi!
Commit University
 
Dockerfiles building docker images automatically v (workdir, env, add, and ...
Dockerfiles   building docker images automatically v (workdir, env, add, and ...Dockerfiles   building docker images automatically v (workdir, env, add, and ...
Dockerfiles building docker images automatically v (workdir, env, add, and ...
ansonjonel
 
Containers, Docker, and Microservices: the Terrific Trio
Containers, Docker, and Microservices: the Terrific TrioContainers, Docker, and Microservices: the Terrific Trio
Containers, Docker, and Microservices: the Terrific Trio
Jérôme Petazzoni
 
The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCl...
The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCl...The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCl...
The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCl...
Docker, Inc.
 
Dockercon EU 2014
Dockercon EU 2014Dockercon EU 2014
Dockercon EU 2014
Rafe Colton
 
O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...
O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...
O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...
Ambassador Labs
 
Fewd week4 slides
Fewd week4 slidesFewd week4 slides
Fewd week4 slides
William Myers
 
Extending DevOps to Big Data Applications with Kubernetes
Extending DevOps to Big Data Applications with KubernetesExtending DevOps to Big Data Applications with Kubernetes
Extending DevOps to Big Data Applications with Kubernetes
Nicola Ferraro
 
Docker intro
Docker introDocker intro
Docker intro
Frei Zhang
 
Behaviour Driven Development con Behat & Drupal
Behaviour Driven Development con Behat & DrupalBehaviour Driven Development con Behat & Drupal
Behaviour Driven Development con Behat & Drupal
sparkfabrik
 
Behaviour Driven Development con Behat & Drupal
Behaviour Driven Development con Behat & DrupalBehaviour Driven Development con Behat & Drupal
Behaviour Driven Development con Behat & Drupal
DrupalDay
 
The Self-Service Developer - GOTOCon CPH
The Self-Service Developer - GOTOCon CPHThe Self-Service Developer - GOTOCon CPH
The Self-Service Developer - GOTOCon CPH
Laszlo Fogas
 
Introduction to Groovy Monkey
Introduction to Groovy MonkeyIntroduction to Groovy Monkey
Introduction to Groovy Monkey
jervin
 
State ofappdevelopment
State ofappdevelopmentState ofappdevelopment
State ofappdevelopment
gillygize
 
Docker module 1
Docker module 1Docker module 1
Docker module 1
Liang Bo
 
Docker puebla bday #4 celebration
Docker puebla bday #4 celebrationDocker puebla bday #4 celebration
Docker puebla bday #4 celebration
Ramon Morales
 
Docker Basic to Advance
Docker Basic to AdvanceDocker Basic to Advance
Docker Basic to Advance
Paras Jain
 
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...
Codemotion
 
CoffeeScript: A beginner's presentation for beginners copy
CoffeeScript: A beginner's presentation for beginners copyCoffeeScript: A beginner's presentation for beginners copy
CoffeeScript: A beginner's presentation for beginners copy
Patrick Devins
 

Similar to MeCab in docker action(OpenWhisk) (20)

IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...
IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...
IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...
 
Microservices, la risposta che (forse) cercavi!
Microservices, la risposta che (forse) cercavi!Microservices, la risposta che (forse) cercavi!
Microservices, la risposta che (forse) cercavi!
 
Dockerfiles building docker images automatically v (workdir, env, add, and ...
Dockerfiles   building docker images automatically v (workdir, env, add, and ...Dockerfiles   building docker images automatically v (workdir, env, add, and ...
Dockerfiles building docker images automatically v (workdir, env, add, and ...
 
Containers, Docker, and Microservices: the Terrific Trio
Containers, Docker, and Microservices: the Terrific TrioContainers, Docker, and Microservices: the Terrific Trio
Containers, Docker, and Microservices: the Terrific Trio
 
The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCl...
The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCl...The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCl...
The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCl...
 
Dockercon EU 2014
Dockercon EU 2014Dockercon EU 2014
Dockercon EU 2014
 
O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...
O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...
O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...
 
Fewd week4 slides
Fewd week4 slidesFewd week4 slides
Fewd week4 slides
 
Extending DevOps to Big Data Applications with Kubernetes
Extending DevOps to Big Data Applications with KubernetesExtending DevOps to Big Data Applications with Kubernetes
Extending DevOps to Big Data Applications with Kubernetes
 
Docker intro
Docker introDocker intro
Docker intro
 
Behaviour Driven Development con Behat & Drupal
Behaviour Driven Development con Behat & DrupalBehaviour Driven Development con Behat & Drupal
Behaviour Driven Development con Behat & Drupal
 
Behaviour Driven Development con Behat & Drupal
Behaviour Driven Development con Behat & DrupalBehaviour Driven Development con Behat & Drupal
Behaviour Driven Development con Behat & Drupal
 
The Self-Service Developer - GOTOCon CPH
The Self-Service Developer - GOTOCon CPHThe Self-Service Developer - GOTOCon CPH
The Self-Service Developer - GOTOCon CPH
 
Introduction to Groovy Monkey
Introduction to Groovy MonkeyIntroduction to Groovy Monkey
Introduction to Groovy Monkey
 
State ofappdevelopment
State ofappdevelopmentState ofappdevelopment
State ofappdevelopment
 
Docker module 1
Docker module 1Docker module 1
Docker module 1
 
Docker puebla bday #4 celebration
Docker puebla bday #4 celebrationDocker puebla bday #4 celebration
Docker puebla bday #4 celebration
 
Docker Basic to Advance
Docker Basic to AdvanceDocker Basic to Advance
Docker Basic to Advance
 
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...
 
CoffeeScript: A beginner's presentation for beginners copy
CoffeeScript: A beginner's presentation for beginners copyCoffeeScript: A beginner's presentation for beginners copy
CoffeeScript: A beginner's presentation for beginners copy
 

Recently uploaded

To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
Paul Groth
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
91mobiles
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
Thijs Feryn
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
Elena Simperl
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Product School
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
Guy Korland
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Tobias Schneck
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdfFIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
Prayukth K V
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
Alison B. Lowndes
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
DianaGray10
 
Generating a custom Ruby SDK for your web service or Rails API using Smithy
Generating a custom Ruby SDK for your web service or Rails API using SmithyGenerating a custom Ruby SDK for your web service or Rails API using Smithy
Generating a custom Ruby SDK for your web service or Rails API using Smithy
g2nightmarescribd
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
Product School
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Ramesh Iyer
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
OnBoard
 

Recently uploaded (20)

To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdfFIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
 
Generating a custom Ruby SDK for your web service or Rails API using Smithy
Generating a custom Ruby SDK for your web service or Rails API using SmithyGenerating a custom Ruby SDK for your web service or Rails API using Smithy
Generating a custom Ruby SDK for your web service or Rails API using Smithy
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
 

MeCab in docker action(OpenWhisk)

  • 1. MECAB IN DOCKERACTION!!! USING DOCKERACTION OF IBM CLOUD FUNCTIONS ( OPENWHISK ) 2017.10.19 Bluemix User Group Tsukiji,Japan#1
  • 2. ABOUT MYSELF  (KUNITO Atsunori)  Atsunori_kunito@niandc.co.jp  I’m a Java programmer at NI and C Soft.inc.  We are Nippon Information and Communication Corporation Group.
  • 4. MY PRIVATE PROJECT  I was creating a personal tool which monitoring google search ranking everyday. It works on Bluemix - OpenWhisk.  Next , I’d like to extract frequently appeared keyword from the top 50 web pages.  I wish I used Watson - Natural Language Understanding(NLU). But the NLU doesn’t support Japanese!!!
  • 5. SOLUTION – WHAT IS MECAB?  MeCab: Yet Another Part-of-Speech and Morphological Analyzer(http://taku910.github.io/mecab/)  MeCab is OSS which is created by Taku Kudo.  Japanese has no separator(space) between word each other.  So I can count frequency of the words with the MeCab.
  • 6. DOCKER ACTION  MeCab is written in C++. How do I run this tool on Cloud? Docker Action !!! wsk action update wsk action invoke /init Init process Run MeCab/run
  • 8. Local Bluemix - OpenWhisk OVERVIEW MeCab MeCab MeCabMeCab exec exec.zip exec STEP 1) Make Dockerfile STEP 2) Push Docker image STEP 3) Write entry point STEP 4) Deploy exec invoke Docker Pull
  • 9. STEP 1) MAKE DOCKERFILE Key Learnings:  Use ‘openwhisk/dockerskeleton’ as base image.  Inject MeCab , dependencys and other tools(if needed) to your docker image.  Your docker image will be uploaded to DockerHub(https://hub.docker.com/) and will be PUBLIC. So You SHOULD NOT include private informations. Be aware to password , API-key etc…  Lunch actionproxy.py. It’s contained in ‘openwhisk/dockerskeleton’ .
  • 10. DOCKERFILE FROM openwhisk/dockerskeleton RUN apk add --no-cache gcc libc-dev g++ RUN apk add --no-cache --virtual .build-deps git make ¥ && apk add --no-cache libxml2-dev libxslt-dev # install mecab RUN cd /tmp && ¥ git clone https://github.com/taku910/mecab.git RUN cd /tmp/mecab/mecab && ¥ ./configure --enable-utf8-only && ¥ make && ¥ make install RUN echo 'export LD_LIBRARY_PATH="/usr/local/lib:$LD_LIBRARY_PATH"' >> ~/.bashrc && ¥ . ~/.bashrc && ¥ cd /tmp/mecab/mecab-ipadic && ¥ ./configure --with-charset=utf8 && ¥ make && ¥ make install RUN pip install --no-cache-dir lxml RUN pip install --no-cache-dir requests RUN pip install --no-cache-dir beautifulsoup4 RUN apk del .build-deps CMD ["/bin/bash", "-c", "cd actionProxy && python -u actionproxy.py"]
  • 11. STEP 2) PUSH YOUR DOCKER IMAGE AND CREATE ACTION  Create an account in DockerHub(if you didn’t have it). docker build -t openwhiskdatest/mecabda . docker login docker push openwhiskdatest/mecabda bx wsk action create wskMeCabDH --docker openwhiskdatest/mecabda
  • 12. STEP 3) WRITE AN ENTRY POINT OF YOUR ACTION  I wrote web crawler with Beautiful Soup and lxml in python.  Execute the MeCab.  Make MeCab’s outputs into single-lined-JSON format. I wrote this process in python too.  Finally , I wrote BASH script which organize those programs described above. This script must be named ‘exec’.
  • 13. STEP 4) CREATE ZIP FILE AND DEPLOY IT  Package programs described in step 3 into a zip file.  Upload it to your action.  Actionproxy.py receive the zip file. Then It extracts and deploys contained programs. bx wsk action update wskMeCabDH exec.zip --docker openwhiskdatest/mecabda
  • 14. NOW WE CAN INVOKE DOCKER ACTION !!! bx wsk action invoke wskMeCabDH --blocking --result -p "url" https://bmxug.connpass.com/event/66961/
  • 15. IT WORKS ON MY PROJECT  Google search word is ‘OpenWhisk’.  Crawling top 50 web pages.  Extract frequently appeared keyword.  It’s represented with wordcloud2 in R.  Can you read ‘Lambda’? It shows many sites about Openwhisk compare with Lambda , I guess.
  • 17. CONCLUSION Key Learnings:  Docker Action is great way to execute some binaries on Bluemix.  Every Openwhisk actions are implemented on Docker Action. In other words , Action Runtime is a pre-defined Docker Action for specific programming language.
  • 18. USECASE OF DOCKER ACTION  Execute some binaries or shell scripts on Bluemix, of course.  You can create your runtime which supports your favorite programing language.  If you wanted to use some dependencies which are unsupported in runtime , You can create a dockerfile to inject those dependencies.
  • 19. REFERENCES  https://hub.docker.com/r/openwhisk/spellcheck/  https://www.ibm.com/blogs/bluemix/2017/01/dock er-bluemix-openwhisk/  http://jamesthom.as/blog/2017/01/16/openwhisk- docker-actions/  DockerHub  https://hub.docker.com/r/openwhisk/dockerskeleton/  https://www.niandc.co.jp/sol/tech/date20170626_6 98.php