SlideShare a Scribd company logo
GUI automation with Sikuli
Ctibor Škuta




               ELAG 2010, Helsinki, Lightning talk
Our problem
• 400 new subject headings picked from 653
  MARC field (keyword) of our bibliographic
  records
• … but then we had to change the 653 field
  with given terms to 650 field (authority
  subject heading)
• Which means for me to fill the same form
  400 TIMES only with one different term
                 ELAG 2010, Helsinki, Lightning talk
Other solution
• Maybe it could be done automatically?
• Aleph administrator: „That‘s not possible.“




                  ELAG 2010, Helsinki, Lightning talk
Totally different solution
• … but maybe we don‘t have to automate the
  system itself
• Maybe we can automate the actual human
  work (typing, clicking)




                ELAG 2010, Helsinki, Lightning talk
Our solution = Sikuli

• Picture-driven programming tool

• Recognizes patterns on graphical output

• Multiplatform (code in Java)

• Using Jython (Python implementation in Java)

                 ELAG 2010, Helsinki, Lightning talk
Simple example
terms = ['"moby is from belgium"', '"moby is not from belgium"']
click( )
time.sleep(5)
for term in terms:
  paste(term)
  type(Key.ENTER)
  time.sleep(5)
  type(Key.BACKSPACE)
  time.sleep(2)
click(    )

                       ELAG 2010, Helsinki, Lightning talk
When to use it?
• you want to automate something, but you
  don‘t have an access to its source

• you want to avoid repeating, boring or
  annoying work




                 ELAG 2010, Helsinki, Lightning talk
Be aware of...
• Unexpected behaiviour of your computer (lags
  – just take your time)

• Inconsistent behaiviour of your software




                 ELAG 2010, Helsinki, Lightning talk
Thanks for your attention!




        ELAG 2010, Helsinki, Lightning talk

More Related Content

Similar to GUI automation with Sikuli

Library, Library Lets Use THAT Library – DFW Mobile
Library, Library Lets Use THAT Library  – DFW MobileLibrary, Library Lets Use THAT Library  – DFW Mobile
Library, Library Lets Use THAT Library – DFW Mobile
Adam Hill
 
Loki: An Opensource Zipkin/Prometheus Mashup written in Go.
Loki: An Opensource Zipkin/Prometheus Mashup written in Go.Loki: An Opensource Zipkin/Prometheus Mashup written in Go.
Loki: An Opensource Zipkin/Prometheus Mashup written in Go.
Weaveworks
 
Vulnerability, exploit to metasploit
Vulnerability, exploit to metasploitVulnerability, exploit to metasploit
Vulnerability, exploit to metasploit
Tiago Henriques
 
Elk meetup boston - logz.io
Elk meetup boston -  logz.ioElk meetup boston -  logz.io
Elk meetup boston - logz.io
tomerlevy9
 
Intelligent System Scripting Language - Conceptual Presentation
Intelligent System Scripting Language - Conceptual PresentationIntelligent System Scripting Language - Conceptual Presentation
Intelligent System Scripting Language - Conceptual Presentation
Cédric Poottaren
 
Functional Programming in PHP
Functional Programming in PHPFunctional Programming in PHP
Functional Programming in PHP
Aurimas Niekis
 
Kotlin from-scratch
Kotlin from-scratchKotlin from-scratch
Kotlin from-scratch
Franco Lombardo
 
Using PowerShell for SharePoint 2013
Using PowerShell for SharePoint 2013Using PowerShell for SharePoint 2013
Using PowerShell for SharePoint 2013
Nikolas Charlebois-Laprade
 
JS Lab`16. Роман Лютиков: "ClojureScript, что ты такое?"
JS Lab`16. Роман Лютиков: "ClojureScript, что ты такое?"JS Lab`16. Роман Лютиков: "ClojureScript, что ты такое?"
JS Lab`16. Роман Лютиков: "ClojureScript, что ты такое?"
GeeksLab Odessa
 
Intro To AOP
Intro To AOPIntro To AOP
Intro To AOP
Donald Belcham
 
Ios-training-institute-in-mumbai
Ios-training-institute-in-mumbaiIos-training-institute-in-mumbai
Ios-training-institute-in-mumbai
vibrantuser
 
Ios-training-institute-in-mumbai
Ios-training-institute-in-mumbaiIos-training-institute-in-mumbai
Ios-training-institute-in-mumbai
vibrantuser
 
Introduction to the intermediate Python - v1.1
Introduction to the intermediate Python - v1.1Introduction to the intermediate Python - v1.1
Introduction to the intermediate Python - v1.1
Andrei KUCHARAVY
 
Hacker 101/102 - Introduction to Programming w/Processing
Hacker 101/102 - Introduction to Programming w/ProcessingHacker 101/102 - Introduction to Programming w/Processing
Hacker 101/102 - Introduction to Programming w/Processing
Dan Chudnov
 
Intro to Kotlin
Intro to KotlinIntro to Kotlin
Intro to Kotlin
Shady Selim
 
Haskell @ HAN Arnhem 2013-2014
Haskell @ HAN Arnhem 2013-2014Haskell @ HAN Arnhem 2013-2014
Haskell @ HAN Arnhem 2013-2014
Tjeerd Hans Terpstra
 
CoreML for NLP (Melb Cocoaheads 08/02/2018)
CoreML for NLP (Melb Cocoaheads 08/02/2018)CoreML for NLP (Melb Cocoaheads 08/02/2018)
CoreML for NLP (Melb Cocoaheads 08/02/2018)
Hon Weng Chong
 
Writing readable Clojure code
Writing readable Clojure codeWriting readable Clojure code
Writing readable Clojure code
Jiří Knesl
 
10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere
10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere
10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere
mfrancis
 
Into the Land of lambda, One Programmer's Journey Into Functional Programming
Into the Land of lambda, One Programmer's Journey Into Functional ProgrammingInto the Land of lambda, One Programmer's Journey Into Functional Programming
Into the Land of lambda, One Programmer's Journey Into Functional Programming
Mike Pence
 

Similar to GUI automation with Sikuli (20)

Library, Library Lets Use THAT Library – DFW Mobile
Library, Library Lets Use THAT Library  – DFW MobileLibrary, Library Lets Use THAT Library  – DFW Mobile
Library, Library Lets Use THAT Library – DFW Mobile
 
Loki: An Opensource Zipkin/Prometheus Mashup written in Go.
Loki: An Opensource Zipkin/Prometheus Mashup written in Go.Loki: An Opensource Zipkin/Prometheus Mashup written in Go.
Loki: An Opensource Zipkin/Prometheus Mashup written in Go.
 
Vulnerability, exploit to metasploit
Vulnerability, exploit to metasploitVulnerability, exploit to metasploit
Vulnerability, exploit to metasploit
 
Elk meetup boston - logz.io
Elk meetup boston -  logz.ioElk meetup boston -  logz.io
Elk meetup boston - logz.io
 
Intelligent System Scripting Language - Conceptual Presentation
Intelligent System Scripting Language - Conceptual PresentationIntelligent System Scripting Language - Conceptual Presentation
Intelligent System Scripting Language - Conceptual Presentation
 
Functional Programming in PHP
Functional Programming in PHPFunctional Programming in PHP
Functional Programming in PHP
 
Kotlin from-scratch
Kotlin from-scratchKotlin from-scratch
Kotlin from-scratch
 
Using PowerShell for SharePoint 2013
Using PowerShell for SharePoint 2013Using PowerShell for SharePoint 2013
Using PowerShell for SharePoint 2013
 
JS Lab`16. Роман Лютиков: "ClojureScript, что ты такое?"
JS Lab`16. Роман Лютиков: "ClojureScript, что ты такое?"JS Lab`16. Роман Лютиков: "ClojureScript, что ты такое?"
JS Lab`16. Роман Лютиков: "ClojureScript, что ты такое?"
 
Intro To AOP
Intro To AOPIntro To AOP
Intro To AOP
 
Ios-training-institute-in-mumbai
Ios-training-institute-in-mumbaiIos-training-institute-in-mumbai
Ios-training-institute-in-mumbai
 
Ios-training-institute-in-mumbai
Ios-training-institute-in-mumbaiIos-training-institute-in-mumbai
Ios-training-institute-in-mumbai
 
Introduction to the intermediate Python - v1.1
Introduction to the intermediate Python - v1.1Introduction to the intermediate Python - v1.1
Introduction to the intermediate Python - v1.1
 
Hacker 101/102 - Introduction to Programming w/Processing
Hacker 101/102 - Introduction to Programming w/ProcessingHacker 101/102 - Introduction to Programming w/Processing
Hacker 101/102 - Introduction to Programming w/Processing
 
Intro to Kotlin
Intro to KotlinIntro to Kotlin
Intro to Kotlin
 
Haskell @ HAN Arnhem 2013-2014
Haskell @ HAN Arnhem 2013-2014Haskell @ HAN Arnhem 2013-2014
Haskell @ HAN Arnhem 2013-2014
 
CoreML for NLP (Melb Cocoaheads 08/02/2018)
CoreML for NLP (Melb Cocoaheads 08/02/2018)CoreML for NLP (Melb Cocoaheads 08/02/2018)
CoreML for NLP (Melb Cocoaheads 08/02/2018)
 
Writing readable Clojure code
Writing readable Clojure codeWriting readable Clojure code
Writing readable Clojure code
 
10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere
10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere
10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere
 
Into the Land of lambda, One Programmer's Journey Into Functional Programming
Into the Land of lambda, One Programmer's Journey Into Functional ProgrammingInto the Land of lambda, One Programmer's Journey Into Functional Programming
Into the Land of lambda, One Programmer's Journey Into Functional Programming
 

Recently uploaded

Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
David Brossard
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
panagenda
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Matthew Sinclair
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
SitimaJohn
 
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
Techgropse Pvt.Ltd.
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
kumardaparthi1024
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
Zilliz
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
CAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on BlockchainCAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on Blockchain
Claudio Di Ciccio
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 

Recently uploaded (20)

Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
 
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
CAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on BlockchainCAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on Blockchain
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 

GUI automation with Sikuli

  • 1. GUI automation with Sikuli Ctibor Škuta ELAG 2010, Helsinki, Lightning talk
  • 2. Our problem • 400 new subject headings picked from 653 MARC field (keyword) of our bibliographic records • … but then we had to change the 653 field with given terms to 650 field (authority subject heading) • Which means for me to fill the same form 400 TIMES only with one different term ELAG 2010, Helsinki, Lightning talk
  • 3. Other solution • Maybe it could be done automatically? • Aleph administrator: „That‘s not possible.“ ELAG 2010, Helsinki, Lightning talk
  • 4. Totally different solution • … but maybe we don‘t have to automate the system itself • Maybe we can automate the actual human work (typing, clicking) ELAG 2010, Helsinki, Lightning talk
  • 5. Our solution = Sikuli • Picture-driven programming tool • Recognizes patterns on graphical output • Multiplatform (code in Java) • Using Jython (Python implementation in Java) ELAG 2010, Helsinki, Lightning talk
  • 6. Simple example terms = ['"moby is from belgium"', '"moby is not from belgium"'] click( ) time.sleep(5) for term in terms: paste(term) type(Key.ENTER) time.sleep(5) type(Key.BACKSPACE) time.sleep(2) click( ) ELAG 2010, Helsinki, Lightning talk
  • 7. When to use it? • you want to automate something, but you don‘t have an access to its source • you want to avoid repeating, boring or annoying work ELAG 2010, Helsinki, Lightning talk
  • 8. Be aware of... • Unexpected behaiviour of your computer (lags – just take your time) • Inconsistent behaiviour of your software ELAG 2010, Helsinki, Lightning talk
  • 9. Thanks for your attention! ELAG 2010, Helsinki, Lightning talk