SlideShare a Scribd company logo
.
Theory of Android Automation
M.Zhanikeev -- maratishe@gmail.com -- Towards Android Automation: Screen Vision and Software Touch -- http://bit.do/marat150525 2/10
...
2/10
.
Traditional Automation (AutoHotkey)
ā€¢ AutoHotkey
ā—¦ mouse click emulation
ā—¦ keyboard emulation
ā—¦ run/switch apps
ā—¦ ...
ā€¢ very useful in research -- software agents replace human operators
M.Zhanikeev -- maratishe@gmail.com -- Towards Android Automation: Screen Vision and Software Touch -- http://bit.do/marat150525 3/10
...
3/10
.
Existing Android Automation (light)
ā€¢ "move app to SD" can only be done
manually
ā€¢ ...so, as a light form of automation, one
can
1. run system.prefs window
2. wait 1s, then, stick a balloon at (x,y)
M.Zhanikeev -- maratishe@gmail.com -- Towards Android Automation: Screen Vision and Software Touch -- http://bit.do/marat150525 4/10
...
4/10
.
Android Tasker
ā€¢ often called "android autohotkey", but grossly lacking in function
M.Zhanikeev -- maratishe@gmail.com -- Towards Android Automation: Screen Vision and Software Touch -- http://bit.do/marat150525 5/10
...
5/10
.
A Screenshot App for Vision
ā€¢ useful, even if you have a native screenshot function
ā€¢ for example, you can take screenshots with a delay
M.Zhanikeev -- maratishe@gmail.com -- Towards Android Automation: Screen Vision and Software Touch -- http://bit.do/marat150525 6/10
...
6/10
.
Task Setup
M.Zhanikeev -- maratishe@gmail.com -- Towards Android Automation: Screen Vision and Software Touch -- http://bit.do/marat150525 7/10
...
7/10
.
Full Auto Design
ā€¢ start ā†’ tasker +
screenshot + PHP/
Python/* script ā†’
decision ā†’ touch
(emulation) ā†’ end
(ā†’next loop)
ā€¢ you can scipt
decision-making using
SL4A (I use PHP)
M.Zhanikeev -- maratishe@gmail.com -- Towards Android Automation: Screen Vision and Software Touch -- http://bit.do/marat150525 8/10
...
8/10
.
The Big Picture
ā€¢ in my case, basically, I need it for 3G/LTE+WiFiDirect virtualization 01
ā—¦ load balancing, background processing
ā€¢ usecase 1: throughput maximization for groups at a university campus 03
ā€¢ usecase 2 : vehicular nets and ultimately clouds 02
.
core idea
..
.
build infrastructure with android devices/OS at the
core
01 myself+0 "Virtual Wireless User: ...Parallel MultiConnect Using WiFi Direct..." 10th MobiQuitous Springer LNICS vol.131 (2013)
03 myself+0 "Group Connect in a New Wireless University Campus" IEICE SRē ” (2014)
02 myself+0 "The Vobile Cloud : Cars are Better for Mobile Clouds and Group Connect" IEICE ITSē ” (2014)
M.Zhanikeev -- maratishe@gmail.com -- Towards Android Automation: Screen Vision and Software Touch -- http://bit.do/marat150525 9/10
...
9/10
.
Thatā€™s all, thank you ...
M.Zhanikeev -- maratishe@gmail.com -- Towards Android Automation: Screen Vision and Software Touch -- http://bit.do/marat150525 10/10
...
10/10

More Related Content

Similar to Towards Android Automation: Screen Vision and Software Touch

Brain storming development in the post pc world
Brain storming development in the post pc worldBrain storming development in the post pc world
Brain storming development in the post pc world
Galit Fein
Ā 
Drupalcamp New York 2009
Drupalcamp New York 2009Drupalcamp New York 2009
Drupalcamp New York 2009
Tom Deryckere
Ā 
project report on IoT
project report on IoTproject report on IoT
project report on IoT
Anshul Sahu
Ā 
Doran-C4L2010
Doran-C4L2010Doran-C4L2010
Doran-C4L2010
Quickoffice Test
Ā 
Android Programming made easy
Android Programming made easyAndroid Programming made easy
Android Programming made easy
Lars Vogel
Ā 
Workshop on Sencha Touch - Part 2 - First application in sencha touch
Workshop on Sencha Touch - Part 2 - First application in sencha touchWorkshop on Sencha Touch - Part 2 - First application in sencha touch
Workshop on Sencha Touch - Part 2 - First application in sencha touch
Nithya Sivakumar
Ā 
2019-12-11-OWASP-IoT-Top-10---Introduction-and-Root-Causes.pdf
2019-12-11-OWASP-IoT-Top-10---Introduction-and-Root-Causes.pdf2019-12-11-OWASP-IoT-Top-10---Introduction-and-Root-Causes.pdf
2019-12-11-OWASP-IoT-Top-10---Introduction-and-Root-Causes.pdf
dino715195
Ā 
(inter)act now!
(inter)act now!(inter)act now!
(inter)act now!
Rik Rijnders
Ā 
Samsung
SamsungSamsung
Samsung
Peter-Paul Koch
Ā 
Bruce lawson-over-the-air
Bruce lawson-over-the-airBruce lawson-over-the-air
Bruce lawson-over-the-air
brucelawson
Ā 
Siruna session at Drupalcon Paris 2009
Siruna session at Drupalcon Paris 2009Siruna session at Drupalcon Paris 2009
Siruna session at Drupalcon Paris 2009
Tom Deryckere
Ā 
I like i phone and android but know .net
I like i phone and android but know .netI like i phone and android but know .net
I like i phone and android but know .net
Chris Love
Ā 
Introduction to Android
Introduction to AndroidIntroduction to Android
Introduction to Android
MUFIX Community
Ā 
DDive- Giuseppe Grasso - mobile su Lotus
DDive- Giuseppe Grasso - mobile su LotusDDive- Giuseppe Grasso - mobile su Lotus
DDive- Giuseppe Grasso - mobile su Lotus
Dominopoint - Italian Lotus User Group
Ā 
The Glass Class: Rapid Prototyping for Wearable Computers
The Glass Class: Rapid Prototyping for Wearable ComputersThe Glass Class: Rapid Prototyping for Wearable Computers
The Glass Class: Rapid Prototyping for Wearable Computers
Mark Billinghurst
Ā 
Why HTML5 is getting on my nervesā€¦
Why HTML5 is getting on my nervesā€¦Why HTML5 is getting on my nervesā€¦
Why HTML5 is getting on my nervesā€¦
Avenga Germany GmbH
Ā 
3-Way Scripts as a Base Unit for Flexible Scale-Out Code
3-Way Scripts as a Base Unit for Flexible Scale-Out Code3-Way Scripts as a Base Unit for Flexible Scale-Out Code
3-Way Scripts as a Base Unit for Flexible Scale-Out Code
Tokyo University of Science
Ā 
Worldkit System
Worldkit SystemWorldkit System
Worldkit System
akash relan
Ā 
Mobile applications for SharePoint using HTML5
Mobile applications for SharePoint using HTML5Mobile applications for SharePoint using HTML5
Mobile applications for SharePoint using HTML5
Christian Heindel
Ā 
Building Applications for HoloLens
Building Applications for HoloLensBuilding Applications for HoloLens
Building Applications for HoloLens
Damir Dobric
Ā 

Similar to Towards Android Automation: Screen Vision and Software Touch (20)

Brain storming development in the post pc world
Brain storming development in the post pc worldBrain storming development in the post pc world
Brain storming development in the post pc world
Ā 
Drupalcamp New York 2009
Drupalcamp New York 2009Drupalcamp New York 2009
Drupalcamp New York 2009
Ā 
project report on IoT
project report on IoTproject report on IoT
project report on IoT
Ā 
Doran-C4L2010
Doran-C4L2010Doran-C4L2010
Doran-C4L2010
Ā 
Android Programming made easy
Android Programming made easyAndroid Programming made easy
Android Programming made easy
Ā 
Workshop on Sencha Touch - Part 2 - First application in sencha touch
Workshop on Sencha Touch - Part 2 - First application in sencha touchWorkshop on Sencha Touch - Part 2 - First application in sencha touch
Workshop on Sencha Touch - Part 2 - First application in sencha touch
Ā 
2019-12-11-OWASP-IoT-Top-10---Introduction-and-Root-Causes.pdf
2019-12-11-OWASP-IoT-Top-10---Introduction-and-Root-Causes.pdf2019-12-11-OWASP-IoT-Top-10---Introduction-and-Root-Causes.pdf
2019-12-11-OWASP-IoT-Top-10---Introduction-and-Root-Causes.pdf
Ā 
(inter)act now!
(inter)act now!(inter)act now!
(inter)act now!
Ā 
Samsung
SamsungSamsung
Samsung
Ā 
Bruce lawson-over-the-air
Bruce lawson-over-the-airBruce lawson-over-the-air
Bruce lawson-over-the-air
Ā 
Siruna session at Drupalcon Paris 2009
Siruna session at Drupalcon Paris 2009Siruna session at Drupalcon Paris 2009
Siruna session at Drupalcon Paris 2009
Ā 
I like i phone and android but know .net
I like i phone and android but know .netI like i phone and android but know .net
I like i phone and android but know .net
Ā 
Introduction to Android
Introduction to AndroidIntroduction to Android
Introduction to Android
Ā 
DDive- Giuseppe Grasso - mobile su Lotus
DDive- Giuseppe Grasso - mobile su LotusDDive- Giuseppe Grasso - mobile su Lotus
DDive- Giuseppe Grasso - mobile su Lotus
Ā 
The Glass Class: Rapid Prototyping for Wearable Computers
The Glass Class: Rapid Prototyping for Wearable ComputersThe Glass Class: Rapid Prototyping for Wearable Computers
The Glass Class: Rapid Prototyping for Wearable Computers
Ā 
Why HTML5 is getting on my nervesā€¦
Why HTML5 is getting on my nervesā€¦Why HTML5 is getting on my nervesā€¦
Why HTML5 is getting on my nervesā€¦
Ā 
3-Way Scripts as a Base Unit for Flexible Scale-Out Code
3-Way Scripts as a Base Unit for Flexible Scale-Out Code3-Way Scripts as a Base Unit for Flexible Scale-Out Code
3-Way Scripts as a Base Unit for Flexible Scale-Out Code
Ā 
Worldkit System
Worldkit SystemWorldkit System
Worldkit System
Ā 
Mobile applications for SharePoint using HTML5
Mobile applications for SharePoint using HTML5Mobile applications for SharePoint using HTML5
Mobile applications for SharePoint using HTML5
Ā 
Building Applications for HoloLens
Building Applications for HoloLensBuilding Applications for HoloLens
Building Applications for HoloLens
Ā 

More from Tokyo University of Science

A Method for Cloud-Assisted Secure Wireless Grouping of Client Devices at Net...
A Method for Cloud-Assisted Secure Wireless Grouping of Client Devices at Net...A Method for Cloud-Assisted Secure Wireless Grouping of Client Devices at Net...
A Method for Cloud-Assisted Secure Wireless Grouping of Client Devices at Net...
Tokyo University of Science
Ā 
Ultrasound Relative Positioning for IoT Devices in Dense Wireless Spaces
Ultrasound Relative Positioning for IoT Devices in Dense Wireless SpacesUltrasound Relative Positioning for IoT Devices in Dense Wireless Spaces
Ultrasound Relative Positioning for IoT Devices in Dense Wireless Spaces
Tokyo University of Science
Ā 
Towards a Packet Traffic Genome Project as a Method for Realtime Sub-Flow Tra...
Towards a Packet Traffic Genome Project as a Method for Realtime Sub-Flow Tra...Towards a Packet Traffic Genome Project as a Method for Realtime Sub-Flow Tra...
Towards a Packet Traffic Genome Project as a Method for Realtime Sub-Flow Tra...
Tokyo University of Science
Ā 
What if We Atomize Student Data and Apps and Put Them on Docker Containers?
What if We Atomize Student Data and Apps and Put Them on Docker Containers?What if We Atomize Student Data and Apps and Put Them on Docker Containers?
What if We Atomize Student Data and Apps and Put Them on Docker Containers?
Tokyo University of Science
Ā 
Large-Scale Crowdsourcing by Vehicular Data Packets in a Sparse Roadside Infr...
Large-Scale Crowdsourcing by Vehicular Data Packets in a Sparse Roadside Infr...Large-Scale Crowdsourcing by Vehicular Data Packets in a Sparse Roadside Infr...
Large-Scale Crowdsourcing by Vehicular Data Packets in a Sparse Roadside Infr...
Tokyo University of Science
Ā 
On Performance Under Hotspots in Hadoop versus Bigdata Replay Platforms
On Performance Under Hotspots in Hadoop versus Bigdata Replay PlatformsOn Performance Under Hotspots in Hadoop versus Bigdata Replay Platforms
On Performance Under Hotspots in Hadoop versus Bigdata Replay Platforms
Tokyo University of Science
Ā 
Taking the Step from Software to Product Development \\ when teaching PBL at ...
Taking the Step from Software to Product Development \\ when teaching PBL at ...Taking the Step from Software to Product Development \\ when teaching PBL at ...
Taking the Step from Software to Product Development \\ when teaching PBL at ...
Tokyo University of Science
Ā 
Design and Implementation of a 3-Party Cloud-Backed Handshake for Secure Grou...
Design and Implementation of a 3-Party Cloud-Backed Handshake for Secure Grou...Design and Implementation of a 3-Party Cloud-Backed Handshake for Secure Grou...
Design and Implementation of a 3-Party Cloud-Backed Handshake for Secure Grou...
Tokyo University of Science
Ā 
The Switchboard Optimization Problem and Heuristics for Cut-Through Networking
The Switchboard Optimization Problem and Heuristics for Cut-Through NetworkingThe Switchboard Optimization Problem and Heuristics for Cut-Through Networking
The Switchboard Optimization Problem and Heuristics for Cut-Through Networking
Tokyo University of Science
Ā 
The Switchboard Traffic Engineering Problem for Mixed Contention/Cut-Through ...
The Switchboard Traffic Engineering Problem for Mixed Contention/Cut-Through ...The Switchboard Traffic Engineering Problem for Mixed Contention/Cut-Through ...
The Switchboard Traffic Engineering Problem for Mixed Contention/Cut-Through ...
Tokyo University of Science
Ā 
Bulk-n-Pick Method for One-to-Many Data Transfer in Dense Wireless Spaces
Bulk-n-Pick Method for One-to-Many Data Transfer in Dense Wireless SpacesBulk-n-Pick Method for One-to-Many Data Transfer in Dense Wireless Spaces
Bulk-n-Pick Method for One-to-Many Data Transfer in Dense Wireless Spaces
Tokyo University of Science
Ā 
Fog Cloud Caching at Network Edge via Local Hardware Awareness Spaces
Fog Cloud Caching at Network Edge via Local Hardware Awareness SpacesFog Cloud Caching at Network Edge via Local Hardware Awareness Spaces
Fog Cloud Caching at Network Edge via Local Hardware Awareness Spaces
Tokyo University of Science
Ā 
On a Hybrid Packets-and-Circuits Switching Logic
On a Hybrid Packets-and-Circuits Switching LogicOn a Hybrid Packets-and-Circuits Switching Logic
On a Hybrid Packets-and-Circuits Switching Logic
Tokyo University of Science
Ā 
Image-Related Uses for Roadside Infrastructure \\ based on Wireless Beacons
Image-Related Uses for Roadside Infrastructure \\ based on Wireless BeaconsImage-Related Uses for Roadside Infrastructure \\ based on Wireless Beacons
Image-Related Uses for Roadside Infrastructure \\ based on Wireless Beacons
Tokyo University of Science
Ā 
Complexity Resolution Control for Context Based on Metromaps
Complexity Resolution Control for Context Based on MetromapsComplexity Resolution Control for Context Based on Metromaps
Complexity Resolution Control for Context Based on Metromaps
Tokyo University of Science
Ā 
The Declarative-Coordinated Model for Self-Optimization of Service Networks
The Declarative-Coordinated Model for Self-Optimization of Service NetworksThe Declarative-Coordinated Model for Self-Optimization of Service Networks
The Declarative-Coordinated Model for Self-Optimization of Service Networks
Tokyo University of Science
Ā 
3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds
3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds
3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds
Tokyo University of Science
Ā 
Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback
Towards Social Robotics on Smartphones with Simple XYZV Sensor FeedbackTowards Social Robotics on Smartphones with Simple XYZV Sensor Feedback
Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback
Tokyo University of Science
Ā 
Back to Rings but not Tokens: Physical and Logical Designs for Distributed Fi...
Back to Rings but not Tokens: Physical and Logical Designs for Distributed Fi...Back to Rings but not Tokens: Physical and Logical Designs for Distributed Fi...
Back to Rings but not Tokens: Physical and Logical Designs for Distributed Fi...
Tokyo University of Science
Ā 
Browser Visualization using PNGs Generated by HTML5 Workers on Multicore
Browser Visualization using PNGs Generated by HTML5 Workers on MulticoreBrowser Visualization using PNGs Generated by HTML5 Workers on Multicore
Browser Visualization using PNGs Generated by HTML5 Workers on Multicore
Tokyo University of Science
Ā 

More from Tokyo University of Science (20)

A Method for Cloud-Assisted Secure Wireless Grouping of Client Devices at Net...
A Method for Cloud-Assisted Secure Wireless Grouping of Client Devices at Net...A Method for Cloud-Assisted Secure Wireless Grouping of Client Devices at Net...
A Method for Cloud-Assisted Secure Wireless Grouping of Client Devices at Net...
Ā 
Ultrasound Relative Positioning for IoT Devices in Dense Wireless Spaces
Ultrasound Relative Positioning for IoT Devices in Dense Wireless SpacesUltrasound Relative Positioning for IoT Devices in Dense Wireless Spaces
Ultrasound Relative Positioning for IoT Devices in Dense Wireless Spaces
Ā 
Towards a Packet Traffic Genome Project as a Method for Realtime Sub-Flow Tra...
Towards a Packet Traffic Genome Project as a Method for Realtime Sub-Flow Tra...Towards a Packet Traffic Genome Project as a Method for Realtime Sub-Flow Tra...
Towards a Packet Traffic Genome Project as a Method for Realtime Sub-Flow Tra...
Ā 
What if We Atomize Student Data and Apps and Put Them on Docker Containers?
What if We Atomize Student Data and Apps and Put Them on Docker Containers?What if We Atomize Student Data and Apps and Put Them on Docker Containers?
What if We Atomize Student Data and Apps and Put Them on Docker Containers?
Ā 
Large-Scale Crowdsourcing by Vehicular Data Packets in a Sparse Roadside Infr...
Large-Scale Crowdsourcing by Vehicular Data Packets in a Sparse Roadside Infr...Large-Scale Crowdsourcing by Vehicular Data Packets in a Sparse Roadside Infr...
Large-Scale Crowdsourcing by Vehicular Data Packets in a Sparse Roadside Infr...
Ā 
On Performance Under Hotspots in Hadoop versus Bigdata Replay Platforms
On Performance Under Hotspots in Hadoop versus Bigdata Replay PlatformsOn Performance Under Hotspots in Hadoop versus Bigdata Replay Platforms
On Performance Under Hotspots in Hadoop versus Bigdata Replay Platforms
Ā 
Taking the Step from Software to Product Development \\ when teaching PBL at ...
Taking the Step from Software to Product Development \\ when teaching PBL at ...Taking the Step from Software to Product Development \\ when teaching PBL at ...
Taking the Step from Software to Product Development \\ when teaching PBL at ...
Ā 
Design and Implementation of a 3-Party Cloud-Backed Handshake for Secure Grou...
Design and Implementation of a 3-Party Cloud-Backed Handshake for Secure Grou...Design and Implementation of a 3-Party Cloud-Backed Handshake for Secure Grou...
Design and Implementation of a 3-Party Cloud-Backed Handshake for Secure Grou...
Ā 
The Switchboard Optimization Problem and Heuristics for Cut-Through Networking
The Switchboard Optimization Problem and Heuristics for Cut-Through NetworkingThe Switchboard Optimization Problem and Heuristics for Cut-Through Networking
The Switchboard Optimization Problem and Heuristics for Cut-Through Networking
Ā 
The Switchboard Traffic Engineering Problem for Mixed Contention/Cut-Through ...
The Switchboard Traffic Engineering Problem for Mixed Contention/Cut-Through ...The Switchboard Traffic Engineering Problem for Mixed Contention/Cut-Through ...
The Switchboard Traffic Engineering Problem for Mixed Contention/Cut-Through ...
Ā 
Bulk-n-Pick Method for One-to-Many Data Transfer in Dense Wireless Spaces
Bulk-n-Pick Method for One-to-Many Data Transfer in Dense Wireless SpacesBulk-n-Pick Method for One-to-Many Data Transfer in Dense Wireless Spaces
Bulk-n-Pick Method for One-to-Many Data Transfer in Dense Wireless Spaces
Ā 
Fog Cloud Caching at Network Edge via Local Hardware Awareness Spaces
Fog Cloud Caching at Network Edge via Local Hardware Awareness SpacesFog Cloud Caching at Network Edge via Local Hardware Awareness Spaces
Fog Cloud Caching at Network Edge via Local Hardware Awareness Spaces
Ā 
On a Hybrid Packets-and-Circuits Switching Logic
On a Hybrid Packets-and-Circuits Switching LogicOn a Hybrid Packets-and-Circuits Switching Logic
On a Hybrid Packets-and-Circuits Switching Logic
Ā 
Image-Related Uses for Roadside Infrastructure \\ based on Wireless Beacons
Image-Related Uses for Roadside Infrastructure \\ based on Wireless BeaconsImage-Related Uses for Roadside Infrastructure \\ based on Wireless Beacons
Image-Related Uses for Roadside Infrastructure \\ based on Wireless Beacons
Ā 
Complexity Resolution Control for Context Based on Metromaps
Complexity Resolution Control for Context Based on MetromapsComplexity Resolution Control for Context Based on Metromaps
Complexity Resolution Control for Context Based on Metromaps
Ā 
The Declarative-Coordinated Model for Self-Optimization of Service Networks
The Declarative-Coordinated Model for Self-Optimization of Service NetworksThe Declarative-Coordinated Model for Self-Optimization of Service Networks
The Declarative-Coordinated Model for Self-Optimization of Service Networks
Ā 
3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds
3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds
3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds
Ā 
Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback
Towards Social Robotics on Smartphones with Simple XYZV Sensor FeedbackTowards Social Robotics on Smartphones with Simple XYZV Sensor Feedback
Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback
Ā 
Back to Rings but not Tokens: Physical and Logical Designs for Distributed Fi...
Back to Rings but not Tokens: Physical and Logical Designs for Distributed Fi...Back to Rings but not Tokens: Physical and Logical Designs for Distributed Fi...
Back to Rings but not Tokens: Physical and Logical Designs for Distributed Fi...
Ā 
Browser Visualization using PNGs Generated by HTML5 Workers on Multicore
Browser Visualization using PNGs Generated by HTML5 Workers on MulticoreBrowser Visualization using PNGs Generated by HTML5 Workers on Multicore
Browser Visualization using PNGs Generated by HTML5 Workers on Multicore
Ā 

Recently uploaded

Christine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptxChristine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptx
christinelarrosa
Ā 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Pitangent Analytics & Technology Solutions Pvt. Ltd
Ā 
Day 2 - Intro to UiPath Studio Fundamentals
Day 2 - Intro to UiPath Studio FundamentalsDay 2 - Intro to UiPath Studio Fundamentals
Day 2 - Intro to UiPath Studio Fundamentals
UiPathCommunity
Ā 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
Ā 
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
DanBrown980551
Ā 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
DianaGray10
Ā 
From Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMsFrom Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMs
Sease
Ā 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Neo4j
Ā 
A Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's ArchitectureA Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's Architecture
ScyllaDB
Ā 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
Ivo Velitchkov
Ā 
Principle of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptxPrinciple of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptx
BibashShahi
Ā 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
Ajin Abraham
Ā 
PoznanĢ ACE event - 19.06.2024 Team 24 Wrapup slidedeck
PoznanĢ ACE event - 19.06.2024 Team 24 Wrapup slidedeckPoznanĢ ACE event - 19.06.2024 Team 24 Wrapup slidedeck
PoznanĢ ACE event - 19.06.2024 Team 24 Wrapup slidedeck
FilipTomaszewski5
Ā 
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
Fwdays
Ā 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
MichaelKnudsen27
Ā 
High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024
Vadym Kazulkin
Ā 
Christine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptxChristine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptx
christinelarrosa
Ā 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Neo4j
Ā 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
Pablo GĆ³mez Abajo
Ā 
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
zjhamm304
Ā 

Recently uploaded (20)

Christine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptxChristine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptx
Ā 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Ā 
Day 2 - Intro to UiPath Studio Fundamentals
Day 2 - Intro to UiPath Studio FundamentalsDay 2 - Intro to UiPath Studio Fundamentals
Day 2 - Intro to UiPath Studio Fundamentals
Ā 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Ā 
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
Ā 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Ā 
From Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMsFrom Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMs
Ā 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Ā 
A Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's ArchitectureA Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's Architecture
Ā 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
Ā 
Principle of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptxPrinciple of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptx
Ā 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
Ā 
PoznanĢ ACE event - 19.06.2024 Team 24 Wrapup slidedeck
PoznanĢ ACE event - 19.06.2024 Team 24 Wrapup slidedeckPoznanĢ ACE event - 19.06.2024 Team 24 Wrapup slidedeck
PoznanĢ ACE event - 19.06.2024 Team 24 Wrapup slidedeck
Ā 
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
Ā 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Ā 
High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024
Ā 
Christine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptxChristine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptx
Ā 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Ā 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
Ā 
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
Ā 

Towards Android Automation: Screen Vision and Software Touch

  • 1.
  • 2. . Theory of Android Automation M.Zhanikeev -- maratishe@gmail.com -- Towards Android Automation: Screen Vision and Software Touch -- http://bit.do/marat150525 2/10 ... 2/10
  • 3. . Traditional Automation (AutoHotkey) ā€¢ AutoHotkey ā—¦ mouse click emulation ā—¦ keyboard emulation ā—¦ run/switch apps ā—¦ ... ā€¢ very useful in research -- software agents replace human operators M.Zhanikeev -- maratishe@gmail.com -- Towards Android Automation: Screen Vision and Software Touch -- http://bit.do/marat150525 3/10 ... 3/10
  • 4. . Existing Android Automation (light) ā€¢ "move app to SD" can only be done manually ā€¢ ...so, as a light form of automation, one can 1. run system.prefs window 2. wait 1s, then, stick a balloon at (x,y) M.Zhanikeev -- maratishe@gmail.com -- Towards Android Automation: Screen Vision and Software Touch -- http://bit.do/marat150525 4/10 ... 4/10
  • 5. . Android Tasker ā€¢ often called "android autohotkey", but grossly lacking in function M.Zhanikeev -- maratishe@gmail.com -- Towards Android Automation: Screen Vision and Software Touch -- http://bit.do/marat150525 5/10 ... 5/10
  • 6. . A Screenshot App for Vision ā€¢ useful, even if you have a native screenshot function ā€¢ for example, you can take screenshots with a delay M.Zhanikeev -- maratishe@gmail.com -- Towards Android Automation: Screen Vision and Software Touch -- http://bit.do/marat150525 6/10 ... 6/10
  • 7. . Task Setup M.Zhanikeev -- maratishe@gmail.com -- Towards Android Automation: Screen Vision and Software Touch -- http://bit.do/marat150525 7/10 ... 7/10
  • 8. . Full Auto Design ā€¢ start ā†’ tasker + screenshot + PHP/ Python/* script ā†’ decision ā†’ touch (emulation) ā†’ end (ā†’next loop) ā€¢ you can scipt decision-making using SL4A (I use PHP) M.Zhanikeev -- maratishe@gmail.com -- Towards Android Automation: Screen Vision and Software Touch -- http://bit.do/marat150525 8/10 ... 8/10
  • 9. . The Big Picture ā€¢ in my case, basically, I need it for 3G/LTE+WiFiDirect virtualization 01 ā—¦ load balancing, background processing ā€¢ usecase 1: throughput maximization for groups at a university campus 03 ā€¢ usecase 2 : vehicular nets and ultimately clouds 02 . core idea .. . build infrastructure with android devices/OS at the core 01 myself+0 "Virtual Wireless User: ...Parallel MultiConnect Using WiFi Direct..." 10th MobiQuitous Springer LNICS vol.131 (2013) 03 myself+0 "Group Connect in a New Wireless University Campus" IEICE SRē ” (2014) 02 myself+0 "The Vobile Cloud : Cars are Better for Mobile Clouds and Group Connect" IEICE ITSē ” (2014) M.Zhanikeev -- maratishe@gmail.com -- Towards Android Automation: Screen Vision and Software Touch -- http://bit.do/marat150525 9/10 ... 9/10
  • 10. . Thatā€™s all, thank you ... M.Zhanikeev -- maratishe@gmail.com -- Towards Android Automation: Screen Vision and Software Touch -- http://bit.do/marat150525 10/10 ... 10/10