SlideShare a Scribd company logo
1 of 8
Download to read offline
International Journal of Electrical and Computer Engineering (IJECE)
Vol. 8, No. 2, April 2018, pp. 1230 – 1237
ISSN: 2088-8708, DOI: 10.11591/ijece.v8i2.pp1230-1237 1230
Tool Use Learning for a Real Robot
Handy Wicaksono1,2
and Claude Sammut1
1
School of Computer Science and Engineering, University of New South Wales; Sydney, Australia
2
Electrical Engineering Department, Petra Christian University; Surabaya, Indonesia
Article Info
Article history:
Received July 24, 2017
Revised December 16, 2017
Accepted December 27, 2017
Keyword:
tool use by a robot
tool use learning
action learning
inductive logic programming
robot software architecture
ABSTRACT
A robot may need to use a tool to solve a complex problem. Currently, tool use must be
pre-programmed by a human. However, this is a difficult task and can be helped if the robot
is able to learn how to use a tool by itself. Most of the work in tool use learning by a robot is
done using a feature-based representation. Despite many successful results, this representa-
tion is limited in the types of tools and tasks that can be handled. Furthermore, the complex
relationship between a tool and other world objects cannot be captured easily. Relational
learning methods have been proposed to overcome these weaknesses [1, 2]. However, they
have only been evaluated in a sensor-less simulation to avoid the complexities and uncer-
tainties of the real world. We present a real world implementation of a relational tool use
learning system for a robot. In our experiment, a robot requires around ten examples to learn
to use a hook-like tool to pull a cube from a narrow tube.
Copyright c 2018 Institute of Advanced Engineering and Science.
All rights reserved.
Corresponding Author:
Handy Wicaksono
School of Computer Science and Engineering, University of New South Wales
handyw@cse.unsw.edu.au
1. INTRODUCTION
Humans use tools to help them complete everyday tasks. The ability to use tools is a feature of humans
intelligence [3]. Like humans, a robot also needs to be able to use a tool to solve a complex task. As an example, in the
RoboCup@Home competition, a robot is asked to demonstrate several tool use abilities such as opening a bottle by
using a bottle opener or watering a plant with a watering can [4]. The robot is given complete knowledge of the tools
and how to use them. When such knowledge is not available, a robot must learn it. Most work in tool use learning has
used a feature-based representation that is dependent on an object’s primitive features. Thus, it is not flexible enough
to be applied to different tools and environments. Learning is also limited to tool selection only. Brown proposed a
relational approach which can overcome these limitations. Furthermore, Wicaksono & Sammut ([5]) have suggested
that this representation has potential to solve a more complex problem, such as tool creation.
We define a tool as an object that is deliberately employed by an agent to help it achieve a goal, which would
be too difficult to achieve without the tool. We want to learn a tool action model that explains changes in the properties
of one or more objects affected by the tool, given that certain preconditions are met. Following Brown [1], learning
is performed by trial and error in an Inductive Logic Programming (ILP) setting [6]. Brown [1] carried out tool use
learning in a sensor-less simulation. This means that a robot has perfect knowledge of the world and uncertainties in
the sensors’ readings are eliminated. In this paper, we would present a complete robotic system for tool use learning
following a relational learning approach. This includes three components:
1. Developing a robot software architecture that consists of primitive and abstract layers and facilitates communi-
cation between them.
2. Creating a mechanism to detect objects and generating primitive behaviors for tool use.
3. Extending relational learning methods and conducting tool use learning experiments using a real robot.
In the following sections, we describe relevant previous work, the knowledge representation formalism used,
the software architecture, and the tool use learning mechanism. Finally, we perform several real world experiments
and draw conclusions.
Journal Homepage: http://iaesjournal.com/online/index.php/IJECE
Int J Elec & Comp Eng ISSN: 2088-8708 1231
2. RELATED WORK
An intelligent robot is identified by its ability to make a decision and learn autonomously in an unstructured
environment. Machine learning techniques could be useful as they can compensate for the imperfect model of a robot
in a real world. For example, control of a robot manipulator can be improved by equipping a PD torque controller
with Adaptive Neuro-Fuzzy Inference System [7] or combining sliding mode control with genetic algorithm [8]. In a
behavior-based robotics approach, reinforcement learning can be used to learn the robot’s behavior [9].
Here we describe the previous work in tool use learning by a robot. Wood et al. [10] use a neural network to
learn appropriate posture of a Sony Aibo robot so it can reach an object by using a tool placed on its back. Stoytchev
[11] learns to select a correct tool via its affordances which are grounded in robot behaviors. However, its result can
not be generalized to other new tools. More recent work by Tikhanoff et al. [12]) combine exploratory behaviors and a
geometrical feature extraction to learn affordances and tool use. Mar et al. [13] extend their work by learning a grasp
configuration which influences the outcome of a tool use action.
The limitations of feature-based representations were mentioned above. Only recently have such representa-
tions been extended so that a robot can learn to grasp a tool [13]. An ILP system can overcome these limitations, as
it represents the tools and other objects in a relational representation [1, 2]. However, previous work has been done
previously in sensor-less simulation only. This means the complexities of acquiring perceptions, generating precise
movements, and dealing with world uncertainties, are avoided. We aim to develop a complete robotic system that
facilitates this learning in a real world.
3. REPRESENTING STATES AND ACTIONS
We maintain two representations of states and actions in primitive and abstract form. Primitive states are the
positions of all objects in the world that are captured by vision sensors using a mechanism described in section 4.2..
As we only use simple objects, they can be detected by their two-dimensional appearance only.
Abstract states are represented as expressions in first-order logic, to be more specific as Horn clauses. Primi-
tive states are translated to an abstract state by calling relevant Prolog programs with the primitive states as their bound
values. To be classified as a tool, an object must possess particular structural properties (e.g. a particular side where
a hook is attached to the handle) and spatial properties (e.g. a hook is touching the back side of a cube). We collect
these properties in a hypothesis, namely tool pose, which is shown below in simplified form.
t o o l p o s e ( Handle , Hook , Box , Tube ): −
a t t a c h e d e n d ( Handle , Hook , back ) , % a s t r u c t u r a l p r o p e r t y
a t t a c h e d s i d e ( Handle , Hook , Side ) , % a s t r u c t u r a l p r o p e r t y
touching ( Hook , Box , back ) . % a s p a t i a l p r o p e r t y
We use an extended STRIPS action model [14] to describe an abstract action and how it affects the world. A
primitive goal that has to be achieved by a robot is added at the end of the model.
Action name
PRE : states that must be true so that an action can be performed
ADD : conditions that become true as a result of the action
DELETE : conditions that are no longer true following the action
CONSTRAINTS : the physical limits of actuators that constrain the action
This action model is also a planning operator. Thus, action learning does not happen in isolation, as it is a part of a
problem solving scenario. Every abstract action is linked to a set of primitive behaviors. This will be discussed later
in section 4.3.
4. ROBOTIC SYSTEM
In this section, we explain our software architecture, objects detection method, and behavior generation
mechanism.
4.1. Software Architecture
We use a relational representation in the high-level layer to take advantage of the expressiveness of first-order
logic (FOL) clauses, especially Horn clauses, and equip a robot with useful symbolic techniques. We implement this
layer in SWI Prolog. Most modern Prolog implementations, including SWI, incorporate a constraint solver which
provide advanced numerical reasoning capabilities.
Tool Use Learning for a Real Robot(Handy Wicaksono)
1232 ISSN: 2088-8708
In the low-level layer, a robot operates in a world of continuous, noisy and uncertain measurements. Its
sensors return readings in numeric ranges, and its actuators operate based on numerical set points. As we aim to use
Robot Operating System (ROS1
) as our middleware, we implement this layer in Python. The Baxter robot, from Re-
think Robotics, has its own ROS-Python API, a set of classes that provides wrappers around the ROS communication.
Another layer, namely the translator, is needed to map primitive to abstract states and to link an abstract
action to corresponding primitive behaviors. It is written in C++ which has an interface to SWI Prolog as the language
for the abstract layer. Communication to the primitive layer is done via ROS using a simple publish and subscribe
mechanism. Our robot software architecture is shown in Fig. 1a.
We give a simple illustration of action execution, namely find tool, which involves communication be-
tween layers. In the primitive layer, a cube and a tube are detected, their corners are acquired and published to-
gether by a ROS node, namely /translator, as a ROS topic, /pri states. The translator also has a ROS
node, /translator, that subscribes to that topic. Then it evaluates the status of the abstract state, in tube
(Cube,Tube), by calling the relevant Prolog predicate. In the abstract layer, in tube(Cube,Tube) is called.
If it is true, as it is the only the precondition of the find tool action, then the translator will publish it as the ROS
topic, /abs action that represents the currently active action. The primitive layer subscribes to this topic. If it
recognizes find tool as an active action, then it triggers the corresponding primitive behaviors. See Fig. 1b for
detail.
Actions translator
Abstract Action 1
Constraint
Solver /
Pose Selector
Abstract layer
Perception
Object
Detection
Behaviors Generation
PID Controller
ILP Learner
States
translator
Logical
State
Planner
Abstract Action n
Translator
Primitive layer
Tool Selector
Tool
identity
Abstract
actions
Tool
pose
Abstract
states
Primitive
states
Pixels
Primitive action +
parameters
Motors
commands
Primitive
states
Real World
(a) Software architecture
objects detection
ROS node: /translator
cube(12,6,15,9)
tube(0,0,15,20)
ROS topic: /pri_states
call Prolog predicate:
in_tube(Cube,Tube)
PRE:
in_tube(Cube,Tube)
STRIPS action model
NAME:
find_tool
EFF:
tool_found(Tool)
ROS node: /primitive
ROS topic: /abs_action
publish "find_tool"
abstract action
behaviors generation
Abstract layer
Translator
Primitive layer
go to tool area,
find correct tool
(b) Simplified example
Figure 1. Robot software architecture and its example
4.2. Object Detection
We detect all objects (i.e. different tools, a cube, and a tube) by their two-dimensional appearances only.
We use the Baxter camera to detect objects locally, while an external web camera provides the global world state.
We combine local and global images from both cameras to achieve our goal. The OpenCV library is used for object
detection.
In pre-processing, Gaussian smoothing is applied. Later, the edges of objects in an image are detected with
a Canny Edge detector, and their contours are found. Each contour is tested to check whether it has the properties of
a particular object. The contour properties include the number of edges, the area, the angle formed by two adjacent
parts, the convexity, and the perimeter. Tools are treated specially as they have more than one shape. After being
detected as a tool, the object’s type is determined, and its handle and hook(s) are acquired. All objects are represented
1http://www.ros.org
Int J Elec & Comp Eng Vol. 8, No. 2, April 2018: 1230 – 1237
Int J Elec & Comp Eng ISSN: 2088-8708 1233
by their minimum and maximum corner points in Cartesian coordinates (Xmin, Ymin, Xmax, Ymax ). The complete
mechanism is shown in Algorithm 1.
Algorithm 1 Objects detection
1: Capture an image
2: Perform Gaussian smoothing
3: Detect the edges of objects using Canny detector
4: Find the contours of the objects
5: for each contour do
6: Classify the contour
7: if The contour is either cube, tube, or tool then
8: Find its minimum & maximum corner points
9: return all minimum & maximum corner points
4.3. Behavior Generation
In our tool use application, a sequence of actions, or a plan, must be performed to achieve the goal, pulling
a cube from a narrow tube. Each action uses different strategies (inverse kinematics solving, visual servoing, gripper
activation, constraints solving) depending on whether the primitive goal is known or not, and whether the visual
information is needed or not. We show an example of the plan in Table 1.
Table 1. The actions sequence for pulling a cube from a narrow tube
No Action Technique Goal Visual
1 Find the tool Inverse Kinematics (IK) solving, visual servoing Known Yes
2 Grip the tool Close the gripper Known No
3 Position the tool Constraints solving, IK solving Unknown Yes
4 Pull the cube IK solving Known No
5 Move the tool away IK solving Known No
6 Ungrip the tool Open the gripper Known No
An abstract action is directly linked to corresponding primitive actions. For simplicity, we only consider
movement in two-dimensional Cartesian coordinates. A primitive goal is a target location in that coordinate system.
When a goal is known, we use an inverse kinematics (IK) solver to compute the angles of all joints and move the arm
towards that goal. However, if it is unknown, we use a constraint solver to create a numerical goal from abstract states
which are the preconditions of an abstract action model (see Fig. 2a). To perform an accurate tool picking action,
we use a simple version of image-based visual servoing [15], where an error signal is determined directly in terms
of image feature parameters. We locate the robot arm in a position where a chosen tool is located in the center of an
image captured by Baxter’s wrist camera, so the arm can move downward perpendicularly to take the tool. See Fig.
2b for the detail.
Action 1
Action 2
Action n
Plan
Action Model
Constraint Solver
IK Solver
Controller
Abstract goal
Primitive goal
Goal in joints movement
Motor commands
(a) Constraints solving to get a primitive goal
Detect a chosen tool in the
image (see Alg. 1)
Is the tool on
the center of image?
Move arm towards center Go down to pick the tool
Yes
No
Start
Finish
(b) A visual servoing process
Figure 2. behaviors generation
Tool Use Learning for a Real Robot(Handy Wicaksono)
1234 ISSN: 2088-8708
5. TOOL USE LEARNING
In this section, we describe relational tool use learning [1, 2]. The robot must learn a tool action model
that describes the properties of a tool and other objects in the world that enable a robot to use a tool successfully.
Specifically, the robot learns a tool pose predicate, which describes the structural and spatial conditions that must
be met for the tool to be used. We maintain a version space of hypotheses, following Mitchell [16]. A version space is
bounded by the most general hypothesis (hg) and the most specific hypothesis (hs). As we explain below, the version
space is used to determine what experiments the robot should perform to generalize or specialize the tool action model.
Initially, a robot may not have a complete action model for a tool, so it cannot construct a plan. After a tutor
gives a tool use example, the robot segments the observed behavior into discrete actions, matches them to actions
already in its background knowledge, and constructs a new action if there is no match for a particular behavior. This
construction may not be sufficient to describe the tool action in a general enough form that it can be applied in different
situations. Therefore, trial and error learning is performed to refine the initial action model.
Trial and error involved performing experiments in which a different tool is selected or its pose is changed to
test the current hypothesis for the tool action. In tool selection, an object that has properties that are most similar to a
previously useful tool is chosen. More specifically, we test whether an object satisfies all primary structural properties,
stored in hg, and most of the secondary ones, stored in hs, given its primitive states.
Having a correct tool is useless when it is located in an incorrect position before it trying to pull the target
object. We select the pose by solving the constraints of the spatial literals in the tool pose predicate. We check whether
the spatial constraints of an object can be solved or not. The unsolved constraints are ignored, and the final result is
used as the numerical goal for the robot.
After a tool use learning experiment is performed, its result, whether successful or not, is passed to the ILP
learner so the hypothesis can be refined. Our learning is derived from Golem [17], an ILP algorithm. Refinement of hs
is done by finding the constrained Least General Generalization (LGG) of a positive examples pair, and hg is refined
by performing the negative-based reduction. The learning algorithm is a modification of Haber [2] and is shown in
Algorithm 2.
Algorithm 2 Tool use learning
1: input: new action model M, hg = true, hs = preconditions of M,N trials, K consecutive success
2: while success < K or index < N do
3: e = generate experiment(hs, hg)
4: tool = select tool(hs, hg) // select a tool with highest rank
5: for each ei in e do
6: pose = select pose(ei) // performing constraint solving on preconditions of the relevant action model
7: if pose = null then
8: break
9: success = execute exp(tool, pose) // performing a tool use experiment
10: if success then
11: label pose positive
12: increment cons success
13: hs = generalise hs // via Least General Generalisation
14: else
15: label pose negative
16: cons success ← 0
17: hg = specialise hg // via negative based reduction
18: add ei to training data
19: increment index
6. RESULTS AND DISCUSSIONS
The Baxter research robot is used in this experiment (see Fig. 3a). The objects in the scene include a cube, a
tube, and five objects of a different shape that potential tools. We also have another set of tool candidates whose hooks
are narrower than their handles (see Fig. 3b). We use Baxter’s wrist camera for visual servoing. We add an external
web camera facing the tube to provide the global state. We divide the experiments into two stages. Initially, we
evaluate the performance of our object detection algorithm and action execution. Later, we conduct tool use learning
experiments.
Int J Elec & Comp Eng Vol. 8, No. 2, April 2018: 1230 – 1237
Int J Elec & Comp Eng ISSN: 2088-8708 1235
web camera
Baxter hand's
camera
5 tools
cube
tube
(a) Scene details (b) All set of tools
Figure 3. Experimental scene
6.1. Object Detection and Execution Experiments
The contours of all objects (cube, tube, and tools) are detected, using Baxter’s wrist camera, and their min-
imum and maximum points are acquired. These are marked with colored dots in the object corners in Fig. 4. Even
when the tool is held by the gripper, our algorithm is still able to detect the object. However, the vision system may
fail in a low-light environment. It also assumes that all objects are aligned perpendicular to each other and the camera.
baxter_cam: cube detection web_cam: cube detection baxter_cam: tube detection web_cam: tube detection
baxter_cam: tools detection web_cam: tools detection 1 web_cam: tools detection 2 web_cam: selected tool detection
Figure 4. Objects detection in images captured by the Baxter hand camera and the web camera
We also evaluate the capability of the Baxter to move accurately to accomplish a tool use task based on the
actions sequence in Table 1. In Fig. 5 we illustrate these actions: find tool action which uses visual servoing,
grip tool action, which activates the robot’s gripper, and position tool actions, which tries to satisfy a prim-
itive goal given by a constraint solver. From these trials, it was determined that the Baxter can move smoothly to
complete the task. We observe that most errors are caused by the perception system, not the action mechanism.
find_tool grip_tool position_tool
Figure 5. Sample of the robot’s actions
6.2. Learning Experiments
In this experiment, we use five different candidate tools with narrow and wide hooks. The cube, as a target
object, is located at a particular position inside the tube, which is changed after the Baxter can withdraw it from the
tube successfully. We adopt an initial action model which was created by Brown ([1]). Learning by trial and error is
then performed to refine the tool pose hypothesis. We stop learning if the robot accomplishes the task three times
consecutively. The complete learning sequence is shown in Fig. 6.
We need at least ten examples to learn to perform this task. However, more experiments may be needed if
any errors occur (e.g. a change in lighting or an invalid move produced by the IK solver). The first positive example
is given by a teacher. In the second example, the robot attempts to imitate the tool and location of the first one, but
it fails as the cube location is changed. In the third example, the robot finds that the attachment side of the hook
should be the same as the location of the cube inside the tube. Later on, the robot still makes mistakes as it tends to
choose a tool with a narrow hook. This narrower property is eliminated from hs in the generalization process in
Tool Use Learning for a Real Robot(Handy Wicaksono)
1236 ISSN: 2088-8708
example 2: - example 3: + example 4: - example 5: -example 1: +
example 7: - example 8: + example 9: + example 10: +example 6: +
Figure 6. Tool use learning experiments in a real world
the eighth example. Finally, learning is completed in the tenth example, as it performs three consecutive successful
experiments. The final hypothesis is shorter (hs is reduced from 14 literals to 11 literals) and more general than the
initial hypothesis.
6.3. Discussions
Our result is similar to the previous relational learning approach [1], they learn tool-use in 12 experiments,
as we use the relatively same mechanism. However, we perform our experiment on a real robot, while the former only
do it on a sensor-less simulation. This includes bridging the gap between low-level layer (in ROS environment) and
high-level layer (in SWI Prolog). We also build objects detection system that has to deal with the noisy environment.
Previous work did not do any detections, as it acquires perfect data from a simulator.
Compared to the line of work in feature-based representation, such as work of Tikhanoff et al. [12], our
approach can learn faster (theirs needs at least 120 trials in various learning stages) as we can easily incorporate
human expertise in the background knowledge. Our experiment scenario is also more complicated, as we locate the
target object inside a tube. We exploit the ability of the relational representation to represent a complex relationship
between objects compactly. We also learn the tool-pose, where the tool should be located to be able to pull the tool
successfully, while it is predefined in their work.
There are limitations in our work, especially in the perception system which can only handle 2D images and
not robust in changing environments. In this area, previous work [12, 13] is better than ours. Despite these limitations,
the experiment demonstrates that the learning system is capable of relational tool use learning in the real world. In
other work of us, we also use a physics simulator to assist a real robot performs tool use learning [18].
7. CONCLUSIONS AND FUTURE WORK
We have developed a complete robot system for relational tool use learning in the real world. The primitive,
translator and abstract layers have been built, along with with their interfaces. We have also implemented a system
to detect objects and generate primitive behaviors by inverse kinematics, a constraint solver and a visual servoing
mechanism. Finally, we have extended a tool use learning system, which has been tested in experiments in the real
world. For a relatively simple tool use scenario, the robot at needs at least ten examples to learn the tool’s properties.
In the future, we want our robot to do tool creation when the available tools cannot be used to solve the
current problem. Those tools can then be tested in the simulator, to save time and materials, and manufactured by
using a 3D printer. The development of a more robust perception and a more accurate movement systems will improve
our system performance. The system will be tested on a wider variety of tool use scenarios, with different tools and
different tasks.
ACKNOWLEDGEMENT
Handy Wicaksono is supported by The Directorate General of Resources for Science, Technology and Higher
Education (DG-RSTHE), Ministry of Research, Technology, and Higher Education of the Republic of Indonesia.
REFERENCES
[1] S. Brown and C. Sammut, “A relational approach to tool-use learning in robots,” in Inductive Logic Programming.
Springer, 2012, pp. 1–15.
[2] A. Haber, “A system architecture for learning robots,” Ph.D. dissertation, School of Computer Science and
Engineering, UNSW Australia, 2015.
Int J Elec & Comp Eng Vol. 8, No. 2, April 2018: 1230 – 1237
Int J Elec & Comp Eng ISSN: 2088-8708 1237
[3] S. H. Johnson-Frey, “What’s so special about human tool use?” Neuron, vol. 39, no. 2, pp. 201 – 204, 2003.
[Online]. Available: http://www.sciencedirect.com/science/article/pii/S0896627303004240
[4] J. Stckler and S. Behnke, “Adaptive tool-use strategies for anthropomorphic service robots,” in 2014 IEEE-RAS
International Conference on Humanoid Robots, Nov 2014, pp. 755–760.
[5] H. Wicaksono and C. Sammut, “A learning framework for tool creation by a robot,” in Proceedings of ACRA,
2015.
[6] S. Muggleton, “Inductive logic programming,” New Generation Computing, vol. 8, no. 4, pp. 295–318, 1991.
[7] O. Bachir and A.-f. Zoubir, “Adaptive neuro-fuzzy inference system based control of puma 600 robot manipula-
tor,” International Journal of Electrical and Computer Engineering, vol. 2, no. 1, p. 90, 2012.
[8] A. R. Firdaus and A. S. Rahman, “Genetic algorithm of sliding mode control design for manipulator robot,”
TELKOMNIKA (Telecommunication Computing Electronics and Control), vol. 10, no. 4, pp. 645–654, 2012.
[9] H. Wicaksono, H. Khoswanto, and S. Kuswadi, “Behaviors coordination and learning on autonomous navigation
of physical robot,” TELKOMNIKA (Telecommunication Computing Electronics and Control), vol. 9, no. 3, pp.
473–482, 2013.
[10] A. Wood, T. Horton, and R. Amant, “Effective tool use in a habile agent,” in Systems and Information Engineer-
ing Design Symposium, 2005 IEEE, April 2005, pp. 75–81.
[11] A. Stoytchev, “Behavior-grounded representation of tool affordances,” in Proceedings of the 2005 IEEE Interna-
tional Conference on Robotics and Automation, April 2005, pp. 3060–3065.
[12] V. Tikhanoff, U. Pattacini, L. Natale, and G. Metta, “Exploring affordances and tool use on the icub,” in 2013
13th IEEE-RAS International Conference on Humanoid Robots (Humanoids), Oct 2013, pp. 130–137.
[13] T. Mar, V. Tikhanoff, G. Metta, and L. Natale, “Self-supervised learning of grasp dependent tool affordances on
the icub humanoid robot,” in 2015 IEEE International Conference on Robotics and Automation (ICRA), May
2015, pp. 3200–3206.
[14] R. Fikes and N. J. Nilsson, “STRIPS: A new approach to the application of theorem proving to problem solving,”
Artif. Intell., vol. 2, no. 3/4, pp. 189–208, 1971.
[15] S. Hutchinson, G. D. Hager, and P. I. Corke, “A tutorial on visual servo control,” IEEE transactions on robotics
and automation, vol. 12, no. 5, pp. 651–670, 1996.
[16] T. M. Mitchell, “Version spaces: A candidate elimination approach to rule learning,” in Proceedings of the 5th
international joint conference on Artificial intelligence-Volume 1. Morgan Kaufmann Publishers Inc., 1977, pp.
305–310.
[17] S. Muggleton and C. Feng, “Efficient induction in logic programs,” in Inductive Logic Programming, S. Mug-
gleton, Ed. Academic Press, 1992, pp. 281–298.
[18] H. Wicaksono and C. Sammut, “Relational tool use learning by a robot in a real and simulated world,” in Pro-
ceedings of ACRA, 2016.
BIOGRAPHY OF AUTHORS
Handy Wicaksono is a Ph.D. student at Artificial Intelligence Group, School of Computer Science
and Engineering, UNSW Australia, with bachelor and master degree in Electrical Engineering from
Institut Teknologi Sepuluh Nopember, Indonesia. He is also a lecturer in Electrical Engineering De-
partment, Petra Christian University, Indonesia. His research is in the area of artificial intelligence,
intelligent robot, and industrial automation.
Claude Sammut is a Professor of Computer Science and Engineering at the University of New
South Wales, Head of the Artificial Intelligence Research Group and Deputy Director of the iCin-
ema Centre for Interactive Cinema Research. Previously, he was a program manager for the Smart
Internet Technology Cooperative Research Centre, the UNSW node Director of the ARC Centre
of Excellence for Autonomous Systems and a member of the joint ARC/NH&MRC project on
Thinking Systems. His early work on relational learning helped to the lay the foundations for the
field of Inductive Logic Programming (ILP). With Donald Michie, he also did pioneering work in
Behavioral Cloning. His current interests include Conversational Agents and Robotics.
Tool Use Learning for a Real Robot(Handy Wicaksono)

More Related Content

What's hot

Object-Oriented Concepts
Object-Oriented ConceptsObject-Oriented Concepts
Object-Oriented ConceptsAbdalla Mahmoud
 
Implementation of Emotional Intelligence in a machine
Implementation of Emotional Intelligence in a machineImplementation of Emotional Intelligence in a machine
Implementation of Emotional Intelligence in a machineIOSR Journals
 
Object-Oriented Programming Concepts
Object-Oriented Programming ConceptsObject-Oriented Programming Concepts
Object-Oriented Programming ConceptsKwangshin Oh
 
Swarm assignment 1
Swarm assignment 1Swarm assignment 1
Swarm assignment 1OmKushwaha7
 
Fundamentals of OOP (Object Oriented Programming)
Fundamentals of OOP (Object Oriented Programming)Fundamentals of OOP (Object Oriented Programming)
Fundamentals of OOP (Object Oriented Programming)MD Sulaiman
 
Introduction to Object Oriented Programming
Introduction to Object Oriented ProgrammingIntroduction to Object Oriented Programming
Introduction to Object Oriented ProgrammingMoutaz Haddara
 
Video surveillance Moving object detection& tracking Chapter 1
Video surveillance Moving object detection& tracking Chapter 1 Video surveillance Moving object detection& tracking Chapter 1
Video surveillance Moving object detection& tracking Chapter 1 ahmed mokhtar
 
Machine learning introduction
Machine learning introductionMachine learning introduction
Machine learning introductionathirakurup3
 
Introduction to oop
Introduction to oop Introduction to oop
Introduction to oop Kumar
 
A Multi-robot System Coordination Design and Analysis on Wall Follower Robot ...
A Multi-robot System Coordination Design and Analysis on Wall Follower Robot ...A Multi-robot System Coordination Design and Analysis on Wall Follower Robot ...
A Multi-robot System Coordination Design and Analysis on Wall Follower Robot ...IJECEIAES
 
Paper id 6122018109
Paper id 6122018109Paper id 6122018109
Paper id 6122018109IJRAT
 
TAAI 2016 Keynote Talk: Intercultural Collaboration as a Multi‐Agent System
TAAI 2016 Keynote Talk: Intercultural Collaboration as a Multi‐Agent SystemTAAI 2016 Keynote Talk: Intercultural Collaboration as a Multi‐Agent System
TAAI 2016 Keynote Talk: Intercultural Collaboration as a Multi‐Agent SystemYi-Shin Chen
 
Spike timing dependent plasticity to make robot navigation more intelligent. ...
Spike timing dependent plasticity to make robot navigation more intelligent. ...Spike timing dependent plasticity to make robot navigation more intelligent. ...
Spike timing dependent plasticity to make robot navigation more intelligent. ...Lietuvos kompiuterininkų sąjunga
 
Lecture Notes in Computer Science (LNCS)
Lecture Notes in Computer Science (LNCS)Lecture Notes in Computer Science (LNCS)
Lecture Notes in Computer Science (LNCS)Lê Anh
 

What's hot (20)

Object-Oriented Concepts
Object-Oriented ConceptsObject-Oriented Concepts
Object-Oriented Concepts
 
Implementation of Emotional Intelligence in a machine
Implementation of Emotional Intelligence in a machineImplementation of Emotional Intelligence in a machine
Implementation of Emotional Intelligence in a machine
 
oop Lecture 16
oop Lecture 16oop Lecture 16
oop Lecture 16
 
Object-Oriented Programming Concepts
Object-Oriented Programming ConceptsObject-Oriented Programming Concepts
Object-Oriented Programming Concepts
 
oop Lecture 11
oop Lecture 11oop Lecture 11
oop Lecture 11
 
Swarm assignment 1
Swarm assignment 1Swarm assignment 1
Swarm assignment 1
 
Fundamentals of OOP (Object Oriented Programming)
Fundamentals of OOP (Object Oriented Programming)Fundamentals of OOP (Object Oriented Programming)
Fundamentals of OOP (Object Oriented Programming)
 
Introduction to Object Oriented Programming
Introduction to Object Oriented ProgrammingIntroduction to Object Oriented Programming
Introduction to Object Oriented Programming
 
oop Lecture 10
oop Lecture 10oop Lecture 10
oop Lecture 10
 
Video surveillance Moving object detection& tracking Chapter 1
Video surveillance Moving object detection& tracking Chapter 1 Video surveillance Moving object detection& tracking Chapter 1
Video surveillance Moving object detection& tracking Chapter 1
 
Machine learning introduction
Machine learning introductionMachine learning introduction
Machine learning introduction
 
oop Lecture19
oop Lecture19oop Lecture19
oop Lecture19
 
Introduction to oop
Introduction to oop Introduction to oop
Introduction to oop
 
A Multi-robot System Coordination Design and Analysis on Wall Follower Robot ...
A Multi-robot System Coordination Design and Analysis on Wall Follower Robot ...A Multi-robot System Coordination Design and Analysis on Wall Follower Robot ...
A Multi-robot System Coordination Design and Analysis on Wall Follower Robot ...
 
Paper id 6122018109
Paper id 6122018109Paper id 6122018109
Paper id 6122018109
 
Object oriented programming
Object oriented programmingObject oriented programming
Object oriented programming
 
TAAI 2016 Keynote Talk: Intercultural Collaboration as a Multi‐Agent System
TAAI 2016 Keynote Talk: Intercultural Collaboration as a Multi‐Agent SystemTAAI 2016 Keynote Talk: Intercultural Collaboration as a Multi‐Agent System
TAAI 2016 Keynote Talk: Intercultural Collaboration as a Multi‐Agent System
 
Oop concept
Oop conceptOop concept
Oop concept
 
Spike timing dependent plasticity to make robot navigation more intelligent. ...
Spike timing dependent plasticity to make robot navigation more intelligent. ...Spike timing dependent plasticity to make robot navigation more intelligent. ...
Spike timing dependent plasticity to make robot navigation more intelligent. ...
 
Lecture Notes in Computer Science (LNCS)
Lecture Notes in Computer Science (LNCS)Lecture Notes in Computer Science (LNCS)
Lecture Notes in Computer Science (LNCS)
 

Similar to Tool Use Learning for a Real Robot

A cognitive robot equipped with autonomous tool innovation expertise
A cognitive robot equipped with autonomous tool  innovation expertise A cognitive robot equipped with autonomous tool  innovation expertise
A cognitive robot equipped with autonomous tool innovation expertise IJECEIAES
 
Artificial intelligence apporoach to robotics
Artificial intelligence apporoach to roboticsArtificial intelligence apporoach to robotics
Artificial intelligence apporoach to roboticsEr. rahul abhishek
 
Object Detection and Tracking AI Robot
Object Detection and Tracking AI RobotObject Detection and Tracking AI Robot
Object Detection and Tracking AI RobotIRJET Journal
 
Learning of robot navigation tasks by
Learning of robot navigation tasks byLearning of robot navigation tasks by
Learning of robot navigation tasks bycsandit
 
LEARNING OF ROBOT NAVIGATION TASKS BY PROBABILISTIC NEURAL NETWORK
LEARNING OF ROBOT NAVIGATION TASKS BY PROBABILISTIC NEURAL NETWORKLEARNING OF ROBOT NAVIGATION TASKS BY PROBABILISTIC NEURAL NETWORK
LEARNING OF ROBOT NAVIGATION TASKS BY PROBABILISTIC NEURAL NETWORKcsandit
 
LEARNING OF ROBOT NAVIGATION TASKS BY PROBABILISTIC NEURAL NETWORK
LEARNING OF ROBOT NAVIGATION TASKS BY PROBABILISTIC NEURAL NETWORKLEARNING OF ROBOT NAVIGATION TASKS BY PROBABILISTIC NEURAL NETWORK
LEARNING OF ROBOT NAVIGATION TASKS BY PROBABILISTIC NEURAL NETWORKcscpconf
 
Wireless Pick & Place Robot
Wireless Pick & Place RobotWireless Pick & Place Robot
Wireless Pick & Place RobotMarmik Kothari
 
Action Recognition using Nonnegative Action
Action Recognition using Nonnegative ActionAction Recognition using Nonnegative Action
Action Recognition using Nonnegative Actionsuthi
 
Java Progamming Paradigms, OOPS Concept, Introduction to Java, Structure of J...
Java Progamming Paradigms, OOPS Concept, Introduction to Java, Structure of J...Java Progamming Paradigms, OOPS Concept, Introduction to Java, Structure of J...
Java Progamming Paradigms, OOPS Concept, Introduction to Java, Structure of J...Sakthi Durai
 
Java Programming Paradigms Chapter 1
Java Programming Paradigms Chapter 1 Java Programming Paradigms Chapter 1
Java Programming Paradigms Chapter 1 Sakthi Durai
 
CNIT 581 - SDR Proposal Report
CNIT 581 - SDR Proposal ReportCNIT 581 - SDR Proposal Report
CNIT 581 - SDR Proposal ReportHaley Jang
 
Towards Machine Learning of Motor Skills
Towards Machine Learning of Motor SkillsTowards Machine Learning of Motor Skills
Towards Machine Learning of Motor Skillsbutest
 
Optimally Learnt, Neural Network Based Autonomous Mobile Robot Navigation System
Optimally Learnt, Neural Network Based Autonomous Mobile Robot Navigation SystemOptimally Learnt, Neural Network Based Autonomous Mobile Robot Navigation System
Optimally Learnt, Neural Network Based Autonomous Mobile Robot Navigation SystemIDES Editor
 
Swarm robotics : Design and implementation
Swarm robotics : Design and implementationSwarm robotics : Design and implementation
Swarm robotics : Design and implementationIJECEIAES
 
Scheme for motion estimation based on adaptive fuzzy neural network
Scheme for motion estimation based on adaptive fuzzy neural networkScheme for motion estimation based on adaptive fuzzy neural network
Scheme for motion estimation based on adaptive fuzzy neural networkTELKOMNIKA JOURNAL
 
Universal Artificial Intelligence for Intelligent Agents: An Approach to Supe...
Universal Artificial Intelligence for Intelligent Agents: An Approach to Supe...Universal Artificial Intelligence for Intelligent Agents: An Approach to Supe...
Universal Artificial Intelligence for Intelligent Agents: An Approach to Supe...IOSR Journals
 
[IROS2017] Online Spatial Concept and Lexical Acquisition with Simultaneous L...
[IROS2017] Online Spatial Concept and Lexical Acquisition with Simultaneous L...[IROS2017] Online Spatial Concept and Lexical Acquisition with Simultaneous L...
[IROS2017] Online Spatial Concept and Lexical Acquisition with Simultaneous L...Akira Taniguchi
 

Similar to Tool Use Learning for a Real Robot (20)

A cognitive robot equipped with autonomous tool innovation expertise
A cognitive robot equipped with autonomous tool  innovation expertise A cognitive robot equipped with autonomous tool  innovation expertise
A cognitive robot equipped with autonomous tool innovation expertise
 
W.doc
W.docW.doc
W.doc
 
liu2019.pdf
liu2019.pdfliu2019.pdf
liu2019.pdf
 
Artificial intelligence apporoach to robotics
Artificial intelligence apporoach to roboticsArtificial intelligence apporoach to robotics
Artificial intelligence apporoach to robotics
 
Object Detection and Tracking AI Robot
Object Detection and Tracking AI RobotObject Detection and Tracking AI Robot
Object Detection and Tracking AI Robot
 
Learning of robot navigation tasks by
Learning of robot navigation tasks byLearning of robot navigation tasks by
Learning of robot navigation tasks by
 
LEARNING OF ROBOT NAVIGATION TASKS BY PROBABILISTIC NEURAL NETWORK
LEARNING OF ROBOT NAVIGATION TASKS BY PROBABILISTIC NEURAL NETWORKLEARNING OF ROBOT NAVIGATION TASKS BY PROBABILISTIC NEURAL NETWORK
LEARNING OF ROBOT NAVIGATION TASKS BY PROBABILISTIC NEURAL NETWORK
 
LEARNING OF ROBOT NAVIGATION TASKS BY PROBABILISTIC NEURAL NETWORK
LEARNING OF ROBOT NAVIGATION TASKS BY PROBABILISTIC NEURAL NETWORKLEARNING OF ROBOT NAVIGATION TASKS BY PROBABILISTIC NEURAL NETWORK
LEARNING OF ROBOT NAVIGATION TASKS BY PROBABILISTIC NEURAL NETWORK
 
Wireless Pick & Place Robot
Wireless Pick & Place RobotWireless Pick & Place Robot
Wireless Pick & Place Robot
 
Action Recognition using Nonnegative Action
Action Recognition using Nonnegative ActionAction Recognition using Nonnegative Action
Action Recognition using Nonnegative Action
 
Java Progamming Paradigms, OOPS Concept, Introduction to Java, Structure of J...
Java Progamming Paradigms, OOPS Concept, Introduction to Java, Structure of J...Java Progamming Paradigms, OOPS Concept, Introduction to Java, Structure of J...
Java Progamming Paradigms, OOPS Concept, Introduction to Java, Structure of J...
 
Java Programming Paradigms Chapter 1
Java Programming Paradigms Chapter 1 Java Programming Paradigms Chapter 1
Java Programming Paradigms Chapter 1
 
Robot nao
Robot naoRobot nao
Robot nao
 
CNIT 581 - SDR Proposal Report
CNIT 581 - SDR Proposal ReportCNIT 581 - SDR Proposal Report
CNIT 581 - SDR Proposal Report
 
Towards Machine Learning of Motor Skills
Towards Machine Learning of Motor SkillsTowards Machine Learning of Motor Skills
Towards Machine Learning of Motor Skills
 
Optimally Learnt, Neural Network Based Autonomous Mobile Robot Navigation System
Optimally Learnt, Neural Network Based Autonomous Mobile Robot Navigation SystemOptimally Learnt, Neural Network Based Autonomous Mobile Robot Navigation System
Optimally Learnt, Neural Network Based Autonomous Mobile Robot Navigation System
 
Swarm robotics : Design and implementation
Swarm robotics : Design and implementationSwarm robotics : Design and implementation
Swarm robotics : Design and implementation
 
Scheme for motion estimation based on adaptive fuzzy neural network
Scheme for motion estimation based on adaptive fuzzy neural networkScheme for motion estimation based on adaptive fuzzy neural network
Scheme for motion estimation based on adaptive fuzzy neural network
 
Universal Artificial Intelligence for Intelligent Agents: An Approach to Supe...
Universal Artificial Intelligence for Intelligent Agents: An Approach to Supe...Universal Artificial Intelligence for Intelligent Agents: An Approach to Supe...
Universal Artificial Intelligence for Intelligent Agents: An Approach to Supe...
 
[IROS2017] Online Spatial Concept and Lexical Acquisition with Simultaneous L...
[IROS2017] Online Spatial Concept and Lexical Acquisition with Simultaneous L...[IROS2017] Online Spatial Concept and Lexical Acquisition with Simultaneous L...
[IROS2017] Online Spatial Concept and Lexical Acquisition with Simultaneous L...
 

More from IJECEIAES

Remote field-programmable gate array laboratory for signal acquisition and de...
Remote field-programmable gate array laboratory for signal acquisition and de...Remote field-programmable gate array laboratory for signal acquisition and de...
Remote field-programmable gate array laboratory for signal acquisition and de...IJECEIAES
 
Detecting and resolving feature envy through automated machine learning and m...
Detecting and resolving feature envy through automated machine learning and m...Detecting and resolving feature envy through automated machine learning and m...
Detecting and resolving feature envy through automated machine learning and m...IJECEIAES
 
Smart monitoring technique for solar cell systems using internet of things ba...
Smart monitoring technique for solar cell systems using internet of things ba...Smart monitoring technique for solar cell systems using internet of things ba...
Smart monitoring technique for solar cell systems using internet of things ba...IJECEIAES
 
An efficient security framework for intrusion detection and prevention in int...
An efficient security framework for intrusion detection and prevention in int...An efficient security framework for intrusion detection and prevention in int...
An efficient security framework for intrusion detection and prevention in int...IJECEIAES
 
Developing a smart system for infant incubators using the internet of things ...
Developing a smart system for infant incubators using the internet of things ...Developing a smart system for infant incubators using the internet of things ...
Developing a smart system for infant incubators using the internet of things ...IJECEIAES
 
A review on internet of things-based stingless bee's honey production with im...
A review on internet of things-based stingless bee's honey production with im...A review on internet of things-based stingless bee's honey production with im...
A review on internet of things-based stingless bee's honey production with im...IJECEIAES
 
A trust based secure access control using authentication mechanism for intero...
A trust based secure access control using authentication mechanism for intero...A trust based secure access control using authentication mechanism for intero...
A trust based secure access control using authentication mechanism for intero...IJECEIAES
 
Fuzzy linear programming with the intuitionistic polygonal fuzzy numbers
Fuzzy linear programming with the intuitionistic polygonal fuzzy numbersFuzzy linear programming with the intuitionistic polygonal fuzzy numbers
Fuzzy linear programming with the intuitionistic polygonal fuzzy numbersIJECEIAES
 
The performance of artificial intelligence in prostate magnetic resonance im...
The performance of artificial intelligence in prostate  magnetic resonance im...The performance of artificial intelligence in prostate  magnetic resonance im...
The performance of artificial intelligence in prostate magnetic resonance im...IJECEIAES
 
Seizure stage detection of epileptic seizure using convolutional neural networks
Seizure stage detection of epileptic seizure using convolutional neural networksSeizure stage detection of epileptic seizure using convolutional neural networks
Seizure stage detection of epileptic seizure using convolutional neural networksIJECEIAES
 
Analysis of driving style using self-organizing maps to analyze driver behavior
Analysis of driving style using self-organizing maps to analyze driver behaviorAnalysis of driving style using self-organizing maps to analyze driver behavior
Analysis of driving style using self-organizing maps to analyze driver behaviorIJECEIAES
 
Hyperspectral object classification using hybrid spectral-spatial fusion and ...
Hyperspectral object classification using hybrid spectral-spatial fusion and ...Hyperspectral object classification using hybrid spectral-spatial fusion and ...
Hyperspectral object classification using hybrid spectral-spatial fusion and ...IJECEIAES
 
Fuzzy logic method-based stress detector with blood pressure and body tempera...
Fuzzy logic method-based stress detector with blood pressure and body tempera...Fuzzy logic method-based stress detector with blood pressure and body tempera...
Fuzzy logic method-based stress detector with blood pressure and body tempera...IJECEIAES
 
SADCNN-ORBM: a hybrid deep learning model based citrus disease detection and ...
SADCNN-ORBM: a hybrid deep learning model based citrus disease detection and ...SADCNN-ORBM: a hybrid deep learning model based citrus disease detection and ...
SADCNN-ORBM: a hybrid deep learning model based citrus disease detection and ...IJECEIAES
 
Performance enhancement of machine learning algorithm for breast cancer diagn...
Performance enhancement of machine learning algorithm for breast cancer diagn...Performance enhancement of machine learning algorithm for breast cancer diagn...
Performance enhancement of machine learning algorithm for breast cancer diagn...IJECEIAES
 
A deep learning framework for accurate diagnosis of colorectal cancer using h...
A deep learning framework for accurate diagnosis of colorectal cancer using h...A deep learning framework for accurate diagnosis of colorectal cancer using h...
A deep learning framework for accurate diagnosis of colorectal cancer using h...IJECEIAES
 
Swarm flip-crossover algorithm: a new swarm-based metaheuristic enriched with...
Swarm flip-crossover algorithm: a new swarm-based metaheuristic enriched with...Swarm flip-crossover algorithm: a new swarm-based metaheuristic enriched with...
Swarm flip-crossover algorithm: a new swarm-based metaheuristic enriched with...IJECEIAES
 
Predicting churn with filter-based techniques and deep learning
Predicting churn with filter-based techniques and deep learningPredicting churn with filter-based techniques and deep learning
Predicting churn with filter-based techniques and deep learningIJECEIAES
 
Taxi-out time prediction at Mohammed V Casablanca Airport
Taxi-out time prediction at Mohammed V Casablanca AirportTaxi-out time prediction at Mohammed V Casablanca Airport
Taxi-out time prediction at Mohammed V Casablanca AirportIJECEIAES
 
Automatic customer review summarization using deep learningbased hybrid senti...
Automatic customer review summarization using deep learningbased hybrid senti...Automatic customer review summarization using deep learningbased hybrid senti...
Automatic customer review summarization using deep learningbased hybrid senti...IJECEIAES
 

More from IJECEIAES (20)

Remote field-programmable gate array laboratory for signal acquisition and de...
Remote field-programmable gate array laboratory for signal acquisition and de...Remote field-programmable gate array laboratory for signal acquisition and de...
Remote field-programmable gate array laboratory for signal acquisition and de...
 
Detecting and resolving feature envy through automated machine learning and m...
Detecting and resolving feature envy through automated machine learning and m...Detecting and resolving feature envy through automated machine learning and m...
Detecting and resolving feature envy through automated machine learning and m...
 
Smart monitoring technique for solar cell systems using internet of things ba...
Smart monitoring technique for solar cell systems using internet of things ba...Smart monitoring technique for solar cell systems using internet of things ba...
Smart monitoring technique for solar cell systems using internet of things ba...
 
An efficient security framework for intrusion detection and prevention in int...
An efficient security framework for intrusion detection and prevention in int...An efficient security framework for intrusion detection and prevention in int...
An efficient security framework for intrusion detection and prevention in int...
 
Developing a smart system for infant incubators using the internet of things ...
Developing a smart system for infant incubators using the internet of things ...Developing a smart system for infant incubators using the internet of things ...
Developing a smart system for infant incubators using the internet of things ...
 
A review on internet of things-based stingless bee's honey production with im...
A review on internet of things-based stingless bee's honey production with im...A review on internet of things-based stingless bee's honey production with im...
A review on internet of things-based stingless bee's honey production with im...
 
A trust based secure access control using authentication mechanism for intero...
A trust based secure access control using authentication mechanism for intero...A trust based secure access control using authentication mechanism for intero...
A trust based secure access control using authentication mechanism for intero...
 
Fuzzy linear programming with the intuitionistic polygonal fuzzy numbers
Fuzzy linear programming with the intuitionistic polygonal fuzzy numbersFuzzy linear programming with the intuitionistic polygonal fuzzy numbers
Fuzzy linear programming with the intuitionistic polygonal fuzzy numbers
 
The performance of artificial intelligence in prostate magnetic resonance im...
The performance of artificial intelligence in prostate  magnetic resonance im...The performance of artificial intelligence in prostate  magnetic resonance im...
The performance of artificial intelligence in prostate magnetic resonance im...
 
Seizure stage detection of epileptic seizure using convolutional neural networks
Seizure stage detection of epileptic seizure using convolutional neural networksSeizure stage detection of epileptic seizure using convolutional neural networks
Seizure stage detection of epileptic seizure using convolutional neural networks
 
Analysis of driving style using self-organizing maps to analyze driver behavior
Analysis of driving style using self-organizing maps to analyze driver behaviorAnalysis of driving style using self-organizing maps to analyze driver behavior
Analysis of driving style using self-organizing maps to analyze driver behavior
 
Hyperspectral object classification using hybrid spectral-spatial fusion and ...
Hyperspectral object classification using hybrid spectral-spatial fusion and ...Hyperspectral object classification using hybrid spectral-spatial fusion and ...
Hyperspectral object classification using hybrid spectral-spatial fusion and ...
 
Fuzzy logic method-based stress detector with blood pressure and body tempera...
Fuzzy logic method-based stress detector with blood pressure and body tempera...Fuzzy logic method-based stress detector with blood pressure and body tempera...
Fuzzy logic method-based stress detector with blood pressure and body tempera...
 
SADCNN-ORBM: a hybrid deep learning model based citrus disease detection and ...
SADCNN-ORBM: a hybrid deep learning model based citrus disease detection and ...SADCNN-ORBM: a hybrid deep learning model based citrus disease detection and ...
SADCNN-ORBM: a hybrid deep learning model based citrus disease detection and ...
 
Performance enhancement of machine learning algorithm for breast cancer diagn...
Performance enhancement of machine learning algorithm for breast cancer diagn...Performance enhancement of machine learning algorithm for breast cancer diagn...
Performance enhancement of machine learning algorithm for breast cancer diagn...
 
A deep learning framework for accurate diagnosis of colorectal cancer using h...
A deep learning framework for accurate diagnosis of colorectal cancer using h...A deep learning framework for accurate diagnosis of colorectal cancer using h...
A deep learning framework for accurate diagnosis of colorectal cancer using h...
 
Swarm flip-crossover algorithm: a new swarm-based metaheuristic enriched with...
Swarm flip-crossover algorithm: a new swarm-based metaheuristic enriched with...Swarm flip-crossover algorithm: a new swarm-based metaheuristic enriched with...
Swarm flip-crossover algorithm: a new swarm-based metaheuristic enriched with...
 
Predicting churn with filter-based techniques and deep learning
Predicting churn with filter-based techniques and deep learningPredicting churn with filter-based techniques and deep learning
Predicting churn with filter-based techniques and deep learning
 
Taxi-out time prediction at Mohammed V Casablanca Airport
Taxi-out time prediction at Mohammed V Casablanca AirportTaxi-out time prediction at Mohammed V Casablanca Airport
Taxi-out time prediction at Mohammed V Casablanca Airport
 
Automatic customer review summarization using deep learningbased hybrid senti...
Automatic customer review summarization using deep learningbased hybrid senti...Automatic customer review summarization using deep learningbased hybrid senti...
Automatic customer review summarization using deep learningbased hybrid senti...
 

Recently uploaded

Seismic Hazard Assessment Software in Python by Prof. Dr. Costas Sachpazis
Seismic Hazard Assessment Software in Python by Prof. Dr. Costas SachpazisSeismic Hazard Assessment Software in Python by Prof. Dr. Costas Sachpazis
Seismic Hazard Assessment Software in Python by Prof. Dr. Costas SachpazisDr.Costas Sachpazis
 
Research Methodolgy & Intellectual Property Rights Series 2
Research Methodolgy & Intellectual Property Rights Series 2Research Methodolgy & Intellectual Property Rights Series 2
Research Methodolgy & Intellectual Property Rights Series 2T.D. Shashikala
 
Insurance management system project report.pdf
Insurance management system project report.pdfInsurance management system project report.pdf
Insurance management system project report.pdfKamal Acharya
 
15-Minute City: A Completely New Horizon
15-Minute City: A Completely New Horizon15-Minute City: A Completely New Horizon
15-Minute City: A Completely New HorizonMorshed Ahmed Rahath
 
Software Engineering - Modelling Concepts + Class Modelling + Building the An...
Software Engineering - Modelling Concepts + Class Modelling + Building the An...Software Engineering - Modelling Concepts + Class Modelling + Building the An...
Software Engineering - Modelling Concepts + Class Modelling + Building the An...Prakhyath Rai
 
Linux Systems Programming: Semaphores, Shared Memory, and Message Queues
Linux Systems Programming: Semaphores, Shared Memory, and Message QueuesLinux Systems Programming: Semaphores, Shared Memory, and Message Queues
Linux Systems Programming: Semaphores, Shared Memory, and Message QueuesRashidFaridChishti
 
5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...archanaece3
 
Online crime reporting system project.pdf
Online crime reporting system project.pdfOnline crime reporting system project.pdf
Online crime reporting system project.pdfKamal Acharya
 
Piping and instrumentation diagram p.pdf
Piping and instrumentation diagram p.pdfPiping and instrumentation diagram p.pdf
Piping and instrumentation diagram p.pdfAshrafRagab14
 
BRAKING SYSTEM IN INDIAN RAILWAY AutoCAD DRAWING
BRAKING SYSTEM IN INDIAN RAILWAY AutoCAD DRAWINGBRAKING SYSTEM IN INDIAN RAILWAY AutoCAD DRAWING
BRAKING SYSTEM IN INDIAN RAILWAY AutoCAD DRAWINGKOUSTAV SARKAR
 
NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024
NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024
NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024EMMANUELLEFRANCEHELI
 
Circuit Breaker arc phenomenon.pdf engineering
Circuit Breaker arc phenomenon.pdf engineeringCircuit Breaker arc phenomenon.pdf engineering
Circuit Breaker arc phenomenon.pdf engineeringKanchhaTamang
 
Theory for How to calculation capacitor bank
Theory for How to calculation capacitor bankTheory for How to calculation capacitor bank
Theory for How to calculation capacitor banktawat puangthong
 
EMPLOYEE MANAGEMENT SYSTEM FINAL presentation
EMPLOYEE MANAGEMENT SYSTEM FINAL presentationEMPLOYEE MANAGEMENT SYSTEM FINAL presentation
EMPLOYEE MANAGEMENT SYSTEM FINAL presentationAmayJaiswal4
 
analog-vs-digital-communication (concept of analog and digital).pptx
analog-vs-digital-communication (concept of analog and digital).pptxanalog-vs-digital-communication (concept of analog and digital).pptx
analog-vs-digital-communication (concept of analog and digital).pptxKarpagam Institute of Teechnology
 
Geometric constructions Engineering Drawing.pdf
Geometric constructions Engineering Drawing.pdfGeometric constructions Engineering Drawing.pdf
Geometric constructions Engineering Drawing.pdfJNTUA
 
Supermarket billing system project report..pdf
Supermarket billing system project report..pdfSupermarket billing system project report..pdf
Supermarket billing system project report..pdfKamal Acharya
 
Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)NareenAsad
 
ChatGPT Prompt Engineering for project managers.pdf
ChatGPT Prompt Engineering for project managers.pdfChatGPT Prompt Engineering for project managers.pdf
ChatGPT Prompt Engineering for project managers.pdfqasastareekh
 
E-Commerce Shopping using MERN Stack where different modules are present
E-Commerce Shopping using MERN Stack where different modules are presentE-Commerce Shopping using MERN Stack where different modules are present
E-Commerce Shopping using MERN Stack where different modules are presentjatinraor66
 

Recently uploaded (20)

Seismic Hazard Assessment Software in Python by Prof. Dr. Costas Sachpazis
Seismic Hazard Assessment Software in Python by Prof. Dr. Costas SachpazisSeismic Hazard Assessment Software in Python by Prof. Dr. Costas Sachpazis
Seismic Hazard Assessment Software in Python by Prof. Dr. Costas Sachpazis
 
Research Methodolgy & Intellectual Property Rights Series 2
Research Methodolgy & Intellectual Property Rights Series 2Research Methodolgy & Intellectual Property Rights Series 2
Research Methodolgy & Intellectual Property Rights Series 2
 
Insurance management system project report.pdf
Insurance management system project report.pdfInsurance management system project report.pdf
Insurance management system project report.pdf
 
15-Minute City: A Completely New Horizon
15-Minute City: A Completely New Horizon15-Minute City: A Completely New Horizon
15-Minute City: A Completely New Horizon
 
Software Engineering - Modelling Concepts + Class Modelling + Building the An...
Software Engineering - Modelling Concepts + Class Modelling + Building the An...Software Engineering - Modelling Concepts + Class Modelling + Building the An...
Software Engineering - Modelling Concepts + Class Modelling + Building the An...
 
Linux Systems Programming: Semaphores, Shared Memory, and Message Queues
Linux Systems Programming: Semaphores, Shared Memory, and Message QueuesLinux Systems Programming: Semaphores, Shared Memory, and Message Queues
Linux Systems Programming: Semaphores, Shared Memory, and Message Queues
 
5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...
 
Online crime reporting system project.pdf
Online crime reporting system project.pdfOnline crime reporting system project.pdf
Online crime reporting system project.pdf
 
Piping and instrumentation diagram p.pdf
Piping and instrumentation diagram p.pdfPiping and instrumentation diagram p.pdf
Piping and instrumentation diagram p.pdf
 
BRAKING SYSTEM IN INDIAN RAILWAY AutoCAD DRAWING
BRAKING SYSTEM IN INDIAN RAILWAY AutoCAD DRAWINGBRAKING SYSTEM IN INDIAN RAILWAY AutoCAD DRAWING
BRAKING SYSTEM IN INDIAN RAILWAY AutoCAD DRAWING
 
NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024
NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024
NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024
 
Circuit Breaker arc phenomenon.pdf engineering
Circuit Breaker arc phenomenon.pdf engineeringCircuit Breaker arc phenomenon.pdf engineering
Circuit Breaker arc phenomenon.pdf engineering
 
Theory for How to calculation capacitor bank
Theory for How to calculation capacitor bankTheory for How to calculation capacitor bank
Theory for How to calculation capacitor bank
 
EMPLOYEE MANAGEMENT SYSTEM FINAL presentation
EMPLOYEE MANAGEMENT SYSTEM FINAL presentationEMPLOYEE MANAGEMENT SYSTEM FINAL presentation
EMPLOYEE MANAGEMENT SYSTEM FINAL presentation
 
analog-vs-digital-communication (concept of analog and digital).pptx
analog-vs-digital-communication (concept of analog and digital).pptxanalog-vs-digital-communication (concept of analog and digital).pptx
analog-vs-digital-communication (concept of analog and digital).pptx
 
Geometric constructions Engineering Drawing.pdf
Geometric constructions Engineering Drawing.pdfGeometric constructions Engineering Drawing.pdf
Geometric constructions Engineering Drawing.pdf
 
Supermarket billing system project report..pdf
Supermarket billing system project report..pdfSupermarket billing system project report..pdf
Supermarket billing system project report..pdf
 
Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)
 
ChatGPT Prompt Engineering for project managers.pdf
ChatGPT Prompt Engineering for project managers.pdfChatGPT Prompt Engineering for project managers.pdf
ChatGPT Prompt Engineering for project managers.pdf
 
E-Commerce Shopping using MERN Stack where different modules are present
E-Commerce Shopping using MERN Stack where different modules are presentE-Commerce Shopping using MERN Stack where different modules are present
E-Commerce Shopping using MERN Stack where different modules are present
 

Tool Use Learning for a Real Robot

  • 1. International Journal of Electrical and Computer Engineering (IJECE) Vol. 8, No. 2, April 2018, pp. 1230 – 1237 ISSN: 2088-8708, DOI: 10.11591/ijece.v8i2.pp1230-1237 1230 Tool Use Learning for a Real Robot Handy Wicaksono1,2 and Claude Sammut1 1 School of Computer Science and Engineering, University of New South Wales; Sydney, Australia 2 Electrical Engineering Department, Petra Christian University; Surabaya, Indonesia Article Info Article history: Received July 24, 2017 Revised December 16, 2017 Accepted December 27, 2017 Keyword: tool use by a robot tool use learning action learning inductive logic programming robot software architecture ABSTRACT A robot may need to use a tool to solve a complex problem. Currently, tool use must be pre-programmed by a human. However, this is a difficult task and can be helped if the robot is able to learn how to use a tool by itself. Most of the work in tool use learning by a robot is done using a feature-based representation. Despite many successful results, this representa- tion is limited in the types of tools and tasks that can be handled. Furthermore, the complex relationship between a tool and other world objects cannot be captured easily. Relational learning methods have been proposed to overcome these weaknesses [1, 2]. However, they have only been evaluated in a sensor-less simulation to avoid the complexities and uncer- tainties of the real world. We present a real world implementation of a relational tool use learning system for a robot. In our experiment, a robot requires around ten examples to learn to use a hook-like tool to pull a cube from a narrow tube. Copyright c 2018 Institute of Advanced Engineering and Science. All rights reserved. Corresponding Author: Handy Wicaksono School of Computer Science and Engineering, University of New South Wales handyw@cse.unsw.edu.au 1. INTRODUCTION Humans use tools to help them complete everyday tasks. The ability to use tools is a feature of humans intelligence [3]. Like humans, a robot also needs to be able to use a tool to solve a complex task. As an example, in the RoboCup@Home competition, a robot is asked to demonstrate several tool use abilities such as opening a bottle by using a bottle opener or watering a plant with a watering can [4]. The robot is given complete knowledge of the tools and how to use them. When such knowledge is not available, a robot must learn it. Most work in tool use learning has used a feature-based representation that is dependent on an object’s primitive features. Thus, it is not flexible enough to be applied to different tools and environments. Learning is also limited to tool selection only. Brown proposed a relational approach which can overcome these limitations. Furthermore, Wicaksono & Sammut ([5]) have suggested that this representation has potential to solve a more complex problem, such as tool creation. We define a tool as an object that is deliberately employed by an agent to help it achieve a goal, which would be too difficult to achieve without the tool. We want to learn a tool action model that explains changes in the properties of one or more objects affected by the tool, given that certain preconditions are met. Following Brown [1], learning is performed by trial and error in an Inductive Logic Programming (ILP) setting [6]. Brown [1] carried out tool use learning in a sensor-less simulation. This means that a robot has perfect knowledge of the world and uncertainties in the sensors’ readings are eliminated. In this paper, we would present a complete robotic system for tool use learning following a relational learning approach. This includes three components: 1. Developing a robot software architecture that consists of primitive and abstract layers and facilitates communi- cation between them. 2. Creating a mechanism to detect objects and generating primitive behaviors for tool use. 3. Extending relational learning methods and conducting tool use learning experiments using a real robot. In the following sections, we describe relevant previous work, the knowledge representation formalism used, the software architecture, and the tool use learning mechanism. Finally, we perform several real world experiments and draw conclusions. Journal Homepage: http://iaesjournal.com/online/index.php/IJECE
  • 2. Int J Elec & Comp Eng ISSN: 2088-8708 1231 2. RELATED WORK An intelligent robot is identified by its ability to make a decision and learn autonomously in an unstructured environment. Machine learning techniques could be useful as they can compensate for the imperfect model of a robot in a real world. For example, control of a robot manipulator can be improved by equipping a PD torque controller with Adaptive Neuro-Fuzzy Inference System [7] or combining sliding mode control with genetic algorithm [8]. In a behavior-based robotics approach, reinforcement learning can be used to learn the robot’s behavior [9]. Here we describe the previous work in tool use learning by a robot. Wood et al. [10] use a neural network to learn appropriate posture of a Sony Aibo robot so it can reach an object by using a tool placed on its back. Stoytchev [11] learns to select a correct tool via its affordances which are grounded in robot behaviors. However, its result can not be generalized to other new tools. More recent work by Tikhanoff et al. [12]) combine exploratory behaviors and a geometrical feature extraction to learn affordances and tool use. Mar et al. [13] extend their work by learning a grasp configuration which influences the outcome of a tool use action. The limitations of feature-based representations were mentioned above. Only recently have such representa- tions been extended so that a robot can learn to grasp a tool [13]. An ILP system can overcome these limitations, as it represents the tools and other objects in a relational representation [1, 2]. However, previous work has been done previously in sensor-less simulation only. This means the complexities of acquiring perceptions, generating precise movements, and dealing with world uncertainties, are avoided. We aim to develop a complete robotic system that facilitates this learning in a real world. 3. REPRESENTING STATES AND ACTIONS We maintain two representations of states and actions in primitive and abstract form. Primitive states are the positions of all objects in the world that are captured by vision sensors using a mechanism described in section 4.2.. As we only use simple objects, they can be detected by their two-dimensional appearance only. Abstract states are represented as expressions in first-order logic, to be more specific as Horn clauses. Primi- tive states are translated to an abstract state by calling relevant Prolog programs with the primitive states as their bound values. To be classified as a tool, an object must possess particular structural properties (e.g. a particular side where a hook is attached to the handle) and spatial properties (e.g. a hook is touching the back side of a cube). We collect these properties in a hypothesis, namely tool pose, which is shown below in simplified form. t o o l p o s e ( Handle , Hook , Box , Tube ): − a t t a c h e d e n d ( Handle , Hook , back ) , % a s t r u c t u r a l p r o p e r t y a t t a c h e d s i d e ( Handle , Hook , Side ) , % a s t r u c t u r a l p r o p e r t y touching ( Hook , Box , back ) . % a s p a t i a l p r o p e r t y We use an extended STRIPS action model [14] to describe an abstract action and how it affects the world. A primitive goal that has to be achieved by a robot is added at the end of the model. Action name PRE : states that must be true so that an action can be performed ADD : conditions that become true as a result of the action DELETE : conditions that are no longer true following the action CONSTRAINTS : the physical limits of actuators that constrain the action This action model is also a planning operator. Thus, action learning does not happen in isolation, as it is a part of a problem solving scenario. Every abstract action is linked to a set of primitive behaviors. This will be discussed later in section 4.3. 4. ROBOTIC SYSTEM In this section, we explain our software architecture, objects detection method, and behavior generation mechanism. 4.1. Software Architecture We use a relational representation in the high-level layer to take advantage of the expressiveness of first-order logic (FOL) clauses, especially Horn clauses, and equip a robot with useful symbolic techniques. We implement this layer in SWI Prolog. Most modern Prolog implementations, including SWI, incorporate a constraint solver which provide advanced numerical reasoning capabilities. Tool Use Learning for a Real Robot(Handy Wicaksono)
  • 3. 1232 ISSN: 2088-8708 In the low-level layer, a robot operates in a world of continuous, noisy and uncertain measurements. Its sensors return readings in numeric ranges, and its actuators operate based on numerical set points. As we aim to use Robot Operating System (ROS1 ) as our middleware, we implement this layer in Python. The Baxter robot, from Re- think Robotics, has its own ROS-Python API, a set of classes that provides wrappers around the ROS communication. Another layer, namely the translator, is needed to map primitive to abstract states and to link an abstract action to corresponding primitive behaviors. It is written in C++ which has an interface to SWI Prolog as the language for the abstract layer. Communication to the primitive layer is done via ROS using a simple publish and subscribe mechanism. Our robot software architecture is shown in Fig. 1a. We give a simple illustration of action execution, namely find tool, which involves communication be- tween layers. In the primitive layer, a cube and a tube are detected, their corners are acquired and published to- gether by a ROS node, namely /translator, as a ROS topic, /pri states. The translator also has a ROS node, /translator, that subscribes to that topic. Then it evaluates the status of the abstract state, in tube (Cube,Tube), by calling the relevant Prolog predicate. In the abstract layer, in tube(Cube,Tube) is called. If it is true, as it is the only the precondition of the find tool action, then the translator will publish it as the ROS topic, /abs action that represents the currently active action. The primitive layer subscribes to this topic. If it recognizes find tool as an active action, then it triggers the corresponding primitive behaviors. See Fig. 1b for detail. Actions translator Abstract Action 1 Constraint Solver / Pose Selector Abstract layer Perception Object Detection Behaviors Generation PID Controller ILP Learner States translator Logical State Planner Abstract Action n Translator Primitive layer Tool Selector Tool identity Abstract actions Tool pose Abstract states Primitive states Pixels Primitive action + parameters Motors commands Primitive states Real World (a) Software architecture objects detection ROS node: /translator cube(12,6,15,9) tube(0,0,15,20) ROS topic: /pri_states call Prolog predicate: in_tube(Cube,Tube) PRE: in_tube(Cube,Tube) STRIPS action model NAME: find_tool EFF: tool_found(Tool) ROS node: /primitive ROS topic: /abs_action publish "find_tool" abstract action behaviors generation Abstract layer Translator Primitive layer go to tool area, find correct tool (b) Simplified example Figure 1. Robot software architecture and its example 4.2. Object Detection We detect all objects (i.e. different tools, a cube, and a tube) by their two-dimensional appearances only. We use the Baxter camera to detect objects locally, while an external web camera provides the global world state. We combine local and global images from both cameras to achieve our goal. The OpenCV library is used for object detection. In pre-processing, Gaussian smoothing is applied. Later, the edges of objects in an image are detected with a Canny Edge detector, and their contours are found. Each contour is tested to check whether it has the properties of a particular object. The contour properties include the number of edges, the area, the angle formed by two adjacent parts, the convexity, and the perimeter. Tools are treated specially as they have more than one shape. After being detected as a tool, the object’s type is determined, and its handle and hook(s) are acquired. All objects are represented 1http://www.ros.org Int J Elec & Comp Eng Vol. 8, No. 2, April 2018: 1230 – 1237
  • 4. Int J Elec & Comp Eng ISSN: 2088-8708 1233 by their minimum and maximum corner points in Cartesian coordinates (Xmin, Ymin, Xmax, Ymax ). The complete mechanism is shown in Algorithm 1. Algorithm 1 Objects detection 1: Capture an image 2: Perform Gaussian smoothing 3: Detect the edges of objects using Canny detector 4: Find the contours of the objects 5: for each contour do 6: Classify the contour 7: if The contour is either cube, tube, or tool then 8: Find its minimum & maximum corner points 9: return all minimum & maximum corner points 4.3. Behavior Generation In our tool use application, a sequence of actions, or a plan, must be performed to achieve the goal, pulling a cube from a narrow tube. Each action uses different strategies (inverse kinematics solving, visual servoing, gripper activation, constraints solving) depending on whether the primitive goal is known or not, and whether the visual information is needed or not. We show an example of the plan in Table 1. Table 1. The actions sequence for pulling a cube from a narrow tube No Action Technique Goal Visual 1 Find the tool Inverse Kinematics (IK) solving, visual servoing Known Yes 2 Grip the tool Close the gripper Known No 3 Position the tool Constraints solving, IK solving Unknown Yes 4 Pull the cube IK solving Known No 5 Move the tool away IK solving Known No 6 Ungrip the tool Open the gripper Known No An abstract action is directly linked to corresponding primitive actions. For simplicity, we only consider movement in two-dimensional Cartesian coordinates. A primitive goal is a target location in that coordinate system. When a goal is known, we use an inverse kinematics (IK) solver to compute the angles of all joints and move the arm towards that goal. However, if it is unknown, we use a constraint solver to create a numerical goal from abstract states which are the preconditions of an abstract action model (see Fig. 2a). To perform an accurate tool picking action, we use a simple version of image-based visual servoing [15], where an error signal is determined directly in terms of image feature parameters. We locate the robot arm in a position where a chosen tool is located in the center of an image captured by Baxter’s wrist camera, so the arm can move downward perpendicularly to take the tool. See Fig. 2b for the detail. Action 1 Action 2 Action n Plan Action Model Constraint Solver IK Solver Controller Abstract goal Primitive goal Goal in joints movement Motor commands (a) Constraints solving to get a primitive goal Detect a chosen tool in the image (see Alg. 1) Is the tool on the center of image? Move arm towards center Go down to pick the tool Yes No Start Finish (b) A visual servoing process Figure 2. behaviors generation Tool Use Learning for a Real Robot(Handy Wicaksono)
  • 5. 1234 ISSN: 2088-8708 5. TOOL USE LEARNING In this section, we describe relational tool use learning [1, 2]. The robot must learn a tool action model that describes the properties of a tool and other objects in the world that enable a robot to use a tool successfully. Specifically, the robot learns a tool pose predicate, which describes the structural and spatial conditions that must be met for the tool to be used. We maintain a version space of hypotheses, following Mitchell [16]. A version space is bounded by the most general hypothesis (hg) and the most specific hypothesis (hs). As we explain below, the version space is used to determine what experiments the robot should perform to generalize or specialize the tool action model. Initially, a robot may not have a complete action model for a tool, so it cannot construct a plan. After a tutor gives a tool use example, the robot segments the observed behavior into discrete actions, matches them to actions already in its background knowledge, and constructs a new action if there is no match for a particular behavior. This construction may not be sufficient to describe the tool action in a general enough form that it can be applied in different situations. Therefore, trial and error learning is performed to refine the initial action model. Trial and error involved performing experiments in which a different tool is selected or its pose is changed to test the current hypothesis for the tool action. In tool selection, an object that has properties that are most similar to a previously useful tool is chosen. More specifically, we test whether an object satisfies all primary structural properties, stored in hg, and most of the secondary ones, stored in hs, given its primitive states. Having a correct tool is useless when it is located in an incorrect position before it trying to pull the target object. We select the pose by solving the constraints of the spatial literals in the tool pose predicate. We check whether the spatial constraints of an object can be solved or not. The unsolved constraints are ignored, and the final result is used as the numerical goal for the robot. After a tool use learning experiment is performed, its result, whether successful or not, is passed to the ILP learner so the hypothesis can be refined. Our learning is derived from Golem [17], an ILP algorithm. Refinement of hs is done by finding the constrained Least General Generalization (LGG) of a positive examples pair, and hg is refined by performing the negative-based reduction. The learning algorithm is a modification of Haber [2] and is shown in Algorithm 2. Algorithm 2 Tool use learning 1: input: new action model M, hg = true, hs = preconditions of M,N trials, K consecutive success 2: while success < K or index < N do 3: e = generate experiment(hs, hg) 4: tool = select tool(hs, hg) // select a tool with highest rank 5: for each ei in e do 6: pose = select pose(ei) // performing constraint solving on preconditions of the relevant action model 7: if pose = null then 8: break 9: success = execute exp(tool, pose) // performing a tool use experiment 10: if success then 11: label pose positive 12: increment cons success 13: hs = generalise hs // via Least General Generalisation 14: else 15: label pose negative 16: cons success ← 0 17: hg = specialise hg // via negative based reduction 18: add ei to training data 19: increment index 6. RESULTS AND DISCUSSIONS The Baxter research robot is used in this experiment (see Fig. 3a). The objects in the scene include a cube, a tube, and five objects of a different shape that potential tools. We also have another set of tool candidates whose hooks are narrower than their handles (see Fig. 3b). We use Baxter’s wrist camera for visual servoing. We add an external web camera facing the tube to provide the global state. We divide the experiments into two stages. Initially, we evaluate the performance of our object detection algorithm and action execution. Later, we conduct tool use learning experiments. Int J Elec & Comp Eng Vol. 8, No. 2, April 2018: 1230 – 1237
  • 6. Int J Elec & Comp Eng ISSN: 2088-8708 1235 web camera Baxter hand's camera 5 tools cube tube (a) Scene details (b) All set of tools Figure 3. Experimental scene 6.1. Object Detection and Execution Experiments The contours of all objects (cube, tube, and tools) are detected, using Baxter’s wrist camera, and their min- imum and maximum points are acquired. These are marked with colored dots in the object corners in Fig. 4. Even when the tool is held by the gripper, our algorithm is still able to detect the object. However, the vision system may fail in a low-light environment. It also assumes that all objects are aligned perpendicular to each other and the camera. baxter_cam: cube detection web_cam: cube detection baxter_cam: tube detection web_cam: tube detection baxter_cam: tools detection web_cam: tools detection 1 web_cam: tools detection 2 web_cam: selected tool detection Figure 4. Objects detection in images captured by the Baxter hand camera and the web camera We also evaluate the capability of the Baxter to move accurately to accomplish a tool use task based on the actions sequence in Table 1. In Fig. 5 we illustrate these actions: find tool action which uses visual servoing, grip tool action, which activates the robot’s gripper, and position tool actions, which tries to satisfy a prim- itive goal given by a constraint solver. From these trials, it was determined that the Baxter can move smoothly to complete the task. We observe that most errors are caused by the perception system, not the action mechanism. find_tool grip_tool position_tool Figure 5. Sample of the robot’s actions 6.2. Learning Experiments In this experiment, we use five different candidate tools with narrow and wide hooks. The cube, as a target object, is located at a particular position inside the tube, which is changed after the Baxter can withdraw it from the tube successfully. We adopt an initial action model which was created by Brown ([1]). Learning by trial and error is then performed to refine the tool pose hypothesis. We stop learning if the robot accomplishes the task three times consecutively. The complete learning sequence is shown in Fig. 6. We need at least ten examples to learn to perform this task. However, more experiments may be needed if any errors occur (e.g. a change in lighting or an invalid move produced by the IK solver). The first positive example is given by a teacher. In the second example, the robot attempts to imitate the tool and location of the first one, but it fails as the cube location is changed. In the third example, the robot finds that the attachment side of the hook should be the same as the location of the cube inside the tube. Later on, the robot still makes mistakes as it tends to choose a tool with a narrow hook. This narrower property is eliminated from hs in the generalization process in Tool Use Learning for a Real Robot(Handy Wicaksono)
  • 7. 1236 ISSN: 2088-8708 example 2: - example 3: + example 4: - example 5: -example 1: + example 7: - example 8: + example 9: + example 10: +example 6: + Figure 6. Tool use learning experiments in a real world the eighth example. Finally, learning is completed in the tenth example, as it performs three consecutive successful experiments. The final hypothesis is shorter (hs is reduced from 14 literals to 11 literals) and more general than the initial hypothesis. 6.3. Discussions Our result is similar to the previous relational learning approach [1], they learn tool-use in 12 experiments, as we use the relatively same mechanism. However, we perform our experiment on a real robot, while the former only do it on a sensor-less simulation. This includes bridging the gap between low-level layer (in ROS environment) and high-level layer (in SWI Prolog). We also build objects detection system that has to deal with the noisy environment. Previous work did not do any detections, as it acquires perfect data from a simulator. Compared to the line of work in feature-based representation, such as work of Tikhanoff et al. [12], our approach can learn faster (theirs needs at least 120 trials in various learning stages) as we can easily incorporate human expertise in the background knowledge. Our experiment scenario is also more complicated, as we locate the target object inside a tube. We exploit the ability of the relational representation to represent a complex relationship between objects compactly. We also learn the tool-pose, where the tool should be located to be able to pull the tool successfully, while it is predefined in their work. There are limitations in our work, especially in the perception system which can only handle 2D images and not robust in changing environments. In this area, previous work [12, 13] is better than ours. Despite these limitations, the experiment demonstrates that the learning system is capable of relational tool use learning in the real world. In other work of us, we also use a physics simulator to assist a real robot performs tool use learning [18]. 7. CONCLUSIONS AND FUTURE WORK We have developed a complete robot system for relational tool use learning in the real world. The primitive, translator and abstract layers have been built, along with with their interfaces. We have also implemented a system to detect objects and generate primitive behaviors by inverse kinematics, a constraint solver and a visual servoing mechanism. Finally, we have extended a tool use learning system, which has been tested in experiments in the real world. For a relatively simple tool use scenario, the robot at needs at least ten examples to learn the tool’s properties. In the future, we want our robot to do tool creation when the available tools cannot be used to solve the current problem. Those tools can then be tested in the simulator, to save time and materials, and manufactured by using a 3D printer. The development of a more robust perception and a more accurate movement systems will improve our system performance. The system will be tested on a wider variety of tool use scenarios, with different tools and different tasks. ACKNOWLEDGEMENT Handy Wicaksono is supported by The Directorate General of Resources for Science, Technology and Higher Education (DG-RSTHE), Ministry of Research, Technology, and Higher Education of the Republic of Indonesia. REFERENCES [1] S. Brown and C. Sammut, “A relational approach to tool-use learning in robots,” in Inductive Logic Programming. Springer, 2012, pp. 1–15. [2] A. Haber, “A system architecture for learning robots,” Ph.D. dissertation, School of Computer Science and Engineering, UNSW Australia, 2015. Int J Elec & Comp Eng Vol. 8, No. 2, April 2018: 1230 – 1237
  • 8. Int J Elec & Comp Eng ISSN: 2088-8708 1237 [3] S. H. Johnson-Frey, “What’s so special about human tool use?” Neuron, vol. 39, no. 2, pp. 201 – 204, 2003. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0896627303004240 [4] J. Stckler and S. Behnke, “Adaptive tool-use strategies for anthropomorphic service robots,” in 2014 IEEE-RAS International Conference on Humanoid Robots, Nov 2014, pp. 755–760. [5] H. Wicaksono and C. Sammut, “A learning framework for tool creation by a robot,” in Proceedings of ACRA, 2015. [6] S. Muggleton, “Inductive logic programming,” New Generation Computing, vol. 8, no. 4, pp. 295–318, 1991. [7] O. Bachir and A.-f. Zoubir, “Adaptive neuro-fuzzy inference system based control of puma 600 robot manipula- tor,” International Journal of Electrical and Computer Engineering, vol. 2, no. 1, p. 90, 2012. [8] A. R. Firdaus and A. S. Rahman, “Genetic algorithm of sliding mode control design for manipulator robot,” TELKOMNIKA (Telecommunication Computing Electronics and Control), vol. 10, no. 4, pp. 645–654, 2012. [9] H. Wicaksono, H. Khoswanto, and S. Kuswadi, “Behaviors coordination and learning on autonomous navigation of physical robot,” TELKOMNIKA (Telecommunication Computing Electronics and Control), vol. 9, no. 3, pp. 473–482, 2013. [10] A. Wood, T. Horton, and R. Amant, “Effective tool use in a habile agent,” in Systems and Information Engineer- ing Design Symposium, 2005 IEEE, April 2005, pp. 75–81. [11] A. Stoytchev, “Behavior-grounded representation of tool affordances,” in Proceedings of the 2005 IEEE Interna- tional Conference on Robotics and Automation, April 2005, pp. 3060–3065. [12] V. Tikhanoff, U. Pattacini, L. Natale, and G. Metta, “Exploring affordances and tool use on the icub,” in 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids), Oct 2013, pp. 130–137. [13] T. Mar, V. Tikhanoff, G. Metta, and L. Natale, “Self-supervised learning of grasp dependent tool affordances on the icub humanoid robot,” in 2015 IEEE International Conference on Robotics and Automation (ICRA), May 2015, pp. 3200–3206. [14] R. Fikes and N. J. Nilsson, “STRIPS: A new approach to the application of theorem proving to problem solving,” Artif. Intell., vol. 2, no. 3/4, pp. 189–208, 1971. [15] S. Hutchinson, G. D. Hager, and P. I. Corke, “A tutorial on visual servo control,” IEEE transactions on robotics and automation, vol. 12, no. 5, pp. 651–670, 1996. [16] T. M. Mitchell, “Version spaces: A candidate elimination approach to rule learning,” in Proceedings of the 5th international joint conference on Artificial intelligence-Volume 1. Morgan Kaufmann Publishers Inc., 1977, pp. 305–310. [17] S. Muggleton and C. Feng, “Efficient induction in logic programs,” in Inductive Logic Programming, S. Mug- gleton, Ed. Academic Press, 1992, pp. 281–298. [18] H. Wicaksono and C. Sammut, “Relational tool use learning by a robot in a real and simulated world,” in Pro- ceedings of ACRA, 2016. BIOGRAPHY OF AUTHORS Handy Wicaksono is a Ph.D. student at Artificial Intelligence Group, School of Computer Science and Engineering, UNSW Australia, with bachelor and master degree in Electrical Engineering from Institut Teknologi Sepuluh Nopember, Indonesia. He is also a lecturer in Electrical Engineering De- partment, Petra Christian University, Indonesia. His research is in the area of artificial intelligence, intelligent robot, and industrial automation. Claude Sammut is a Professor of Computer Science and Engineering at the University of New South Wales, Head of the Artificial Intelligence Research Group and Deputy Director of the iCin- ema Centre for Interactive Cinema Research. Previously, he was a program manager for the Smart Internet Technology Cooperative Research Centre, the UNSW node Director of the ARC Centre of Excellence for Autonomous Systems and a member of the joint ARC/NH&MRC project on Thinking Systems. His early work on relational learning helped to the lay the foundations for the field of Inductive Logic Programming (ILP). With Donald Michie, he also did pioneering work in Behavioral Cloning. His current interests include Conversational Agents and Robotics. Tool Use Learning for a Real Robot(Handy Wicaksono)