GAIuS Tutorial


Published on

GAIuS (General Artificial Intelligence using Software) is a blackbox software framework for developing applications with artificial intelligence.

Posted by Intelligent Artifacts.

Published in: Technology, Education
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

GAIuS Tutorial

  1. 1. General Artificial Intelligence using Software A Software Framework forCreating, Running, Managing, and Evolving Intelligent Agents Tutorial Desktop Development Edition (1.0 Nibiru Pre-Release) Revision 5.0 Sevak Avakians July 2012
  2. 2. “GAIuS”, the GAIuS logo, “Mind your application.”, “Intelligent Artifacts”, and this document © 2012 by Sevak Avakians All rights reserved. 2
  3. 3. Introduction! 8 Who should read this! 9 Knowledge Prerequisites! 9 About this Document! 9 About this Tutorial! 9Part 1 - Quick-Start Section! 11 Installation! 11 Update your installation! 11 Run the Demo! 11 Application! 12 Start the agent! 12 Connect to the manager! 13 Connect to a primitive! 14 Getting comfortable with the Console! 14 Sequence Learning! 16 Option 1:! 17 Option 2:! 18 Option 3:! 18 Exercise 2! 19 Saving Knowledge! 19 Sleeping and Off-line Processes! 19 Prediction Potentials! 20 Evidence! 21 Confidence! 21 Prediction Results! 21 3
  4. 4. Prediction Result 1! 21 Prediction Result 2! 22 Prediction Result 3! 23 Prediction Result 4! 24 Prediction Result 5! 25Part 2 - Creating your own GAIuS Agents! 27 Your App! 27 Genome Creation! 28 Create! 29 Gene! 29 Assign! 30 Serial and Parallel Information Pathways! 34 Create your agent! 35 Run your agent! 35 GAIuS Console! 36 Show! 37 Stream! 37 Toggle! 37Part 3 - Overview! 39 Intelligent Agents! 40 Lets get started!! 42 Caveats and Solutions! 44 Exercise 1! 46 Adding Manipulatives! 46 Playing Notes! 47 4
  5. 5. Creating Intelligent Agents in GAIuS! 49 The Application! 49 Creating Manipulatives! 50 Overview of Manipulatives! 50 INPUT Manipulatives! 52 OUTPUT Manipulatives! 54 Action Weights and Costs! 54 UTILITY Manipulatives! 55 Manipulative as Application! 56 Genetically Configurable Parameters! 57 Primitive genes! 58 Primitive Information Services! 60 GAIuS Decision Engine (GDE)! 61 Topological Structures! 61 Global Structures! 61 Hierarchal Structures! 62 Network Structures! 62 Hybrid Structures! 62 Automating Agent Creation! 63 Genetic Algorithms! 63Part 4 – Overview for Scientists & Researchers! 64 Key Concepts! 64 Representations! 64 Knowledge Bases! 65 Primitive! 65 5
  6. 6. Manipulative! 66Genetically Configured Parameters! 66Agent Genome! 67Food for Thought! 67Formal System! 67Frames in GAIuS! 68Meta-knowledge Processing! 68Internal Representations! 70GKR Objects! 70Vector Objects! 70Hypersphere Objects! 70Hyperblob Objects! 70Sequence Objects! 70Prediction Objects! 70Actions! 70Utilities! 70Finding Invariants! 70Finding Canonical Vectors! 71Hyperblob Classifiers! 71Finding Isomorphic Sequences! 73Analogies between Sequences! 73Off-line Auxiliary Functions! 73Dream Function! 74Prune Function! 74Reorg Function! 74 6
  7. 7. Adding Functionality to the Framework! 74 Adding Machine Learning Algorithms! 74Appendix! 75 Primitive Information Services API! 75 Console Commands! 79 7
  8. 8. IntroductionGAIuS is a “Strong AI” framework for developing intelligent applications. It separates theapplication layer from the intelligence layer allowing independent development branches foreach. An application developer need not worry about the internal workings of the artificialintelligence. After configuring her application to use GAIuS, any improvements and/or newtechnologies incorporated into GAIuS will be transparently used by the application.GAIuS algorithms are knowledge domain independent. Regardless of the task, GAIuS agentscan be created using the same methodology. GAIuS symbolically processes the world presentedto it by either a standard set of manipulatives or custom manipulatives created by a developer.The symbolic knowledge representations are created internally by GAIuS as the agentexperiences its environment. This provides a consistent foundation for the agent to learn,recognize, perform operations on the information, make decisions, and carry out those decisions.As a framework, GAIuS can be used to create either strong AI general domain applications orspecial “expert systems” used in a narrow domain.GAIuS agents are easily mashable. The combination of primitives, manipulatives, andknowledge bases are portable. They can be combined with primitives, manipulatives, andknowledge bases of dissimilar agents to create new “expert systems” of each domain.Additionally, this allows for a simple path to sharing knowledge between systems.GAIuS agents are breedable. Configurable attributes define a GAIuS agent. These attributes arecontained in a way that allows for genetic algorithms (and other automated methods) to evolveagents based on their behavior.GAIuS agents can be “simple”. Using only one primitive, powerful intelligent agents can bemade quickly providing machine learning, pattern recognition, prediction, object inspection,and decision making, among other AI capabilities that usually require much work to create.GAIuS agents can be “complex”. Using many primitives wired together in networks,hierarchies, or combinations of both, agents can be made that, for example, share analogiesbetween senses to gain context, make better predictions, and creatively solve problems.For developers, GAIuS is a powerful tool that softens the learning curve of developingartificial intelligence applications.For researchers, GAIuS provides a common platform to share, test, and compare ideas.For end-users, GAIuS applications enable intuitive training methods for making machines dothe things you want them to do. 8
  9. 9. Who should read thisThis is a document for learning how to develop artificial intelligence applications using GAIuS.Part 1 is intended to get you up and running a sample GAIuS agent quickly. Part 2 contains atutorial designed to provide application developers general and detailed information aboutGAIuS that may be useful to quickly begin adding intelligence to an application.Researchers and scientists in artificial intelligence and/or cognitive science will also benefit fromthis tutorial although some details about the algorithms, formulas, and techniques are left for Part2. It is encouraged that researchers and scientists review the source code to gain more details.Of course, contributers to GAIuS will benefit from both sections. It is recommended thatcontributers are familiar with both the practical and theoretical aspects of the framework.This tutorial is not for end-users. End-users should read any documentation that comes with theintelligent agent or intelligent artifact product(s). Normally, this should be minimal since GAIuSallows end-users to interact with machines in a natural, intuitive way.Knowledge PrerequisitesYou should already be familiar with any programming language that provides XML-RPC webservices. Some familiarity with Python will be useful.Developers need not be familiar with any details of artificial intelligence. Nor, do they need tolearn anything more than what is provided within this tutorial. Jargon and theoretical details willbe kept to a minimum in Part 1. You just need to “mind your application”. :-)Researchers, scientists, and contributors should be very familiar with modern AI techniques andtechnologies. You should be comfortable with concepts and practical implementations ofmachine learning algorithms, artificial neural networks, Bayesian statistics and belief networks,and lots of math. The more you know, the better you can contribute!About this DocumentChanges made to this document are reflected in the documents version number on the title page.Any major changes such as additional chapters or large changes within existing sections willwarrant a new major version number that precedes the “.” in the version number. Minor changeswill warrant an incremental change in the minor version number that follows the “.” in theversion number. For example, the version number “2.5” will be the second major version releaseof this document with five minor change releases. Additionally, the month and year of thechange are also provided on the title page.About this Tutorial 9
  10. 10. This tutorial covers the GAIuS Desktop Development Edition. This is a VMWare virtualmachine that includes a complete GAIuS installation, the Eclipse IDE, and a sample agent named"Amadeus". Part 1 of this tutorial is designed to get you up and running quickly with a GAIuSagent. Part 2 teaches how to create your own GAIuS agent. Part 3 provides deeper details andinsight into GAIuS. Part 4 is intended to provide scientists and researchers with the theoreticalbackground of GAIuS. This symbol and highlighting is used to emphasize an important theoretical point thatmay have practical pitfalls." This symbol and highlighting is used to provide useful information or tips and tricks.Ψ This symbol and highlighting is used to provide insight into GAIuS psychology, i.e. thepsychology of an artificial intelligence built on GAIuS.Shell commands, shell outputs, and software code in-line with the text of a paragraph will use this font. Shell commands that are required as the next step in the tutorial, or software code will be highlighted inthis shade of blue.The exercises provided are designed to get you familiar with the key concepts that are importantfor developing GAIuS intelligent agents. The output of some of these exercises are needed forlater sections of the tutorial, so it is worth your time to try them out.This tutorial is a work-in-progress. Material may be added or deleted as is appropriate.A couple of things to note: 1. GAIuS is available pre-installed on a Ubuntu Linux virtual machine. This allows us to have better control over the quality of the requisite libraries and versions. 2. IPython is the basis of the GAIuS consoles. You can use command completion using <TAB>. 10
  11. 11. Part 1 - Quick-Start SectionAs the name implies, this section is to get you up and running GAIuS quickly. This section doesnot provide any explanations. For a deeper understanding and explanations about any topics,please read the relevant sections in Part 1 and/or Part 2 of the guide.InstallationGAIuS is provided as a virtual machine. To use it, you need either VMWare Workstation,Fusion, or Player. VMWare Player is a free download from VMWares website, default login credentials for the VM are: Account Name: gaius Password: G@!u$1234For your security, please change the password after your first login using the command: passwd <new-password>Update your installationAs a first step after starting your new installation, open a console and update the system using thecommand: g-update-gaius allThis will download and install the latest GAIuS framework, standard agents, and standardapplications. Alternatively, you can update the framework, agents, or applications separately bychanging the “all” option to “framework”, “agents”, or “applications”, respectively.Run this command periodically to get the latest versions.Run the DemoYour GAIuS installation comes with some useful applications and the “Amadeus” agent. Thisagent is designed to demonstrate many of the features of GAIuS. It is used throughout thistutorial. 11
  12. 12. ApplicationAlong with this agent, are some useful application-side programs. Applications must be startedbefore starting an agent. From the top-left menu button, navigate to “Applications” → “StartApps” → “Virtual Piano”.This will pop open a virtual keyboard. This is the “application” that will provide the agent withreal-time perception data. If the agent is started before the application is available to provide data, GAIuS willcomplain with an “Agent.SystemError - ERROR - Source signal not found!” error. Terminatethe agent (see below), start the application, then restart the agent.Start the agentOpen a console and enter: g-start-agent AmadeusThe g-start-agent command can be used to start any agent that has been installed on the system.At this point, you will see something similar to below: 12
  13. 13. An agent manager has been created. The manager is responsible for initiating, interacting, andterminating the agent. Pressing Ctrl+C within the console will terminate the agent and themanager. This is the acceptable way of terminating an agent. When the agent is terminated using Ctrl+C, the framework cleans up resources and filesthat would otherwise be left behind. If the agent is terminated in any other way, for example byclosing the console, then a file named “.<agent-name>_agent” must be manually deleted. Usethe command: rm ~/.Amadeus_agentto remove this file. Otherwise, any attempts to restart the agent will fail with a message statingthat the agent is currently running.Connect to the managerOpen a new console and type: g-console connect manager 13
  14. 14. This will open a GAIuS console and connect to the agents manager. From the manager, you cansend commands to all the primitives within the agent. Use the “manager” keyword followed bya “.”, and <tab> for autocompletion to discover the available commands: manager [#]> manager.<tab>Commands sent through the manager are applied to all active primitives.Connect to a primitiveLets again, open a console. This time, enter: g-console connect AmadeusInstead of connecting to anything, this will return a list of primitives that this agent exposes. Available Primitives for Amadeus Agent P_HearSeeDoU P_LowerPredictionAmadeus exposes two primitives, “P_HearSeeDoU” and “P_LowerPrediction”. Lets connect to“P_HearSeeDoU”: g-console connect Amadeus P_HearSeeDoUThis will open another GAIuS console. P_HearSeeDoU [#]>The name of the primitive is given in the console’s prompt. To get a list of available commandsfor the primitive, you can type: P_HearSeeDoU [#]> primitives.<tab>Alternatively, since the console allows for either ‘show’, ‘stream’, or ‘toggle’ functions, you cantype any of those three words followed by <tab> for a listing of the available commands.Getting comfortable with the ConsoleThe console is a useful tool for developing, troubleshooting, and interacting with a live GAIuSsystem. It is an IPython shell, so users of IPython should feel comfortable using it. Consolesconnect either to an agent’s manager or a single primitive of the agent.You can start a new console using the following command: g-console connect <agent-name> <primitive-name>When the console starts, it provides some helpful information: 14
  15. 15. GAIuS Console Commands/queries must end with closed parentheses. Available commands and queries are: showHelp() To see whats available: show<TAB> stream<TAB> toggle<TAB> Additional commands and queries: (The following are intended for use by manipulatives.) getData() getLTM() getSTM() getPredictedSequences() getSample() setFeedback()There are three basic command types. Each command type begins with one of the following: show - shows information stream - shows continuously updated information at regular intervals toggle - toggles a conditionalTwo additional command types are available. These types are usually intended for use byautomated processes, but may also be useful during development and testing of agents: get - returns data set - sets internal valueTyping any of these prefixes and hitting the “Tab” button will give you a list of availablecommands.The commands that are most useful for the remainder of this tutorial are: showVectorKB() - Returns a dictionary of the vector_KB showSequenceKB() - Returns a dictionary of the sequence_KB showPredictions() - Shows information regarding the active sequences, their potentials, predicted objects, and objects missing from observation. toggleLEARN(n) - Toggles the LEARN flag. If an integer “n” is provided, the learned sequence will include n prior events. For example, if n=3, then the sequence being learned will also include the 3 latest past 15
  16. 16. events. If n is not provided or n=0, then sequence learning will begin from the current event.Additionally, there is a “hidden” function that is useful for analyzing prediction results: plotSequencePotentials()Sequence LearningPreviously, we saw how GAIuS learns about time-independent objects. These objects provide astatic view of the state of the world. They are essentially a snapshot in time. On the other hand,sequences provide a dynamic understanding of the world. Sequences are time-dependentrepresentations of the state of static objects.Sequence learning is different from vector learning. Vector learning is automatic; there is nooption to turn vector learning on or off. Sequence learning, however, requires toggling a switchto delimit the sequence that is to be learned. The console provides a convenient function totoggle this switch, “toggleLEARN()”. The status of the “LEARN” flag is displayed in theshowStatus() or streamStatus() result on the “LEARN” line. When sequence learning has beenenabled, the LEARN value will be “True”.It is intended that for more advanced agents, sequence learning is automatically toggled bymanipulatives triggered by either some internal or external conditions. Sequence learning can be triggered by many different processes or on many differentconditions. One example is that it can be toggled on or off via a feedback manipulative based onthe quality of the predictions that the agent is making. If the predictions are accurate, then thereis nothing further to learn. If the predictions become inaccurate, then the agent can activatesequence learning to better predict those events the next time it experiences them. Remember toturn off sequence learning when the predictions are back on track.Other examples of triggers for toggling sequence learning include updated UTILITY values, i.e.reinforcement learning.Lets go ahead and start a new instance of the Amadeus agent. (Kill any remaining processes.)For this section, we will only use the 8 primary notes of the virtual keyboard. After restarting theagent, we will train it to learn three sequences.In a console, type: toggleLEARN()to initiate sequence learning. Now, play the following sequence of notes on the keyboard: 16
  17. 17. C4, D4, E4, F4, G4, A4, B4, C5In the console, type again: toggleLEARN()You can verify that a sequence has been learned using any of the following methods:1. In a console, type: showStatus() SequenceKB size should now equal one more than previous.2. In a console, type: showSequenceKB() This will return a dictionary of sequences in the sequence knowledge base. Look for the learned sequence as a dictionary value. Its key is a random UUID. Now, lets train a second sequence. Again, toggleLEARN() to initiate sequence learning, then playthe following sequence on the virtual keyboard: C5, B4, A4, G4, F4, E4, D4, C4Stop sequence learning using the toggleLEARN() command.Finally, train a third sequence. If you happen to know a song that you can play using the notesprovided, feel free to train the system to learn that song! In my example, I have used the song,“When the Saints Come Marchin In” as the third sequence.There are several options available for reviewing sequence learning and prediction. We willexamine each below.Option 1:From a console, type: streamStatus()and observe the lines beginning with: 17
  18. 18. Active Sequence: Sequence potential: Predicting: Missing:As GAIuS discovers which sequence it is currently observing, the “Active Sequence” line willprovide the name of that sequence.The “Sequence potential” is a numeric value corresponding to the “potential”, i.e. GAIuS beliefthat the sequence is the observed one.“Predicting” is the number of objects left in the sequence.“Missing” is the number of objects that should have already occurred, but have not been observedby the system.Option 2:For more detail, from a console, type: streamPredictions()This command provides greater detail. The “Active Sequences” are a list of full sequence namessorted from most-likely to least-likely.The “potentials” are a list of potential values corresponding to the above “Active Sequences”.The “Predictions” are a list of predictions of objects to be observed for each corresponding activesequence.Finally, the “Missing” are a list of missing objects for each corresponding active sequence.Option 3:From a console, type: plotSequencePotentials(N)where “N” is an integer indicating the number of time intervals to sample for the resulting plot.For example, plotSequencePotentials(10) 18
  19. 19. will sample data for 10 time intervals.Exercise 2Using any of the three options above, play back each sequence in turn and observe GAIuS guessof the sequence it is currently observing, the prediction of what it is yet to observe, and itsnotification of any objects that were missed from its observation.Saving KnowledgeSo youve created an agent and spent a good deal of time training it. What happens when youclose the agents Homunculus shell? If you havent saved your agents knowledge bases (KBs),then everything that it has learned is lost. GAIuS agents dont have a “Save” button that willretain everything that has been learned. Instead, each primitive has to save its own KBs. This isdone via a call to the primitives SLEEP function. In the console, type: toggleSLEEP()This command sets off a chain of processes that start by saving all the KBs into files in theprimitives “databases” folder. If there are more than one primitives, the toggleSLEEP() commandmust be called on each primitive. GAIuS primitives can be in the sleep or awake state independently of each other.When the agent is restarted, GAIuS checks for saved KBs in the “databases” folder and loadsthem if found. If you want to the agent to forget everything it has learned, simply delete the files in thedatabase folder.Sleeping and Off-line ProcessesThis brief section is informational, only. Developers can skip this section if they wish. A deeperexplanation is provided in Part 2 of this tutorial.When a primitive is put into the sleep state, aside from saving the KBs, it also runs multipleprocesses that are designed to optimize, categorize, and correct errors that have built while theagent was awake. The processes reduce the size of the KBs by finding invariants between theVectorObjects and the SequenceObjects. 19
  20. 20. Ψ If your agent is never given the opportunity to sleep, it may become slower, less efficient,and prone to the same mistakes that it may have been making from when its knowledge baseswere small. Downtime allows off-line processes to analyze, prune, and fix errors in the data thatthe agent has acquired during its online period.When a set of vectors represent the same real-world object, the entire set can be replaced by asingle “canonical vector”.Similarly, when multiple sequences contain relatively similar events, they are analogous or“isomorphs” of each other. Each sequence can be replaced with its isomorph to be used in theprediction process.Prediction PotentialsOne of GAIuS core objectives is to provide predictions. There are many ways that GAIuS canpredict. Some of these are already provided in the framework. Others can be added by adeveloper.The most important prediction algorithm used in GAIuS is the only default method providedsince it can be used to initiate more accurate techniques. This method predicts based on learnedsequences.Several things to note:1. All newly created sequences have a prediction potential equal to 1. This means that even without having observed any information, there is a potential that each sequence is currently being experienced or “active”.2. As more information comes in, the potentials for each sequence changes. Potentials that drop below 1 are considered unlikely to be currently active. Potentials greater than one are proportionally more likely to be active.3. Sequences that have been experienced more than once start with an offset proportional to the natural log of the frequency. GAIuS rightfully considers these sequences to be more likely to be currently active or to become active.4. The potentials are derived using Bayesian statistics. Objects that are seen more frequently contribute different weights to the potential than objects that are seen less frequently. This also factors into the adjustments made by missing objects. For example, if a less frequently seen object is found missing in a sequence, the potential for that sequence is less affected than if the missing object had a higher frequency of being observed.5. Active sequence frequencies are continuously updated using a fraction of the frequency of objects observed in the current event. Partially observed sequences, therefore, will have non- integer frequency values. The fractional values for a fully observed sequence will add up to the integer value of “1”. 20
  21. 21. 6. At the end of observing a full sequence, the potential for that sequence drops back to its offset described in #3 above. However, since its frequency just increased, the offset has increased as well.EvidenceAnother value provided by a primitive related to the prediction potential is the evidence value.This value is indicates how much evidence for a particular sequence (predicted or not) isobserved by the agent. The values range from zero to one. Zero, obviously, means there is noevidence observed for that sequence. A value of one indicates that all the expected objects forthat sequence have been observed. Evidence looks across the entire known sequence todetermine its value.ConfidenceConfidence is another value related to the prediction potential. Unlike evidence, confidence isdetermined based only on what is currently observed within the window size determined by aprimitives max_sequence_length gene. (See Primitive Genes section.) Basically, if a portionof a known sequence is observed with all of the objects in the correct order within that portion,then confidence will equal one, even though evidence may be low.Prediction ResultsThe following are results from tests using the plotSequencePotentials() command. The sequenceswere trained once in the order presented above, and not repeated.Each node in the graph is an event within the agents configured time interval. An event cancontain one or more objects including the “null object” or “null-vector”. The timing of theplayback for this test was synchronized to the configured time interval for clarity in the resultinggraphs.Prediction Result 1For the ascending notes, C4, D4, E4, F4, G4, A4, B4, C5 (UID starts with “46...”), the patternwas accurately predicted after the second note was played: 21
  22. 22. Prediction Result 2For the descending notes, C5, B4, A4, G4, F4, E4, D4, C4 (UID starts with “5d...”), the patternwas also accurately predicted after the second note was played. 22
  23. 23. Prediction Result 3For the song, “When the Saints Come Marchin In” (UID starts with “27...”), the pattern was alsoaccurately predicted after the second note was played. Here, there are occasional secondary andtertiary matches corresponding to similar parts of ascending and descending notes with “Whenthe Saints Come Marchin In”. This is an example of associations made between learnedsequences. The associations are recalled at the appropriate times, but only momentarily. 23
  24. 24. Prediction Result 4This test included a mix of two sequences. Both ascending notes and descending notes wereplayed alternating keys from each sequence. As can be seen by the graph below, at first GAIuSalternated its prediction that either sequence was being played before deciding that bothsequences are currently active.Once again, the sequence for the “Saints” song (UID starts with ca...) was also ruled out after thethird note was played. Throughout the remainder of the test, “Saints” was not considered as anoption. 24
  25. 25. Prediction Result 5This last test included a mix of all the sequences at different moments in time. As each sequencecompletes, its potential drops. However, the potential stays above the previous lower limitbecause the frequency of the sequence just increased by one. 25
  26. 26. 26
  27. 27. Part 2 - Creating your own GAIuS AgentsThis chapter details how to create an intelligent agent using GAIuS. Heres an overview of thisprocess: 1. Create an application with XML-RPC web services to communicate with GAIuS. The application is the data source that feeds GAIuS. The source can also be a flat file of data used for testing or developmental purposes. (See "Your App" section below.) 2. Design your information pathways. Basically, this means figure out what operations need to be done to the source data before feeding it into a primitive. For example, you may want to filter the data, multiply it, aggregate it, etc. The operators are chosen using either custom manipulatives or manipulatives from the standard library. If nothing needs to be done to the data then your design will only use the "dataPassThrough" manipulative. 3. Create a “DNA” file. The “DNA” file is a script defining the configuration and topology of the GAIuS agent. The file must be named using your agents name prepended to “.dna”. For example, if your agent is named, “Alice”, then the DNA file must be named “Alice.dna”. The included agents can be used as examples. 1. Create custom manipulatives, if needed. 2. Create genes for variables that will be used by manipulatives. 3. Assign primitives and configure the parameters. 4. Assign genes with specific values to manipulatives. 5. Attach manipulatives to primitives or other manipulatives. This sets the topology of the agent. 4. Open a console and run g-agent create <agent-name>.dna At this point, your agents new folder structure is created under the “gaius-agents”directory using the agents name. 5. Move/Add the custom manipulatives to the agents manipulatives folder.Your AppSo, youve already written or plan to write an application (also called the "app") that will provideGAIuS with a stream of data. This data will, at a minimum, consist of numeric arrays or strings.It will provide GAIuS with a view to its environment (a.k.a "percepts"). Your application mayoptionally provide GAIuS with outputs that allow it to interact with its environment. Finally, 27
  28. 28. your application may optionally provide a stream of scalar values that will be used forreinforcement learning.Your choices in providing the data streams include: 1. XML-RPC web services 2. A local program with outputs used as the data stream. (See “Manipulative as Application” section.) 3. Specially named flat files: a. "*.gaf": Contains comma-separated arrays of numbers with line-separated events b. "*.gsf": Contains comma-separated arrays of strings with line-separated eventsThese streams can provide input data and/or reinforcement learning data. The files are best usedfor development and testing purposes.Your application will likely expect some outputs from GAIuS. For example, processinformation, decision (or full decision tables), or commands may be returned to the application.Your choices in output interfaces consist of: 1. XML-RPC web services 2. A local program with inputs (e.g. parameters) 3. An OUTPUT (or “Action”) manipulative that handles all of the action details.In any of the three cases for output interfaces, an OUTPUT manipulative must be used. Theamount of details needed in the manipulative depends only on the chosen option.To create an agent, you must have a data feed. As seen above, the application can simply be asource of data.For example, the data feed may be an application that sequentially grabs data from a databaseand presents it for consumption via an XML-RPC. Optionally, the application may takecommands (OUTPUT) from the agent.Next, you must create an agents "DNA". The DNA is a configuration file that defines theagent’s configuration and topology. The DNA file is converted into an “agent genome”. Whenan agent is activated, it is instantiated using its genome.Genome CreationGenome creation consists of first creating the DNA file followed by running the g-agent createcommand using the DNA file as the argument: g-agent create my_agent.dnaThis command creates the agent’s folder structure, creates the agent’s genome, andinstallsnecessary files. 28
  29. 29. The DNA file contains two sections: create and assign. In the first section, you create “genes”that are used by manipulatives. You can create custom manipulatives or use manipulatives fromthe standard library. In either case, each manipulative needs to have some special genes.(INPUT and UTILITY manipulatives require at least two genes: ‘source’ and ‘data’. OUTPUTmanipulatives require at least four genes: ‘source’, ‘data’, ‘cost’, and ‘weight’.) If any of themanipulatives have other variable parameters, then it is necessary to create genes for thoseparameters, too.In the second section, you assign the genes to the manipulatives. During assignment, you canchange any default values for the genetic parameters. In this section, you also assignmanipulatives to primitives or other manipulatives. This creates the agent’s topology.Primitives also have genes. These genes have already been created and assigned. Only theirvalues can be changed. This is done in the assignment section.Let’s go ahead an create an agent’s DNA file now. Create a file named "my_agent.dna". Withinthe file, include two sections:my_agent.dna: [CREATE] # Creation steps below: [ASSIGN] # Assignment steps below:All parts of the creation step should be included in lines after "[CREATE]", but before"[ASSIGN]". All parts of the assignment step should be included in lines after "[ASSIGN]".CreateIf your agent requires the creation of custom manipulatives, you should create them now. (Seethe "Manipulative Creation" section.) Here, we will assume that you will be using somemanipulatives from the standard library.Lets say you plan on using the "dataPassThrough" manipulative from the standard library. Thisis an INPUT manipulative without any extra variable parameters. As stated earlier, at least twospecial genes need to be created. For an INPUT manipulative, the two genes are "data" and"source". (For an OUTPUT manipulative, the four genes are "data", "target", "weight", and"cost".)GeneTo create a gene, call the following function: 29
  30. 30. createGeneTemplate(<name>, <manipulative_name>, <value>, <alleles>, <mutability>,<volatility>)where, <name> is the name of this gene. (string) <manipulative_name> is the name of the manipulative that this gene will be assigned. (string) <value> is the default value for this gene. (any) <alleles> is a list of possible/acceptable values. This may also be a range or xrange for numeric values. <mutability> is a value between 0 and 1 inclusive that defines how frequently the value parameter will change to one of the allele values. <volatility> is a value between 0 and 1 inclusive that defines how far of a jump from the current value that a mutation can take. (Note, that the new value must still exist in the alleles.)This information is used to define a "gene" in GAIuS.For our example, we will create two genes for the dataPassThrough manipulative: createGeneTemplate(data, dataPassThrough, None, [], 0, 0) createGeneTemplate(source, dataPassThrough, , [], 0, 0)The first gene, data, has a value of None because we are not going to assign a default data set.We expect that the manipulative will grab the data from the source or be fed data. Whichsource? That gets defined in the second gene, source. Notice that in the creation step we do notneed to assign even an actual source. Here, we have left the source as an empty string. We coulddefine it now, but we choose to define it during the assignment step.AssignOur second step is to assign. First, we need to assign at least one primitive to the agent, andassign values to that primitive. Use the "myArtifactGenome.addPrimitive" function. Place thefollowing into the "[ASSIGN]" section of "my_agent_creation.script": primitive1_id = myArtifactGenome.addPrimitive(name=my_agent_name, port=4100, host=localhost, perceptime_interval=1.0, min_prediction_potential=1, classifier=CV, search_depth=100, # PCA=True, # JTC=100, f_multiplier=100, 30
  31. 31. # base_anticipate=1, # anticipate_without_evidence=False, # lower_prediction_count_threshold=6, upper_prediction_count_threshold=6, max_sequence_length=5)Now, any default values can be overwritten by the assigned values. Note that the primitives hostname and port number must be assigned. All other parameters can keep their default values or bechanged. You can use the # symbol to comment out any sections that can remain at their defaultvalues. If youd like to retain all the default values, simply use this instead: primitive1_id = myArtifactGenome.addPrimitive(name=my_agent_name, port=4100, host=localhost)We keep the primitives returned ID so that we can assign manipulatives to this primitive usingits ID.Next, we assign manipulatives by using the "myArtifactGenome.addManipulative" function: manipulative_id = myArtifactGenome.addManipulative(name, attach_point, **kwargs)Here, "name" is the name of the gene that has previously been created"attach_point" is the ID of the primitive or manipulative that this manipulative should beattached toand **kwargs can optionally be used to change any value of a gene that was used. For example,if the gene is "source" and we want to change the default "value" and default alleles to somethingelse, we do so here by: manipulative_id = myArtifactGenome.addManipulative(name, attach_point, source={value: my_new_source, alleles: [my_new_source,another_source]})If there were more genes that needed changin, they can also be changed at this time: manipulative_id = myArtifactGenome.addManipulative(name, attach_point, source={value: my_new_source, alleles: [my_new_source,another_source]}, data={value: 5, mutability:0.2, volatility:1.0})The above changes the data genes default value to 5, its mutability to 0.2, and its volatility to1.0.For our current example, we will assign the "dataPassThrough" manipulative to our onlyassigned primitive: 31
  32. 32. manipulative_id = myArtifactGenome.addManipulative(name=dataPassThrough, attach_point=primitive1_id, source={value: http://localhost:3900?getData?array})Notice that we also provided a new value to the source gene. In this case, the source of data isan XML-RPC named "getData" that resides at the localhost server on port 3900 and returns arraydata. 32
  33. 33.  A note on sources: Sources of data can be a file, an XML-RPC server (including primitives), or anothermanipulative. If the source of data is another manipulative, then the value should remain an emptystring, . If the source is a file, then the files extension dictates the type of data that fileprovides. For example, the file extension ".gaf" provides line-delimited, comma-separatednumerical arrays. An excerpt from “Agents/Amadeus/Application/Data/musical_notes_vectors.gaf”: 1,0,0,0,0,0,0,0 0,1,0,0,0,0,0,0 0,0,1,0,0,0,0,0 shows three time-separated events. Each event contains the component values of avector. (The basis for each component is implied.) A file with extension, ".gsf" provides line-delimited, comma-separated strings. An excerpt from “Agents/Amadeus/Application/Data/musical_notes_vectors.gaf”: note,B note,C,E D,car A note,A shows five time-separated events. The first, third, and fifth events contain two objects,each. The second event contains three objects. The fourth event contains only one object. Anempty line would represent the lack of any objects for that event in time. For XML-RPCs, the data type is defined by the word after the last ? in the address. For example, http://localhost:3900?getData?array provides array data while http://localhost:3900?getData?string provides string data. XML-RPCs also can provide utility values. This is done using the utility keyword: http://localhost:3900?getData?utility 33
  34. 34. Serial and Parallel Information PathwaysIn the example above, a single manipulative that sources data from a single point feeds that data -without altering it- into a single primitive. This is a serial process. What if there were multipleoperations that we wanted to make on the data? Or, what if there were multiple data sources thatwe wanted to feed the primitive? Both of these can be easily done during the assign step bychoosing different attach points.For example, say our source of data gives us arrays. Arrays are automatically converted intoVectorObjects in GAIuS. If we wanted to take these vectors and find deltas between vectors ofthe current event and the prior event, we could use the "vectorDelta" manipulative from thestandard library. If, for some reason, we wanted to then sum the components of the output of thismanipulative before feeding the result into the primitive, then we could use the "vectorSum"manipulative (Figure 1):Figure 1: Manipulatives connected in series to a single primitive.To get this topology, we would first assign the "vectorSum" manipulative to the primitive byattaching it to the primitives ID. Then, we would assign the "vectorDelta" manipulative to the"vectorSum" manipulative by attaching it to the "vectorDeltas" ID. The vectorDeltas "source"genes value would point to the data_source. (vectorSums "source" genes value would be since it is getting its data directly from vectorDelta.) This topology has a single serialinformation pathway.What if in addition to the vectorSum of the vectorDelta, we also wanted to feed the raw data intothe same primitive? In this case, we would additionally attach another manipulative,"dataPassThrough", to the same primitive. The dataPassThroughs "source" genes value wouldalso point to the same data_source (Figure 2).Figure 2: Manipulatives connected in series and parallel to a single primitive.Now, there are two parallel information pathways from the same source to the primitive.Of course, we could have assigned a new primitive and attached the dataPassThrough to it usingthe same data_source (Figure 3). Both topologies may exhibit similar behavior or showvariations in behavior depending on what factors such as the kind of information extracted from 34
  35. 35. the environment, the level of noise in the data, what is done with the information after it isprocessed by the primitive, and likely many others. The efficacy of the topology is highlydependent on what you are trying to achieve with the agent.Figure 3: Manipulatives (in series) connected to independent primitives (in parallel).Create your agentOnce the above is completed, you are ready to create your agent. In the frameworks "Tools"folder, there is a program named "".If you are creating a new agent, run it with the following parameters: g-agent create <path-to-agent-dna-file>This will install the agent, and create directories and files under your system environments"gaius-agents" folder. One of the files is the agents genome. You can review the agents genomeusing the g-agent command: g-agent show <agent-name>If you have any custom manipulatives, you can add them now to your agents "manipulatives"folder. These must exist in that folder prior to activating the agent.Run your agentOnce youve created an agent, you are ready to activate it. g-start-agent <agent-name>An agent is activated from its genome. The genome must reside in a directory having the samename as the agent and its genome. For example, the agent name "MyExampleAgent" has adirectory: ../MyExampleAgent/The genome resides at: ../MyExampleAgent/MyExampleAgent.genome 35
  36. 36. The g-agent command has already taken care of these details for you when it installed the agent.You should be able to locate your agent and its genome in a folder with your agents name under“gaius-agents”.GAIuS ConsoleTo view the internals of a running system, you need to open up GAIuS consoles. Theseconsoles communicate with a specific primitive via the primitives web services. If you knowthe primitives name, you can connect to it directly: g-console connect <agent-name> <primitive-name>The GAIuS console is an iPython shell. If you are already comfortable with iPython, then youshould feel at home with the GAIuS console. All the regular iPython capabilities and featuressuch as tab-completion are included.The console has some GAIuS specific pre-built commands. Typing showHelp()The console allows connecting to:1. An agent manager. This exposes the manager object. Typing manager.<tab> will display commands for sending to all the primitives of an active agent. g-console connect manager2. An agents primitive. This exposes the primitive object. Typing primitive.<tab> will display commands available for interacting with a primitive. g-console connect <agent-name> <primitive-name>3. An agent. This mode allows for discovering the available primitives before connecting. Use primitives.<tab> to display the available primitives. g-console connect <agent-name>When connecting to a primitive, the available commands fall into 3 categories: show, stream, andtoggle. Typing, for example, "" followed by the <TAB> key will list all the"show" commands. Similarly for stream and toggle.These commands correspond to the Primitive Information Services API exposed by everyprimitive. All of the consoles commands and the Primitive Information Services API aredescribed in the appendix. 36
  37. 37. ShowThe "show" commands show you the current information of a primitive. For example,"showStatus()" will return a print of:Figure 4: Output of showStatus() command.Similarly, "showDecisionTable()" will return a table of actions with normalized weight scoresprinted to the console. These scores indicate the probability of each action being chosen for thenext event.StreamFor some of the "show" commands, an alternative "stream" command exists. While the "show"commands print information on a single event to the screen, the "stream" commands willperiodically print that information. This is useful to view updates. The frequency of the streamevents is set by an interval option (-i or --interval) when executing By default, thisvalue is 1 second. The maximum interval should never be set to anything shorter than half ofthe primitives perceptime_interval. Anything quicker is unnecessary XML-RPC overhead.Figure 5: Output of streamDecisionTable() command. All weights are equal, but will change asthe agent learns.ToggleToggle commands allow you to toggle on or off some switches within a primitive. Theseswitches include: LEARN – When “on”, the primitive records the current sequence. When this switch is toggled “off”, the recorded sequence is converted into an internal representation and stored in a knowledge base. This is how GAIuS learns sequences.ACT – When “on”, the primitive will make decisions and take actions available via OUTPUTmanipulatives. When “off”, the decision engine will not be used and no actions will be made. 37
  38. 38. SLEEP – When “on”, the primitive will block all incoming percepts and reallocate its resourcesto processing data and/or information “offline”. When this switch is “off”, the primitive willprocess incoming data as usual.Toggle commands can also be used by other automated processes including other manipulatives,agents, or applications. 38
  39. 39. Part 3 - OverviewBefore diving in, lets take a high-level look at what we will be doing. Say that we have anapplication that needs some intelligence. The application could be a vision system such as anOCR, a stock price predictor, an autonomous car, a weather forecaster, a robot, etc. The goal ofthe intelligence is to take quantified data from the application and output appropriate responses.A response may be to provide the end-user with information such as “There is a 35% chance ofrain tomorrow at noon in the NYC area”. The response may also be an action that is taken suchas turn-steering-wheel-left-40-degrees.For more advanced agents, the response may be an internal action that operates further on theinformation. Such a system would also likely shuttle that information to another internalcomponent for additional processing. Eventually, all this internal activity must produce anoutput that is useful for an end-user.The agent can be taught new things by a teacher, or can learn on its own by observing itsenvironment.The agent will be trainable. That means, an end-user will be able to provide positive or negativereinforcement to the agent based on its behavior. The reinforcement is a numeric value called a“utility”. An agent will always try to maximize its utility by choosing behaviors that havehistorically received positive reinforcement and avoiding behaviors that have historicallyreceived negative reinforcement.The agent can also provide feedback to itself using utilities. For example, if an internal actionA1 performs better at predicting events than other internal actions, the agent can assign a higherpositive utility value to the sequence that produced the action A1 which will result in the agentusing A1 more frequently than the other actions when needing to make a similar prediction.Utilities can be created for any goal-setting scheme. The ones described above serve a purposesimilar to “happiness”. A self-preservation utility similar to “fear” can be used, for example, toprevent an autonomous car from crashing into a wall. Another use would be similar to “hunger”where the autonomous car realizes it must fill its tank with gas. The values of utilities, like otherinputs, can also be predicted. For an autonomous car, this is useful when it must decide if it candrive to its next destination on its remaining fuel or if it should first fill up its tank.Although there are many algorithms that run while an agent is “online”, there are otheralgorithms that must be run while the agent is “off-line”. These auxiliary processes arenecessary to make the agent more efficient while online. GAIuS primitives have a “sleep”function that puts the primitive into an off-line state and activates processes that review and re-categorize data that was seen while “awake”.Over time, the agent will have built large knowledge bases. Real-world data often hasfluctuations in measurements. GAIuS agents build their initial knowledge bases with eachfluctuation. Clearly, this can become an overwhelming amount of data for machines with limited 39
  40. 40. resources. Wherever possible, it is useful to replace this large set of data with canonical datapoints that represent the entire spectrum of fluctuations for that point.The internal structure of an agent with more than one “primitive” may be hierarchal, networked,or a hybrid of the two. The structure is defined by the paths that information takes as it isshuttled through the agent. (For many situations, a single “primitive” will accomplish therequired job; those agents will not have an internal structure.) This structure affects the overallbehavior and intelligence of the agent. Genetic algorithms can be used to improve the internalstructure and generate novel new GAIuS agents.Some final notes about this process:Although the act of creating an artificial intelligence using GAIuS is relatively simple,“growing” the AI requires patience and (...gulp!) understanding. General intelligence is often aslow process that takes many years for humans to grasp, and many more for humans to use insuch ways that are useful to others. For example, children take a few years to learn a language.Though some four-year-olds are bilingual, they arent used as translators at the United Nations.Ψ You will need to carefully observe your AI and your reactions to it. GAIuS artifacts learnby observing and by positive or negative reinforcement. GAIuS AIs work on the “monkey see,monkey do”, wherein they repeat behavior (if capable of it) that they observe even withoutreinforcement training.Train your AI. Just because you gave it the tools to do what you want it to do, doesnt mean thatit will do it! You need to train your AI through both observations of your behavior andreinforcement learning. Repeating the desired results of the behavior youre expecting from theAI will give the AI the impetus to do it, too. So will positive reinforcement whenever it observesthat behavior (sometimes, even if its not the one doing it!).If you are constantly providing negative reinforcement, the AI will eventually stop doing what itwas trained to do. Positive reinforcement is more affective to get the AI on the “path of theacceptable sequence”. Negative reinforcement should only be used to deter the AI from goingon the “path of the unacceptable sequence”. More on this in the tutorial.Intelligent AgentsLets begin with defining an intelligent agent as used within the GAIuS framework. This willintroduce the concepts of manipulatives, primitives, and some important configurations.An "agent" consists of at least one user-supplied application and at least one Manipulativeactivating at least one Primitive. 40
  41. 41. Figure 7: An intelligent agent. Blue arrows are IO data feed directions.The application may be an entire virtual environment or a parser of external real-worldenvironments.Figure 8: An intelligent agent connected to the world.Essentially, the user-supplied application provides the input data to the framework and may alsoexpress the output actions decided by the framework. 41
  42. 42. A Primitive is the main algorithm of GAIuS. A Primitive only exists if a manipulative has beenassociated with it. At the very least, one manipulative must exist in the system. Thismanipulativefeeds the system with data it gets from the user-supplied application. The Primitive processesthis data in a consistent and methodical way, regardless of the source of the data. The criticalprocessed information is exposed via XML-RPC web services.Manipulatives are similar to plug-ins. They can be configuration files or software code.Manipulatives are so-named because they can manipulate the states of the system. At the veryleast, they provide the system with the data that drives a Primitive. Additional manipulativesprovide greater functionality to the system such as providing the actions that the system can take,inputting utilities that the system can use for goal-setting, shuttling of data between primitives,and other more advanced functions. Within the GAIuS framework, the quality and usefulness ofthe intelligence is dictated by the features and configurations of the manipulatives.Lets get started!Now lets do something interesting. First, well play a few notes on the virtual keyboard andwatch the status changes in a console. Make sure you have two consoles open. In one console,type streamStatus()and in the other console type: streamPCA()The streamStatus() command will show the status of the following values of the primitive: Primitive P_HearSeeDoU SLEEP: False LEARN: False Items in WM: 0 Events in STM: 0 Active Sequence: None Sequence potential: 0 Predicting: None Missing: None LTM Length: 0 vector_KB size: 1 SequenceKB size: 0For now, notice that the vector_KB (vector Knowledge Base) size equal “1”. This means thatGAIuS has already learned of an object. The first object that every primitive learns is the nullvector for the data provided by the input manipulatives. Therefore, the vector object in thevector_KB is the vector with component values all equaling zero and basis that are unique to this 42
  43. 43. primitive. (Recall that a vector consists of a basis and value such as <x:0>, where the basis is“x” and xs value is “0”.)The other console should show: PCA over Learned State Vectors: EMPTYThe streamPCA() command will show the results of Principle Component Analysis of the inputvectors over known vectors. Currently, only one vector is known (the null vector) and no othervectors have been provided as inputs. Therefore, the current output is “EMPTY”.Now, in the virtual piano keyboard, click on the “C4” button. This will generate a tonecorresponding to the “C4” note. The output of the streamPCA() console will now show: PCA over Learned State Vectors: Classifier: Input vector:bb06a9fd93f5e91b15ed654a60d30fbbfbe47871and the vector_KB size in the streamStatus() console will have incremented to “2”. The systemhas just learned the vector corresponding to the “C4” note. The hash of the vector is used as thevectors name, “bb06a9fd93f5e91b15ed654a60d30fbbfbe47871”. Since there are no other non-nullvectors in the vector KB, the result of classification is that the vector“bb06a9fd93f5e91b15ed654a60d30fbbfbe47871” is the smallest component of an object.Continue by clicking on “D4” followed by “E4”, then stop. Keep in mind to space apart yourkey presses by the clock cycle! Key presses faster than the clock speed will not be noticed bythe system.At this point, the vector_KB size should equal “4”. The last item shown by streamPCA() shouldbe: Classifier: Input vector:f41f8ad8893c9faa2e765d96f957ec467934e467corresponding to the note “E4”.Next, lets click on the “ChCE” chord button. This will generate data consisting of acombination of “C4” and “E4”. Therefore, the PCA should return not one, but two componentvectors and their names should match the names for “C4” and “E4”:C4 = bb06a9fd93f5e91b15ed654a60d30fbbfbe47871E4 = f41f8ad8893c9faa2e765d96f957ec467934e467The streamPCA() console now returns: 43
  44. 44. PCA over Learned State Vectors: Known: bb06a9fd93f5e91b15ed654a60d30fbbfbe47871 f41f8ad8893c9faa2e765d96f957ec467934e467 New:What this tells us is that the “ChCE” chord was recognized by GAIuS as consisting of twoseparate known components, “C4” and “E4”.Now, we will try to infer an unknown vector by passing GAIuS a chord containing at least oneknown and one unknown vector. Click the “ChCEG” chord button. The streamPCA() consolenow returns: PCA over Learned State Vectors: Known: bb06a9fd93f5e91b15ed654a60d30fbbfbe47871 f41f8ad8893c9faa2e765d96f957ec467934e467 New: 5e18ec0fd68142faf65fef27ec0eb6d68ca55b8eThe PCA correctly guessed that the “ChCEG” chord consists of three components, two of whichare already known, and the third of which is a new object. The new object has been labeled“5e18ec0fd68142faf65fef27ec0eb6d68ca55b8e”. Lets verify that GAIuS remains consistent withthe naming of this object as the “G4” note by clicking on “G4” and reviewing the output of thestreamPCA() console: PCA over Learned State Vectors: Classifier: Input vector:5e18ec0fd68142faf65fef27ec0eb6d68ca55b8eAs expected, the vector representation of the “G4” note is consistent with the previouslydiscovered vector from the composite chord.Caveats and SolutionsNext, we will explore two caveats with this method and review ways of mitigating the problems.If you have not followed along each step above, please do so now to get the system in the samestate necessary for the rest of this section. 44
  45. 45. Now, click the “ChFA” button. Notice that neither of the components have been previouslyintroduced to the system. The output of streamPCA() is: PCA over Learned State Vectors: Known: 5e18ec0fd68142faf65fef27ec0eb6d68ca55b8e New: 64c2cdcb5bf600a9b49e94e423e6b43b34bdb651GAIuS has incorrectly inferred the components of this composite vector. Without havingpreviously seen either “F4” or “A4”, GAIuS cannot guess both unknown componentssimultaneously. GAIuS cannot infer multiple unknown components simultaneously.A way around this problem is re-processing PCA after the system has individually observed thecomponents. In GAIuS, this is done off-line through the “Dreamer” function. The chance of thesystem having individually observed the components is greatest at the “end of the day”.Additionally, when the system is off-line, it has greater resources to power through the Iterative-Deepening-Depth-First Search (IDDFS) algorithm needed for the PCA.The second caveat arises from ambiguous data. This problem occurs when there is more thanone possible way of observing the provided data. On the virtual keyboard, click the “ChGABC”chord button. The output from streamPCA() is now: PCA over Learned State Vectors: Known: bb06a9fd93f5e91b15ed654a60d30fbbfbe47871 5e18ec0fd68142faf65fef27ec0eb6d68ca55b8e New: fbe89e972830f7609cfd5a8f10357e2397b606c8The reason GAIuS did not infer the correct component vectors from this input vector is due to adegeneracy of the input data. GAIuS cannot distinguish between multiple objects if the input data is degenerate.To view the input data, open the “” file in an editor and look at the data for the“ChGABC” chord. Notice that the value it provides: # G => [0,0,1,2,3,2,1,0] # A => + [0,0,0,1,2,3,2,1] 45
  46. 46. # B => + [0,0,0,0,1,2,3,2] # C => + [3,2,1,0,0,0,0,0] array = [3,2,2,3,6,7,6,3]can be achieved through multiple combinations of the learned vector components. The rootcause of this defect is that the components objects contain values that unnecessarily overlap. Forexample, “C4” data is: array = [3,2,1,0,0,0,0,0]Instead, a better representation for “C4” would be: array = [1,0,0,0,0,0,0,0]Similarly for all the other single notes. Once those notes have been fixed, “ChGABC” isrepresented as: array = [1,0,0,0,1,1,1,0]which is now a unique set in the system. This obvious defect was designed into this tutorial toillustrate this problem and can easily be removed. Care must be taken, however, in real-worldapplications to prevent such a degeneracy from occurring. There are two methods for preventing the above problem. 1. Reducing the number of overlapping bases. 2. “Rich-contexting” or increasing the number of independent bases.Both are essentially the same solution applied from different angles.Exercise 1Fix the arrays in the “” file to prevent degeneracy. Test that each note isrepresented by its own unique and consistent vector hash using the streamPCA() command.(HINT: More than one solution is possible using either of the two mitigating methods describedin the previous paragraph. Just make sure that the length of each array is the same.)Adding ManipulativesThe Amadeus agent comes with ten manipulatives attached in parallel to one primitive(“P_HearSeeDoU”) and one manipulative attached to another primitive (“P_lowerPrediction”).Nine of these manipulatives are output (or “action”) manipulatives. They are easily identified bytheir names which always start with “ACTION”. In this agent, each output manipulative plays akey on the virtual piano keyboard, except for one. This last output manipulatives action 46
  47. 47. (“ACTIONnone “) is to perform “no action”. Whenever this action is selected, the agent will notplay any of the virtual piano keys.The remaining two are input manipulatives. Although they are both the same “dataPassThrough”manipulative, they are each linked to a different data source. The P_HearSeeDoUsdataPassThrough manipulative is linked to the virtual piano (at the application layer) andretrieves the currently played note. The P_lowerPredictions dataPassThrough is linked toP_HearSeeDoU. Specifically, it grabs P_HearSeeDoUs next predicted sequence name. (Thepurpose for this is will be explained later.)From this list, there is an important manipulative missing. It is the utility manipulative thatextracts a utility score from the application seen in Figure 13.Figure 9: The Utility application as a simple "Good" or "Bad" classification with arbitraryutility scores of +50 and -50, respectively.As part of this tutorial, we will create the genes for this manipulative and attach it to theappropriate primitive.Playing NotesWhen a baby is born, no one needs to tell her to start wiggling his toes or moving his arms. Overtime, the baby learns what appendages she has and what they do. Well, since an AI may be usedin some application that may be problematic for the agent to start off flailing its arms, GAIuSstarts its agents off in a state that prohibits them from acting. Here, the word “acting” is used todescribe any “action” that the agent may take. Like sequence learning, acting is also a toggleswitch.From a GAIuS console, type: toggleACT() 47
  48. 48. This will enable the agent to take one of the nine actions that are available to it. (You canmonitor that it has learned all nine actions by watching if it has learned all eight notes and the“no note” (appears as the null-vector) in the vector_KB line of the showStatus() or streamStatus()view. When the count equals 9, the additional being the null-vector, then the agent has alsolearned to associate its actions with the observed results. This is, of course, assuming that youhavent pressed any of the keys yourself. If you have, just wait for a minute or two to be certainthat the agent has itself pressed all the keys.) Since the first thing that any agent learns is thenull-vector, it may be hard to know if the agent learned the results of the “ACTIONnone”manipulative. So, either pay careful attention to hear a moment of silense while the agents“ACT” parameter is “True”, or just wait long enough (3-4 minutes should be good enough) toassure that the “ACTIONnone” manipulative was played.When the agent has pressed all of the eight tone keys by itself (and chosen “ACTIONnone”),turn off acting by again sending the following command: toggleACT()Now that the agent has figured out which of its actions correspond to which of its observables,lets go ahead and re-train the agent our three songs from before. This time, we want to make theagent finish playing a song that we start. The “” file in the Amadeus directory will help you to train without having tomanually stroke the keys, yourself. Available commands are: train_ascending() train_descending() train_saints() play_ascending() play_descending() play_saints()Run it in a separate command shell using the command: python autotrain.pyAfter training has completed, start playing one of the songs. Play four or five notes of the song(the more notes, the higher the active sequence potential, therefore the better the agent willrecognize the song). Then, use the toggleACT() command again and wait for the agent to finishplaying the song you started.Note that this may not work on your first few attempts. If it doesnt, then try playing the fullsong for the agent several more times. You can also add reinforcement during your playback byusing the RL_BAR applications “Good” button. In this agent, reinforcement works on thesequence and not just the actions. By giving the positive reinforcement to the agent duringplayback, the agent understands that the sequence of that song is desirable. It will attempt tomimic desirable sequences. 48
  49. 49.  If it appears that your agent is a bit hesitant to react to your prompts, you may want toincrease the JTC value in the primitives genome. Increasing the value of this parameter has theaffect of making the agent more prone to aligning actions with a perceived sequence with lowerconfidence of being correct. In short, “jumping to conclusions”. This parameter will beexplained in more detail later.Creating Intelligent Agents in GAIuSIn GAIuS, the logic and symbolic representations for the intelligence portion is separated fromthe input/output logic of the sensors and actuators. To create agents in GAIuS, you must firstcreate an application that provides GAIuS with data.Next, you must create a “genome” for the agent. The genome is essentially a configuration file.GAIuS uses “Genetically Configurable Parameters” (GCP) for every configurable item. Theadvantage to using GCPs is that genetic algorithms can be used to improve agents.The genome includes configuration values of each primitive in the agent, the attach points ofeach manipulative, and configuration values for the manipulatives if they have any.(Please review the “Create Genome” section in the Quick-Start part of this tutorial.)The ApplicationThe application must expose an XML-RPC web service function named “getData()” that returnsa list of numeric values. The application is responsible for updating the list, assuring that thelength of the list remains a constant, and assuring that all the values in the list are positive realnumbers.There can be more than one application associated with an agent.Lets take a brief look at the “” application as an example. Notice that it exposes aweb service function named “getData()”: def getData(self): # print, "data requested." data = = [0,0,0,0,0,0,0,0] # reset return dataThe function returns a list of numeric values, These values are set through otherfunctions within the “Piano” class. For example, 49
  50. 50. def beepC4(self): self.playbeep(261.6) array = [3,2,1,0,0,0,0,0] self.observe(array) returnplays a beeping sound and sets the list values to “[3,2,1,0,0,0,0,0] ”. When the getData() functionis requested through the web service (via an Input manipulative), it returns the list value andresets the values to “[0,0,0,0,0,0,0,0] ”.Creating ManipulativesThe second necessary component of a GAIuS agent is the Manipulative. Manipulatives are thedriving force of the GAIuS framework. They are designed to be atomic and reusable.Manipulatives serve to shuttle data into and throughout a GAIuS instance, operate on processeddata, provide the actions available to GAIuS, and score the effectiveness of actions taken byGAIuS. Much of this work is done using functions exposed by web services.Overview of ManipulativesThere are three basic types of Manipulatives: INPUT manipulatives grab data from a source and pass it through. In transition, the manipulative may or may not transform or operate on the data.Each input manipulative requires, at a minimum, the creation of a data and source genes: createGeneTemplate(data, <manipulative_name>, None, [], 0, 0) createGeneTemplate(source, <manipulative_name>, None, [], 0, 0)The data gene is a container for data that passes through manipulative. Without it, no data wouldpass through.The source gene points to the source of the data. If its value parameter is left as a blank string,, the data is expected from a serially attached manipulative. Otherwise, the value parametermust contain the source of the data, such as a path to a file or an URL with port and methodinformation for an XML-RPC service.Some examples of input manipulatives from the standard manipulative library (GAIuS/lib/Manipulatives) are: dataPassThrough - retrieves the data from a “source”, transforming the data into an internal representation if needed, and passes the data along. 50
  51. 51. sumOfVector - retrieves the data from a “source” of VectorObjects, adds the componentsof each vector, and returns the result as a new VectorObject. traceOfMatrix – retrieves the data from a “source” of groups of VectorObjects, combines the VectorObjects together as a matrix, and returns the trace of the matrix as a new VectorObject. rotateXformSearchByBrute – retrieves the data from a “source” of groups of VectorObjects, iteratively rotates all the vectors within the group between two set angles attempting a match of the resultant group (as a sequence) against known sequences in a sequence knowledgebase. (This is a brute-force search.)The examples above show the flexibility of the framework in operating on, or searching throughdata while passing it through a network. This functionality is not limited to vectors.Manipulatives can be created that operate and/or search through strings and sequences, too. OUTPUT manipulatives provide the system with available actions and their methods. In general, actions are specific to a problem domain. Therefore, no action manipulatives are provided in the standard library except for “ACTIONnone”.Each output manipulative requires, at a minimum, the creation of four genes: createGeneTemplate(data, <manipulative_name>, None, [], 0, 0) createGeneTemplate(target, <manipulative_name>, , [], 0, 0) createGeneTemplate(weight, <manipulative_name>, 1.0, [], 0, 0) createGeneTemplate(cost, <manipulative_name>, 1, [], 0, 0)Again, the data gene is a container for data that passes through the manipulative.The target gene points to the source of the data. Unlike with the source gene, the valueparameter for a target gene must contain the XML-RPC service and method information. UTILITY manipulatives are identical to INPUT manipulatives. The only difference is the type of data that it processes. An input manipulative is called a “utility manipulative” when the data that it processes is a scalar value, i.e. numeric values that are not given in an array. Scalar values are interpreted by GAIuS as values for reinforcement learning. These scalar values can be any real number. Internally, positive utility values are associated with desirable states, while negative utility values are associated with undesirable states.Other types are possible by combining these types. They are not given “official” names, but cangenerally be classed as follows: 51
  52. 52. FEEDBACK or FEEDFORWARD manipulatives are a hybrid of INPUT and OUTPUT manipulatives. They can be used to review the current state of one or more primitives and provide instructions (by taking actions) to one or more primitives. In the case for FEEDBACK, the provider and recipient is the same primitive. In the case for FEEDFORWARD, the provider and recipient are different manipulatives. Essentially, these shuttle information around.Other custom manipulatives are possible and encouraged, keeping in mind the atomic andreusable nature of manipulative design. To be reusable, custom manipulatives should not beapplication specific. Any operations performed by manipulatives should be knowledge domain independent.Some are provided as examples of the flexibility of manipulatives that operate within theframework. You are encouraged to write your own manipulatives, too! Useful and uniquemanipulatives submitted to the GAIuS project team will be considered for inclusion in futureversions. You also have the ability to share manipulatives with other developers or researchers.Since GAIuS data is domain-independent, manipulatives may be useful for a large number ofdisparate applications.INPUT ManipulativesLets take a look at the simplest input manipulative, “” located in thestandard library. Only the functional parts are included here for easy reading. Everything thathas been removed is not essential for its operation: from GAIuS.Workers.Manipulator import * @Manipulator def dataPassThrough(**kwargs): returnAs can be seen above, the file and structure is very simple. The only rules are that the file namemust match the function name and the function must be decorated with the “Manipulator” object.This manipulative is an “INPUT” type. All INPUT manipulatives have the “@Maniplator”decorator. At this point, you are probably wondering what actually happens in this function.Well, all of the work is actually done within the Manipulator decorator keeping the actualmanipulative very simple. The type of data that the input manipulative sees determines thebehavior of the manipulative. For example, if it sees array data it will convert the data into aVectorObject before passing it along. If it sees scalar data, it will convert that data into a utilityscore before passing it along. If it sees string data, it will convert the data into a GKR objectbefore passing it along. These types are processed differently when they reach a primitive. 52
  53. 53. The instructions of what this function must do, i.e. the variables of where to get data from, areincluded in the genes attached to it when you create the agent. You will recall that inputmanipulatives require at least two genes, “source” and “data”. The GCP value contained in thesource gene tells the manipulative where the data comes from. The Manipulator function knowshow to extract that data and process it into internal representations. Once processed, thismanipulative does not alter that data in any other way.Lets take a look at a manipulative that does alter the data, such as “”: from GAIuS.Workers.Manipulator import * from GAIuS.Representations.VectorObject import VectorObject from numpy import average @ManipulatorException def myOperation(vectors): return map(lambda vector: VectorObject([average(vector.vector)], vector.time), vectors) @Manipulator def avgOfVector(**kwargs): data = kwargs[data][value] new_data = myOperation(data) kwargs[data][value] = new_data returnThis manipulative takes incoming vector data and returns a new vector that is an average of theoriginal vectors components. Most noticeable is the addition of a new function named“myOperation”. (This can be named anything that you want.) This function is called from themanipulatives primary function, “avgOfVector”.There are five steps to keep in mind when developing your own manipulatives that operate ondata:1. Always put the @Manipulator decorator over the manipulatives primary function. For example, the primary function in this manipulative is the “avgOfVector”. It is also the same as the name of the manipulative.2. Add the @ManipulatorException decorator over your operator functions. This willallow capturing and logging of any exceptions from the operator into the agents log file, ratherthan crashing the agent.3. You must first request the current data “from the line” using the data = kwargs[data][value] statement.4. Operate on the data outside of the primary function using something like “myOperation” and call it within your primary function, e.g.new_data = myOperation(data)5. After you have operated on the data, you must put the new data back “on the line” using the kwargs[data][value] = new_data statement. 53
  54. 54. Thats all there is to it!OUTPUT ManipulativesNow, we turn to OUTPUT manipulatives. Here is an example from the Amadeus agent: from GAIuS.Workers.Manipulator import * @ActionManipulator def ACTIONPlayA4(**kwargs): returnOnce more, anything that isnt necessary has been removed for easy readability.OUTPUT manipulatives use the “@ActionManipulator” decorator.Since it looks like the actions function doesnt do anything, you may be wondering where theaction is done?Once again, this information is stored in a gene. This time, it is the “target” gene that containsthe address and method of a remote server. The action here is done using an XML remoteprocedure call (XML-RPC) to the application. In this case, the external application, which is avirtual piano keyboard, knows that when it receives that particular call, it must play that specificnote.In addition to the “target” gene, OUTPUT manipulatives must also have “data”, “weight”, and“cost” genes. The data gene is similar to that of the input manipulatives.Action Weights and CostsFor each defined action function, there is a weight and cost associated with it.Weights are the agents tendency to some action over other actions proportional to the weighsrelative value. Generally, an agent starts off with actions having equal weights. Over time, theagent develops preferences for some actions over others. 54
  55. 55. Ψ These preferences develop over time due to positive/negative utilities associated with theactions. As some actions elicit greater positive feedback, their action weights will increase. Asothers elicit negative feedback, their action weights will decrease. The weights are relative tothe values of other weights within the same OUTPUT manipulative; action weights from twodifferent OUTPUT manipulatives are independent.The cost is what it would cost the agent to do that action. This should be a real-world value suchas the amount of energy it would cost to do the action. Every action costs some resource. Try to figure it what that resource is for yourapplication. You can even create an agent to figure that out for you! But, typically it issomething that is obvious and would take less effort if you just manually provided theinformation. If you cannot figure out a cost, set the value to “1” rather than “0”. This willprevent the agent from thinking that it has a free-lunch to do the same action(s) as frequently asit wishes without consequences other than from reinforcement learning.Notice that the actions of this manipulative are to call the functions of the “”application. This is how the agent plays the virtual keyboard.UTILITY ManipulativesThe UTILITY manipulatives are an important part of GAIuS goal-setting and reinforcementlearning mechanisms.Passing a utility instead of other input data is simply a matter of using a scalar value as the input.For example, if you use the “dataPassThrough” manipulative and point its source gene to asource of data that provides scalar real values, then the data is automatically converted into aUTILITY. At this point, the manipulative is also called a “UTILITY manipulative”.Just like the INPUT manipulative, the utility values for this manipulative are provided by theexternal application, “”.Utility values mostly affect SequenceObjects. SequenceObjects contain a utility score that iscumulatively changed each time a UTILITY stimulus is provided. Positive reinforcementincreases the score; negative reinforcement decreases the score.When a utility stimulus is provided to the framework, i.e. when a UTILITY manipulativeprovides data to one or more primitives, all of the current active sequences within thoseprimitives are affected. The amount that each is affected by the utility is proportional to itsprediction potential. In practice, this means that when reinforcement training an agent, allbehaviors associated with the active sequences will be affected similarly. If two equipotential 55