A Novel Technique For Creative Problem-Solving By Using Q-Learning And Association Algorithm

International Journal of Research in Computer and
Communication Technology, Vol 3, Issue 8, August - 2014
ISSN (Online) 2278- 5841
ISSN (Print) 2320- 5156
www.ijrcct.org Page 809
A Novel Technique for Creative Problem-Solving by using Q-learning and
Association algorithm
1
Venkata Phanikrishna B, 2
T. Srinivasarao
1Assistant Professor, Dept of C.S.E, D.N.R. College of Engineering & Technology, Bhimavaram, Andhra Pradesh (India)
2Assistant Professor, Dept of C.S.E, S.R.K.R ENGINEERING COLLEGE, Bhimavaram, Andhra Pradesh (India)
Abstract— for a single problem there are multiple solutions
or multiple ideas from different users. Means every one gave
an idea (solution) for solving the given problem if we
distribute that particular problem to the different users
(people). Finally we got set of ideas (solutions) for a
particular problem, but individual users don’t know his/her
idea is useful or not for solving the problem, if useful how
many people are accepting their idea’s for solving the
problem and what is the best idea for that problem among
them. Means this is I- Query Solver is the concept which help
to employs computer-mediated electronic communication to
replace verbal communications. This paper integrates the
unique association thinking of humans with an intelligent
agent technique to devise an automated decision. Agent can
represent a session participant who is actively participating
in problem solving process. This agent grounded on the three
association capabilities of human thinking (similarity,
contiguity, and contrast) Furthermore, a Collective ideation
Decision System (CIDS) is built to construct an environment
where agent can learn and share their knowledge with each
other.
Keywords: - ideation, CIDS, intelligent agent, Data
summarization, I-Query, Q-learning algorithm,
association algorithm.
I. INTRODUCTION
For a single problem there are multiple ideas from
multiple peoples. Due to the increasing availability of
large amount of data (ideas) in many fields of science,
business environment and in many IT applications, most
recent researches tend to use semantic knowledge. Query
Solver is help to employs computer-mediated electronic
communication to replace verbal communications. This
utilizes special software that gathers employees’ ideas and
shares them with other group members to encourage faster
collaboration. This semantic information/knowledge
derived to improve exploring new valuable knowledge
(creative information) which hidden around data.
The elementary theory of exploring new knowledge, is
to integrate the learning process based on the principles of
information association (Similarity, Contiguity, Contrast
and Causality), with an efficient structure of representing
the knowledge. Therefore, the fundamental goal of
knowledge representation is to represent knowledge in a
manner that facilitates the inferencing (Inference is the act
or process of deriving logical conclusions from premises
known or assumed to be true. The conclusion drawn is also
called an idiomatic) from knowledge, which based on the
principles of information association as linking principle
between information. These principles have an important
behavior for generating diverse information through the
dynamic exchange of varied knowledge. By combining
much information with many relations, more novel and
useful information are found. Representing such
combination requires a special structure
CIDS is integrated into an intelligent care project for the
purpose of innovation e-service recommendation.
Evaluation results indicate that the proposed system
advances e-brainstorming by crossing the three key
boundaries of human ideation capability understanding,
cognition boundary, and endurance.
This System works with intelligent agents based
environment with privacy preferences and mapping is
done with different domains areas. The agents are filtered
and grouped according to their knowledge domain.
II. THE BASIC CONCEPTS
Query Solver is help to employs computer-mediated
electronic communication to replace verbal
communications. Appropriate Information/knowledge
plays a very prominent role in effective Business
Intelligence systems that combine operational data with
analytical tools to present complex and competitive
information to planners and decision makers.
The objective is to improve the timeliness and quality of
inputs to the decision process. Business intelligence is a
form of knowledge that simplifies information discovery
and analysis, making it possible for decision makers at all
levels of an organization to more easily access,
understand, analyze, collaborate, and act on information,

ISSN (Print) 2320- 5156
anytime and anywhere. Business intelligence system
provides the consolidation and analysis of data, and the
capacity of processing data into the executable decision
making information. In this scenario knowledge
repositories and knowledge management systems have
evolved for generating and managing the knowledge. BI is
defined as a broad category of applications and
technologies for gathering, storing, analyzing, and
providing access to data to help enterprise users make
better business decisions.
An intelligent agent capable of generating ideas,
the preliminary work investigates three fundamental
human’s association capabilities (similarity, contiguity,
and contrast) during idea generation, implements these
capabilities in an agent’s inference mechanism, and
unfolds the design of Intelligent Agent, which performs
idea associations (based on a devised ontology-based
representation of ideas).
An Intelligent Agent represents an ideation
participant that can learn to understand the task and adopt
external stimuli, free from limits in working memory and
from attention exhaustion. (CIDS) is the ideation
architecture and environment with which Intelligent Agent
(IAs) could learn and share their knowledge with each
other. This study shows the preliminary results of this
agent-based ideation architecture and inference
mechanism. Osborn noted that almost all idea generation
activities rely on the association of ideas. Association
encourages imagination and causes one thought to lead to
another.
The ancient Greeks laid down the three patterns of
associations:
1. Association by contiguity. This indicates
“proximity,” like a baby’s shoe reminding
someone of an infant.
2. Association by similarity. An example of this is a
picture of a lion might remind someone of their
cat.
3. Association by contrast. An example of this is a
midget who might remind the viewer of a giant.
The three association patterns are accordingly introduced
to the inference mechanism of INTELLIGENT AGENT
(IA) ideation agents in order to allow autonomous idea
generation. This INTELLIGENT AGENT (IA) design can
further enhance the number of ideas generated (that is, the
dominant measure of e- brainstorming effectiveness) by
removing the limitations, as mentioned in the Bounded
Ideation Theory. We argue that this concept of
autonomous idea generation can create a new perspective
of recommender systems.
III. SYSTEM ARCHITECTURE
FIG1 : Collective ideation Decision System (CIDS)
Based on an automated decision agent called Intelligent
Agent (IA), in which the diverse ideas generated from this
inference mechanism founded on the principles of
information associations; similarity, contrast and
contiguity. That considered as a novel information
protocol which is adopted to facilitate the progressive
construction of actionable information map with elevating
both information quantity and information quality.
This system aims at constructing a new e-brainstorming
decision model, integrating semantic expression with the
three ideation association capabilities into
INTELLIGENT AGENT (IA) agents that are supported
and facilitated with the system architecture of CBDS. As
depicted in above figure, the architecture and the
environment have several components. Above figure
provides an overall high-level description of the
interactions between the main components of the system.
This System provides an environment in which agents can
learn and share their knowledge based on an inference
mechanism for recommending the intelligent agent to
generate diverse information that was adapted as in Figure
1.
The IA is equipped to experience learning capability by
utilizing a reinforcement learning method based on
Q-learning (design agent’s inference engine),together with
the capability of semantic information association of
human thinking (similarity, contiguity and contrast).
Information association sought to comprehend the

ISSN (Print) 2320- 5156
emerging unity of reason and cause by means of linking.
So the association principles provide the effective
information-linking strategies that used for autonomous
information generation based on the causality among the
generated information, where information association
plays an important role in linking and generating diverse
creative information.
These parts and their interactions are described as follows:
1. UI Module:
This module has two components:
 Open Problem and Participants
Component.
 Valued Ideas Notification
Component.
Open Problem and Participants Component:
An “open problem” is an initial topic or issue
given to the system. This study defines an open problem as
an idea instance represented by a specification defined in
our idea ontology. “Participants” means the clients
attending the brainstorming session, where every client is
associated with an intelligent agent (INTELLIGENT
AGENT (IA)) that serves as the client’s representative to
deal with idea association. The Open Problem and
Participants Component represents the given open
problem and the participants and sends two input
parameters, Initial Idea and Clients, to CBDS. That is, this
study assumes the existence of an initial idea instance
(possibly obtained from a user’s needs) and a given set of
ideation participants.
Valued Ideas Notification Component:
This receives the set of valued ideas and the
ideation map from the Idea Chosen Module and delivers
them to the session participants. Hence, the participants
cannot only obtain successful results from brainstorming
but also discover the causality among ideas.
2. Profile Module:
This comprises two parts: User Profile and Idea
Knowledge Base
The User Profile stores fundamental information
about a client. The Idea Knowledge Base records every
client’s domain knowledge, which comprises idea
instances and the relationships between these instances.
That is, this study assumes the existence of a given set of
ideation participants and their relevant knowledge.
3. Bookkeeping Agent:
This forwards valued ideas to every session
participants’ Idea Knowledge Base, thus increasing the
number of instances in each participant’s Idea Knowledge
Base. Accordingly, clients can enhance their domain
knowledge with each round of ideation.
4. Collective Brainstorming Blackboard:
In conventional brainstorming, a blackboard is
prepared for participants to depict and share their creative
ideas. The Collective Brainstorming Blackboard acts as
the brainstorming platform. The Collective Brainstorming
Blackboard receives system parameters Initial Idea and
Clients from the Open Problem and Participants
Component, then spontaneously initializes each client’s
INTELLIGENT AGENT (IA), and builds a
communication platform to provide an environment in
which INTELLIGENT AGENT (IA)s can learn and share
their knowledge. The objective of this module is to build a
tree-like ideation map comprising creative ideas generated
by INTELLIGENT AGENT (IA)s for use by the
Idea-Chosen Module (which selects qualified ideas, called
Valued Ideas, from the ideation map).
5. Intelligent Agent:
A INTELLIGENT AGENT (IA) is an intelligent agent
that represents its client to attend the brainstorming
session and manage the process of ideas association. The
functions of a INTELLIGENT AGENT (IA) are listed as
follows:
 Receiving the Input Idea from the Collective
Brainstorming Blackboard,
 Accessing a client’s domain knowledge from
Personal Data Manger Component,
 Creating a creative idea based on its inference
engine, and
 Returning creative ideas to the Collective
Brainstorming Blackboard.
Hence, an ideation map is gradually constructed by
repeated interactions between the Collective
Brainstorming Blackboard and INTELLIGENT AGENT
(IA)s.
6. Idea Chosen Module:
The Collective Brainstorming Blackboard assigns a
numeric value, called the Idea Chosen Indicator (ICI), for
every creative idea on the ideation map when constructing
the ideation map. Accordingly, the Idea-Chosen Module
can perform a valued idea selection according to a user-set
criterion after a brainstorming process finishes. For
instance, a creative idea is selected as a valued idea if its
Idea Chosen Value (IEV) is over a particular bound.

ISSN (Print) 2320- 5156
IV. ALGORITHMS
The amin of this paper is getting the
 Every participant can get the,
 what’s the exact answer (i.e. admin answer) for
given question
 how much, his or her answer related to the given
question
 Find the active participant by
 Number of questions he posted
 Answers given to the other questions
The IA is equipped to experience learning capability by
utilizing a reinforcement learning method based on
Q-learning (design agent’s inference engine), together
with the capability of semantic information association of
human thinking (similarity, contiguity and contrast).
 Final answer (admin answer) can be retrieved by
using ASSOCIATIN ALGORITHM
 Active user can be obtained by using Q-Learning
ALGORITHM
Q-Learning ALGORITHM:
The Q-Learning algorithm was proposed as a
way to optimize solutions in Markov (Russian
mathematician and mark off) decision process problems.
Q-Learning is in its capacity to choose between
immediate rewards and delayed rewards. It is a model-free
reinforcement learning technique that works by learning
an action-value function that gives the expected utility of
taking a given action in a given state and following a fixed
policy thereafter.
One of the strengths of Q-learning is that it is able
to compare the expected utility of the available actions
without requiring a model of the environment. A recent
variation called delayed Q-learning has shown substantial
improvements, bringing probably approximately correct
learning (PAC) bounds to Markov decision processes.
The problem model consists of an agent, states S
and a set of actions per state A. By performing an action,
the agent can move from state to state. Each state provides
the agent a reward (a real or natural number). The goal of
the agent is to maximize its total reward. It does this by
learning which action is optimal for each state.
The algorithm therefore has a function which calculates
the Quality of a state-action combination:
S
Q : × A→ R
At each step of time, an agent observes the
Vector of state xt,
Then chooses and applies an action ut.
As the process moves to state xt+1, the agent receives a
reinforcement(rewards) r(xt, ut).
Q(st,at)← Q(st-1,at-1)+α (st,at)×[Rt+1 + γ
maxat+1Q(st+1,at+1)-Q(st-1,at-1)]
Where
Rt+1 is the reward observed after performing at in st
α (st,at) (0 < α ≤ 1) the learning rate (may be the same for
all pairs)
The discount factor γ is such that (0 ≤ γ < 1)
Q(st,at)← Q(st-1,at-1)+(1-α t(st,at))+α t(st,at) [Rt+1 + γ
maxat+1Q(st+1,at+1)]
Q(st,at) ← Q(st,st)(1-α t(st,at)) + α t(st,at)[Rt+1 +γ maxat+1
Q(St+1,at+1)]
Learning rate α t(s,a)
The learning rate determines to what extent the newly
acquired information will override the old information. A
factor of 0 will make the agent not learn anything, while a
factor of 1 would make the agent consider only the most
recent information.
A factor of 1/nsa
Where nsa is the number of times that action has been
taken from state , gives a uniform average over all
attempts and yields the true expectation given enough
samples, and given that the reward function is stationary.
Discount factor γ
γ is such that (0 ≤ γ < 1)
The discount factor determines the importance of future
rewards. A factor of 0 will make the agent "opportunistic"
(or short-sighted) by only considering current rewards,
while a factor approaching 1 will make it strive for a
long-term high reward. If the discount factor meets or
exceeds 1, the Q values may diverge. The transition rule of
Q learning is a very simple formula:
Q(state, action) = R(state, action) + gamma *
Max[Q(next state, all actions)]
If gamma is closer to zero, the agent will tend to consider
only immediate rewards.
If gamma is closer to one, the agent will consider future
rewards with greater weight, willing to delay the reward.
ASSOCIATION ALGORITHM:
Association algorithm provided by Analysis Services
that is useful for recommendation engines. A
recommendation engine recommends products to
customers based on items they have already bought, or in
which they have indicated an interest.

ISSN (Print) 2320- 5156
Association models are built on datasets that contain
identifiers both for individual cases and for the items that
the cases contain. A group of items in a case is called an
itemset. An association model consists of a series of
itemsets and the rules that describe how those items are
grouped together within the cases. The rules that the
algorithm identifies can be used to predict a customer's
likely future purchases, based on the items that already
exist in the customer's shopping cart
Three fundamental human’s association capabilities
1. similarity,
2. contiguity, and
3. contrast
During idea generation
All idea generation activities rely on the association of
ideas Association encourages imagination and causes one
thought to lead to another.
The ancient Greeks laid down the three patterns of
associations
 Association by contiguity. This indicates
“proximity,” like a baby’s shoe reminding
someone of an infant.
 Association by similarity. An example of this is a
picture of a lion might remind someone of their
cat.
 Association by contrast. An example of this is a
midget who might remind the viewer of a giant.
The three association patterns are accordingly introduced
to the inference mechanism of ideation agents in order to
allow autonomous idea generation.
TABLE 1: Instance Association Relationships
Instance
Association
Relationship
Definition
Similarity
Association
Relationship
If sibling instances are of the same
parent instance with the
is-a-Relationship, they are
mutually connected by similarity
association relationship
Contiguity
association
relationship
parent instance with the Part-of
relationship, they are mutually
connected by contiguity
association relationship
Contrast association
relationship
parent instance with the
is-a-relationship (together with the
order information within their
instance information), one sibling
instance (the farthest) can then be
identified being connected by
contrast association relationship
with respect to a given idea
instance.
V. CONCLUSION & FUTURE ENHANCEMENTS
CONCLUSION
In this project we implement Humanistic brainstorm
and Artificial E-Brainstorm as single system. With the
productivity of CIDS (Artificial E-Brainstorm) and
Decision of Humanistic brainstorm this solution gives us
effective result.
This paper presents the use of Intelligent agents (IAs) in
the e-brainstorming process in order to reach automatic
collective decisions by e-brainstorming. In this process,
IAs collaborates with CIDS e-brainstorming system
architecture. IA can learn to understand the task and utilize
external stimuli without restrictions in the working
memory or attention span. CIDS is the ideation
architecture and environment, with which IAs can learn
and share knowledge.
In our future Enhancement we will implement auto
retrieval of answer related to posted question from
previous results. If they are satisfied with that answers they
can get else they can continue posting session.
FUTURE ENHANCEMENTS
However, the proposed mechanism still has limitations,
owing to the confined scopes of IA’s association
capabilities and ideation ontologies, and the rigidity of the
protocol for ideation map construction. Future work will
incorporate new association techniques and flexible agent
ideation protocols and explore different semantic ideation
ontologies and idea evaluation systems. Additionally, the
dynamic evolution of ideation ontologies and possible
scopes of future applications will be addressed.
REFERENCES
[1] Soe-Tsyr Yuan and Yen-Chuan Chen “Semantic
Ideation Learning for Agent-Based E-
Brainstorming”, IEEETRANSACTIONS ON
KNOWLEDGE AND DATA ENGINEERING,
VOL. 20, NO. 2, FEBRUARY 2008.

ISSN (Print) 2320- 5156
[2] I-Query Solver by Desale Sanket S International
Conference on Recent Trends in Engineering &
Technology, Mar-2012.
[3] Structuring Ideation Map using Oriented Directed
Acyclic Graph with Privacy Preferences S.Manju,
M.Punithavalli ACEEE Int. J. on Information
Technology , Vol. 3, No. 4, Dec 2013
[4] Geoffrey.J Rawlinson, “Creative Thinking and
Brainstorming”, Halsted Press, 1981.
[5] Applied Imagination: Principles and Procedures of
Creative Problem-Solving,” Creative Education
Foundation, third revised, 1993
[6] Sacco.O and Passant.A ,A Privacy Preference
Ontology for linked data, LDOW,2011.
Authors
VENKATA PHANIKRISHNA B1
ASST. PROFESSOR,
DNR COLLEGE OF ENGINEERING &
TECHNOLOGY
b.phanikrishna@gmail.com
TOTTEMPUDI SRINIVASARAO2
ASST. PROFESSOR
S.R.K.R. ENGINEERING COLLEGE
srinu.tottempudi@gmail.com

A Novel Technique For Creative Problem-Solving By Using Q-Learning And Association Algorithm

Recommended

Recommended

More Related Content

Similar to A Novel Technique For Creative Problem-Solving By Using Q-Learning And Association Algorithm

Similar to A Novel Technique For Creative Problem-Solving By Using Q-Learning And Association Algorithm (20)

More from Andrew Parish

More from Andrew Parish (20)

Recently uploaded

Recently uploaded (20)

A Novel Technique For Creative Problem-Solving By Using Q-Learning And Association Algorithm