This presentation (part of Semantech InnovationWorx's Redefining IT series) explores what the next generation of Content Management and Search Engines will look like and what we need to do to reach intelligent computing...
The Future Begins… with an idea Web Servers are one thing, Nuclear Reactors are a bit more dangerous – there are systems which absolutely must not FAIL. <ul><li>Innovation represents the deliberate attempt to change current reality. </li></ul><ul><li>Those who believe that things can be better or can be done better have the motivation to pursue change – Innovation is the roadmap or blueprint for that change. </li></ul><ul><li>The Future consists of thousands of such blueprints coming together to build a new reality – Innovation Worx is dedicated to providing consistent and actionable Innovation for you both through our consulting and our presentation series. </li></ul>
Introduction “ Pattern recognition and association make up the core of our thought. These activities involve millions of operations carried out in parallel, outside the field of our consciousness. If AI appeared to hit a brick wall after a few quick victories, it did so owing to its inability to emulate these processes.” - Daniel Crevier <ul><li>Intelligence is the most misunderstood concept in Information Technology. This fundamental misunderstanding is why analytics, artificial intelligence and search capabilities have failed to meet initial expectations. </li></ul><ul><li>A modest redefinition will change all of that… </li></ul>
What is Intelligence anyway ? Is it aptitude or retained knowledge, intuition or pattern recognition ? How can we measure, reproduce or automate what we don’t fully understand yet?
The Premise <ul><li>This presentation is dedicated to illustrating a new paradigm for machine intelligence in the context of two real-world IT applications. </li></ul><ul><li>Those applications are; Content Management and computer aided discovery (e.g. Search or Internet Search). </li></ul><ul><li>This briefing is separated into several sections: </li></ul><ul><ul><ul><li>The Promise & Potential of Intelligent Content & Search </li></ul></ul></ul><ul><ul><ul><li>Defining Intelligent Content </li></ul></ul></ul><ul><ul><ul><li>Defining Intelligent Search </li></ul></ul></ul><ul><ul><ul><li>Semantic Integration & Intelligence </li></ul></ul></ul>
Understanding Intelligence <ul><li>Machine Intelligence designed to mimic human cognition has failed to reach its potential. These solution approaches have also to some extent distracted us from the real value of Intelligence. </li></ul><ul><li>Stripped of irrelevant value judgments, Intelligence is in fact a set of inter-related tools and capabilities – tools which are then applied to higher cognitive Human capabilities. The goal is not a sentient machine, but rather enhanced human cognition . </li></ul><ul><li>Super-cognition is the ability extract meaning from stores of human knowledge and apply it immediately in problem-solving contexts. </li></ul>
Content & Search Management <ul><li>Content & Search are intertwined and interdependent upon one another. However few solutions have effectively combined these capabilities beyond keyword recognition. </li></ul><ul><li>Keywords, metadata and database indexing have changed over the past 25 years but not as much as many seem to believe. </li></ul><ul><li>Search engines have introduced inference algorithms to help improve relevance of search returns but then again this has always been limited by the nature of the querying process & SQL. </li></ul>
Section 1: The Promise & Potential Copyright 2011, All Rights Reserved – Teksouth Corporation
A Ten Year Stalemate <ul><li>During the late 1990’s we experienced an expansion of innovation related to both content management and search / discovery. </li></ul><ul><li>This innovation was fueled by new technical standards and the emergent global Internet infrastructure. </li></ul><ul><li>Rather abruptly, in the early 2000’s, the pace of innovation slowed and has now nearly ground to a halt. The biggest gains in the past five years have revolved around integration of content management and publishing. The stalemate is not due to a lack of technology or standards, but of vision… </li></ul>
The world has been transformed by the Web – yet it seems as though its full Potential has somehow eluded us…
The Promise <ul><li>The idea was that the Internet would change everything; it would make the world a smaller, more dynamic, connected community. </li></ul><ul><li>Much of the promise was based on the notion that content or information would be more easily accessible, more easily discovered and managed. </li></ul><ul><li>It turns out most of the promise wasn’t realized – while we have made great strides with social networking; information remains as difficult to find and manage as its ever been. In fact, the Internet may have actually succeeded in making our data management challenges even more daunting… </li></ul>
The Potential <ul><li>The breakthrough potential remains. Having access to more information is both empowering and limiting simultaneously. However, if we can manage larger stores of information intelligently we might finally be able to move to the next level. </li></ul><ul><li>Information is content and also data; it is both structured and unstructured. It becomes knowledge only when it has been assimilated and applied. Value can be added both collectively and by the individual, value can be derived through discovery and aggregation. The potential of Intelligent Content & Search is tied to the ability to target information to real-world motivations and tasks. </li></ul>
A static Visualization of the Internet. The actual scope and nature of Internet Is now much more dynamic in nature. This doesn’t imply intelligence in itself but does add to its potential.
The Future of Intelligence <ul><li>The future has already been predicted – in fact dozens of new Internet Standards based on that vision have been published over the past decade. </li></ul><ul><li>Unfortunately, most have ignored those standards and / or have failed to realize the potential of the vision – known as the “Semantic Web.” </li></ul><ul><li>The main premise behind the Semantic Web is the ability manage content globally across the internet through an ecosystem of both predefined and dynamic relationships and patterns. </li></ul><ul><li>This doesn’t make the Internet Intelligent; but it does allow us to exploit it intelligently. </li></ul>
Semantic Intelligence part 1 <ul><li>So what’s the big deal; how does the Semantic Web lead to Intelligence ? </li></ul><ul><li>Semantics provide context for information that can allow it to be assessed, compared or otherwise analyzed against any number of information sources across the world as long as they adhere to the same standards – in an automated fashion. </li></ul><ul><li>Let’s say we do a search on “Turtles” and get a 18 million results. What we really want to know is longevity across related species – with the right criteria the search itself can directly answer the question. We still ask the questions and reach our conclusions, but the time to an answer is cut 100 fold . </li></ul>
Knowledge is achieved only through understanding; Understanding is a dialectic process – asking questions to solve a problem or mystery. Semantic Intelligence is the ability to facilitate that process by defining relationships and patterns - allowing for dynamic relationship discovery. The result is supercharged dialectic .
Section 2: Defining Intelligent Content Copyright 2011, All Rights Reserved – Teksouth Corporation
Why Content isn’t Intelligent <ul><li>Why isn’t Content today Intelligent ? </li></ul><ul><li>It’s stovepiped across far too many systems, formats and data exchange models. </li></ul><ul><li>It’s not fully interactive at either the community or individual level. </li></ul><ul><li>It’s too hard to discover and when it is discovered it still must be manually parsed too often. Getting to the answer is even more difficult considering how many sources must be assessed in this manner. </li></ul><ul><li>It still costs more to create, store and manage than is necessary and much of if is redundant without necessary reconciliation. </li></ul>
While having a library at your disposal is a nice thought – having to plough through 20 books to answer one simple question isn’t the most efficient approach. We now have access to many of those resources online – but still spend more time searching for and assembling results than we might otherwise have to…
When Content Becomes Intelligent <ul><li>How can Content possibly be considered Intelligent ? </li></ul><ul><li>Let’s examine this question by defining characteristics of what Intelligent Content ought to be. Intelligent Content… </li></ul><ul><ul><li>Should be interoperable. In other words, it has to be able to exist in any necessary format or system. </li></ul></ul><ul><ul><li>Should support aggregation and aggregation rules. </li></ul></ul><ul><ul><li>Should support metadata description on 3 levels; collective, individual and automated dynamic. </li></ul></ul><ul><ul><li>Should support visualization technologies. Topics must be diagrammable in relation to one another. </li></ul></ul><ul><ul><li>Should support evolution resulting from collective collaboration. </li></ul></ul>
Current Content Management <ul><li>Content Management solutions have been on the market since the late 1990’s. Most of these solutions have focused on proprietary and / or simplistic methods for organizing, indexing, retrieving and managing content lifecycles. </li></ul><ul><li>Document Management has both been viewed as part of content management and separate – the trend has been moving towards convergence. </li></ul><ul><li>Over the past 5 years some solutions have gradually begun to adopt vocabulary, ontology and taxonomy management (e.g. Semantic) capabilities. Content Management has also begun merging with Content Publishing solutions. It is still not “Intelligent”… </li></ul>
Content Management & Search <ul><li>Content can be viewed narrowly – as the sum of all content contained within one or more linked content management systems / repositories, or it could be viewed more openly. </li></ul><ul><li>If we view content openly, then potentially every, document, every email, every discussion board, every web page etc. everywhere in the world has the potential to become shared content. Obviously some of this will remain private, but there will be many opportunities to share or make content available for global search. </li></ul><ul><li>As content is aggregated into larger and larger pools; Intelligent search will be the only way to manage it. </li></ul>
Section 3: Defining Intelligent Search Copyright 2011, All Rights Reserved – Teksouth Corporation
What Constitutes Relevant Search ? <ul><li>Context is nearly impossible to guess. Let’s say you decide to run a search on the impact of Lead Poisoning; you type in “lead poisoning.” </li></ul><ul><li>How does the search engine know if you’re writing a paper, satisfying a personal inquiry, looking for medical information, examining historical implications etc. Well, it doesn’t and it can’t. </li></ul><ul><li>The best that it can do is allow you choose something and then click on a related link that opens up results similar to that one aspect of the response (typified by that link). </li></ul>
It’s all about Context <ul><li>Relevance is based entirely on the context of the question. Search Engines, at least today’s search engines, cannot know our context. That cannot be hardwired and despite some inference capability it will never effectively be guessed. </li></ul><ul><li>Unless – we build a different type of Discovery architecture, one that allows us to build our context into the discovery engine over time. One that allows us to construct searches based upon discrete problems, common tasks & long term goals. </li></ul><ul><li>Our context is what makes Searching Intelligent. </li></ul>
Discovery is like finding pieces to a puzzle; except that often it is a picture that is based on our perceptions or expectations.
Understanding Search Engines <ul><li>Search engines are based upon very mature database technology. They are primarily focused upon structured data and using indexing, metadata and various search algorithms to help improve search speed and relevance. </li></ul><ul><li>Some (few) search engines have adopted semantic technology or inference algorithms to try to evolve – others have begun mining unstructured content / data. These are small rather than revolutionary improvements and still have largely ignored the question of context – from both an individual and collective / global perspective. </li></ul>
Context can include temporal consideration – information can change given different time related constraints…
Search: What’s Missing <ul><li>The problem with the current generation of discovery tools is not limited to actual discovery – it has more to do with the lack of continuity between discovery sessions (searches) and the complete inability to build upon those results over time. </li></ul><ul><li>Bookmarks or Favorites are among the few continuity mechanisms we do have and they’ve changed little since the first web browsers were introduced during the mid-1990’s. </li></ul><ul><li>We are missing an entire architecture that would allow for effective integration of search results over time with our day to day tasks. </li></ul>
Discovery Threads <ul><li>In helping to redefine search or to define intelligent search we must begin envisioning aspects of a new architecture. </li></ul><ul><li>One of those aspects has to do with Context in terms of being able to capture past discovery assigned to specific objectives. This is a Search or “ Discovery Thread .” A Discovery Thread allows one to track their progress much like a newsgroup captures the evolution of a discussion in a Discussion Thread. </li></ul><ul><li>A Discovery Thread could include dozens or hundreds or more related searches linked together by a common goal. These can also be shared within communities. </li></ul>
Discovery Trees <ul><li>If we were to build an ontology or topic map from a Discovery Thread what we might end up with is a Discovery Tree . ( a discovery tree is a static or dynamic visual pattern ) </li></ul><ul><li>This represents the ability to condense and visualize the discovery path and possible results to more quickly ascertain how one got from A to B. </li></ul><ul><li>Discovery Trees, like Threads can be a personal or a shared resource. They would have embedded links and automatically extracted meta-data enhanced by personal context annotations. </li></ul>
Section 4: Semantic Integration Copyright 2011, All Rights Reserved – Teksouth Corporation
Semantic Integration , Defined <ul><li>The other side of the coin towards making content and search more intelligent is how underlying resources are organized in order to help facilitate content and search exploitation. Much of this is pattern-based. </li></ul><ul><li>This is handled through Semantic Integration . </li></ul><ul><li>Semantic Integration is a methodology, an IT practice and set of technical expectations. The goal of all of this is to ensure that data is easier to find, has richer relationships and is as openly available as possible. Semantic Integration is based on the premise that information integration can not be wholly predefined, but most rather support dynamic recombination. </li></ul>
Semantics is more than the current set of Semantic Web standards.
Semantic Computing <ul><li>Let’s talk a little a bit about the difference between Semantic Computing principles and the Semantic Web as it is currently constructed. </li></ul><ul><li>Semantic Computing is based on the premise that context, reasoning, output and visualization are all based upon a combination of static and dynamic Contexts and can be represented through patterns. </li></ul><ul><li>The Semantic Web is based on the notion that if data resources share some or all of the characteristics defined within Semantic Web standards that they can be exploited like a global database. The Semantic Web adheres to principles of Semantic Computing but the reverse is not always the case. </li></ul>
An important part of Semantic Integration is the ability to define, apply and share rules.
Semantic Intelligence part 2 <ul><li>Semantic Intelligence exists at two levels; first it represents the immediate (local) ability to process data and information-based semantic patterns and rules in order to gain insight or add value. Secondly it represents on a larger (global) perspective the ability to harness shared knowledge in a more efficient, automated fashion. </li></ul><ul><li>The second part comes later as more organizations make value enhanced information available in global communities. This process has already begun through development and sharing of Ontologies but Ontologies are only the tip of the iceberg. </li></ul>
Semantic Integration : Methodology This represents an example of how principles of Semantic Computing can be applied to solution lifecycles through Semantic Integration.
Semantic Architecture <ul><li>Another crucial part of Semantic Integration is Semantic Architecture. Some of what we hinted at previously in regards to Intelligent Content Management and Search could be considered direct examples of applied Semantic Architecture. </li></ul><ul><li>The primary notion behind Semantic Architecture is the ability to exploit Semantic Computing principles and Semantic Integration methodology to produce specific results on the system or system of systems level. Semantic Architectures are developed with interoperability expectations up front rather than as an added afterthought. </li></ul>
Vision is driven by motivation. Our motivation is to make computing more effective in order to enhance our performance.
Applied Semantics <ul><li>Intelligent Content and Search will not come through a simple application of Semantic Web standards. It will not come by making computers sentient in attempts to mimic our expectations and reasoning processes. It will come through human / machine partnership. </li></ul><ul><li>Intelligent Content & Search will come when we recognize where and how we can change or evolve our process and effectively reduce information overload by better managing context. </li></ul><ul><li>It requires shifting goals, redefining architectures and applying a new methodology. </li></ul>
Conclusion <ul><li>This presentation is only one in a larger series examining how the IT industry is likely to evolve and what our new-term opportunities for improvement are. </li></ul><ul><li>Any or all explorations for Redefining IT that we present are based upon key limitations or challenges facing our industry today. </li></ul><ul><li>Semantech Innovation Worx intends to provide dedicated presentations for any new terms we introduce as well… </li></ul>
Semantech Inc. is InnovationWorx <ul><li>Semantech Inc. is a solutions provider founded in 2007. Our company is located in the Dayton, Ohio metro area. Since our inception we have supported clients in a more than half a dozen industries nationwide. </li></ul><ul><li>Our company represents a unique approach – we’re not offering just IT or Management Consulting. We specialize in facilitating complex organizational Transformations. This is why Innovation Worx was created. Semantech was founded to facilitate change… </li></ul>