Let’s get started. When you present this yourself, make sure you insert your own name and title in this slide.
But there are challenges to derive value from unstructured data80% of enterprise information is Unstructured And it’s messy with no structured model.Lack of Data Skills: 51% say they have shortage of skilled people to analyze the data.46% lack the time to interpret unstructured Data Information – Especially unstructured is growing at exponential rates – 36% difficulty managing unstructured dataBottom line: Decision Makers are strugglingDecision Makers struggle with the following:Lack of analytical skills within the organizationLack of IT support and tools to provide analysis beyond traditional finance, supply chain, and HRLack coherent, accessible structure for the diverse data they’ve collectedKey Idea: But, unstructured data is a big problem for IT.First, this data isn’t all captured in a single place. It’s common knowledge at this point that barely 20% of enterprise data is maintained in traditional data management systems.[build]Which means that over 80% of the world’s data is not. it’s spread out everywhere, on social media sites and in the internet, on file shares and your email, or maybe embedded in the text fields in your enterprise applications.[build]Most of this information is “unstructured”, and it turns out that it is really very different from traditional structured data. Unstructured data has no pre-defined data model and/or does not fit well into relational tables. Typically, there is no identifiable structure – it can have complex, hierarchical structures, and is often text-heavy. Due to the explosion and proliferation of the internet and social media, unstructured data is growing exponentially in volume and diversity – and organizations are looking for better ways to manage and analyze this data.The second challenge is the quality of this data. The truth is that while there are golden nuggets of insight buried in these mountains of data, most of it will be messy, and may not be all that valuable. How do you decide what matters to your business? There’s not necessarily a right answer: Different user groups will find value in different data sets.The bottom line is most organizations believe they lack the time and skills to make sense of it all, so they struggle to get started and never realize the benefits of unstructured analys
In the example we shared, the benefits doing this seem so obvious. So why aren’t more companies doing it? What makes usage of data so overwhelming? 1) One of the factors is that the number of data sources relevant to decision making is growing exponentially. We are currently experiencing a phenomenon that could be called the “Datafication of everything” which describes how everything that we do leaves a digital trail. Within the enterprise alone, data comes from new places like Sensor data from machines installed on equipment, multiple CRM systems from acquisitions, survey data from employees, usage statistics of company assets. There is so much data out there that oftentimes you may not know what you are looking for inside the data. You might not be able to describe how to integrate new data into your existing reports, but you know there are insights worth exploring. You’ll know it when you see it. 2) Related to this concept of the datafication of everything is that much of the interesting data is outside reach of current analytic systems, but still very relevant to how you need to fully understand your business. There is a lot of information within your enterprise which is not captured in your reports, and this is usually qualitative data such as customer service verbatims, and other types of Dialog. There is also lots of data External to the organization, which comes in all forms: Government Data, Social Media Information, competitor websites. These are messy sources of information, which are difficult to analyze alongside the highly curated enterprise information within your organization. Nonetheless, as you know, sources of information help you more fully understand the business, and in the recent past you may have asked your IT teams to help get a hold of this data, so that it could be used in decision making. 3) Finally, In the information age, we are all now much savvier with data. Computing is something that affects all aspects of our lives. Systems have evolved since the early days but in some cases still need work. You’ve been burned by IT systems which didn’t work or were very difficult to use, but you knew the keys to information were in there so you kept on trying. Why does the experience need to be so difficult? In addition, No longer are we sufficient with having the analytic access to only 5 % power users out there. You are all familiar with data, comfortable with it, you know that data is important and demand to have access to tools that help you to confidently make decisions about your business.
Now, the flip side of every challenge is the opportunity that challenge provides. With the three challenges we have discussed, the opportunity lies in the impact an organization will have to accelerate data driven action, to view big data as not just exhaust of business processes, but as a valuable commodity to enable enterprise effectiveness. It is a shift of mentality from seeing data as something which is merely stored and forgotten about, to viewing it as a change agent which can actively impact the business.We know that there is much more information which is relevant to our decision making. BUILD What if we had a way to make more accurate business decisions from actual enterprise data, instead of having to rely on intuition? What if we had the ability to quickly combine data together, visualize patterns and expose concepts, and use those new learnings to continuously ask new questions to better understand the factors which are driving business results? We know that much of data which is relevant to us is outside the reach of current analytics. But being able to harness this information will give us a more complete view of the factors affecting the business. BUILD What if we had an easy way to connect unrelated information, to leverage data external to the organization, and to improve upon it by deriving sentiment and other factors? Finally, we know that users have very high expectations when it comes to how you interact with data. BUILD But what that means is that there is an interest in empowering people to make better decisions with data, at all levels of the organization. To do this, we need to provide people the ability to work independently within a framework of governance and security, without having to add burden to IT’s workload. And we need something that is easy, even delightful to use. We’ll explore all of these possibilities in the time to come.
One of the most common questions we hear from customers is ‘what is the difference between BI and Data Discovery?’ It is important to understand that these two types of technologies optimize for different scenarios, but can be used together quite effectively.Think of a spectrum that spans the upfront knowledge a user brings to their analysis. How well do they understand which questions will be important, versus whether or not they have identified which data they need to focus on? In many cases, the questions are well known - they are driven from the metrics and dimensions that are used to describe key business processes; but the specific data that most impacts these models are unknown.[build]This is where BI platforms really excel, because they allow end users to query incredibly large and broad swaths of data, using reports and metrics to identify what slice of data matters the most.Data Discovery tools play at the opposite end of the spectrum.[build]They excel when the user has a hunch as to what data is important, but they want to amplify their understanding of this data as much as possible, and have the tool help guide them towards asking the new questions they haven’t thought of yet.And these two approaches are very complementary to each other. [build]Users can start with BI, identify the most productive places to spend their time across all areas of the enterprise, and then leverage Data Discovery to gain a deeper, personal understanding of the shape and characteristics of the data. Or, a business user can start by using the self-service aspect of Data Discovery to build new data mashups and discover new questions which can then be rolled back into enterprise models.
Let's consider a solution to help quality engineers analyze, predict and avoid costly product recalls.To figure this out, a reliability engineer needs to ask some additional questions. “What parts receive the most claims? What other products contain those parts? Who supplies those parts? What did the customer say was wrong? What are industry experts and other consumers saying about our products?”The information sources holding these answers are diverse, so where do we start?[build]As in most Endeca solutions, we start by incorporating the gold standard data and metrics in your existing data warehouse and BI solutions. Unstructured data from the other systems will then enrich and enhance this data to provide new insights.[build]One of the first places to look is your enterprise applications. Lots of valuable information gets left behind from the analysis you're doing in the warehouse, such as the text of your service requests or warranty claims. [build]Then there are all of the disparate unstructured sources within and beyond your organization. Government websites and public data sets, industry safety forums, and consumer chatter on popular social media sites.By seeing all of this information together, business users can now ask the new questions that will help them understand what's happening, what could happen next, and where to go for more information. They've turned all of this disparate information into insight.
We now introduce Oracle’s unique offering in the data discovery space – Endeca Information Discovery. This is a purpose built, enterprise ready platform for agile data discovery on both structured and unstructured content.Roughly speaking, the Endeca Information Discovery solution has three layers. [build][build]Structured, semi-structured, and unstructured data to be used for discovery is brought together by EID Integrator, an industrial-strength agile ETL tool with extensions for text enrichment and file and web crawling. This tool is used by IT to put data into Endeca Server – which is the backbone of the whole solution, and we’ll discuss it in more detail in a moment. [build][build]Studio is the product in which discovery applications are composed, consumed, shared and now, in addition to providing the most intuitive data discovery interface in the market, pioneered from Endeca’s rich history in easy to use consumer facing websites, studio also enables end-user self-service data provisioning, meaning that business user can upload data and build discovery applications without needing to consider the lower elements of the stack and involve IT in any way.
Key Point – Add new data for IT. Use EID integrator for powerful ETL capabilities, robust, scalable, multi-source, ‘production grade’EID Integrator provides a robust, yet agile, GUI based ETL tool. Connect to all common enterprise data sources (files, RDBMS, XML, JSON, etc), join sources together, write powerful data transforms from an extensive library of pre-built components and leverage CTL to write even more powerful transforms, load to Endeca server through a bulk loader or key-value pair loader. Schedule and monitor batch jobs using Integrator Server. Use Information Acquisition System (IAS) for crawl file stores and web sites (HTML)Integrator extended to handle subtle nuances of unstructured / semi structured content, including multi-assign values, jagged/sparse records.Also includes Text Enrichment and Sentiment analysis components (purchased separately)
Lets now talk about why it is important to enable data discovery as an enterprise capability[build]A data discovery platform should enable business users to be independent from IT when needed, so we should introduce the concept of ‘Self-Service Discovery’. Self-service discovery means that regular non-technical users, are enabled to add new data and content to the system themselves, ask questions of this content by leveraging pre-built interfaces to help them easily navigate and search the data, and finally see new patterns by selecting from a wide range of intuitive and interactive data visualizations, right out of the box that allow them to see insights for themselves and further more share these insights amongst colleagues and other small communities within the organization[build]But enterprise data discovery should not be thought of as just a self-service capability, performed by business users in isolation. True enterprise data discovery needs to provide all the sophisticated capabilities for highly trained and technical IT resources to compose complex applications that bring together many more diverse data sources and enable advanced features, hand-written metrics, views, and visualizations. These deeper applications are then published to large communities of end-users—with the requisite security policies in place --who can use these ‘IT provisioned’ applications to discover patterns without pausing to consider the composition of the app.We can now highlight a symbiotic relationship between these 2 very different types of discovery app.[build]Our platform should allow a complex ‘IT provisioned’ application to be enhanced or extended by individual analysts and business users in the field.[build]Similarly, an analyst who, after building their own self-service application and thinks it provides a particularly valuable perspective, can ask IT to formalize or “harden” their discovery application for use by the masses.. In this way, an enterprise’s data discovery environment organically evolves toward an optimal balance of innovation and control. This equilibrium stands in stark contrast to the chaos of desktop silos and vigilante analysts that are frequently associated with pure desktop discovery tools.
As we've mentioned--With large varieties of data, you oftentimes don’t know what you don’t know, but you know that the information is important. This is starkly in contrast how you would approach reporting for a traditional analytic system, as when you are creating heath check or status reports that run the business, you generally have a good idea of the information you are looking for. When it comes to harnessing the varied sources of information that explain what is affecting the business, there are three critical capabilities. BUILDFirst is the ability to mash up data, to rapidly combine it and to draw relationships from unrelated information. In our consumer products case, they didn’t know how the comments on the bulletin board would be related to the sales numbers for processed cheese, but they thought there might be a connection. So they needed a way to quickly add that bulletin board data to the system and explore it to see if there were connections and dependencies with their sales numbers. BUILDSecond is the ability to have a way to understand data, through visualization. Using powerful visualization tools, a user can easily see where there are outliers in the data, visually understand whether there are relationships between various factors, and whether certain events happen with more frequency than others. BUILDFinally, when discoveries happen, oftentimes new questions continue to occur which might need more data to answer. So you need the ability for your technology to be flexible, to allow you to continuously iterate and add more information, then visualize again, to get closer and closer to the heart of those questions.
The best example of a customer who harnessed the power of all the varied sources of information available to them is Delphi Automotive. Delphi, a $15B global supplier of electronics and technologies for the automotive industry, faced the challenge of determining whether their parts were meeting contractual levels of performance when warranty claims were filed. To do this, they had to wade through more than a dozen sources of warranty data from internal teams, partners, suppliers, and customers to determine the root cause of warranty claims, all within a 30-day window. Once the 30-day window closes, Delphi would be potentially billed for cost incurred by their customers regardless of who held the true responsibility for a failed part. Delphi recognized that the real problem was in the variety of issues. Every issue meant a new investigation, and the integration and transformation of new types of data. Using Oracle business analytics technology, Delphi was able to combine data from 16 different systems, including customer transaction data from SAP, parts attribute data from internal catalogs, incident data from a problem tracking system, and verbatim text descriptions from field performance issues. The records from this wide variety of systems were linked by Delphi part numbers in some cases, customer part numbers in others, and service part numbers in still others. This new combination of diverse data is accessible to warranty engineers through a zero-training interface that combines simple searches with sophisticated analytics.In the first month of deployment, engineers discovered the root cause of three field performance issues. Since then, Delphi has told us that their solution has paid for itself many times over in speed of response, allowing Delphi to detect and fix field issues faster for its customers and potentially faster and better than any of its competitors. Since adopting information discovery into its warranty and quality processes, Delphi no longer sees data variety as an obstacle to information value, but as an opportunity. Delphi now can simultaneously manage specific warranty plans for each of the 20,000 individual part numbers it manufactures and ships at a rate of 7 million pieces each month. It can more deeply investigate the field performance data from their products, each of which may have a thousand component parts in its assembly, and promptly solve any issues found.CUSTOMER PERSPECTIVE“With Endeca, we can put our best problem-solving resources to work much sooner by finding field performance issues in hours rather than weeks, great for both Delphi and our customers.”William GugginaVice President, OperationsDelphi Electronics & SafetyCOMPANY OVERVIEWA leading global supplier of electronics and technologiesIndustry: AutomotiveEmployees: 118,000Revenue: US$15 billion
As we mentioned, much of the data both inside and outside of the enterprise is inaccessible to analytic systems because it is text based, and qualitative in nature. Traditional analytic systems don’t have the capability to classify, or find common elements and themes within dialog information like customer service verbatims or bulletin board text. External systems generally take a large amount of effort to model and conform to the enterprise standards which are required by traditional analytic systems. When it comes to finding the key to this historically inaccessible data, there are three critical capabilities. BUILDFirst is the ability to take completely unrelated information, and find out the connections that exist between that data, regardless of where it came from. By finding common points and dependencies between the information, you can understand how various factors interact with one another. Going back to our consumer products example, analysts found a relationship between processed cheese sales and the comments moms made about processed cheese in the bulletin boards, despite the fact that the information resided in different systems. BUILDSecond is the ability to specifically access external data. In today’s day and age, much of the information about your business such as customer opinions in product reviews, employee feedback on job boards, government standards data, all reside outside the corporate walls, but provide facets of the reasons why business events happen. Businesses need a way for this information to be easily brought into the enterprise for analysis, regardless of the fact that there aren’t common models. BUILDFinally, you need the ability to analyze this qualitative data, to find themes that are buried within the data itself, to apply sentiment analysis to understand whether people are feeling positive or negative depending on what they’ve written, and to be able to derive classifications to the dialog and text which will allow it to be analyzed side by side with the quantitative data within the enterprise. Ideally, you need the ability to easily search upon your analysis to find all of the information about surfaced terms that may be of interest to you.
The best example of a customer who was able to put external information to good use is Agentschap Telecom, a Dutch inspection service for radio frequency bands. In the Netherlands, and across the Euro Zone, you may know that there is a strong culture of peer to peer radio which illicitly uses government frequency bands to create secret radio stations, or allow Lorry drivers to communicate with each other. Agentschap telecom was tasked with the objective to monitor and police the radio frequency bands to prevent that broadcast piracy. The real impacts of this work were underscored in one case where a large passenger airplane lost contact with the radio tower because of interference in the broadcast channels. Through analysis and inspection of external sources, Agentschap telecom was able to avert disaster. To survive in the Inspection World, you must have the right information in right time so you are able to make the right inspections and act fast. The challenge that Agentschap Telecom faced with was characterized by a need to quickly react to interferences and trends. For that reason having better knowledge of what to Inspect and Prevent has became one of the most important functions. In the beginning, They only had their monitoring system for the Fixed Antenna Network and Mobile Network, it was not integrated with the monitoring center or the organizational business analytics. Realizing that this break in information reduced their agility to react to events, they proposed an integrated solution that improved their ability to react much quicker to interferences and are able to model risk scenarios and prevent events. Using business analytics technology, they were able to leverage information from social media, and marketplace information like ebay to understand who was buying equipment that could be used to broadcast. By being more targeted in their inspection, they were able to improve their inspection success as well as reduce costs, but more importantly they were able to prevent more interferences before critical occurrences happened.
As you know, we as a culture are much more savvy with data and we have higher expectations for the applications we interact with on a day to day basis. Today’s user expects a consumer style experience that is intuitive and requires little or no effort to use. When it comes to exceeding user expectations, there are three main capabilities. BUILDFirst is the ability to find independence from IT. With IT budgets and resources constrained, users don’t have the patience to wait in queue while IT finds the resources to satisfy individual requests. Users need the ability to do Self-service upload of information for analytical analysis, to Create diverse data mash-ups on personal data files, external web content, and any other information you find useful, and take action on that information before it becomes stale. BUILDThat said, the role of IT is still critical. The quality of insights relates directly to the quality of information which is being analyzed, and in an ideal world, IT would have the ability to provide a framework of governance, enterprise standards, and security for the exploration and discovery being conducted. IT needs a way to provide access to trusted enterprise data sets, based on a users roles and responsibilities, but allow users to explore those sets and add new data independently. Not doing so will create the same disconnected silos of insight that we saw in the old days. BUILDFinally, users need an experience that is truly delightful, something that doesn’t require training, that is intuitive, and effortless so that anyone in the organization, regardless of technical capability, can make data based decisions.
The best example of a customer who has surpassed user expectations and transformed the way they conduct business with data, is Land O Lakes corporation, which is the leading producer of dairy products and distributor of crop protection products and agricultural seed in the united states. One division of the corporation sells and distributes new strains of seed to increase farmers' crop yields, and faced the challenge of wanting maximizing profitability by positioning higher margin products. They had lots of data available to them to convince farmers to buy their specialized strains of seed: data from their transaction warehouse that indicated which farmer had bought what, data from "answer plots“, or fields Land O'Lakes plants all around the US at different latitudes in different soils with different seeds to demonstrate the actual yields, and government data on how many acres are planted with which crops. Using business analytics technology, they combined all of these sources in a user interface that was easy enough for farmers with no technical background to use. This application is now being used by thousands of salespeople -- many of them former farmers themselves. Land O'Lakes realized a significant increase in profits as well as volumes but most impressively, they saved $4 Million in implementation costs and last year, won the Gartner BI Excellence Award for their solution. 4000 sales and marketing usersOVERVIEWLand O’Lakes, with annual revenues of $12.8B, operates in the dairy products and agricultural services segments. It is owned by, and serves more than 300,000 agricultural producers and some 1,000 member cooperatives.CHALLENGES/OPPORTUNITIESEnsure customer requests are properly handledLeverage detailed regional seed and crop protection data to drive salesCorrelate marketing initiatives with sales performanceSOLUTIONOracle Endeca Information Discovery (formerly Latitude) for interactive analytics on 20 data sources forOracle BI EE and Oracle BI Applications for enterprise reporting and dashboards, on JD Edwards and other transactional systemsCOMPANY OVERVIEWLeading distributor of crop protection products and agricultural seed in the United States.Industry: AgricultureEmployees: Revenue: $4B US BENEFITSThe reason we can help these salespeople use complex data in a simple way is that the discovery app delivers consumer ease-of-use:It provides integrated search, faceted browsing, and analytics. These familiar tools, Google-like search, eCommerce browsing, and interactive visualizations, let non-technical users build up complex queries through simple, incremental steps.Faceted browsing across diverse data is an innovation pioneered in ecommerce. You can see it in the left-hand navigation at sites like HomeDepot.com, ESPN.com and Target.com. Facets are attribute value pairs like “color equals blue” or “part number equals 1234”. Faceted browsing shows all the possible, but only the valid, next refinements for your query. This is crucial when browsing across records that come from different sources and have different structures.The faceted analytical model encourages exploration. Because creating queries is just a matter of chaining facets together, users express their questions easily, asking more questions.CUSTOMER FEEDBACK“We’re very pleased with the results Endeca has given us. Combined with other activities, in the Winfield group (the crop inputs group), our volumes are up 20% this year and our profit is up nearly 30%”.“What we’ve learned is the Endeca tool allowed us to do it quickly, on the fly, at a much cheaper cost than the alternatives”Dan Knutson, CFO, Land O’Lakes20% increase in volumes, 30% increase in profitHelps salespeople use complex data in a simple way: integrated search, faceted browsing and analyticsSaved 1.5 years and $4M by solving this problem with discovery technologyEasily combined internal and external information in a variety of formats
Now, you are probably wondering what business analytics technology is at the heart of all of these success stories. It is Oracle Endeca Information Discovery, an acquired product which has its roots in consumer web commerce. As we’ve described, Oracle Endeca is able to easily combine any diverse data sets, such as taking the qualitative customer verbatim from field service interactions, with related dialogs on twitter, and combining it with quantitative data from a warranty claims system. Bringing together all the relevant content into a single interface provides users with extremely powerful insights to quickly get to the root of business occurences being examined. Endeca applications are designed to be incredibly easy to build, evolve and maintain. The inherent value of Endeca Information Discovery is that users have a self-service way to add new data sources and to visualize and expose the information in the way you need as investigations around business events progress. The iterative nature of the solution will allow you to find answers to the questions you have in a matter of weeks, instead of months or years with traditional BI tools. Endeca allows you to sit down and explore data in unanticipated manners without having any technical background. The tool is both flexible and intuitive, and allows you to explore all of the relevant data, without any training. If you are comfortable using a web browser and a search box, you can immediately start to get value from an application. But you don’t have to take my word for it. Let’s take a look at a quick demo.
Please do not delete this slides, we are obliged to present these due to the sponsoring with Intel.Intel and Oracle have collaborated together for more than 20 years to optimised Oracle software to run best on Intel ArchitectureIntel have a team of engineers sat onsite at Oracle working hand in hand with Oracle developers. Starting at the top of the spiral and working around to the rightOracle provide feedback into Intel’s next generation chip designs and we then work together to build those chips into Oracle’s serversOracle use Intel’s compilers and software development tools when developing there software and we have teams of people working on Oracle Operating systems – OVM, Solaris x86 and OLEAs well as for the database, fusion middle wear and applicationsThe final result is a machine like Exadata that is optimised at every level, from silicon to machine for performance, energy efficiency and reliability
Please do not delete this slides, we are obliged to present these due to the sponsoring with Intel.We work together to meet the growing demands of IT such as readiness for cloud computing, Big Data and mobility. Intel has focused on improving the processor that lies at the heart of a next generation data center, so that we can deliver amazing experiences to end-users. We have ongoing focused efforts to help ensure those experiences are secure and optimized.
The slides after this placeholder can be added to the presentation if questions are asked, or if more background about the product is desired.
"Explore Data and Discover Hidden insight with Information Discovery" - Michał Grochowski, Senior Sales Consultant, Oracle Polska