From Data to Knowledge
•
1 like•595 views
John Morton has extensive experience in leveraging big data and analytics for business value. This includes roles as CTO for SAS and advising organizations on big data strategies. The document discusses differences between data, information, and knowledge. Data becomes information when processed, while knowledge allows predictions and depends on experience. It also outlines challenges in moving from data to knowledge, such as framing questions and addressing assumptions and data quality.
Recommended
Recommended
More Related Content
What's hot
What's hot (14)
Similar to From Data to Knowledge
Similar to From Data to Knowledge (20)
Recently uploaded
Recently uploaded (20)
From Data to Knowledge
- 1. DATA TO KNOWLEDGE John Morton Computational Intelligence Unconference London Jul 2014
- 2. Copyright © 02/08/2014, Computers, Processes and Management Limited.. All rights reserved. EUR. ING. JOHN MORTONCENG. FBCS CITP JOHN.MORTON@CONSULTCPM.COM John’s experience includes: • Former regional CTO for SAS :The Business Analytics Company delivering Big data and new analytic solutions • CTO and Technical Assurance Advisor for transforming legacy systems to multi-channel, multi-product services leveraging existing investment. Circa 650M GBP. • A key member of the Overall Design Authority for the delivery of transformation of IT systems in the UK National Health Service (NHS) for Southern England and London. • Innovation director for a Technology company, focussed on healthcare, financial services and retail industry sectors. • Worldwide consolidation of P&L accounts to leverage analytic assessment of grey markets and timely delivery of corporate accounts. • Rationalisation of Global MIS and data warehousing systems from over 40 separate data systems to three key information platforms. • Worldwide CRM architecture for a Mobile Telecommunications company. • Defining the IT Strategy for Central Government Revenue Management and accounting including call centre consolidation, channel specification for customer self-service covering WAP, SMS, internet, iDTV, mobilization. John is a IT profession skills assessor specialising in Strategy and Architecture; Portfolio, programme and project management; and Applications Design and Development He is also a GIC Director IP3-GIC. Global GDP is over 70 Trillion USD for the Computing profession and the global program for computing as spearheaded by IP3 and IP3-GIC will be a catalyst for a more than a 20% increase in global GDP in the next 10 years to over 85 Trillion USD. Big Data Advisor John's role is to advise and counsel CxOs, Directors and Business Managers on the capabilities and value of new methods and technologies in providing business enhancing services. John uses practical enterprise architecture approaches to create and implement large- scale solutions and transformations for a number of Blue Chip companies. John has particular focus on Big Data and High Performance Analytics where data is being used to drive business yield, business growth, monetize data, redefine competitive advantage and exploit information
- 3. Copyright © 02/08/2014, Computers, Processes and Management Limited.. All rights reserved.
- 4. Copyright © 02/08/2014, Computers, Processes and Management Limited.. All rights reserved. INFORMATION VERSUS KNOWLEDGE Information Processed data Simply gives us facts Clear, crisp, structured and simplistic Easily expressed in written form Obtained by condensing, correcting, contextualizing, and calculating data Devoid of owner dependencies Static and fixed data Knowledge Actionable information Allows making predictions, casual associations, or predictive decisions Muddy, fuzzy, partly unstructured Intuitive, hard to communicate, and difficult to express in words and illustration Depends on the owner Evolves through greater experience, information and context
- 5. Copyright © 02/08/2014, Computers, Processes and Management Limited.. All rights reserved. DECISION MAKING ?
- 6. Copyright © 02/08/2014, Computers, Processes and Management Limited.. All rights reserved. COMMON VIEW Selection Data Transformed Data Pattern Pre processed Data Target Data Knowledge Data Mining Transfor- mation Pre- Processing Insight
- 7. Copyright © 02/08/2014, Computers, Processes and Management Limited.. All rights reserved. Based on 450 responses from 109 respondents who report practicing Big Data analytics; 4.1 responses per respondent on average . Source: TDWI Big Data Analytics Report. Structured data ( tables, records ) Semi-structured data ( XML and similar standards ) Complex data ( hierarchical or legacy sources ) Event data ( messages, usually in real time ) Unstructured data ( human language, audio, video ) Web logs and click streams Social media data ( blogs, tweets, social networks ) Other Spatial data ( long / lat coordinates, GPS output ) Machine-generated data ( sensors, RFID, devices ) Scientific data ( astronomy, genomes, physics ) Data being used today?
- 8. Copyright © 02/08/2014, Computers, Processes and Management Limited.. All rights reserved. INSPECTION/INVESTIGATIONS Legislations and regulation Legislations and regulation Legislations and regulation Legislations and regulation Inspection Resources Knowledge for Inspections Evidence required from inspection Resourced Inspection Plan Pre-Inspect Inspect and Evidence Gathering Judgement Outcome Best Practice and Improvement Schedule of Inspections Services to be assesse d Inspecti on Plans Policies Standards and Procedures Inspection Benchmark
- 9. Copyright © 02/08/2014, Computers, Processes and Management Limited.. All rights reserved. INSPECTION/INVESTIGATIONS Legislations and regulation Legislations and regulation Legislations and regulation Legislations and regulation Inspection Resources Knowledge for Inspections Evidence required from inspection Resourced Inspection Plan Pre-Inspect Inspect and Evidence Gathering Judgement Outcome Best Practice and Improvement Schedule of Inspections Services to be assesse d Inspecti on Plans Policies Standards and Procedures Inspection Benchmark Taxono my and Semanti c analysis Taxono my and Semantic analysis Taxono my and Semantic analysis Optimisa tion Predictio n Analytics Data renderin g Productivi ty optimisati on
- 10. Copyright © 02/08/2014, Computers, Processes and Management Limited.. All rights reserved. CAN WE USE THE KNOWLEDGE GENERATED TO INFORM A BETTER INSPECTION PROCESS? Data points can be facts that have a yes/no answer A data point that is partially factual “is this present seen” , or Not and for the way the service is being delivered, volume of service or demand from service users is this appropriate A data point that is based on the knowledge and expertise of the inspector. Can knowledge lead to “common wisdom” or “intelligence”?
- 11. Copyright © 02/08/2014, Computers, Processes and Management Limited.. All rights reserved. INSPECTION/INVESTIGATIONS Legislations and regulation Legislations and regulation Legislations and regulation Legislations and regulation Inspection Resources Knowledge for Inspections Evidence required from inspection Resourced Inspection Plan Pre-Inspect Inspect and Evidence Gathering Judgement Outcome Best Practice and Improvement Schedule of Inspections Services to be assesse d Inspecti on Plans Policies Standards and Procedures Inspection Benchmark Taxono my and Semanti c analysis Taxono my and Semantic analysis Taxono my and Semantic assessm ent Optimis_ ation Predictio n Analytics Data renderin g Opinion Analytics Behaviour al Analysis Network analyticsProductivi ty optimisati on
- 12. Copyright © 02/08/2014, Computers, Processes and Management Limited.. All rights reserved. CHALLENGES Framing the question(s) Assumptive thou shall not be! Selecting the right methods Data Quality or Raw Data Perceptions – Yes you are right!
- 13. Copyright © 02/08/2014, Computers, Processes and Management Limited.. All rights reserved. QUESTIONS Eur. Ing. John Morton CEng, FBCS, CiTP, John.Morton@ConsultCPM.com +44 7771740203
Editor's Notes
- In the past : I have come up from the mathematics and programming background accreting qualifications and honours through time. Yes I have Maths, Astronomy and Information Systems engineering , including satallite communications in may background. I have spent over 20 years in the IT industry working on large scale industrialised solutions. The last 10 years of so I have spent my life specifically looking at industrialised architecture and systems.
- So if you can see something that no one else can, does that give you better knowledge and intelligence? Or is that intelligence.
- Information Information refers to data that has been given some meaning by way of connection to other data. In computing terms it is data that has been processed. The ‘meaning’ applied to the data may not be useful. For instance, data stored in a database can be processed by some code to give information about something, for example a banking application can determine how a particular account balance increased by returning the record of the credit that occurred to that account, thus ‘information’ is retrieved about that transaction. Please note that without information, you will not have knowledge. Knowledge Knowledge is the concise and appropriate collection of information in a way that makes information useful. Knowledge refers to a deterministic process where patterns within a given set of information are discovered. We can also positively say that when a person memorizes some information about something, then they have knowledge about it. That knowledge will have some use, however that knowledge doesn’t in itself infer further knowledge. Take the example of school kids who memorize knowledge of the multiplication table (times table), for instance like the result of 3 times 3 is 9 (3*3=9), because they have amassed knowledge of the table. However, the kids will not be able to respond when asked the result of 2300*150 as that entry isn’t in the table. It takes true analytical ability and the ability to reduce the question to empirical factual knowledge, not just some memorized set of knowledge, to solve the problem.
- The selection of mining methodology is very important in case of data mining. Different mining techniques are available based on the type of data to be mined. Also, factors including type of knowledge required as outcome of data mining, amount of data noise, size of data to be mined etc are also to be considered before selecting data mining method. The correct selection of data mining method is very important to get correct results. Performance issues are also to be resolved as performance is the most expected factor in case of data mining. Though there are many different methods available for data mining, most of them do not perform as expected especially in case of large quantities of data. Also, data mining needs a skilled user to provide noise free data, select the data mining algorithm, make proper conclusions out of the output etc. Thus, data mining cannot stand alone by itself. Moreover, the predictions made with the help of data might not necessarily become true always. It could get affected by many other factors that were not available during the data mining process.