0
Enabling Information Discovery by
Unifying Search and Data Management
Amir Halfon, CTO
Global Financial Services
Slide 2
Some Recent History
 1994: First full text web search engines become available
 1998: Google is founded
 2003-2...
Slide 3
Some Not So Recent History
 1960s: Navigational and hierarchical databases (IMS, IDMS)
 1970s: Edgar Codd introd...
Slide 4
What if the Two Shall Meet?
SEARCHDATABASE
Slide 5
Schema-Agnostic, Hierarchical Data Model
Trade
Cashflows
Payment
Date
Net
Payment
Payer
Party
Receiver
Party
Payme...
Slide 6
Vs. the Relational Approach
Slide 7
Universal Index
Words and phrases
... Semantic Web is a collaborative
movement led by the World Wide Web
Consortiu...
Slide 8
PDF
Word txt
Use Case: 360 Degree Customer View
UNIFIED DATA
SEARCH
Load and index data “as is”
On-boarding docs,
...
Slide 9
Use Case: Fraud Prevention
Analytics
Profile Configuration
Profile Data Extracted
from Claims
Provider and benefic...
Slide 10
Use Case: Regulatory Reporting
AUTOMATED LINKAGE
SEARCH; WORKLIST
PDF
Word
Pre-Trade
Communications
Trade
Data
Re...
Slide 11
What’s Next?
 Semantic technology
 Even more power – graph traversal, inference
Slide 12
Amir Halfon
amir.halfon@marklogic.comQuestions?
Upcoming SlideShare
Loading in...5
×

MarkLogic - Open Analytics Meetup

473

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
473
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Schema-agnostic Scalable Scale out on commodity hardware Document-centric Can handle multitude of data types Fully integrated search When organizations are looking for infrastructure to manage and leverage Big Data, they look for three things:A database that can handle unstructured and multi-structured data with ease. Great search capabilities so users can find the data they are looking for and leverage it to make better decisions for the business.Application services and tools that allows developers to build applications quickly and easily so that the data turns into usable information.There are plenty of best of breed technologies out there to serve each one of these functions – but cobbling together a system to do that is time and resource intensive – not only to build, but more so to maintain.MarkLogic provides all three of those capabilities. And, we have the added bonus of having 11 years under our belt to ensure that the system is enterprise hardened with the security, back up, recovery, high availability and data integrity you come to expect from an Enterprise data management system.
  • Cashflow-matching fpml message exampleSystemautomatically determines how to index data as the data is loaded into the databaseNo a prioriknowledge of data structureNo need for up-front logical data modeling… but some modeling is still importantAdding new data elements or changing data elements is not disruptiveSearching millions of records still has sub-second response time
  • Every time you take hierarchical data and put it into a traditional database you have to put repeating groups in separate tables and use SQL “joins” to reassemble the data
  • Key points:Quickly aggregate interaction history from diverse systems across LoBs, as well as onboarding docs (loan origination, etc.)Traverse personal connections graph (social and commercial) to glean new information.Receive alerts based on suspicious activities (fraud) or personal connections (AML), as well as marketing opportunities (targeted offers).Key technical featuresUnstructured content support (onboarding, loan origination docs, etc.)Search (interaction model: quickly grab all customer info based on name, etc.)Semantics (traverse social connections linking customer to corporate entities and other individuals)Schema-on-read (quickly aggregate info across diverse systems/products)Event processing (fraud alert, product targeting suggestions)
  • Key points:Quickly aggregate interaction history from diverse systems across LoBs, as well as onboarding docs (loan origination, etc.)Traverse personal connections graph (social and commercial) to glean new information.Receive alerts based on suspicious activities (fraud) or personal connections (AML), as well as marketing opportunities (targeted offers).Key technical featuresUnstructured content support (onboarding, loan origination docs, etc.)Search (interaction model: quickly grab all customer info based on name, etc.)Semantics (traverse social connections linking customer to corporate entities and other individuals)Schema-on-read (quickly aggregate info across diverse systems/products)Event processing (fraud alert, product targeting suggestions)
  • Find all the ISDA CSAs that are affected by a rating change, and aggregate credit risk based on existing positions
  • Transcript of "MarkLogic - Open Analytics Meetup"

    1. 1. Enabling Information Discovery by Unifying Search and Data Management Amir Halfon, CTO Global Financial Services
    2. 2. Slide 2 Some Recent History  1994: First full text web search engines become available  1998: Google is founded  2003-2004: GFS, MapReduce and BigTable whitepapers  1999-2005: Lucene, Nutch and Hadoop
    3. 3. Slide 3 Some Not So Recent History  1960s: Navigational and hierarchical databases (IMS, IDMS)  1970s: Edgar Codd introduces the relational database model; System R, INGRESS, and Oracle follow  1980s: Object databases and ORM tools  2000s: NoSQL databases
    4. 4. Slide 4 What if the Two Shall Meet? SEARCHDATABASE
    5. 5. Slide 5 Schema-Agnostic, Hierarchical Data Model Trade Cashflows Payment Date Net Payment Payer Party Receiver Party Payment Amount tradeId Party Identifier Party Reference currency amount
    6. 6. Slide 6 Vs. the Relational Approach
    7. 7. Slide 7 Universal Index Words and phrases ... Semantic Web is a collaborative movement led by the World Wide Web Consortium (W3C) ... Structure Label Author Ing Comp ID Para Org Values name:sorbitol date:2012-06-04 company:Roche Entities and positions ... ACE inhibitors, since the risk of lithium toxicity is very high in such patients... Geospatial <location> <lat>46.946584</lat> <lng>93.076172</lng> </location> Universal Index
    8. 8. Slide 8 PDF Word txt Use Case: 360 Degree Customer View UNIFIED DATA SEARCH Load and index data “as is” On-boarding docs, call center logs Personal Connections CardsDDA Mortgages
    9. 9. Slide 9 Use Case: Fraud Prevention Analytics Profile Configuration Profile Data Extracted from Claims Provider and beneficiary profiles
    10. 10. Slide 10 Use Case: Regulatory Reporting AUTOMATED LINKAGE SEARCH; WORKLIST PDF Word Pre-Trade Communications Trade Data Reference Data
    11. 11. Slide 11 What’s Next?  Semantic technology  Even more power – graph traversal, inference
    12. 12. Slide 12 Amir Halfon amir.halfon@marklogic.comQuestions?
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×