Downloaded 17 times
Uploaded bySingleStore
Gartner Catalyst 2017: Image Recognition on Streaming Data
This document discusses using MemSQL to perform real-time image recognition on streaming data. Key points include: - Feature vectors extracted from images using models like TensorFlow can be stored in MemSQL tables for analysis. - MemSQL allows querying these feature vectors to find similar images based on cosine similarity calculations. - This enables applications like detecting duplicate or illegal images in real-time streams.
More Related Content
Similar to Gartner Catalyst 2017: Image Recognition on Streaming Data
Gartner Catalyst 2017: Image Recognition on Streaming Data
- 1. Image Recognition on StreamingData Neil Dahlke, Senior Solutions Engineer 21 August 2017
- 2. AT MEMSQL Senior SolutionsEngineer, San Francisco BEFORE MEMSQL I worked on Globus, a high performance data transfer tool for research scientists, out of the University of Chicago in coordination with Argonne National Lab. PREVIOUS TALKS Real Time, Geospatial, Maps (slides) Streaming in the Enterprise (slides) Real Time Analytics with Spark and MemSQL (slides) 2 Me at a Glance
- 3. The future of computingis visual…
- 4.
- 5.
- 6.
- 7. Mapping Social ImageryHandwriting and many more.
- 8. and it isalso mathematical.
- 9. But first, let’screate a shared vocabulary.
- 10. Easy to setupreal-time data pipelines with exactly-once semantics Streaming Data Ingest Memory optimized tables for analyzing real-time events Live Data Disk optimized tables with up to 10x compression and vectorized queries for fast analytics Historical Data 10 MemSQL at a Glance
- 11. Data Loading QueryLatency Concurrency FAST LOW Vectorized queries Real-time dashboards Live data access Multi-threaded processing Transactions and Analytics Scalable performance HIGH Stream data Real-time loading Full data access 11
- 12. • Distributed, ANSISQL, database • Full ACID features • Lock free, shared nothing • Compiled queries • Massively parallel • Geospatial and JSON • In-memory and on-disk • MySQL protocol • Streaming • HTAP (rowstore and columnstore) MemSQL in One Slide 12
- 13. Architecture: MemSQL BuildingBlocks memsqld 13
- 14. Architecture: Aggregators andLeaves Agg 1 Agg 2 Leaf 1 Leaf 2 Leaf 3 Leaf 4 14
- 15. Architecture: Aggregators Aggregate Agg1 Agg 2 Leaf 1 Leaf 2 Leaf 3 Leaf 4 15
- 16. Architecture: Leaves HoldPartitions Agg 1 Agg 2 Leaf 1 Leaf 2 Leaf 3 Leaf 4 16
- 17. Architecture: It’s SQLAll The Way Down Agg 1 Agg 2 agg1> select avg(price) from orders; leaf1> using memsql_demo_0 select count(1), sum(price) from orders; leaf2> using memsql_demo_12 select count(1), sum(price) from orders; ... Leaf 1 Leaf 2 Leaf 3 Leaf 4 17
- 18. Now, back toimage recognition.
- 19.
- 20.
- 21. How does itwork?
- 22. 22 Real-Time Image RecognitionWorkflow ▪ Train a model with Spark and TensorFlow ▪ Use the Model to extract feature vectors from images • Model + Image => FV ▪ You can store every feature vector in a MemSQL table CREATE TABLE features ( id bigint(11) NOT NULL, image binary(4096) DEFAULT NULL, KEY id (id)USING CLUSTERED COLUMNSTORE );
- 23. 4,996 POINTS CLASSIFICATION DE-DUPLICATION MATCHING
- 24.
- 25.
- 26. 26 Working with FeatureVectors For every image we store an ID and a normalized feature vector in a MemSQL table called features. ID | Feature Vector x | 4KB To find similar images using cosine similarity, we use this SQL query: SELECT id FROM feature_vectors WHERE DOT_PRODUCT(image, 0xDEADBEEF) > 0.9
- 27.
- 28. 28 Understanding Dot Product ▪Dot Product is an algebraic operation • X = (x1, …, xN), Y = (y1, …, yN) • (X*Y) = SUM(Xi * Yi) ▪ With the specific model and normalized feature vectors DOT PRODUCT results in a similarity score. • The closer the score is to 1 the more similar are the images
- 29. 29 Performance numbers ▪ Memoryspeed: ~50GB/sec ▪ Vector size: 4KB ▪ 12.5 Million Images a second per node ▪ 1 Billion images a second on 100 node cluster
- 30. 30 Performance Enhancing Techniques Achievingbest-in-class dot product implementation ▪ SIMD-powered ▪ Data compression ▪ Query parallelism ▪ Scale out ▪ Result: Processing at Memory Bandwidth Speed
- 31. 31 MemSQL gives us… ▪Performance ▪ Scalability ▪ High concurrency ▪ Real-time (operational) ▪ Compatibility (BI, Spark, Kafka, ETL, etc) ▪ Hybrid deployment ▪ Robustness, durability, security ▪ And also…
- 32. Load Ingest from ApacheKafka, Amazon S3 or Kinesis Guarantee message delivery with exactly-once semantics Transform Map and enrich data with user defined or Apache Spark transformations MemSQL Streaming Extract 32
- 33. memsql> CREATE PIPELINEfeatures_pipeline AS -> LOAD DATA KAFKA "public-kafka.memcompute.com:9092/features" -> INTO TABLE feature_vectors -> (id, feature); -> FIELDS TERMINATED BY ',’ -> (id, @image) -> SET image = UNHEX(SUBSTRING(@image, 3)); Query OK, (0.89 sec) memsql> START PIPELINE features_pipeline; Query OK, (0.01 sec) 33 Simple Streaming Setup with CREATE PIPELINE
- 34. 34 MemSQL Pipelines Sequence DataSources MemSQL 1. Extract 2. Transform extracted data 3. Load into Database tables Pipelines
- 35. 35 MemSQL Pipelines Architecture:Kafka Kafka Broker MemSQL LeafPipelines Kafka Broker MemSQL LeafPipelines Kafka Broker MemSQL LeafPipelines MemSQL AggPipelines 1. Extract 2. Transform 3. Load Data reshuffle Metadata query 1. Extract 2. Transform 3. Load 1. Extract 2. Transform 3. Load
- 36. Real-Time Application New Image Stream Real-Time Processing Reference Image Store TensorFlow Streaming RealTime Image Recognition Workflow
- 37.
- 38. The Data WarehouseBlueprint for ML, AI and Hybrid Cloud GARY ORENSTEIN, SVP PRODUCT Tuesday, August 22 | 1:45p.m.-2:05p.m. | TechZone Theater at
Editor's Notes
- #26 Normalize each feature vector by dividing each element in the vector by the length of the vector, which gives us a length of 1 The similarity is higher if the dot product is close to one. is an algebraic operation that takes two equal-length sequences of numbers (usually coordinate vectors) and returns a single number. In Euclidean geometry, the dot product of the Cartesian coordinates of two vectors is widely used and often called inner product (or rarely projection product); see also inner product space. Algebraically, the dot product is the sum of the products of the corresponding entries of the two sequences of numbers.