An overview of Mobile Image Recognition, a presentation of Moodstocks' approach and a focus on Moodstocks SDK core library for Paris Tech Talks Meetup #6.
* https://moodstocks.com/
* http://www.meetup.com/Paris-Tech-Talks/events/165819522/
* https://github.com/Moodstocks/jpec
5. Our approach: thick client
Mobile App
Moodstocks API
client
server
reference images
up to millions images
Moodstocks SDK
image signatures
up to 10k images
query match ID or nil
server-side search
for large DB
sync
9. Design Goal #1
Portability
• core library written in C99
• +99% code in common between iOS / Android
• e.g supporting Intel CPU-s on Android was… no effort!
10. Design Goal #2
Efficiency
• in-memory data structures for maximum speed
• leverage NEON capabilities (SIMD) where applicable
• a typical search runs in ~ 30 ms / 10k images (iPhone 5S)
11. Design Goal #3
Low footprint
• 100% home made, lightweight implementation
• implement & embed only what we need!
• around 800 kB for the iOS ARMv7s release build (-Os)
17. Internal Modules (1/4)
• master-slave sync engine for image signatures
• efficient: works with diff-s & data bundles (less requests)
• fail-safe: retries + the client always stays consistent
Sync StorageLogs API client
18. Internal Modules (2/4)
Sync StorageLogs API client
• collects search results & metadata (OS, etc) along time
• a long-running thread posts batches every 30s
• tolerant to network errors -> HTTP 409 (Already Exists)
19. Internal Modules (3/4)
• persists image signatures on-disk
• persists search results events on-disk
• used as a temp on-disk buffer at sync time
Sync StorageLogs API client
20. Internal Modules (4/4)
• used for server-side recognition (send image via HTTP)
• includes a pool to re-use connections
• also used to post logs batches to the server (HTTP)
Sync StorageLogs API client
22. 3rd Party Libraries (1/4)
libcurl Tokyo Cabinet jpec msgpack
• the +500M users file transfer library
• multi API -> multiple transfers in the same thread
• we use it as the backend of our sync engine + HTTP reqs.
23. 3rd Party Libraries (2/4)
libcurl Tokyo Cabinet jpec msgpack
• key / value embedded database (hash / table / B+tree)
• … and also a fantastic C library (see tcutil.h)
• we use it as the backend of our storage engine
24. 3rd Party Libraries (3/4)
libcurl Tokyo Cabinet jpec msgpack
• our home-made, tiny JPEG encoder (600 LOC-s)
• we open sourced it -> github.com/Moodstocks/jpec
• we use it for server-side requests (send image)
25. 3rd Party Libraries (4/4)
libcurl Tokyo Cabinet jpec msgpack
• efficient binary serialization format
• we use it to pack the data we synchronize
27. Building
• we use simple Makefile-s and shell scripts
• we cross-compile w/ the iOS and Android toolchains
• we combine all the arch-s into a fat static lib (armv7, …)
28. Testing
• we write unit tests with ct (Easy Unit Testing) by @krarick
• we run them on device! e.g jailbreaked iPhone + openssh
• we use a mock to test the sync engine
29. Quality
• no warnings: we compile with -Wall -Werror
• static analysis with Clang Static Analyzer
• memory leak checks with Valgrind