SlideShare a Scribd company logo
1 of 29
Download to read offline
SOLR + Joomla powering
the catalog of Asia's Largest Auction House
Parth Lawate
Strategic Marketing Manager Joomla
CEO, Techjoomla, Tekdi Web Solutions
@parthlawate, @techjoomla
www.techjoomla.com
Cook
Bookworm Gardener
JUG Pune
Joomla Freak
Trekking
Entrepreneur
Joomla Day India
Open Source
Software Architecht
Marketing
Content Strategy
Hiking
Tekdi Technologies Pvt. Ltd.
@tekdinet
tekdi.net
IOS Apps
CRM
Magento
E Learning
Ecommerce
Joomla
Custom Apps
Android
CMS
HTML5
Social Networks
Techjoomla. For All things
Joomla
@techjoomla
techjoomla.com
jGive
People Suggest
jomLike
JTicketing
J!Bolo
Broadcast
Invitex
Email Beautifier
SocialAds
J!MailAlerts
REST API
Payments API
Social API
Quick2Cart
Quick Facts
●
The client is a major Art &
Auction house in India & is
one of the largest in Asia.
●
Data collation over a
period of 20+ years
●
Over 500,000 records with
complex interrelations.
Quick Facts
●
Complex data structure
with 100+ parameters
/fields on each data type
●
Graphics Heavy – All
artifacts have High
resolution images
The Technical Challenge
●
Over 100,000 records in
the first phase of migration
●
Extremely complex data
relations
●
Complex Data types &
Record parameter volume
& complexity
The Human Challnge
●
Use of MS Excel for years to manage
their knowledge base before we
came on board
●
Working with the client's research &
archivist team who had almost no
knowledge of any kind of web
technologies
●
Getting the team of traditional
archivists to adopt a modern system.
The Solution
●
The data complexity &
relations called for using a
CCK
●
We Chose Zoo to serve as a
base for all the
customisations to come
●
Custom apps based on this
architechture.
Term Glossary
●
Classification – First level cartegorisation eg. ANTQ
●
Sub Classification – arm
●
Artifact – Actual Record
●
Masterlists – Records that can be used as Associated
records or as a link between 2 or more records
Starting Small
●
9 Classifications
●
50 + Subsclassifications
●
50,000 Artifacts
The Work with Zoo
●
Custom field types
●
Custom association
plugins in order to create
records from relations
●
Custom views
The Early Search
●
Custom extension for
parametric search
●
One table per classificaton
●
CRON based indexer
●
MySQL powered with Natural
language support
●
Using MySQL soundex for
'did you mean' feature
We Want Excel !
●
Though we got the archivists to use web forms.. they still missed
the ease of excel
●
So we gave it to them ! With Hanson table based Mass Edit view
for Zoo.
Bulk Processing's gotta be there !
●
Bulk Edit
●
Bulk Delete
●
Bulk Add
●
Custom Importing Tools
with volume processing &
automapping
The Data today
●
12 Classifications
●
100 + sub classifications
●
8 Masterlists
●
200,000 artifacts
The Baby's growing up !
More Data called for an
architechture upgrade
Need for a better search
●
200,000 Records
●
Zoo Data structure isnt
optimised for search
●
MYSQL based indexer would
hit limits down the line.
Unions across 9 tables (which
could increase) would make
it slower
Need for a better search...
●
Single & 2 letter
autosuggest not supported
by Mysql (3 char min limit
for LIKE)
●
Normal search was not as
fast as expected (Brought
down load time for ~0.8secs
to 0.3 secs)
Getting the data ready for SOLR
●
MYSQL Indexer from
earlier phase modified to
create a Data normaliser
to push data to SOLR
●
CLI script that reads
records to populate
SOLR index
●
Using the PHP-SOLR
library
Osianama.com
PHP-SOLR Library
Browser
SOLR
Main
Index
Suggestions
Index
planned
Getting SOLR into
the picture
●
Custom Search replaced
by SOLR
●
SOLR hosted on Separate
Amozon instance
●
Initial Implementation
was only for search
Benefits
●
Much better natural
language search,
●
Better relevance scoring
●
Full reindex everyday
●
Even browsing is now
SOLR powered which
means MORE SPEED !
●
Record counts per category
& sub-category easily
achieved using faceting
●
Now using SOLR's suggester
module
●
Using separate 'cores' for
main index and suggest
terms index
Whats coming ?
●
Autosuggest directly works off
SOLR (currently piped through
PHP)
●
Implement delta indexing,
currently not implemented due
to multitude of relational data.
●
Change in a bottom level record
needs to flow through to all
associations
What else is so
awesome about
this ?
HTML5 Local Storage
●
HTML5 Local storage is
being used to cache data
locally & load used data
faster
●
Sets the road for offline
use in the future !
Google Image Search anyone ?
●
Ajax Grid pagination like Google Images
●
Preloading & caching of images, CDN backed delivery
IOS App for IPad
●
Powered by RESTful
Webservices writen on top
of Joomla using com_api
●
Initial version developed in
HTML5+Cordova (Phonegap)
●
Supports offline use of
alredy viewed data
Even More !
The Project is under continuous development. The features here only cover development at the point this
presentation was made.
●
Online Sale of Images ,
Downloads & Rights
Managements
●
Research & Teaching tools
●
Social Network
●
Subscription based
privileged access
Thank You !
●
Questions ?
●
Interested in developing something similar ?
Drop us an email ! contact@techjoomla.com
Twitter @techjoomla | @parthlawate

More Related Content

What's hot

Solr Under the Hood at S&P Global- Sumit Vadhera, S&P Global
Solr Under the Hood at S&P Global- Sumit Vadhera, S&P Global Solr Under the Hood at S&P Global- Sumit Vadhera, S&P Global
Solr Under the Hood at S&P Global- Sumit Vadhera, S&P Global Lucidworks
 
Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Search Intelligently - Liferay Symposium North America 2016, Chicago, USASearch Intelligently - Liferay Symposium North America 2016, Chicago, USA
Search Intelligently - Liferay Symposium North America 2016, Chicago, USAAndré Ricardo Barreto de Oliveira
 
Anatomy of Data Frame API : A deep dive into Spark Data Frame API
Anatomy of Data Frame API :  A deep dive into Spark Data Frame APIAnatomy of Data Frame API :  A deep dive into Spark Data Frame API
Anatomy of Data Frame API : A deep dive into Spark Data Frame APIdatamantra
 
What is NoSQL and CAP Theorem
What is NoSQL and CAP TheoremWhat is NoSQL and CAP Theorem
What is NoSQL and CAP TheoremRahul Jain
 
SQL Analytics for Search Engineers - Timothy Potter, Lucidworksngineers
SQL Analytics for Search Engineers - Timothy Potter, LucidworksngineersSQL Analytics for Search Engineers - Timothy Potter, Lucidworksngineers
SQL Analytics for Search Engineers - Timothy Potter, LucidworksngineersLucidworks
 
Building Software Backend (Web API)
Building Software Backend (Web API)Building Software Backend (Web API)
Building Software Backend (Web API)Alexander Goida
 
PHP WITH MYSQL TRAINING IN HYDERABAD
PHP WITH MYSQL TRAINING IN HYDERABADPHP WITH MYSQL TRAINING IN HYDERABAD
PHP WITH MYSQL TRAINING IN HYDERABADven88
 

What's hot (7)

Solr Under the Hood at S&P Global- Sumit Vadhera, S&P Global
Solr Under the Hood at S&P Global- Sumit Vadhera, S&P Global Solr Under the Hood at S&P Global- Sumit Vadhera, S&P Global
Solr Under the Hood at S&P Global- Sumit Vadhera, S&P Global
 
Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
Search Intelligently - Liferay Symposium North America 2016, Chicago, USASearch Intelligently - Liferay Symposium North America 2016, Chicago, USA
Search Intelligently - Liferay Symposium North America 2016, Chicago, USA
 
Anatomy of Data Frame API : A deep dive into Spark Data Frame API
Anatomy of Data Frame API :  A deep dive into Spark Data Frame APIAnatomy of Data Frame API :  A deep dive into Spark Data Frame API
Anatomy of Data Frame API : A deep dive into Spark Data Frame API
 
What is NoSQL and CAP Theorem
What is NoSQL and CAP TheoremWhat is NoSQL and CAP Theorem
What is NoSQL and CAP Theorem
 
SQL Analytics for Search Engineers - Timothy Potter, Lucidworksngineers
SQL Analytics for Search Engineers - Timothy Potter, LucidworksngineersSQL Analytics for Search Engineers - Timothy Potter, Lucidworksngineers
SQL Analytics for Search Engineers - Timothy Potter, Lucidworksngineers
 
Building Software Backend (Web API)
Building Software Backend (Web API)Building Software Backend (Web API)
Building Software Backend (Web API)
 
PHP WITH MYSQL TRAINING IN HYDERABAD
PHP WITH MYSQL TRAINING IN HYDERABADPHP WITH MYSQL TRAINING IN HYDERABAD
PHP WITH MYSQL TRAINING IN HYDERABAD
 

Similar to Using Joomla, Zoo & SOLR to power Asia's Largest Auction House

Word Press at Scale - WordCamp Minneapolis
Word Press at Scale - WordCamp MinneapolisWord Press at Scale - WordCamp Minneapolis
Word Press at Scale - WordCamp MinneapolisDrew Gorton
 
Introduction to SQL Alchemy - SyPy June 2013
Introduction to SQL Alchemy - SyPy June 2013Introduction to SQL Alchemy - SyPy June 2013
Introduction to SQL Alchemy - SyPy June 2013Roger Barnes
 
When to Use MongoDB
When to Use MongoDBWhen to Use MongoDB
When to Use MongoDBMongoDB
 
Scaling Pinterest
Scaling PinterestScaling Pinterest
Scaling PinterestC4Media
 
Big data on google platform dev fest presentation
Big data on google platform   dev fest presentationBig data on google platform   dev fest presentation
Big data on google platform dev fest presentationPrzemysław Pastuszka
 
MongoDB meetup at Hike
MongoDB meetup at HikeMongoDB meetup at Hike
MongoDB meetup at HikeBharvi Dixit
 
Real time analytics on deep learning @ strata data 2019
Real time analytics on deep learning @ strata data 2019Real time analytics on deep learning @ strata data 2019
Real time analytics on deep learning @ strata data 2019Zhenxiao Luo
 
Presto at Hadoop Summit 2016
Presto at Hadoop Summit 2016Presto at Hadoop Summit 2016
Presto at Hadoop Summit 2016kbajda
 
2 one spot redshift bigdatacamp 1.02
2 one spot redshift bigdatacamp 1.022 one spot redshift bigdatacamp 1.02
2 one spot redshift bigdatacamp 1.02Valerie Akinson Brown
 
2 one spot redshift bigdatacamp 1.02
2 one spot redshift bigdatacamp 1.022 one spot redshift bigdatacamp 1.02
2 one spot redshift bigdatacamp 1.02BigDataCamp
 
2 one spot redshift bigdatacamp 1.02
2 one spot redshift bigdatacamp 1.022 one spot redshift bigdatacamp 1.02
2 one spot redshift bigdatacamp 1.02Valerie Akinson Brown
 
O365Engage17 - How to Automate SharePoint Provisioning with PNP Framework
O365Engage17 - How to Automate SharePoint Provisioning with PNP FrameworkO365Engage17 - How to Automate SharePoint Provisioning with PNP Framework
O365Engage17 - How to Automate SharePoint Provisioning with PNP FrameworkNCCOMMS
 
An Introduction to Pentaho Kettle
An Introduction to Pentaho KettleAn Introduction to Pentaho Kettle
An Introduction to Pentaho KettleDan Moore
 
ResourceSpace: Recent pains and future gains
ResourceSpace: Recent pains and future gainsResourceSpace: Recent pains and future gains
ResourceSpace: Recent pains and future gainsResourceSpace
 

Similar to Using Joomla, Zoo & SOLR to power Asia's Largest Auction House (20)

Ilyin drupal-solr 0
Ilyin drupal-solr 0Ilyin drupal-solr 0
Ilyin drupal-solr 0
 
Word Press at Scale - WordCamp Minneapolis
Word Press at Scale - WordCamp MinneapolisWord Press at Scale - WordCamp Minneapolis
Word Press at Scale - WordCamp Minneapolis
 
Introduction to SQL Alchemy - SyPy June 2013
Introduction to SQL Alchemy - SyPy June 2013Introduction to SQL Alchemy - SyPy June 2013
Introduction to SQL Alchemy - SyPy June 2013
 
When to Use MongoDB
When to Use MongoDBWhen to Use MongoDB
When to Use MongoDB
 
Maximizing the Impact of Institutional Knowledge Using DSpace
Maximizing the Impact of Institutional Knowledge Using DSpaceMaximizing the Impact of Institutional Knowledge Using DSpace
Maximizing the Impact of Institutional Knowledge Using DSpace
 
MongoDB Basics
MongoDB BasicsMongoDB Basics
MongoDB Basics
 
Scaling Pinterest
Scaling PinterestScaling Pinterest
Scaling Pinterest
 
Drupal 7 and SolR
Drupal 7 and SolRDrupal 7 and SolR
Drupal 7 and SolR
 
JSON-LD and SHACL for Knowledge Graphs
JSON-LD and SHACL for Knowledge GraphsJSON-LD and SHACL for Knowledge Graphs
JSON-LD and SHACL for Knowledge Graphs
 
Big data on google platform dev fest presentation
Big data on google platform   dev fest presentationBig data on google platform   dev fest presentation
Big data on google platform dev fest presentation
 
MongoDB meetup at Hike
MongoDB meetup at HikeMongoDB meetup at Hike
MongoDB meetup at Hike
 
Real time analytics on deep learning @ strata data 2019
Real time analytics on deep learning @ strata data 2019Real time analytics on deep learning @ strata data 2019
Real time analytics on deep learning @ strata data 2019
 
Introduction To Pentaho Kettle
Introduction To Pentaho KettleIntroduction To Pentaho Kettle
Introduction To Pentaho Kettle
 
Presto at Hadoop Summit 2016
Presto at Hadoop Summit 2016Presto at Hadoop Summit 2016
Presto at Hadoop Summit 2016
 
2 one spot redshift bigdatacamp 1.02
2 one spot redshift bigdatacamp 1.022 one spot redshift bigdatacamp 1.02
2 one spot redshift bigdatacamp 1.02
 
2 one spot redshift bigdatacamp 1.02
2 one spot redshift bigdatacamp 1.022 one spot redshift bigdatacamp 1.02
2 one spot redshift bigdatacamp 1.02
 
2 one spot redshift bigdatacamp 1.02
2 one spot redshift bigdatacamp 1.022 one spot redshift bigdatacamp 1.02
2 one spot redshift bigdatacamp 1.02
 
O365Engage17 - How to Automate SharePoint Provisioning with PNP Framework
O365Engage17 - How to Automate SharePoint Provisioning with PNP FrameworkO365Engage17 - How to Automate SharePoint Provisioning with PNP Framework
O365Engage17 - How to Automate SharePoint Provisioning with PNP Framework
 
An Introduction to Pentaho Kettle
An Introduction to Pentaho KettleAn Introduction to Pentaho Kettle
An Introduction to Pentaho Kettle
 
ResourceSpace: Recent pains and future gains
ResourceSpace: Recent pains and future gainsResourceSpace: Recent pains and future gains
ResourceSpace: Recent pains and future gains
 

More from Parth Lawate

Quizzo - NDEAR Compliant , Beckn Driven, Gamified Learning Application
Quizzo - NDEAR Compliant , Beckn Driven, Gamified Learning Application Quizzo - NDEAR Compliant , Beckn Driven, Gamified Learning Application
Quizzo - NDEAR Compliant , Beckn Driven, Gamified Learning Application Parth Lawate
 
Minimum viable product @headstart pune Feb 2020
Minimum viable product @headstart pune Feb 2020Minimum viable product @headstart pune Feb 2020
Minimum viable product @headstart pune Feb 2020Parth Lawate
 
Wow! you built this with joomla!
Wow! you built this with joomla!Wow! you built this with joomla!
Wow! you built this with joomla!Parth Lawate
 
The Unite Framework
The Unite Framework The Unite Framework
The Unite Framework Parth Lawate
 
The Spirit of Opensource - contribution as a strategy for growth and innova...
The Spirit of Opensource   - contribution as a strategy for growth and innova...The Spirit of Opensource   - contribution as a strategy for growth and innova...
The Spirit of Opensource - contribution as a strategy for growth and innova...Parth Lawate
 
Techjoomla Infrastructure Extensions - Adding an Enterprise Layer to Joomla!
Techjoomla Infrastructure Extensions - Adding an Enterprise Layer to Joomla!Techjoomla Infrastructure Extensions - Adding an Enterprise Layer to Joomla!
Techjoomla Infrastructure Extensions - Adding an Enterprise Layer to Joomla!Parth Lawate
 
Powering india's largest learning platform for school kids with joomla
Powering india's largest learning platform for school kids with joomlaPowering india's largest learning platform for school kids with joomla
Powering india's largest learning platform for school kids with joomlaParth Lawate
 
The spirit of Opensource - lets plan to contribute ! @JWC16
The spirit of Opensource - lets plan to contribute ! @JWC16The spirit of Opensource - lets plan to contribute ! @JWC16
The spirit of Opensource - lets plan to contribute ! @JWC16Parth Lawate
 
Shika 1.0 Launch Presentation
Shika 1.0 Launch PresentationShika 1.0 Launch Presentation
Shika 1.0 Launch PresentationParth Lawate
 
Getting Involved with Joomla - Why and how to contribute
Getting Involved with Joomla - Why and how to contributeGetting Involved with Joomla - Why and how to contribute
Getting Involved with Joomla - Why and how to contributeParth Lawate
 
Building it right with Joomla 3 !
Building it right with Joomla 3 !Building it right with Joomla 3 !
Building it right with Joomla 3 !Parth Lawate
 
Using the Joomla Framework for Internet of Things (IoT) Case for Lighting Con...
Using the Joomla Framework for Internet of Things (IoT) Case for Lighting Con...Using the Joomla Framework for Internet of Things (IoT) Case for Lighting Con...
Using the Joomla Framework for Internet of Things (IoT) Case for Lighting Con...Parth Lawate
 
Joomla 3.3 Presentation for the First Joomla Google Hangout ever !
Joomla 3.3 Presentation for the First Joomla Google Hangout ever ! Joomla 3.3 Presentation for the First Joomla Google Hangout ever !
Joomla 3.3 Presentation for the First Joomla Google Hangout ever ! Parth Lawate
 
Joomla Day Netherlands - Building the Joomla Marketing Strategy..
Joomla Day Netherlands - Building the Joomla Marketing Strategy..Joomla Day Netherlands - Building the Joomla Marketing Strategy..
Joomla Day Netherlands - Building the Joomla Marketing Strategy..Parth Lawate
 
Socialads Webinar for CloudAccess on 4th September 2013 valid upto SAv2.9
Socialads Webinar for CloudAccess on 4th September 2013 valid upto SAv2.9Socialads Webinar for CloudAccess on 4th September 2013 valid upto SAv2.9
Socialads Webinar for CloudAccess on 4th September 2013 valid upto SAv2.9Parth Lawate
 
Invitex Webinar Presentation -11th September '13 for CloudAccess Webinar - Va...
Invitex Webinar Presentation -11th September '13 for CloudAccess Webinar - Va...Invitex Webinar Presentation -11th September '13 for CloudAccess Webinar - Va...
Invitex Webinar Presentation -11th September '13 for CloudAccess Webinar - Va...Parth Lawate
 
Monetize Your Joomla Website with Techjoomla !
Monetize Your Joomla Website with Techjoomla !Monetize Your Joomla Website with Techjoomla !
Monetize Your Joomla Website with Techjoomla !Parth Lawate
 
Joomla Appstore/ JoomRepo/ a new JED
Joomla Appstore/ JoomRepo/ a new JEDJoomla Appstore/ JoomRepo/ a new JED
Joomla Appstore/ JoomRepo/ a new JEDParth Lawate
 
Joomla presentation for PHPCamp 2011
Joomla presentation for PHPCamp 2011Joomla presentation for PHPCamp 2011
Joomla presentation for PHPCamp 2011Parth Lawate
 
REST API for Joomla
REST API for JoomlaREST API for Joomla
REST API for JoomlaParth Lawate
 

More from Parth Lawate (20)

Quizzo - NDEAR Compliant , Beckn Driven, Gamified Learning Application
Quizzo - NDEAR Compliant , Beckn Driven, Gamified Learning Application Quizzo - NDEAR Compliant , Beckn Driven, Gamified Learning Application
Quizzo - NDEAR Compliant , Beckn Driven, Gamified Learning Application
 
Minimum viable product @headstart pune Feb 2020
Minimum viable product @headstart pune Feb 2020Minimum viable product @headstart pune Feb 2020
Minimum viable product @headstart pune Feb 2020
 
Wow! you built this with joomla!
Wow! you built this with joomla!Wow! you built this with joomla!
Wow! you built this with joomla!
 
The Unite Framework
The Unite Framework The Unite Framework
The Unite Framework
 
The Spirit of Opensource - contribution as a strategy for growth and innova...
The Spirit of Opensource   - contribution as a strategy for growth and innova...The Spirit of Opensource   - contribution as a strategy for growth and innova...
The Spirit of Opensource - contribution as a strategy for growth and innova...
 
Techjoomla Infrastructure Extensions - Adding an Enterprise Layer to Joomla!
Techjoomla Infrastructure Extensions - Adding an Enterprise Layer to Joomla!Techjoomla Infrastructure Extensions - Adding an Enterprise Layer to Joomla!
Techjoomla Infrastructure Extensions - Adding an Enterprise Layer to Joomla!
 
Powering india's largest learning platform for school kids with joomla
Powering india's largest learning platform for school kids with joomlaPowering india's largest learning platform for school kids with joomla
Powering india's largest learning platform for school kids with joomla
 
The spirit of Opensource - lets plan to contribute ! @JWC16
The spirit of Opensource - lets plan to contribute ! @JWC16The spirit of Opensource - lets plan to contribute ! @JWC16
The spirit of Opensource - lets plan to contribute ! @JWC16
 
Shika 1.0 Launch Presentation
Shika 1.0 Launch PresentationShika 1.0 Launch Presentation
Shika 1.0 Launch Presentation
 
Getting Involved with Joomla - Why and how to contribute
Getting Involved with Joomla - Why and how to contributeGetting Involved with Joomla - Why and how to contribute
Getting Involved with Joomla - Why and how to contribute
 
Building it right with Joomla 3 !
Building it right with Joomla 3 !Building it right with Joomla 3 !
Building it right with Joomla 3 !
 
Using the Joomla Framework for Internet of Things (IoT) Case for Lighting Con...
Using the Joomla Framework for Internet of Things (IoT) Case for Lighting Con...Using the Joomla Framework for Internet of Things (IoT) Case for Lighting Con...
Using the Joomla Framework for Internet of Things (IoT) Case for Lighting Con...
 
Joomla 3.3 Presentation for the First Joomla Google Hangout ever !
Joomla 3.3 Presentation for the First Joomla Google Hangout ever ! Joomla 3.3 Presentation for the First Joomla Google Hangout ever !
Joomla 3.3 Presentation for the First Joomla Google Hangout ever !
 
Joomla Day Netherlands - Building the Joomla Marketing Strategy..
Joomla Day Netherlands - Building the Joomla Marketing Strategy..Joomla Day Netherlands - Building the Joomla Marketing Strategy..
Joomla Day Netherlands - Building the Joomla Marketing Strategy..
 
Socialads Webinar for CloudAccess on 4th September 2013 valid upto SAv2.9
Socialads Webinar for CloudAccess on 4th September 2013 valid upto SAv2.9Socialads Webinar for CloudAccess on 4th September 2013 valid upto SAv2.9
Socialads Webinar for CloudAccess on 4th September 2013 valid upto SAv2.9
 
Invitex Webinar Presentation -11th September '13 for CloudAccess Webinar - Va...
Invitex Webinar Presentation -11th September '13 for CloudAccess Webinar - Va...Invitex Webinar Presentation -11th September '13 for CloudAccess Webinar - Va...
Invitex Webinar Presentation -11th September '13 for CloudAccess Webinar - Va...
 
Monetize Your Joomla Website with Techjoomla !
Monetize Your Joomla Website with Techjoomla !Monetize Your Joomla Website with Techjoomla !
Monetize Your Joomla Website with Techjoomla !
 
Joomla Appstore/ JoomRepo/ a new JED
Joomla Appstore/ JoomRepo/ a new JEDJoomla Appstore/ JoomRepo/ a new JED
Joomla Appstore/ JoomRepo/ a new JED
 
Joomla presentation for PHPCamp 2011
Joomla presentation for PHPCamp 2011Joomla presentation for PHPCamp 2011
Joomla presentation for PHPCamp 2011
 
REST API for Joomla
REST API for JoomlaREST API for Joomla
REST API for Joomla
 

Recently uploaded

Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 

Recently uploaded (20)

20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 

Using Joomla, Zoo & SOLR to power Asia's Largest Auction House

  • 1. SOLR + Joomla powering the catalog of Asia's Largest Auction House
  • 2. Parth Lawate Strategic Marketing Manager Joomla CEO, Techjoomla, Tekdi Web Solutions @parthlawate, @techjoomla www.techjoomla.com Cook Bookworm Gardener JUG Pune Joomla Freak Trekking Entrepreneur Joomla Day India Open Source Software Architecht Marketing Content Strategy Hiking
  • 3. Tekdi Technologies Pvt. Ltd. @tekdinet tekdi.net IOS Apps CRM Magento E Learning Ecommerce Joomla Custom Apps Android CMS HTML5 Social Networks
  • 4. Techjoomla. For All things Joomla @techjoomla techjoomla.com jGive People Suggest jomLike JTicketing J!Bolo Broadcast Invitex Email Beautifier SocialAds J!MailAlerts REST API Payments API Social API Quick2Cart
  • 5. Quick Facts ● The client is a major Art & Auction house in India & is one of the largest in Asia. ● Data collation over a period of 20+ years ● Over 500,000 records with complex interrelations.
  • 6. Quick Facts ● Complex data structure with 100+ parameters /fields on each data type ● Graphics Heavy – All artifacts have High resolution images
  • 7. The Technical Challenge ● Over 100,000 records in the first phase of migration ● Extremely complex data relations ● Complex Data types & Record parameter volume & complexity
  • 8. The Human Challnge ● Use of MS Excel for years to manage their knowledge base before we came on board ● Working with the client's research & archivist team who had almost no knowledge of any kind of web technologies ● Getting the team of traditional archivists to adopt a modern system.
  • 9. The Solution ● The data complexity & relations called for using a CCK ● We Chose Zoo to serve as a base for all the customisations to come ● Custom apps based on this architechture.
  • 10. Term Glossary ● Classification – First level cartegorisation eg. ANTQ ● Sub Classification – arm ● Artifact – Actual Record ● Masterlists – Records that can be used as Associated records or as a link between 2 or more records
  • 11. Starting Small ● 9 Classifications ● 50 + Subsclassifications ● 50,000 Artifacts
  • 12. The Work with Zoo ● Custom field types ● Custom association plugins in order to create records from relations ● Custom views
  • 13. The Early Search ● Custom extension for parametric search ● One table per classificaton ● CRON based indexer ● MySQL powered with Natural language support ● Using MySQL soundex for 'did you mean' feature
  • 14. We Want Excel ! ● Though we got the archivists to use web forms.. they still missed the ease of excel ● So we gave it to them ! With Hanson table based Mass Edit view for Zoo.
  • 15. Bulk Processing's gotta be there ! ● Bulk Edit ● Bulk Delete ● Bulk Add ● Custom Importing Tools with volume processing & automapping
  • 16. The Data today ● 12 Classifications ● 100 + sub classifications ● 8 Masterlists ● 200,000 artifacts The Baby's growing up !
  • 17. More Data called for an architechture upgrade
  • 18. Need for a better search ● 200,000 Records ● Zoo Data structure isnt optimised for search ● MYSQL based indexer would hit limits down the line. Unions across 9 tables (which could increase) would make it slower
  • 19. Need for a better search... ● Single & 2 letter autosuggest not supported by Mysql (3 char min limit for LIKE) ● Normal search was not as fast as expected (Brought down load time for ~0.8secs to 0.3 secs)
  • 20. Getting the data ready for SOLR ● MYSQL Indexer from earlier phase modified to create a Data normaliser to push data to SOLR ● CLI script that reads records to populate SOLR index ● Using the PHP-SOLR library
  • 21. Osianama.com PHP-SOLR Library Browser SOLR Main Index Suggestions Index planned Getting SOLR into the picture ● Custom Search replaced by SOLR ● SOLR hosted on Separate Amozon instance ● Initial Implementation was only for search
  • 22. Benefits ● Much better natural language search, ● Better relevance scoring ● Full reindex everyday ● Even browsing is now SOLR powered which means MORE SPEED ! ● Record counts per category & sub-category easily achieved using faceting ● Now using SOLR's suggester module ● Using separate 'cores' for main index and suggest terms index
  • 23. Whats coming ? ● Autosuggest directly works off SOLR (currently piped through PHP) ● Implement delta indexing, currently not implemented due to multitude of relational data. ● Change in a bottom level record needs to flow through to all associations
  • 24. What else is so awesome about this ?
  • 25. HTML5 Local Storage ● HTML5 Local storage is being used to cache data locally & load used data faster ● Sets the road for offline use in the future !
  • 26. Google Image Search anyone ? ● Ajax Grid pagination like Google Images ● Preloading & caching of images, CDN backed delivery
  • 27. IOS App for IPad ● Powered by RESTful Webservices writen on top of Joomla using com_api ● Initial version developed in HTML5+Cordova (Phonegap) ● Supports offline use of alredy viewed data
  • 28. Even More ! The Project is under continuous development. The features here only cover development at the point this presentation was made. ● Online Sale of Images , Downloads & Rights Managements ● Research & Teaching tools ● Social Network ● Subscription based privileged access
  • 29. Thank You ! ● Questions ? ● Interested in developing something similar ? Drop us an email ! contact@techjoomla.com Twitter @techjoomla | @parthlawate