SlideShare a Scribd company logo
1 of 13
Lightning Talk
   7 minutes to talk about

   Exporting and importing
   company-level data
   Sébastien Le Marchand




Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012
PROBLEMATIC




Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012   2
Problematic
   A classic data import/export use case

   + Transfer data between multiple environnements
      •Need for
                  « Selective » export or import, not all data…      Integration
                  Incremental import


      • don’t push DB dump !
                                                                        Q/A
   + Available tools
      •« Site scope » data
                  LAR export / import
                  Remote stagging                                    Acceptance
                  Resource Importer Plugin
      •Data outside « site scope » (company-level data)
                  ?... that’s the problem !
                                                                     Production

Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012                 3
Problematic
   What are company-level data ?

                                                              « Site-scope » data

                                                               Covered by Liferay ARchive native export /
                                                              import




                                                                « Company-scope » data

                                                                 No native feature to help us…




Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012                                       4
FRONTLINE FEEDBACK




Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012   5
Frontline feedback
   Example : roles permissions definitions

   + Default permissions for roles
      •A critical security configuration

                                                                                                        2
            1
                                                                                                    3

   + How-to ?
      •Without any tool (very painful) :
                  you need to write specific configuration procedure documentation (potentially huge)
                  You need to « replay » procedure manually (UI) for each environement (re-)initi
                  Unsafe (human error)


      Build a tool to do the work (go back at home earlier !)
                  Fast to replay
                  Repeatable (error-prone)
Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012                                      6
Frontline feedback
   Best practices to write import/export tools

   + Don’t work directly with database (SQL extract & load)
      •Possible application-level data integrity violations


   + Only use portal services to read and write data
      •Choose your weapon
                  Plain-old plugin, written in Java
                  Admin script, written in Groovy, Ruby, etc.


   + Be carreful about IDs
      •An entity ID is not the same in source environment and target environment
                  Do not rely on entity ID
                  Relations between entities must not be stored with IDs




Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012                 7
OUT-OF-THE-BOX TOOL




Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012   8
Out-of-the-box tool
   Discover a comprehensive ready-to-use tool

   + Hook Plugin
      •No user interface, actions only by file manipulation
                  Easy to use for Ops
      •Input and output files in the same directory (configurable)
                  ${liferay.home}/deploy/imex
   + Export
      Input
                  xxxx.export.properties (options)
      Output
                  xxxx.imex.zip (data archive)
   + Import
      Input
                  xxxx.import.properties (options)
                  xxxx.imex.zip (data archive)


Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012   9
Out-of-the-box tool
   Options overview

                                                                     xxxx.export.properties




Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012                            10
Out-of-the-box tool
   Data format overview




Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012   11
Out-of-the-box tool
   Get it !

   + Historically, internal developpement targeting Liferay 5.2.x
   + Now open-source project targeting Liferay 6.1
      •Beta release


   + Coverage :
      •Now
                  Roles & roles default permissions definitions
                  Sites (using LAR mechanisms)
      •Next releases
                  Extended coverage…


             http://sqli-imex.github.com

Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012   12
That's all folks!
   Thanks!

   +Im/Ex tool                                                       +Sébastien Le Marchand
      •http://sqli-imex.github.com                                    •slemarchand@sqli.com
                                                                      •@slemarchand


   +SQLI Group
      •http://www.sqli.com
      •http://www.entreprise-digitale.com
      •@SQLIEENTREPRISE

                                        > 1800 employees
                                        €170 M turnover
                                        Capability to deploy projects
                                        internationally



Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012                            13

More Related Content

Similar to Les2012 lightning talk_exporting_and_importing_company-level_data

E-GEN/iCAN
E-GEN/iCANE-GEN/iCAN
E-GEN/iCAN
teddi22
 
SQL Server Konferenz 2014 - SSIS & HDInsight
SQL Server Konferenz 2014 - SSIS & HDInsightSQL Server Konferenz 2014 - SSIS & HDInsight
SQL Server Konferenz 2014 - SSIS & HDInsight
Tillmann Eitelberg
 
RedisGraph A Low Latency Graph DB: Pieter Cailliau
RedisGraph A Low Latency Graph DB: Pieter CailliauRedisGraph A Low Latency Graph DB: Pieter Cailliau
RedisGraph A Low Latency Graph DB: Pieter Cailliau
Redis Labs
 

Similar to Les2012 lightning talk_exporting_and_importing_company-level_data (20)

OpenIO A3techlive in London - March 2016
OpenIO A3techlive in London - March 2016OpenIO A3techlive in London - March 2016
OpenIO A3techlive in London - March 2016
 
Zia: Case Management
Zia: Case ManagementZia: Case Management
Zia: Case Management
 
Complete open source IAM solution
Complete open source IAM solutionComplete open source IAM solution
Complete open source IAM solution
 
Digital Library Software
Digital Library SoftwareDigital Library Software
Digital Library Software
 
E-GEN/iCAN
E-GEN/iCANE-GEN/iCAN
E-GEN/iCAN
 
Recover 30% of your day with IBM Development Tools (Smarter Mainframe Develop...
Recover 30% of your day with IBM Development Tools (Smarter Mainframe Develop...Recover 30% of your day with IBM Development Tools (Smarter Mainframe Develop...
Recover 30% of your day with IBM Development Tools (Smarter Mainframe Develop...
 
10 Reasons Your Software Sucks 2014 - Tax Day Edition!
10 Reasons Your Software Sucks 2014 - Tax Day Edition!10 Reasons Your Software Sucks 2014 - Tax Day Edition!
10 Reasons Your Software Sucks 2014 - Tax Day Edition!
 
What's new in Elasticsearch v5
What's new in Elasticsearch v5What's new in Elasticsearch v5
What's new in Elasticsearch v5
 
SQL Server Konferenz 2014 - SSIS & HDInsight
SQL Server Konferenz 2014 - SSIS & HDInsightSQL Server Konferenz 2014 - SSIS & HDInsight
SQL Server Konferenz 2014 - SSIS & HDInsight
 
Confoo-Montreal-2016: Controlling Your Environments using Infrastructure as Code
Confoo-Montreal-2016: Controlling Your Environments using Infrastructure as CodeConfoo-Montreal-2016: Controlling Your Environments using Infrastructure as Code
Confoo-Montreal-2016: Controlling Your Environments using Infrastructure as Code
 
DevOps and the C64: what's your excuse
DevOps and the C64: what's your excuseDevOps and the C64: what's your excuse
DevOps and the C64: what's your excuse
 
Enterprise-Grade DevOps Solutions for a Start Up Budget
Enterprise-Grade DevOps Solutions for a Start Up BudgetEnterprise-Grade DevOps Solutions for a Start Up Budget
Enterprise-Grade DevOps Solutions for a Start Up Budget
 
RedisGraph A Low Latency Graph DB: Pieter Cailliau
RedisGraph A Low Latency Graph DB: Pieter CailliauRedisGraph A Low Latency Graph DB: Pieter Cailliau
RedisGraph A Low Latency Graph DB: Pieter Cailliau
 
Prometheus for the traditional datacenter
Prometheus for the traditional datacenterPrometheus for the traditional datacenter
Prometheus for the traditional datacenter
 
EclipseConEurope2012 SOA - Models As Operational Documentation
EclipseConEurope2012 SOA - Models As Operational DocumentationEclipseConEurope2012 SOA - Models As Operational Documentation
EclipseConEurope2012 SOA - Models As Operational Documentation
 
4Developers 2015: Sylius - E-Commerce framework for PHP - Pawel Jedrzejewski
4Developers 2015: Sylius - E-Commerce framework for PHP - Pawel Jedrzejewski4Developers 2015: Sylius - E-Commerce framework for PHP - Pawel Jedrzejewski
4Developers 2015: Sylius - E-Commerce framework for PHP - Pawel Jedrzejewski
 
Yii Framework - Do we really need another php framework?
Yii Framework - Do we really need another php framework?Yii Framework - Do we really need another php framework?
Yii Framework - Do we really need another php framework?
 
Doug McCune - Using Open Source Flex and ActionScript Projects
Doug McCune - Using Open Source Flex and ActionScript ProjectsDoug McCune - Using Open Source Flex and ActionScript Projects
Doug McCune - Using Open Source Flex and ActionScript Projects
 
Amplexor Drupal for the Enterprise seminar - evaluating Drupal for the Enterp...
Amplexor Drupal for the Enterprise seminar - evaluating Drupal for the Enterp...Amplexor Drupal for the Enterprise seminar - evaluating Drupal for the Enterp...
Amplexor Drupal for the Enterprise seminar - evaluating Drupal for the Enterp...
 
Storing the mental Model
Storing the mental ModelStoring the mental Model
Storing the mental Model
 

More from Sébastien Le Marchand

More from Sébastien Le Marchand (12)

Client Extensions - Episode 2 : Custom Element - 2024-04-30 - Liferay France ...
Client Extensions - Episode 2 : Custom Element - 2024-04-30 - Liferay France ...Client Extensions - Episode 2 : Custom Element - 2024-04-30 - Liferay France ...
Client Extensions - Episode 2 : Custom Element - 2024-04-30 - Liferay France ...
 
Let's revive the Liferay Marketplace together - /dev/24 2021
Let's revive the Liferay Marketplace together - /dev/24 2021Let's revive the Liferay Marketplace together - /dev/24 2021
Let's revive the Liferay Marketplace together - /dev/24 2021
 
Convert to liferay forms! - Liferay DEVCON 2021
Convert to liferay forms! - Liferay DEVCON 2021Convert to liferay forms! - Liferay DEVCON 2021
Convert to liferay forms! - Liferay DEVCON 2021
 
Convertissez vos formulaires ! - Liferay France Symposium 2020
Convertissez vos formulaires ! - Liferay France Symposium 2020Convertissez vos formulaires ! - Liferay France Symposium 2020
Convertissez vos formulaires ! - Liferay France Symposium 2020
 
Let’s discover and extend the Content Dashboard - /dev/24 2020
Let’s discover and extendthe Content Dashboard - /dev/24 2020Let’s discover and extendthe Content Dashboard - /dev/24 2020
Let’s discover and extend the Content Dashboard - /dev/24 2020
 
Relever les défis techniques d'un site internet multi-pays et multilingue - L...
Relever les défis techniques d'un site internet multi-pays et multilingue - L...Relever les défis techniques d'un site internet multi-pays et multilingue - L...
Relever les défis techniques d'un site internet multi-pays et multilingue - L...
 
Liferay Gogo Scripts - Liferay France User Group - meetup #13
Liferay Gogo Scripts -  Liferay France User Group - meetup #13Liferay Gogo Scripts -  Liferay France User Group - meetup #13
Liferay Gogo Scripts - Liferay France User Group - meetup #13
 
Back from Liferay DEVCON 2018 (Liferay France User Group Meetup #10 - 15/11/2...
Back from Liferay DEVCON 2018 (Liferay France User Group Meetup #10 - 15/11/2...Back from Liferay DEVCON 2018 (Liferay France User Group Meetup #10 - 15/11/2...
Back from Liferay DEVCON 2018 (Liferay France User Group Meetup #10 - 15/11/2...
 
Recap Liferay France Symposium 2018
Recap Liferay France Symposium 2018Recap Liferay France Symposium 2018
Recap Liferay France Symposium 2018
 
Instances multiples : les pièges à éviter (Liferay User Group France)
Instances multiples : les pièges à éviter (Liferay User Group France)Instances multiples : les pièges à éviter (Liferay User Group France)
Instances multiples : les pièges à éviter (Liferay User Group France)
 
Developing faster than ever (Liferay DEVCON 2017)
Developing faster than ever (Liferay DEVCON 2017)Developing faster than ever (Liferay DEVCON 2017)
Developing faster than ever (Liferay DEVCON 2017)
 
Développement : mettez le turbo ! - Liferay France Symposium 2017
Développement : mettez le turbo ! - Liferay France Symposium 2017Développement : mettez le turbo ! - Liferay France Symposium 2017
Développement : mettez le turbo ! - Liferay France Symposium 2017
 

Les2012 lightning talk_exporting_and_importing_company-level_data

  • 1. Lightning Talk 7 minutes to talk about Exporting and importing company-level data Sébastien Le Marchand Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012
  • 2. PROBLEMATIC Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012 2
  • 3. Problematic A classic data import/export use case + Transfer data between multiple environnements •Need for « Selective » export or import, not all data… Integration Incremental import • don’t push DB dump ! Q/A + Available tools •« Site scope » data LAR export / import Remote stagging Acceptance Resource Importer Plugin •Data outside « site scope » (company-level data) ?... that’s the problem ! Production Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012 3
  • 4. Problematic What are company-level data ? « Site-scope » data  Covered by Liferay ARchive native export / import « Company-scope » data  No native feature to help us… Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012 4
  • 5. FRONTLINE FEEDBACK Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012 5
  • 6. Frontline feedback Example : roles permissions definitions + Default permissions for roles •A critical security configuration 2 1 3 + How-to ? •Without any tool (very painful) : you need to write specific configuration procedure documentation (potentially huge) You need to « replay » procedure manually (UI) for each environement (re-)initi Unsafe (human error) Build a tool to do the work (go back at home earlier !) Fast to replay Repeatable (error-prone) Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012 6
  • 7. Frontline feedback Best practices to write import/export tools + Don’t work directly with database (SQL extract & load) •Possible application-level data integrity violations + Only use portal services to read and write data •Choose your weapon Plain-old plugin, written in Java Admin script, written in Groovy, Ruby, etc. + Be carreful about IDs •An entity ID is not the same in source environment and target environment Do not rely on entity ID Relations between entities must not be stored with IDs Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012 7
  • 8. OUT-OF-THE-BOX TOOL Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012 8
  • 9. Out-of-the-box tool Discover a comprehensive ready-to-use tool + Hook Plugin •No user interface, actions only by file manipulation Easy to use for Ops •Input and output files in the same directory (configurable) ${liferay.home}/deploy/imex + Export Input xxxx.export.properties (options) Output xxxx.imex.zip (data archive) + Import Input xxxx.import.properties (options) xxxx.imex.zip (data archive) Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012 9
  • 10. Out-of-the-box tool Options overview xxxx.export.properties Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012 10
  • 11. Out-of-the-box tool Data format overview Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012 11
  • 12. Out-of-the-box tool Get it ! + Historically, internal developpement targeting Liferay 5.2.x + Now open-source project targeting Liferay 6.1 •Beta release + Coverage : •Now Roles & roles default permissions definitions Sites (using LAR mechanisms) •Next releases Extended coverage… http://sqli-imex.github.com Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012 12
  • 13. That's all folks! Thanks! +Im/Ex tool +Sébastien Le Marchand •http://sqli-imex.github.com •slemarchand@sqli.com •@slemarchand +SQLI Group •http://www.sqli.com •http://www.entreprise-digitale.com •@SQLIEENTREPRISE > 1800 employees €170 M turnover Capability to deploy projects internationally Liferay Europe Symposium – Wiesbaden, Germany | 16-17 october 2012 13