SlideShare a Scribd company logo
1 of 21
The LAKE Experience
Stefano Cossu, Director of Application Services, the Art Institute of Chicago
Kevin Ford, Senior Software Architect, the Art Institute of Chicago
Museums and The Web 2017—Cleveland OH, April 22nd 2017
A Voyage Into Unknown Museum DAM Waters
(And Back)
Rise & Decline of City–States
● Home-grown software
● Home-grown cataloging practices
● Proprietary vocabularies and authority lists
● Department-specific file & metadata stores
Not So Long Ago:
https://commons.wikimedia.org/wiki/File:Cinta_muraria_di_Palmanova.jpg
Perfect for a self-contained system
Rise & Decline of City–States
Suddenly:
https://commons.wikimedia.org/wiki/File:16th_century_Portuguese_Spanish_trade_routes.png
https://commons.wikimedia.org/wiki/File:Ralamb_Sipahi.jpg
Rise & Decline of City–States
● Data fragmentation
● Changing cataloging practices
● Integration is a challenge (if
ever possible)
● Data preserwhat?
● Spreadsheet infestation
Hence:
https://commons.wikimedia.org/wiki/File:MS_Ghent_-_Battle_of_Tewkesbury.jpg
Here We Go
● Clear separation of purposes
● Emphasis on communication channels
● As little custom code as possible
Alright then:
https://commons.wikimedia.org/wiki/File:Ship%27s_limited_channel_effects_(Blockage_Effects)_NT.PNG
Here We Go
● Collection Management
● Digital Asset Management
● Preservation
● Image ordering, production and delivery
● Publishing to Web, trusted parties and other channels
We Will Integrate:
Here We Go
● Images (pretty and ugly)
● Interpretive media
● Curatorial and Registration paperwork
● Conservation documents
● Institutional archives
● And much more...
We Will Store:
● Works (Objects)
● Agents
● Places
● Exhibitions
● Transactions
● Shipments
Related To:
Here We Go
And Qualify Relationships:
Here We Go
Here We Go
● LAKEshore → DAMS (Hydra)
● Combine → ETL (custom Python code)
● LAKE repo → Institutional repository (Fedora)
● Geneva → Event framework (Apache Camel)
● Indices → Solr, triplestore (Blazegraph)
● LPM → LAKE Public Mirror (Fedora + Solr
+ IIIF server)
Resulting in:
Whither LAKE?
● Research & Design learning curve
Completely new concepts
Completely new technologies
For Starters:
● Implementation issues
Alpha software
Upstream development timeline
Legacy systems grumbling
Followed By:
Whither LAKE?
● Data Mapping (empathy please)
Decades of changing conventions
● Data migration
285K images, 50Tb
Scaling: tiny issues become huge
Performance & monitoring
On the Side:
● After full-institutional release
Long-term sustainability
Adoption management
Scheduling digitization projects
For Dessert:
Whither LAKE?
Whither LAKE?
● Beneficial only in complex setups
● Deployment templates are imperative
● Find tools to monitor systems collectively
● Adapt layout from ongoing monitoring
Distributed Architecture: Lessons Learned
Whither LAKE?
Whither LAKE?
● Necessary with complex workflows
● Beautiful when it works
● Use tools to monitor the path of messages
● Users too must understand UX implications
Asynchronous Architecture: Lessons Learned
Was It Worth It?
YES BUT
Was It Worth It?
Yes:
● Deeper understanding of our data & systems
● It's working! (>600K resources in production)
● Achieved better data quality and access
● It is expandable
Was It Worth It?
But:
● (Significant) specialized skills required
● Still a lot of work to do
● Long-term payoff → Hard to sell
Was It Worth It?
But, Yes?
● Long-term advantages are just beginning to show
● LAKE is open source
● Container technology can simplify adoption
● Anyone interested?
Thank You
Stefano Cossu scossu@artic.edu @industrievisive
Kevin Ford kford1@artic.edu @3windmills
https://github.com/aic-collections/aicdams-lakeshore/

More Related Content

Similar to Cossu ford the_lake_experience_mw2017

Labours of Love & Convenience - Open Repositories 2018
Labours of Love & Convenience - Open Repositories 2018Labours of Love & Convenience - Open Repositories 2018
Labours of Love & Convenience - Open Repositories 2018Stefano Cossu
 
Scala Days Highlights | BoldRadius
Scala Days Highlights | BoldRadiusScala Days Highlights | BoldRadius
Scala Days Highlights | BoldRadiusBoldRadius Solutions
 
Apache Web Services in the Real World, an E-Science Perspective
Apache Web Services in the Real World, an E-Science PerspectiveApache Web Services in the Real World, an E-Science Perspective
Apache Web Services in the Real World, an E-Science PerspectiveSrinath Perera
 
Understanding Hadoop
Understanding HadoopUnderstanding Hadoop
Understanding HadoopAhmed Ossama
 
Technologie Proche: Imagining the Archival Systems of Tomorrow With the Tools...
Technologie Proche: Imagining the Archival Systems of Tomorrow With the Tools...Technologie Proche: Imagining the Archival Systems of Tomorrow With the Tools...
Technologie Proche: Imagining the Archival Systems of Tomorrow With the Tools...Artefactual Systems - AtoM
 
The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...
The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...
The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...Docker, Inc.
 
Archival Technologies
Archival TechnologiesArchival Technologies
Archival TechnologiesCliff Landis
 
Southwickc lampert lodlam_training
Southwickc lampert lodlam_trainingSouthwickc lampert lodlam_training
Southwickc lampert lodlam_trainingssouthwick
 
Think Big - How to Design a Big Data Information Architecture
Think Big - How to Design a Big Data Information ArchitectureThink Big - How to Design a Big Data Information Architecture
Think Big - How to Design a Big Data Information ArchitectureInside Analysis
 
The Legion Programming Model for HPC
The Legion Programming Model for HPCThe Legion Programming Model for HPC
The Legion Programming Model for HPCinside-BigData.com
 
UK Community day 20180427 Microsoft Flow hackathon
UK Community day 20180427 Microsoft Flow hackathonUK Community day 20180427 Microsoft Flow hackathon
UK Community day 20180427 Microsoft Flow hackathonPenny Coventry
 
Lambda architecture @ Indix
Lambda architecture @ IndixLambda architecture @ Indix
Lambda architecture @ IndixRajesh Muppalla
 
Architecting Agile Data Applications for Scale
Architecting Agile Data Applications for ScaleArchitecting Agile Data Applications for Scale
Architecting Agile Data Applications for ScaleDatabricks
 
Intro to Front End Frameworks
 Intro to Front End Frameworks Intro to Front End Frameworks
Intro to Front End FrameworksProduct School
 
Spca2014 practical large scale migration guidance v1.0 andries den haan
Spca2014 practical large scale migration guidance v1.0 andries den haanSpca2014 practical large scale migration guidance v1.0 andries den haan
Spca2014 practical large scale migration guidance v1.0 andries den haanNCCOMMS
 
Practical large scale migration guidance
Practical large scale migration guidancePractical large scale migration guidance
Practical large scale migration guidanceAndries den Haan
 

Similar to Cossu ford the_lake_experience_mw2017 (20)

Labours of Love & Convenience - Open Repositories 2018
Labours of Love & Convenience - Open Repositories 2018Labours of Love & Convenience - Open Repositories 2018
Labours of Love & Convenience - Open Repositories 2018
 
Scala Days Highlights | BoldRadius
Scala Days Highlights | BoldRadiusScala Days Highlights | BoldRadius
Scala Days Highlights | BoldRadius
 
Apache Web Services in the Real World, an E-Science Perspective
Apache Web Services in the Real World, an E-Science PerspectiveApache Web Services in the Real World, an E-Science Perspective
Apache Web Services in the Real World, an E-Science Perspective
 
BDIA Findings
BDIA FindingsBDIA Findings
BDIA Findings
 
Understanding Hadoop
Understanding HadoopUnderstanding Hadoop
Understanding Hadoop
 
Technologie Proche: Imagining the Archival Systems of Tomorrow With the Tools...
Technologie Proche: Imagining the Archival Systems of Tomorrow With the Tools...Technologie Proche: Imagining the Archival Systems of Tomorrow With the Tools...
Technologie Proche: Imagining the Archival Systems of Tomorrow With the Tools...
 
The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...
The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...
The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...
 
Archival Technologies
Archival TechnologiesArchival Technologies
Archival Technologies
 
Southwickc lampert lodlam_training
Southwickc lampert lodlam_trainingSouthwickc lampert lodlam_training
Southwickc lampert lodlam_training
 
Think Big - How to Design a Big Data Information Architecture
Think Big - How to Design a Big Data Information ArchitectureThink Big - How to Design a Big Data Information Architecture
Think Big - How to Design a Big Data Information Architecture
 
The Legion Programming Model for HPC
The Legion Programming Model for HPCThe Legion Programming Model for HPC
The Legion Programming Model for HPC
 
UK Community day 20180427 Microsoft Flow hackathon
UK Community day 20180427 Microsoft Flow hackathonUK Community day 20180427 Microsoft Flow hackathon
UK Community day 20180427 Microsoft Flow hackathon
 
Lambda architecture @ Indix
Lambda architecture @ IndixLambda architecture @ Indix
Lambda architecture @ Indix
 
Changing Platforms
Changing PlatformsChanging Platforms
Changing Platforms
 
Transformers in 2021
Transformers in 2021Transformers in 2021
Transformers in 2021
 
Architecting Agile Data Applications for Scale
Architecting Agile Data Applications for ScaleArchitecting Agile Data Applications for Scale
Architecting Agile Data Applications for Scale
 
Intro to Front End Frameworks
 Intro to Front End Frameworks Intro to Front End Frameworks
Intro to Front End Frameworks
 
Spca2014 practical large scale migration guidance v1.0 andries den haan
Spca2014 practical large scale migration guidance v1.0 andries den haanSpca2014 practical large scale migration guidance v1.0 andries den haan
Spca2014 practical large scale migration guidance v1.0 andries den haan
 
Practical large scale migration guidance
Practical large scale migration guidancePractical large scale migration guidance
Practical large scale migration guidance
 
Upgrading maps with Linked Data
Upgrading maps with Linked DataUpgrading maps with Linked Data
Upgrading maps with Linked Data
 

More from Stefano Cossu

Stefano_Cossu_OR23_deck.pdf
Stefano_Cossu_OR23_deck.pdfStefano_Cossu_OR23_deck.pdf
Stefano_Cossu_OR23_deck.pdfStefano Cossu
 
The Oxford Common File Layout
The Oxford Common File LayoutThe Oxford Common File Layout
The Oxford Common File LayoutStefano Cossu
 
Scossu gdi iiif_r+d_report_2019
Scossu gdi iiif_r+d_report_2019Scossu gdi iiif_r+d_report_2019
Scossu gdi iiif_r+d_report_2019Stefano Cossu
 
Brace yourselves, the Archives are Coming – Code4Lib 2020, Pittsburgh
Brace yourselves, the Archives are Coming – Code4Lib 2020, PittsburghBrace yourselves, the Archives are Coming – Code4Lib 2020, Pittsburgh
Brace yourselves, the Archives are Coming – Code4Lib 2020, PittsburghStefano Cossu
 
IIIF at the Getty: Vision & Tactics
IIIF at the Getty: Vision & TacticsIIIF at the Getty: Vision & Tactics
IIIF at the Getty: Vision & TacticsStefano Cossu
 
Reconciliation is a Necessity – IIIF Meeting, Edinburgh 2018
Reconciliation is a Necessity – IIIF Meeting, Edinburgh 2018 Reconciliation is a Necessity – IIIF Meeting, Edinburgh 2018
Reconciliation is a Necessity – IIIF Meeting, Edinburgh 2018 Stefano Cossu
 
A Little Sweat Goes A Long Way - Museums and The Web 2016
A Little Sweat Goes A Long Way - Museums and The Web 2016A Little Sweat Goes A Long Way - Museums and The Web 2016
A Little Sweat Goes A Long Way - Museums and The Web 2016Stefano Cossu
 
Libraries, Archives, Museums discussion - MCN 2015
Libraries, Archives, Museums discussion - MCN 2015Libraries, Archives, Museums discussion - MCN 2015
Libraries, Archives, Museums discussion - MCN 2015Stefano Cossu
 
AIC Linked Open Data panel Museums and the Web 2015
AIC Linked Open Data panel Museums and the Web 2015AIC Linked Open Data panel Museums and the Web 2015
AIC Linked Open Data panel Museums and the Web 2015Stefano Cossu
 
Stefano Cossu, The Art Institute of Chicago - Open Repositories 2014 presenta...
Stefano Cossu, The Art Institute of Chicago - Open Repositories 2014 presenta...Stefano Cossu, The Art Institute of Chicago - Open Repositories 2014 presenta...
Stefano Cossu, The Art Institute of Chicago - Open Repositories 2014 presenta...Stefano Cossu
 

More from Stefano Cossu (11)

Stefano_Cossu_OR23_deck.pdf
Stefano_Cossu_OR23_deck.pdfStefano_Cossu_OR23_deck.pdf
Stefano_Cossu_OR23_deck.pdf
 
The Oxford Common File Layout
The Oxford Common File LayoutThe Oxford Common File Layout
The Oxford Common File Layout
 
Scossu gdi iiif_r+d_report_2019
Scossu gdi iiif_r+d_report_2019Scossu gdi iiif_r+d_report_2019
Scossu gdi iiif_r+d_report_2019
 
Brace yourselves, the Archives are Coming – Code4Lib 2020, Pittsburgh
Brace yourselves, the Archives are Coming – Code4Lib 2020, PittsburghBrace yourselves, the Archives are Coming – Code4Lib 2020, Pittsburgh
Brace yourselves, the Archives are Coming – Code4Lib 2020, Pittsburgh
 
Behind 12 sunsets
Behind 12 sunsetsBehind 12 sunsets
Behind 12 sunsets
 
IIIF at the Getty: Vision & Tactics
IIIF at the Getty: Vision & TacticsIIIF at the Getty: Vision & Tactics
IIIF at the Getty: Vision & Tactics
 
Reconciliation is a Necessity – IIIF Meeting, Edinburgh 2018
Reconciliation is a Necessity – IIIF Meeting, Edinburgh 2018 Reconciliation is a Necessity – IIIF Meeting, Edinburgh 2018
Reconciliation is a Necessity – IIIF Meeting, Edinburgh 2018
 
A Little Sweat Goes A Long Way - Museums and The Web 2016
A Little Sweat Goes A Long Way - Museums and The Web 2016A Little Sweat Goes A Long Way - Museums and The Web 2016
A Little Sweat Goes A Long Way - Museums and The Web 2016
 
Libraries, Archives, Museums discussion - MCN 2015
Libraries, Archives, Museums discussion - MCN 2015Libraries, Archives, Museums discussion - MCN 2015
Libraries, Archives, Museums discussion - MCN 2015
 
AIC Linked Open Data panel Museums and the Web 2015
AIC Linked Open Data panel Museums and the Web 2015AIC Linked Open Data panel Museums and the Web 2015
AIC Linked Open Data panel Museums and the Web 2015
 
Stefano Cossu, The Art Institute of Chicago - Open Repositories 2014 presenta...
Stefano Cossu, The Art Institute of Chicago - Open Repositories 2014 presenta...Stefano Cossu, The Art Institute of Chicago - Open Repositories 2014 presenta...
Stefano Cossu, The Art Institute of Chicago - Open Repositories 2014 presenta...
 

Recently uploaded

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)wesley chun
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxKatpro Technologies
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?Antenna Manufacturer Coco
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?Igalia
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonetsnaman860154
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityPrincipled Technologies
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 

Recently uploaded (20)

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 

Cossu ford the_lake_experience_mw2017

  • 1. The LAKE Experience Stefano Cossu, Director of Application Services, the Art Institute of Chicago Kevin Ford, Senior Software Architect, the Art Institute of Chicago Museums and The Web 2017—Cleveland OH, April 22nd 2017 A Voyage Into Unknown Museum DAM Waters (And Back)
  • 2. Rise & Decline of City–States ● Home-grown software ● Home-grown cataloging practices ● Proprietary vocabularies and authority lists ● Department-specific file & metadata stores Not So Long Ago: https://commons.wikimedia.org/wiki/File:Cinta_muraria_di_Palmanova.jpg Perfect for a self-contained system
  • 3. Rise & Decline of City–States Suddenly: https://commons.wikimedia.org/wiki/File:16th_century_Portuguese_Spanish_trade_routes.png https://commons.wikimedia.org/wiki/File:Ralamb_Sipahi.jpg
  • 4. Rise & Decline of City–States ● Data fragmentation ● Changing cataloging practices ● Integration is a challenge (if ever possible) ● Data preserwhat? ● Spreadsheet infestation Hence: https://commons.wikimedia.org/wiki/File:MS_Ghent_-_Battle_of_Tewkesbury.jpg
  • 5. Here We Go ● Clear separation of purposes ● Emphasis on communication channels ● As little custom code as possible Alright then: https://commons.wikimedia.org/wiki/File:Ship%27s_limited_channel_effects_(Blockage_Effects)_NT.PNG
  • 6. Here We Go ● Collection Management ● Digital Asset Management ● Preservation ● Image ordering, production and delivery ● Publishing to Web, trusted parties and other channels We Will Integrate:
  • 7. Here We Go ● Images (pretty and ugly) ● Interpretive media ● Curatorial and Registration paperwork ● Conservation documents ● Institutional archives ● And much more... We Will Store: ● Works (Objects) ● Agents ● Places ● Exhibitions ● Transactions ● Shipments Related To:
  • 8. Here We Go And Qualify Relationships:
  • 10. Here We Go ● LAKEshore → DAMS (Hydra) ● Combine → ETL (custom Python code) ● LAKE repo → Institutional repository (Fedora) ● Geneva → Event framework (Apache Camel) ● Indices → Solr, triplestore (Blazegraph) ● LPM → LAKE Public Mirror (Fedora + Solr + IIIF server) Resulting in:
  • 11. Whither LAKE? ● Research & Design learning curve Completely new concepts Completely new technologies For Starters: ● Implementation issues Alpha software Upstream development timeline Legacy systems grumbling Followed By:
  • 12. Whither LAKE? ● Data Mapping (empathy please) Decades of changing conventions ● Data migration 285K images, 50Tb Scaling: tiny issues become huge Performance & monitoring On the Side: ● After full-institutional release Long-term sustainability Adoption management Scheduling digitization projects For Dessert:
  • 14. Whither LAKE? ● Beneficial only in complex setups ● Deployment templates are imperative ● Find tools to monitor systems collectively ● Adapt layout from ongoing monitoring Distributed Architecture: Lessons Learned
  • 16. Whither LAKE? ● Necessary with complex workflows ● Beautiful when it works ● Use tools to monitor the path of messages ● Users too must understand UX implications Asynchronous Architecture: Lessons Learned
  • 17. Was It Worth It? YES BUT
  • 18. Was It Worth It? Yes: ● Deeper understanding of our data & systems ● It's working! (>600K resources in production) ● Achieved better data quality and access ● It is expandable
  • 19. Was It Worth It? But: ● (Significant) specialized skills required ● Still a lot of work to do ● Long-term payoff → Hard to sell
  • 20. Was It Worth It? But, Yes? ● Long-term advantages are just beginning to show ● LAKE is open source ● Container technology can simplify adoption ● Anyone interested?
  • 21. Thank You Stefano Cossu scossu@artic.edu @industrievisive Kevin Ford kford1@artic.edu @3windmills https://github.com/aic-collections/aicdams-lakeshore/