A talk given at Eclipse Con France in 2018 about microservices used in Geoscience. Similar talk was also given at Landmark LIFE conference.

1. GeoScience and µServices

2. Background
▌ Landmark would be one of the top 50 largest software companies
▌ Scientific and Engineering software for Energy sector
 Exploration
 Construction
 Production
▌ Neftex
 Exploration

4. What kind of developer are you?
▌ Experienced with Java (since 1.1 which is ~1998)
▌ Python hacker (what other kind is there?)
▌ Javascript noob (!! How hard can it be? Typescript, Uglify)
▌ Fortran deny all (literally don’t tell anyone)
▌ C++ dangerous (enough to be)
▌ C# not much (linux?)
▌ Wrote much UI and Server Architecture 

5. History Lesson

6. Happy Developers?
▌ Modern Programming Languages like Java and Python
 Developers became happy
 They write more code
Thing Why we liked it? What Happened?
3G/4G
Languages
(Java, C#, Python, C++)
Manage memory,
Powerful APIs, UI
toolkits
We wrote lots of code
IDEs
(Xemacs, JCreator,
Eclipse, IntelliJ,
VisualStudio)
It make our code
more colourful and
we could write more
We wrote lots of code
Frameworks
(RCP, Struts, JSF,
Django, ASP, etc. etc.)
Do more stuff,
Organise things
We wrote lots of code and
it was more efficient
Incremental
Programs
(OSGi, Jigsaw, etc.)
Our massive
programs still worked
We wrote lots of code and
it didn’t all have to fit in
memory!!

7. Introducing the Monolith
▌ Single Tier Software Application
▌ Horizontal Approach to Features
▌ Modularity is Code-Based
▌ Examples:
 Scientific Desktop Application
 Some Types of Server Application
− Thick client
− Multiple web page single server

8. Some problems I have hit with…
▌ Developers working horizontally, less efficiently
▌ Encapsulation of modules – entropy increases
▌ Multiple APIs doing the same task
 CORBA/RMI/JMS
 Swing/SWT/JavaFX
 DOM+SAX/Xpath/Castor/Gson/Jackson
▌ Ability to scale in various directions (scale cube)
▌ And… Deployment / Testing / Understanding / Agility
Despite all this
 Powerful,
 Well used and understood,
 Profitable
Monolithic, or nearly monolithic, software

9. Some I think which can be okay with…
▌Testing
 Well defined interactions
 Limited set of interoperable features
▌OSGi Services – one monolith multiple services
▌Developer ownership, interoperability
▌Cost
Monolithic, or nearly monolithic, software

10. Summary
▌ We have lots and lots and lots of code
▌ We find it hard to change it
▌ Customers/Users get
 releases infrequently
 more defects
 lower scale solutions
 …depressed

11. Service Oriented Architecture (SOA)
▌ Application is a collection of loosely coupled services
▌ One service
 Logically represents a business activity with a specified outcome.
 Self-contained.
 black box for its consumers.
 May consist of other underlying services
▌ Idea been around in one form or another since early 2000’s
µServices are one type of SOA

12. µServices
▌ Fine grained and light weight
▌ Modularity is enforced, each component easy to understand, develop
and test
▌ Small autonomous development teams
 Vertical development rather than horizontal
 Less for one developer to deal with / understand / have to compile
▌ IMPORTANT Services can scale independently and on the fly
▌ Technology choices weakly linked and easy to change

13. Team structure
 Product Management (exactly 1)
 Development Lead (exactly 1)
 Developers (1-10) (UI, µservice, database)
 Testing (0-3)
 UX (As needed)
 DevOps (1)
 Architect (As needed)

15. This is all very well but…
▌ How do we apply it to scientific algorithms?
▌ How will they scale?
 Different algorithms have different requirements
 Scale out in the cloud
▌ What kind of cloud? We are looking at DC/OS, Kubernetes and native:
AWS, Azure, Google Cloud etc.
▌ What about Big Data?

16. Case Study – Automatic Lithostratigraphy
▌ Python Research Code
 Machine learning
 Parallelizable algorithm each python process can work on one part of the problem
prediction
▌ Java µService using JSON-REST and Kafka deployed in a Jetty Server to a docker container
▌ Large amounts of data but highly parallel each run being only O(10)xO(100,000), for instance
9 1D datasets of size 25,000.
▌ Scale in:
 Y - (web service front end and data splitter) + (analysis)
 X - Many (analysis) processes
 Z – No need to scale

18. Standard Python Project Containing
R&D Code
Module for exposing running one facet
of the code by Kafka message
(competitive consumer)
A readme / getting started and other docs are
contained in the project
Jenkins build runs the tests and
deploys the python consumer to the
cluster as a docker image

19. Python client
µservice code using JSON-REST Spring
Boot, etc
Self contained set of unit tests and resources
Separate build which
• Downloads dependencies for developer
• Builds on Jenkins a ‘fat’ jar
• Deploys to DCOS
The build is deployed to docker image for
running on the cluster
A readme / getting started and other docs are
contained in the project

20. 2ms
▌ Binary artifacts can be published and consumed = modularity old skool
 E.g. IO packages
▌ µservice design patterns
 E.g. Aggregator
Bit of an side note: Artifactory and Aggregation
20ms
20ms 20ms
2ms
= 62ms
2ms
20ms
2ms
= 28ms
2ms
Aggregator

21. ▌ week6-alith-zoom.png1
Results of autolith in plotly (one prediction)

22. Thanks!
▌ Landmark for moving to microservices
▌ Gael and the team for organising the conference
▌ Devs and product management
▌ YOU for listening