The document discusses the 'Peak of Data Integration 2023' presentation by Michiel Schram, focusing on effective data handling without requiring high-performance machines. It outlines key strategies for achieving peak performance, including data reduction, utilizing database capabilities, and promoting collaboration. Various case studies illustrate practical data integration applications across environmental and municipal projects.

1. Peak performance

2. The
Peak
of
Data
Integration
20
23
You don’t need a super-duper
machine
to peak perform!

3. Presenter
The
Peak
of
Data
Integration
20
23
Michiel Schram
Operational Geo-datamanager
Sweco Nederland B.V.
I help organizations with the verifiable and reproducible exchange of data.

4. The
Peak
of
Data
Integration
20
23
Agenda
● Introduction
● What we do
● How you can do it
● Conclusion

5. The
Peak
of
Data
Integration
20
23
What we do

6. The
Peak
of
Data
Integration
20
23
Peak performance examples

7. The
Peak
of
Data
Integration
20
23
Validation platform
Enterprise validation platform for accepting
nature data from provinces for national
reporting.
1

8. The
Peak
of
Data
Integration
20
23
Green City Challenge
Proces lot’s of open data into information to
convince municipalities to create a more green
environment.
2

9. The
Peak
of
Data
Integration
20
23
Blockholm
Download platform for Swedisch Minecraft
worlds.
3

10. The
Peak
of
Data
Integration
20
23
Reporting
Processing large GML’s into
Word-documents for impact analyzes for
nature reserves.
4

11. The
Peak
of
Data
Integration
20
23
GaragePark
Scanning all industrial areas in the Netherlands
for possible extensions of garage parks.
5

12. The
Peak
of
Data
Integration
20
23
Environmental engineering
Making input data for environmental models.
6

13. The
Peak
of
Data
Integration
20
23
How you can do it

14. The
Peak
of
Data
Integration
20
23
Case
Setting up a workspace for calculating 1.6 million points

15. The
Peak
of
Data
Integration
20
23
Get receptors Run model
Get and
proces results
Write result
Do some
pre-pocessing

16. The
Peak
of
Data
Integration
20
23

17. The
Peak
of
Data
Integration
20
23

18. The
Peak
of
Data
Integration
20
23
1. Reduce amount of data

19. The
Peak
of
Data
Integration
20
23

20. The
Peak
of
Data
Integration
20
23

21. The
Peak
of
Data
Integration
20
23

22. The
Peak
of
Data
Integration
20
23

23. The
Peak
of
Data
Integration
20
23

24. The
Peak
of
Data
Integration
20
23

25. The
Peak
of
Data
Integration
20
23
2. Let the database do the job

26. The
Peak
of
Data
Integration
20
23

27. The
Peak
of
Data
Integration
20
23
o Tables
o Relations
o Constraints
o Views
o Queries
o Indexing
o Multiple connections

28. The
Peak
of
Data
Integration
20
23

29. The
Peak
of
Data
Integration
20
23
3. Be in controle

30. The
Peak
of
Data
Integration
20
23

31. The
Peak
of
Data
Integration
20
23

32. The
Peak
of
Data
Integration
20
23

33. The
Peak
of
Data
Integration
20
23

34. The
Peak
of
Data
Integration
20
23

35. The
Peak
of
Data
Integration
20
23

36. The
Peak
of
Data
Integration
20
23

37. The
Peak
of
Data
Integration
20
23

38. The
Peak
of
Data
Integration
20
23

39. The
Peak
of
Data
Integration
20
23

40. The
Peak
of
Data
Integration
20
23
Collaberate

41. The
Peak
of
Data
Integration
20
23

42. The
Peak
of
Data
Integration
20
23

43. The
Peak
of
Data
Integration
20
23
You don’t need a super-duper machine
to peak perform!
So what about….

44. The
Peak
of
Data
Integration
20
23
Intel Xeon W-2123
• Cores 4
• CPU threads 8
• Base clock speed3.6GHz
• Turbo clock speed 3.9GHz
• Memory 64GB
• Calculation time 1 day
Intel Xeon Gold 6138
• Cores 20
• CPU threads 40
• Base clock speed2GHz
• Turbo clock speed 3,7GHz
• Memory 16GB
• Calculation time 0.5 day

45. The
Peak
of
Data
Integration
20
23
Conclusion

46. The
Peak
of
Data
Integration
20
23
How you can peak perform
1. Reduce amount of data
2. Let the database do the job
3. Be in controle
4. Collaberate

47. ThankYou!
michiel.schram@sweco.nl