Data processing concept applied to AEC project - migration of drawings from an old CAD system to modern 3D software

1. Brought to you by Knowledge Partner
To Infinity and beyond:
transforming 1000000 sq.m. of CAD
data to Revit models
Nikolay Gerasimov

2. 2
To Infinity and beyond
Content
0. About us
1. About the project
2. Data processing concept
3. 3 must-have skills
4. Project realisation
5. Lessons learnt
6. Q&A

3. 3
We are:
▪ 30+ employees from 8 countries
▪ Consultants in areas of BIM and data
management
▪ Developers of custom solutions and common
toolbox
▪ Providers of teaching courses for engineers and
PMs and web-based source
About us

4. 4
About me
▪ PhD in physics
▪ In AEC since 2006
▪ HVAC engineer, BIM manager, MEP BIM Expert…
▪ Worked in Russia, Germany, Austria

5. 5
About the project
▪ Cannot name the client because of legal reasons
▪ 60 buildings
▪ 27000 rooms
▪ 4 CAD systems (the main one is 40 years old!)
▪ 10 databases
▪ 2 Million objects on the drawings
▪ 1 drawings file with 1.4 Mln objects and 27000
viewports

6. 6
Data transfer goal
▪ The final goal is to continue the planning (i.e.
drawing) and FM in Revit
▪ Existing graphics must be converted into Revit
models with editable elements
▪ There were no clear requirements to the
models: client expected smooth and easy
transition without any efforts from his side
▪ The task is not to create something new but find
the way to transform existing things to another
form
▪ When you create new, you can always say: “Ok,
that’s what you get. Where’s my money?”
▪ Data processing is different – you can really fail.
But this is the main fun ☺

7. 7
Data Processing Concept
▪Data is a mathematical representation
of information
▪ Mathematics is a universal language
▪ It allows one to solve any problem predictably,
completely, reproducibly and robust
▪ It is not empirical
▪ Only mathematical problems can be automated
▪ Finding a proper representation can be the most
difficult step

8. 8
Fundamental skills for data processing
Diagramming
▪„Pathfinding“ problem
▪ Teaches you to think sequentially and
pragmatically
▪ Helps you to think of the Data as a whole
▪ Helps you to see Patterns and Bottle Necks
▪ Helps you to measure how close you are to the
goal
▪ Pretty means correct

9. 9
Fundamental skills for data processing
Database programming
▪ Tables are one of the most convenient form
to store data
▪ Databases is a 50 years old technology of
storing and processing data
▪ Helps you to further improve the vision of
data as a whole
▪ Gives you the idea of „relations“ between
data and the instruments to use these
relations
▪ Teaches you to ask questions
▪ Teaches you one of the easiest programming
languages - SQL

10. 10
Fundamental skills for data processing
Scripting
▪ The main problem of all „programming“ courses:
they teach you the language, not how it is
related to your problems
▪ Gives you the power to do absolutely anything
one can do with computers. And to be in charge
for that
▪ Allows you to write algorithms – instructions
how exactly you want to process the data – to
control performance
▪ Teaches you to control robustness of the
processing – what can go wrong and what
happens this case

11. 11
▪ Programming consists of a lot of Technologies
▪ You choose a proper one depending on the goal
▪ Python allows you to get the result
▪ C-languages allow you to build the road
Fundamental skills for data processing
Python C (#, ++)

12. 12
Project planning
0. Have you done this before?
1. To what area does your problem belong to?
2. How different are your incomes and desired outcomes?
3. How to represent them as data? Which data structures are common in the area?
4. What kind of math does operate these data?
5. What tools are common to operate your data by proper math?
6. How to measure the completeness and correctness of the result?
7. What happens if something is changed?

13. 13
Project planning
BPMN Diagrams

14. 14
Solution 0
▪ Link DWG to Revit
▪ Import DWG to Revit
0. We have done this before
1. We expect to see the same graphics in Revit as
on DWGs
2. Both incomes and outcomes formats are
supported by Revit
3. We already have our data in proper formats
4. We rely on native Revit functionality
5. We use only well known Revit
6. We can check results only visually. We know,
we loose all text. Result are lines, not model
objects.
7. In case of a change we need to reimport DWG
again. Some changes cannot be implemented
at all.

15. 15
Solution 0.3
▪ Link DWG to Revit
▪ Read DWG content as curves with coordinates
▪ Place families of line and arc so on the same
place
0. We need to deal with Revit API
1. We expect to see the same graphics in Revit as
on DWGs
2. Both incomes and outcomes formats are
supported by Revit
3. We already have our data in proper formats
4. We rely on Revit API and need to deal with
coordinates transformations
5. We use only Revit and Dynamo (macros)
6. We can check results only visually. We know,
we loose all text. Result are “models by
curves”. We’ve lost all styles.
7. In case of a change we need to reimport DWG
again. Some changes cannot be implemented
at all.

16. 16
Solution 0.9
▪ To restore line styles and make result “more
Revit” we create families for each block from
DWG and place them accordingly
▪ AutoCAD has native feature to export blocks
data: name, position, rotation, scale – to Excel
table
▪ There is also a custom LISP script (from the
Internet) to batch export blocks as separate
DWGs
▪ Overall we expected about 400.000 blocks as
their names come from original CAD system
▪ Blocks DWGs are imported into detail item
template and saved as separate families with the
same name as block

17. 17
Solution 0.9
▪ The process becomes more advanced
▪ We generate new data which are used only inside the
process itself (a lot of data)
▪ We reuse some parts of previous iterations
0. We are still within our competencies
1. We expect to see the same graphics in Revit as on
DWGs
2. We use Excel as input for Revit and need to deal
with it
3. We have intermediate data between preprocess and
core (though in proper format)
4. We keep on challenging our knowledge in Revit API
5. We are still within well known tools
6. We can check results only visually. We loose all text.
Result are 2D models.
7. Changes still require reimport

18. 18
EPIC FAIL
▪ Blocks of the same name have different
geometry and insertion points
▪ They look the same but the data behind are
different!
▪ Can we process our data not as coordinates but
as we see them – as graphics itself?
▪ How to process pictures?

19. 19
Solution 1
▪ One cannot process a picture – he needs to
represent it as an image
▪ Image is a data structure(s) which is widely used
to process graphical information
▪ Image processing is a classical part of computer
science, well known and well developed. If only
we could use CPython…
▪ Thanks to BiLT 2018 and Mariam Osman we can
connect CPython to Revit with just 3 (or more)
lines of code
▪ This opens the Pandora Box…

20. 20
Solution 1
▪ We need to return to idea of processing data
only from Revit
▪ We already know all the steps on Revit side
▪ We have not done image processing before – it
was a very heavy lifting to set it up
▪ We lost line styles again and still have no text
▪ The expected amount of resulting families was
about the same number as of elements while we
cannot compare graphics between DWGs, only
within one set (usually a building)

21. 21
Solution 2
▪ To compare graphics between different sets of
DWGs we need to store the intermediate results
▪ We are going to store about 100-400 thousand
2D geometries and need to process them as fast
as possible
▪ GIS – technology to process massive 2D
geometry data
▪ PostGIS is a free and powerful DB for 2D graphics
▪ QGIS is a free tool to visualise geometry data
stored in PostGIS (or other sources)
▪ Apart from that, the client switched from the
DWG to XML format
▪ We also need to further improve grouping of
different graphics into families

22. 22
Solution 2
▪ As expected, adding a storage dimension to the
process made it more mature, robust, scalable
and adjustable
▪ GIS was a right guess (or logical choise) – it‘s a
whole universe of tools and concepts to process
geometry
▪ To orient yourself in an unknown universe you
use simple metrics – performance
▪ Adding complexity to the process must be
balanced with benefits: completeness,
reproducibility, robustness, performance,
scalability
▪ We managed to migrate all data, including text
and dimensions, to native Revit objects
▪ The time to migrate one building dropped down
from a week to 3-4 hours

23. 23
Solution 2
▪ We are even able to reconstruct Revit walls from set
of lines including gaps for doors and windows…

24. 24
Lessons learnt
▪ It took 1,5 year to go from DWG import to the walls
recognition algorithm
▪ There is nothing wrong to go back sometimes. The
key is - proper balance between things that you know
and that you’re learning
▪ You make the next improvement only when you
understand what exactly doesn’t work. No trial-and-
error on live projects!
▪ Look around. AEC industry is a bad place for
inspiration
▪ You must be ready for “project of your life”, don’t sit
and wait for it!

25. Brought to you by Knowledge Partner
Nikolay Gerasimov
nikolay.gerasimov@plandata.eu
Thank you for joining us!
To Infinity and beyond…