Kang Taewook. Ph.D
laputa99999@gmail.com
AI Agent and Robotics
in Smart Construction
KICT
BIM, Facility Management, DX, Scan to BIM
12 books publication https://github.com/mac999
Contents
LLM & AI Agent Trend
Robotics Trend
Smart Construction Usecase
K-Smart Construction
AI Agent & Robotics in R&D
Challenge & Conclusion
LLM & AI Agent Trend
AX revolusion
Members of the Nobel
Committee for Chemistry at the
Royal Swedish Academy of
Sciences explain the work of
2024 Nobel Prize in Chemistry
winners David Baker, Demis
Hassabis and John M.
Jumper.JONATHAN
NACKSTRAND/AFP via Getty
Images
AI Pioneers Geoffrey
Hinton And John
Hopfield Win Nobel
Prize For Physics |
Latest News | WION
LLM
Large Language Models as General Pattern Machines
LLM
LLM
Google and OpenAI’s AI
models win milestone
gold at global math
competition - The
Business Times
LLM
China 이니셔티브
Link: 딥시크(deep seek) 오픈소스 코드 및 구조 분석하기
AX agent 멀티모달
AI agent 오픈소스 sLLM
Link: Gemma3 기반
Ollama 활용 AI 에이
전트 개발 핵심
Function Call 구현해
보기
Gemma 3
구글이 개발해 2025년 3월 10일에 출시한 LLM(1B, 4B,12B, 27B). 차세대 경량 오픈 멀티모달 AI 모델, 텍스트와
이미지를 동시에 처리할 수 있는 기능 지원. 컨텍스트 크기는 32k(1B), 128k(4B-27B) 지원
Gemma 3 Release - a google Collection
AI agent with MCP
MCP (Model Context Protocol)
Link: 인공지능 AI 에이전트 표준 프로토콜 MCP 개념, 사용, 개발 및 동작 구조 분석하기
https://github.com
/modelcontextprot
ocol/servers
AI Agent with coding
바이브
코딩
This Game Created by AI 'Vibe Coding' Makes $50,000 a Month. Yours Probably Won’t, Wix Acquires Six-month-old AI “Vibe Coding” Startup Base44 for
$80M Cash, Cognizant’s Vibe Coding Lesson for Indian IT, Vibe Coded a Website With My Daughter Using an AI Tool Called Bolt - Business Insider
Vibe Coding: The Future of
Software Development or Just a
Trend? - Lovable Blog
Build Apps with AI in Minutes | Base44
Robotics Trend
Robotics
BIM principle and Digital transformation: 2023년 하반기 해외 스마트건설 및 BIM 기술 트랜드
두산건설 XiteCloud
Robotics
Trimble 의 건설장비 가이던스 Earthworks 플랫폼
두산건설 XiteCloud
Robotics
Digital Roads of the Future Partnership 프로젝트 사례
두산건설 XiteCloud
Robotics
3D 스캔 AI 기반 스마트 인스펙션(Pomerleau)
두산건설 XiteCloud
Robotics with ROS
Robotics
Robotics
Robot Operating System Software — ECI documentation
Robotics
Robot Operating System Software — ECI documentation
Daddy Makers: 벨로다인 LiDAR로 SLAM 개발
Robotics
ZiwenZhuang/parkour: [CoRL 2023] Robot Parkour Learning
Robotics
Design Your Robot on Hardware-in-the-Loop with NVIDIA Jetson |
NVIDIA Technical Blog
Robotics
Design Your Robot on Hardware-in-the-Loop with NVIDIA Jetson |
NVIDIA Technical Blog
Robotics
Design Your Robot on Hardware-in-the-Loop with
NVIDIA Jetson | NVIDIA Technical Blog
Robotics
HIL on NVIDIA Orin NX with Isaac ROS vslam and Nvblox
Robotics + AI + IoT
Open Neural Network eXchange
Robotics + AI + IoT
Figure AI
AI + Robotics
isaac-sim/OmniIsaacGymEnvs: Reinforcement Learning Environments for Omniverse Isaac Gym
AI + Robotics
AEC Challenge
COVID 이후 건설 노동력 문제
강화된 건설 현장 안전 문제
지속가능성 및 친환경 건설에 대한 요구
건설 DX 채택 어려움과 뒤쳐진 산업 경쟁력
Smart Construction
Use case
AI in Construction
Global Smart Construction Market Overview(Market
Research Future)
AI in Construction
건설AI 글로벌 시장 규모 예상(The Business Research
Company)
• Concept design generation
• Reasoning for Smart construction
• Anomaly detection in contract documents
• Optimization in Construction Management
• Simulation of construction planning scenarios with Digital Twin
• Query and Decision making using LLM
• Construction Robot with LLM
• Personalised safety education and guidance system
AI in Construction
Smart Construction Scenario
Digital
Twin
Structural
health
monitoring
Track and
trace
Remote
diagnosis
Remote
services
Remote
control
Condition
monitoring
Systems
health
monitoring
BIM
as i-DB
IoT…
AI
Sensor device
ICBM
Simulation
Robotics
Scan-Vision
Smart contract
based on Blockchain
Gen AI
Multi AI
Agent
AI Agent with Robotics
Robotics
Ask.
Safety?
Performace?
Site inspection?
Autonomous construction?
ReAct
FuncCall
AI Agent
GIS
BIM
Docs
Drawing
…
Simulation
Context={
Site,
Project,
Resource…
}
Tools={
getSite(),
getWorkingArea(),
getInspectDevices(),
robot.detectObjects(),
getLimitZones(),
getEnvSensors()…
}
LLM
Communication
Answer.
Safety is …
Performace …
Site inspection …
Autonomous construction
…
LLM
Reasoning
Cursor + Autodesk
미래 개별 소프트웨어 애드인이 지능형 에이전트로 대체될 것 (The Building Coder, 2025).
LLM 에이전트와 APS의 접목(Autodesk)
Document Crunch
Document Crunch
LLM 기반 계약서, 시방서 비정형 문서 분석. 리스크 관리, 정보 검색 시간 단축.
프로젝트 관리자, 법무팀, 하청업체가 복잡한 문서 검토 시 위험 관리 및 시간 절약.
TOGAL.AI
Autodesk Assistant
Autodesk Assistant를 통한 Agent 기능 실행(Autodesk)
자연어를 통해 캐드 데이터 접근 및 질의 답변
Doxel
Funding $56.5M, 2021
Builtdots
Builddots
컴퓨터 비전 기술 기반 BIM 모델 비교 변화 추적
이스라엘 인공지능 건설 소프트웨어 스타트업. Intel Capital 주도. 1,500만 달러의 투자 유치
Procore Helix
Procore Helix
AI, 에이전트 워크플로, 분석 지원하는 인텔리전스 시스템 지원
ALICE
ALICE Technologies
다양한 시공 시나리오 시뮬레이션 최적 공정 계획 지원
ANYbotics
ANYbotics AG
Hilti Jaibot
Hilti 천장 드릴링 로봇
Hilti Jaibot(천장 드릴링), ACR의 TyBot(철근 결속)과 같은 전문 로봇은 특정 반복 작업을 자동화
Dusty Robotics
Dusty Robotics
BIM 도면을 현장 바닥에 1:1로 인쇄하여 먹매김 오류를 방지
Trimble Construction One
Trimble Construction One 기반 측량/스캔 로봇
(Trimble)
Connected Construction 전략 통해 현장 하드웨어와 사무실 소프트웨어를 플랫폼으로 통합.
3차원 라이다, 로봇, 스캔 및 GPS 등 하드웨어 기술 활용, AI 데이터 파이프라인을 구축.
Built Robotics
www.builtrobotics.com
Site Management Infra Kit
Site Management Infra Kit
K-Smart Construction
Scenario – IoT based Road Pavement Quality Management R&D
IoT
Big data
management
AI + Simulation using LLM
Cloud
platform
Machine control
Field monitoring
IoT
sensor
Usecase
for safety, accuracy, productivity
sensing
Data analysis
& prediction
GIS IoT based monitoring
Field control
Infra IoT
service
connection
Plant control system (SCADA)
Field monitoring system
LoRA, BLE,
WiFi…
Layer 8 | IISL
(Infra IoT Service Layer)
Worker
Agency
<device_definition id=‘dd#1’>
<device id=‘T#1’name=‘temp’type=‘temperature’>
<maker name=‘CH korea’ email=‘laputa99999_9@gmail.com’ tel=‘82-0330-0802-1013’ location=‘…’/>
<specification>
<op_range name=‘voltage’ unit=‘V’type=‘real’value=‘3.3’/>
<op_range name=‘temperature’ unit=‘degree’ type=‘real’begin=‘-10.0’end=’60.0’/>
<op_range name=‘humidity’unit=‘%R.H’type=‘real’begin=‘0.0’end=’50.0’/>
<op_range name=‘GPS’unit=‘WGS84’type=‘vector2D’begin=‘(0,0)’ end=‘(127, 32)’/>
<op_range name=‘characteristic_curve’unit1=‘temperature’ unit2=‘voltage’ type=‘vector2D’>
(0,0), (1.2, 2.4), (3.5, 6.2), (4.1, 7.2)
</op_range>
<op_range name=‘period’unit=‘year’value=‘2’/>
</specification>
</device>
</device_definition>
Intelligent IoT sensor
•Self diagnose
•IISL protocol
•Security
•Availability
1
1
2
3
4
5
6
7
8
Plant sensing
Scenario – IoT based Road Pavement Quality Management R&D
Scenario – IoT based Road Pavement Quality Management R&D
Scenario – IoT based Road Pavement Quality Management R&D
Scenario – IoT based Road Pavement Quality Management R&D
Scenario – Smart construction R&D
Scenario
Scenario
Scenario
Scenario
AI Agent & Robotics in R&D
https://github.com/mac999
Smart Inspection with Robotics
Trimble
2021.3
Trimble
GPS
카메라
카메라
스캐너
IMU
DMI
KICT
Smart Inspection with Robotics
원거리 영상 전송, 조명 추가. 주행 안전성(BLDC모터 적용) 및 장치 추가 편의성 개선. 1회 충전 4시간 운영 가능
현재, 협력기관과 함께 SLAM기반 장애물 자동 회피 개발
로버 기반 실시간 스캔 경로(좌) 및 수집된 포인트 클
라우드 데이터(우). 작업생산성 231% 개선
SLAM, LiDAR데이터 비교 검증. 평균 오차 18mm, 최
대오차 735mm
로버 장비(좌. 개발 협업 중), 개발 완료 후 테스트 중인 로버(중) 및 원격 운행 영상 모니터링(우)
원거리 영상 전송
테스트 장면
Smart Inspection with Robotics
Δ 1.37%
Δ 21.48%
Δ 16.76%
Δ 2.34%
UNF
Purdue
Scan to BIM for Smart Inspection
3D Scan Data Level of Detail Massive 3D Data Reducer
Noise Filtering
Classification
Geometry Mapping
BIM Mapping
GIS Mapping
Service
Scan to BIM for Smart Inspection
3D Scan Data Level of Detail Massive 3D Data Reducer
Noise Filtering
Classification Building facade
Bridge Element
Building Indoor
Road Element
Classification
Model
Geometry Mapping
BIM Mapping
GIS Mapping
Service
…
Scan to BIM
LocSE
AP
Aggregation
features (N, d’)
Input
point features
(N, 3+ d)
Local spatial encoding (LocSE) & attentive pooling (AP)
LocSE
AP Dilated residual
block
lrelu
Train point
features
Training sequence
Scan data classification
Scan to BIM
Affine transform
GeoTiff conversion
Property
Geometry
object
BIM
object
Scan to BIM pipeline
(SBDL)
Scan to BIM
Scan to BIM
mac999/scan_to_bim_pipeline: scan to
bim pipieline
BIM-based DT - case study
Space Environment Management. Ex. Temperature, Humidity, Light …
Easy system maintenance
Open source usage
UNF Prototype Research
BIM-based DT - Framework
Digital World View
D1. BIM database
D2. Property
Database
D3. Real World
Connector
D4. Open API
D5. Data Analysis &
Simulation
Digital Twin Application
Objective Definition
Requirement Definition
Architecture Design
Development
Operation & Maintenance
Real World
R1. As-Built BIM
development
R3. Digital World
Connector
D6. Dashboard
Link
Realtime
R4. IoT
Data flow
Dependency
Digital Twin
Development Flow
R2. Field &
Legacy dataset
Physical world Virtual world
BIM-based DT - Framework
Digital World View
D1. BIM database
D2. Property
Database
D3. Real World
Connector
D4. Open API
D5. Data Analysis &
Simulation
Digital Twin Application
Objective Definition
Requirement Definition
Architecture Design
Development
Operation & Maintenance
Real World
R1. As-Built BIM
development
R3. Digital World
Connector
D6. Dashboard
Link
Realtime
R4. IoT
Data flow
Dependency
Digital Twin
Development Flow
R2. Field &
Legacy dataset
Physical world Virtual world
BIM-DT
Connector
BIM-based DT – LoD development
BIM modeling
Lightweight model development
LoD modeling
LoD 200
Model (20 Mb)
BIM (Over 1 Gb)
BIM-based DT – LoD development
BIM modeling
Lightweight model development
LoD modeling
LoD 200
Model (20 Mb)
BIM (Over 500 Mb)
Building.floor.room
bldg#1.1ST FLOOR.OFFICE 1204
BIM-based DT – IoT database development
Setup IoT
database
Setup IoT
device &
metadata
Install &
connect IoT
device in space
Collect & Store
IoT dataset
Learning
prediction &
anomaly
detection
model
Deploy &
Operate deep
learning model
Dev & Ops
for IoT dataset train
BIM-based DT – IoT database development
Open API server
IoT device IoT database
BIM
deep learning
anomaly detection
Building envornment monitoring service
BIM-based DT – IoT database development
{
"ID": "IoT ID string",
"name": "IoT device name",
"description": "device description",
"MAC_address": "IP MAC address",
"sensors": [
{
"ID": "sensor ID",
"sensor_name": "temperature",
"sensor_type": "real",
"sensor_value": "27.3683",
"sensor_position":
"building#1.storey#1.room#106"
},
{
...
}
]
}
BIM-based DT – IoT database development
{
"ID": "IoT ID string",
"name": "IoT device name",
"description": "device description",
"MAC_address": "IP MAC address",
"sensors": [
{
"ID": "sensor ID",
"sensor_name": "temperature",
"sensor_type": "real",
"sensor_value": "27.3683",
"sensor_position":
"building#1.storey#1.room#106"
},
{
...
}
]
}
BIM-based DT – IoT database development
{
"ID": "IoT ID string",
"name": "IoT device name",
"description": "device description",
"MAC_address": "IP MAC address",
"sensors": [
{
"ID": "sensor ID",
"sensor_name": "temperature",
"sensor_type": "real",
"sensor_value": "27.3683",
"sensor_position":
"building#1.storey#1.room#106"
},
{
...
}
]
}
BIM-based DT – train & prediction
LSTM unit LSTM unit LSTM unit LSTM unit
Input Building Environment Data
Dense layer (1) Predict Data
LSTM Layer
Loss = 6.5e-4
BIM-based DT – train & prediction
Dataset Count RMSE
train 1223 0.134
test 815 0.136
BIM-based DT - train & prediction
anomaly data count = 45 (2.2%. σ=3)
BIM, IoT and Digital Twin
Trimble
2021.3
Deep Learning
IoT
GIS
BIM, IoT and Digital Twin
mac999/digital_twin_BIM_IoT: digital twin
based building env management etc
mac999/citygml_parser: CityGML 3.0
(Python version) parser for reading,
writing, and converting CityGML files into
JSON using Python.
mac999/landxml_parser: LandXML parser
Energy Usage Optimization using AI
Seoul Energy Dream Center
Energy Usage Optimization using AI
𝑦𝑡 = 𝜇𝑡 + 𝛾𝑡 + 𝛹𝑡 + ෍
𝑖=1
𝑝
𝜙𝑖𝑦𝑖−1 + ෍
𝑗=1
𝑚
𝛽𝑗𝑥𝑗𝑡 + 𝜀𝑡
Energy Usage Optimization using AI
No Number of Nodes in Hidden Layers Dropout Batch size
M1 [12, 16, 32, 16, 12, 6] 0.00 32
M2 64
M3 0.01 32
M4 64
M5 [18, 24, 48, 24, 18, 12] 0.00 32
M6 64
M7 0.01 32
M8 64
M9 [24, 32, 64, 32, 24, 18] 0.00 32
M10 64
M11 0.01 32
M12 64
M13 [12, 16, 32, 64, 32, 16, 12] 0.00 32
M14 64
M15 0.01 32
M16 64
Energy Usage Optimization using AI
Energy Usage Optimization using AI
Energy Usage Optimization using AI
mac999/building_energy_prediction_model:
building_energy_prediction_model for
research project
AI Agent LLAMA & Langchain
Langchain CEO.
2022.
30M$ in 2023.
AI Agent with BIM
AI Agent with BIM
AI Agent with BIM
AI Agent with BIM
AI Agent with BIM
AI Agent with BIM
mac999/BIM_LLM_code_agent: BIM agent
using RAG
AI Agent with GIS
AI Agent with GIS
AI Agent with GIS
mac999/geo-llm-agent-dashboard: Geo
Map AI Agent Dashboard Web App for
example
LLM in Engineering
LLM in Engineering
LLM in Engineering
ENA Model ID Train No Loss Accuracy Model size
(kb)
Time performance
(minutes)
M1.1. MLP 1650 0.0870 0.9494 61 0:02:34
M1.2. MLP 1714 0.0852 0.9519 84 0:02:36
M1.3. MLP 1716 0.0882 0.9544 93 0:02:42
M1.4. MLP 1718 0.1322 0.9507 30 0:02:25
M2.1. LSTM 1730 0.0889 0.9408 812 0:02:13
M2.2. LSTM 1732 0.0851 0.9420 850 0:02:17
M2.3. LSTM 1734 0.0886 0.9408 3,312 0:02:00
M3.1. Transformers 2003 0.3533 0.7744 74,557 0:11:06
M3.2.Transformers 2014 0.3551 0.7719 74,557 0:07:48
M3.3.Transformers 2021 0.3596 0.7423 74,557 0:06:25
M4.1. LLM 0103 0.0587 0.9507 427,783 3:12:05
M4.2. LLM 2334 0.0534 0.9531 427,783 7:59:00
BERT
LLM in Engineering
LLM in Engineering
mac999/earthwork-net-model · Hugging
Face
Conclusion
• AI Agent와 로보틱스 기술 급격한 발전
• AI Agent 기반 건설 테크 기업의 증가
• LLM은 멀티모달, 다중 에이전트의 OS로써 역할
• LLM은 Edge Computer 에 내장
• AI기반 디지털트윈 시뮬레이션을 통한 재작업 감소, 비용개선
Smart Construction Challenge
Smart Construction Challenge
Smart Construction Challenge
Smart Construction Challenge
Smart Construction Challenge
Smart Construction Challenge
• 이기종 시스템 간 상호운용성 부족
• 열악한 현장 환경과 네트워크 불안전성
• 스마트 건설 데이터 품질 문제
• 건설 분야 학습 데이터의 부족
• 데이터 보안 및 활용시 책임 문제
• 높은 초기 기술 도입 비용
• 전문 인력 부족
• 법 제도 기반 미흡
Q&A
laputa99999@gmail.com
https://www.linkedin.com/in/tae-wook-
kang-64a83917

건설산업 경쟁력 확보를 위한 AI에이전트와 로보틱스기반 스마트건설 동향 및 기술 세미나

  • 1.
    Kang Taewook. Ph.D laputa99999@gmail.com AIAgent and Robotics in Smart Construction
  • 2.
    KICT BIM, Facility Management,DX, Scan to BIM 12 books publication https://github.com/mac999
  • 3.
    Contents LLM & AIAgent Trend Robotics Trend Smart Construction Usecase K-Smart Construction AI Agent & Robotics in R&D Challenge & Conclusion
  • 4.
    LLM & AIAgent Trend
  • 5.
    AX revolusion Members ofthe Nobel Committee for Chemistry at the Royal Swedish Academy of Sciences explain the work of 2024 Nobel Prize in Chemistry winners David Baker, Demis Hassabis and John M. Jumper.JONATHAN NACKSTRAND/AFP via Getty Images AI Pioneers Geoffrey Hinton And John Hopfield Win Nobel Prize For Physics | Latest News | WION
  • 6.
    LLM Large Language Modelsas General Pattern Machines
  • 7.
  • 8.
    LLM Google and OpenAI’sAI models win milestone gold at global math competition - The Business Times
  • 9.
    LLM China 이니셔티브 Link: 딥시크(deepseek) 오픈소스 코드 및 구조 분석하기
  • 10.
  • 11.
    AI agent 오픈소스sLLM Link: Gemma3 기반 Ollama 활용 AI 에이 전트 개발 핵심 Function Call 구현해 보기 Gemma 3 구글이 개발해 2025년 3월 10일에 출시한 LLM(1B, 4B,12B, 27B). 차세대 경량 오픈 멀티모달 AI 모델, 텍스트와 이미지를 동시에 처리할 수 있는 기능 지원. 컨텍스트 크기는 32k(1B), 128k(4B-27B) 지원 Gemma 3 Release - a google Collection
  • 12.
    AI agent withMCP MCP (Model Context Protocol) Link: 인공지능 AI 에이전트 표준 프로토콜 MCP 개념, 사용, 개발 및 동작 구조 분석하기 https://github.com /modelcontextprot ocol/servers
  • 13.
    AI Agent withcoding 바이브 코딩 This Game Created by AI 'Vibe Coding' Makes $50,000 a Month. Yours Probably Won’t, Wix Acquires Six-month-old AI “Vibe Coding” Startup Base44 for $80M Cash, Cognizant’s Vibe Coding Lesson for Indian IT, Vibe Coded a Website With My Daughter Using an AI Tool Called Bolt - Business Insider Vibe Coding: The Future of Software Development or Just a Trend? - Lovable Blog Build Apps with AI in Minutes | Base44
  • 14.
  • 15.
    Robotics BIM principle andDigital transformation: 2023년 하반기 해외 스마트건설 및 BIM 기술 트랜드 두산건설 XiteCloud
  • 16.
    Robotics Trimble 의 건설장비가이던스 Earthworks 플랫폼 두산건설 XiteCloud
  • 17.
    Robotics Digital Roads ofthe Future Partnership 프로젝트 사례 두산건설 XiteCloud
  • 18.
    Robotics 3D 스캔 AI기반 스마트 인스펙션(Pomerleau) 두산건설 XiteCloud
  • 19.
  • 20.
  • 21.
    Robotics Robot Operating SystemSoftware — ECI documentation
  • 22.
    Robotics Robot Operating SystemSoftware — ECI documentation Daddy Makers: 벨로다인 LiDAR로 SLAM 개발
  • 23.
  • 24.
    Robotics Design Your Roboton Hardware-in-the-Loop with NVIDIA Jetson | NVIDIA Technical Blog
  • 25.
    Robotics Design Your Roboton Hardware-in-the-Loop with NVIDIA Jetson | NVIDIA Technical Blog
  • 26.
    Robotics Design Your Roboton Hardware-in-the-Loop with NVIDIA Jetson | NVIDIA Technical Blog
  • 27.
    Robotics HIL on NVIDIAOrin NX with Isaac ROS vslam and Nvblox
  • 28.
    Robotics + AI+ IoT Open Neural Network eXchange
  • 29.
    Robotics + AI+ IoT Figure AI
  • 30.
    AI + Robotics isaac-sim/OmniIsaacGymEnvs:Reinforcement Learning Environments for Omniverse Isaac Gym
  • 31.
  • 32.
    AEC Challenge COVID 이후건설 노동력 문제 강화된 건설 현장 안전 문제 지속가능성 및 친환경 건설에 대한 요구 건설 DX 채택 어려움과 뒤쳐진 산업 경쟁력
  • 33.
  • 34.
    AI in Construction GlobalSmart Construction Market Overview(Market Research Future)
  • 35.
    AI in Construction 건설AI글로벌 시장 규모 예상(The Business Research Company)
  • 36.
    • Concept designgeneration • Reasoning for Smart construction • Anomaly detection in contract documents • Optimization in Construction Management • Simulation of construction planning scenarios with Digital Twin • Query and Decision making using LLM • Construction Robot with LLM • Personalised safety education and guidance system AI in Construction
  • 37.
    Smart Construction Scenario Digital Twin Structural health monitoring Trackand trace Remote diagnosis Remote services Remote control Condition monitoring Systems health monitoring BIM as i-DB IoT… AI Sensor device ICBM Simulation Robotics Scan-Vision Smart contract based on Blockchain Gen AI Multi AI Agent
  • 38.
    AI Agent withRobotics Robotics Ask. Safety? Performace? Site inspection? Autonomous construction? ReAct FuncCall AI Agent GIS BIM Docs Drawing … Simulation Context={ Site, Project, Resource… } Tools={ getSite(), getWorkingArea(), getInspectDevices(), robot.detectObjects(), getLimitZones(), getEnvSensors()… } LLM Communication Answer. Safety is … Performace … Site inspection … Autonomous construction … LLM Reasoning
  • 39.
    Cursor + Autodesk 미래개별 소프트웨어 애드인이 지능형 에이전트로 대체될 것 (The Building Coder, 2025). LLM 에이전트와 APS의 접목(Autodesk)
  • 40.
    Document Crunch Document Crunch LLM기반 계약서, 시방서 비정형 문서 분석. 리스크 관리, 정보 검색 시간 단축. 프로젝트 관리자, 법무팀, 하청업체가 복잡한 문서 검토 시 위험 관리 및 시간 절약.
  • 41.
  • 42.
    Autodesk Assistant Autodesk Assistant를통한 Agent 기능 실행(Autodesk) 자연어를 통해 캐드 데이터 접근 및 질의 답변
  • 43.
  • 44.
    Builtdots Builddots 컴퓨터 비전 기술기반 BIM 모델 비교 변화 추적 이스라엘 인공지능 건설 소프트웨어 스타트업. Intel Capital 주도. 1,500만 달러의 투자 유치
  • 45.
    Procore Helix Procore Helix AI,에이전트 워크플로, 분석 지원하는 인텔리전스 시스템 지원
  • 46.
    ALICE ALICE Technologies 다양한 시공시나리오 시뮬레이션 최적 공정 계획 지원
  • 47.
  • 48.
    Hilti Jaibot Hilti 천장드릴링 로봇 Hilti Jaibot(천장 드릴링), ACR의 TyBot(철근 결속)과 같은 전문 로봇은 특정 반복 작업을 자동화
  • 49.
    Dusty Robotics Dusty Robotics BIM도면을 현장 바닥에 1:1로 인쇄하여 먹매김 오류를 방지
  • 50.
    Trimble Construction One TrimbleConstruction One 기반 측량/스캔 로봇 (Trimble) Connected Construction 전략 통해 현장 하드웨어와 사무실 소프트웨어를 플랫폼으로 통합. 3차원 라이다, 로봇, 스캔 및 GPS 등 하드웨어 기술 활용, AI 데이터 파이프라인을 구축.
  • 51.
  • 52.
  • 53.
  • 54.
  • 55.
    Scenario – IoTbased Road Pavement Quality Management R&D IoT Big data management AI + Simulation using LLM Cloud platform Machine control Field monitoring IoT sensor Usecase for safety, accuracy, productivity sensing Data analysis & prediction GIS IoT based monitoring Field control Infra IoT service connection Plant control system (SCADA) Field monitoring system LoRA, BLE, WiFi… Layer 8 | IISL (Infra IoT Service Layer) Worker Agency <device_definition id=‘dd#1’> <device id=‘T#1’name=‘temp’type=‘temperature’> <maker name=‘CH korea’ email=‘laputa99999_9@gmail.com’ tel=‘82-0330-0802-1013’ location=‘…’/> <specification> <op_range name=‘voltage’ unit=‘V’type=‘real’value=‘3.3’/> <op_range name=‘temperature’ unit=‘degree’ type=‘real’begin=‘-10.0’end=’60.0’/> <op_range name=‘humidity’unit=‘%R.H’type=‘real’begin=‘0.0’end=’50.0’/> <op_range name=‘GPS’unit=‘WGS84’type=‘vector2D’begin=‘(0,0)’ end=‘(127, 32)’/> <op_range name=‘characteristic_curve’unit1=‘temperature’ unit2=‘voltage’ type=‘vector2D’> (0,0), (1.2, 2.4), (3.5, 6.2), (4.1, 7.2) </op_range> <op_range name=‘period’unit=‘year’value=‘2’/> </specification> </device> </device_definition> Intelligent IoT sensor •Self diagnose •IISL protocol •Security •Availability 1 1 2 3 4 5 6 7 8 Plant sensing
  • 56.
    Scenario – IoTbased Road Pavement Quality Management R&D
  • 57.
    Scenario – IoTbased Road Pavement Quality Management R&D
  • 58.
    Scenario – IoTbased Road Pavement Quality Management R&D
  • 59.
    Scenario – IoTbased Road Pavement Quality Management R&D
  • 60.
    Scenario – Smartconstruction R&D
  • 61.
  • 62.
  • 63.
  • 64.
  • 65.
    AI Agent &Robotics in R&D https://github.com/mac999
  • 66.
    Smart Inspection withRobotics Trimble 2021.3 Trimble GPS 카메라 카메라 스캐너 IMU DMI KICT
  • 67.
    Smart Inspection withRobotics 원거리 영상 전송, 조명 추가. 주행 안전성(BLDC모터 적용) 및 장치 추가 편의성 개선. 1회 충전 4시간 운영 가능 현재, 협력기관과 함께 SLAM기반 장애물 자동 회피 개발 로버 기반 실시간 스캔 경로(좌) 및 수집된 포인트 클 라우드 데이터(우). 작업생산성 231% 개선 SLAM, LiDAR데이터 비교 검증. 평균 오차 18mm, 최 대오차 735mm 로버 장비(좌. 개발 협업 중), 개발 완료 후 테스트 중인 로버(중) 및 원격 운행 영상 모니터링(우) 원거리 영상 전송 테스트 장면
  • 68.
    Smart Inspection withRobotics Δ 1.37% Δ 21.48% Δ 16.76% Δ 2.34% UNF Purdue
  • 69.
    Scan to BIMfor Smart Inspection 3D Scan Data Level of Detail Massive 3D Data Reducer Noise Filtering Classification Geometry Mapping BIM Mapping GIS Mapping Service
  • 70.
    Scan to BIMfor Smart Inspection 3D Scan Data Level of Detail Massive 3D Data Reducer Noise Filtering Classification Building facade Bridge Element Building Indoor Road Element Classification Model Geometry Mapping BIM Mapping GIS Mapping Service …
  • 71.
    Scan to BIM LocSE AP Aggregation features(N, d’) Input point features (N, 3+ d) Local spatial encoding (LocSE) & attentive pooling (AP) LocSE AP Dilated residual block lrelu Train point features Training sequence Scan data classification
  • 72.
    Scan to BIM Affinetransform GeoTiff conversion Property Geometry object BIM object Scan to BIM pipeline (SBDL)
  • 73.
  • 74.
  • 75.
    BIM-based DT -case study Space Environment Management. Ex. Temperature, Humidity, Light … Easy system maintenance Open source usage UNF Prototype Research
  • 76.
    BIM-based DT -Framework Digital World View D1. BIM database D2. Property Database D3. Real World Connector D4. Open API D5. Data Analysis & Simulation Digital Twin Application Objective Definition Requirement Definition Architecture Design Development Operation & Maintenance Real World R1. As-Built BIM development R3. Digital World Connector D6. Dashboard Link Realtime R4. IoT Data flow Dependency Digital Twin Development Flow R2. Field & Legacy dataset Physical world Virtual world
  • 77.
    BIM-based DT -Framework Digital World View D1. BIM database D2. Property Database D3. Real World Connector D4. Open API D5. Data Analysis & Simulation Digital Twin Application Objective Definition Requirement Definition Architecture Design Development Operation & Maintenance Real World R1. As-Built BIM development R3. Digital World Connector D6. Dashboard Link Realtime R4. IoT Data flow Dependency Digital Twin Development Flow R2. Field & Legacy dataset Physical world Virtual world BIM-DT Connector
  • 78.
    BIM-based DT –LoD development BIM modeling Lightweight model development LoD modeling LoD 200 Model (20 Mb) BIM (Over 1 Gb)
  • 79.
    BIM-based DT –LoD development BIM modeling Lightweight model development LoD modeling LoD 200 Model (20 Mb) BIM (Over 500 Mb) Building.floor.room bldg#1.1ST FLOOR.OFFICE 1204
  • 80.
    BIM-based DT –IoT database development Setup IoT database Setup IoT device & metadata Install & connect IoT device in space Collect & Store IoT dataset Learning prediction & anomaly detection model Deploy & Operate deep learning model Dev & Ops for IoT dataset train
  • 81.
    BIM-based DT –IoT database development Open API server IoT device IoT database BIM deep learning anomaly detection Building envornment monitoring service
  • 82.
    BIM-based DT –IoT database development { "ID": "IoT ID string", "name": "IoT device name", "description": "device description", "MAC_address": "IP MAC address", "sensors": [ { "ID": "sensor ID", "sensor_name": "temperature", "sensor_type": "real", "sensor_value": "27.3683", "sensor_position": "building#1.storey#1.room#106" }, { ... } ] }
  • 83.
    BIM-based DT –IoT database development { "ID": "IoT ID string", "name": "IoT device name", "description": "device description", "MAC_address": "IP MAC address", "sensors": [ { "ID": "sensor ID", "sensor_name": "temperature", "sensor_type": "real", "sensor_value": "27.3683", "sensor_position": "building#1.storey#1.room#106" }, { ... } ] }
  • 84.
    BIM-based DT –IoT database development { "ID": "IoT ID string", "name": "IoT device name", "description": "device description", "MAC_address": "IP MAC address", "sensors": [ { "ID": "sensor ID", "sensor_name": "temperature", "sensor_type": "real", "sensor_value": "27.3683", "sensor_position": "building#1.storey#1.room#106" }, { ... } ] }
  • 85.
    BIM-based DT –train & prediction LSTM unit LSTM unit LSTM unit LSTM unit Input Building Environment Data Dense layer (1) Predict Data LSTM Layer Loss = 6.5e-4
  • 86.
    BIM-based DT –train & prediction Dataset Count RMSE train 1223 0.134 test 815 0.136
  • 87.
    BIM-based DT -train & prediction anomaly data count = 45 (2.2%. σ=3)
  • 88.
    BIM, IoT andDigital Twin Trimble 2021.3 Deep Learning IoT GIS
  • 89.
    BIM, IoT andDigital Twin mac999/digital_twin_BIM_IoT: digital twin based building env management etc mac999/citygml_parser: CityGML 3.0 (Python version) parser for reading, writing, and converting CityGML files into JSON using Python. mac999/landxml_parser: LandXML parser
  • 90.
    Energy Usage Optimizationusing AI Seoul Energy Dream Center
  • 91.
    Energy Usage Optimizationusing AI 𝑦𝑡 = 𝜇𝑡 + 𝛾𝑡 + 𝛹𝑡 + ෍ 𝑖=1 𝑝 𝜙𝑖𝑦𝑖−1 + ෍ 𝑗=1 𝑚 𝛽𝑗𝑥𝑗𝑡 + 𝜀𝑡
  • 92.
    Energy Usage Optimizationusing AI No Number of Nodes in Hidden Layers Dropout Batch size M1 [12, 16, 32, 16, 12, 6] 0.00 32 M2 64 M3 0.01 32 M4 64 M5 [18, 24, 48, 24, 18, 12] 0.00 32 M6 64 M7 0.01 32 M8 64 M9 [24, 32, 64, 32, 24, 18] 0.00 32 M10 64 M11 0.01 32 M12 64 M13 [12, 16, 32, 64, 32, 16, 12] 0.00 32 M14 64 M15 0.01 32 M16 64
  • 93.
  • 94.
  • 95.
    Energy Usage Optimizationusing AI mac999/building_energy_prediction_model: building_energy_prediction_model for research project
  • 96.
    AI Agent LLAMA& Langchain Langchain CEO. 2022. 30M$ in 2023.
  • 97.
  • 98.
  • 99.
  • 100.
  • 101.
  • 102.
    AI Agent withBIM mac999/BIM_LLM_code_agent: BIM agent using RAG
  • 103.
  • 104.
  • 105.
    AI Agent withGIS mac999/geo-llm-agent-dashboard: Geo Map AI Agent Dashboard Web App for example
  • 106.
  • 107.
  • 108.
    LLM in Engineering ENAModel ID Train No Loss Accuracy Model size (kb) Time performance (minutes) M1.1. MLP 1650 0.0870 0.9494 61 0:02:34 M1.2. MLP 1714 0.0852 0.9519 84 0:02:36 M1.3. MLP 1716 0.0882 0.9544 93 0:02:42 M1.4. MLP 1718 0.1322 0.9507 30 0:02:25 M2.1. LSTM 1730 0.0889 0.9408 812 0:02:13 M2.2. LSTM 1732 0.0851 0.9420 850 0:02:17 M2.3. LSTM 1734 0.0886 0.9408 3,312 0:02:00 M3.1. Transformers 2003 0.3533 0.7744 74,557 0:11:06 M3.2.Transformers 2014 0.3551 0.7719 74,557 0:07:48 M3.3.Transformers 2021 0.3596 0.7423 74,557 0:06:25 M4.1. LLM 0103 0.0587 0.9507 427,783 3:12:05 M4.2. LLM 2334 0.0534 0.9531 427,783 7:59:00 BERT
  • 109.
  • 110.
  • 111.
    Conclusion • AI Agent와로보틱스 기술 급격한 발전 • AI Agent 기반 건설 테크 기업의 증가 • LLM은 멀티모달, 다중 에이전트의 OS로써 역할 • LLM은 Edge Computer 에 내장 • AI기반 디지털트윈 시뮬레이션을 통한 재작업 감소, 비용개선
  • 112.
  • 113.
  • 114.
  • 115.
  • 116.
  • 117.
    Smart Construction Challenge •이기종 시스템 간 상호운용성 부족 • 열악한 현장 환경과 네트워크 불안전성 • 스마트 건설 데이터 품질 문제 • 건설 분야 학습 데이터의 부족 • 데이터 보안 및 활용시 책임 문제 • 높은 초기 기술 도입 비용 • 전문 인력 부족 • 법 제도 기반 미흡
  • 118.