Boosting Urban Collective Intelligence: When (Generative)AI met 6G. Agya Workshop: Smart & Sustainable Cities.

1. tii.ae
Boosting Urban Collective Intelligence:
When (Generative) AI met 6G…
Thierry Lestable, Ph.D, Dipl-lng.
Chief Researcher (acting),
Telecom & Cybersecurity Units
Artificial Intelligence & Digital Science Research Center

2. 2
TII – Technology Innovation Institute
Combating Climate Change
The Need for Smart and Sustainable Infrastructures!
Current level is at 1.2 C. finding the global solutions to reduce
emissions by 45% by 2030 and reach net
zero by 2050.
In Europe, transportation is the main source of carbon emissions:
- It represents 27% of total emissions in the EU
- 45% of these emissions come from cars.
Road transport is the main contributor to these emissions: 77%.
In France: 121 million metric tons of CO2 in 2018.
The European Green Deal aims at reducing the emissions of
the transport sector by 90% by 2050
Road congestion has economic, environmental, and human costs:
• Commuter: time, fuel, quality of life, vehicle maintenance
• Safety: crashes, emergency response times
• Freight: supply chain delays, fuel, shipping costs
• Environmental & Public Health: CO2, fuel, death and illness
• Economic Competitiveness: access to jobs, airport delays
Every 24s, someone dies on the road = 1.35 Million Road deaths every
year (source: World Health Organization)
United Nations Intergovernmental Panel
on Climate Change commits to limit
increase in global surface temperature to
1.5 to 2 C by 2050.
“By 2050, the Earth’s population is expected to increase from 7 to 10 billion and the
world economy is projected to nearly quadruple, creating a growing demand for energy
and natural resources. This new economy is projected to use 80% more energy by
2050.”
“ 55% of the world’s population lives in urban areas and this percentage is expected to
rise to 68% by 2050 (.. addition of 2.5 billion people to urban areas worldwide.).
Additionally, freight volumes are expected to grow by 40% during this time period”
A recent study published in Environmental Research Letters found that 100 cities
drive 18% of global CO2 emissions

3. 3
TII – Technology Innovation Institute
1in7
5G Technology across 4 high-emitting sectors could create annual emissions
savings equal to taking 1 in 7 cars (over 35 million cars) off EU roads.
GHG Emissions (Indirect) Abatement Thanks to 5G
By 2030: 550MtCO2 each year = 15% of total EU emission in 2017
“Communications technology, digitalization, and
cloud technologies are central to the transition to
a zero-emissions economy.”
“The avoided emissions that they enable is
substantial.”
Dr Ajay Gambhir,
Grantham Institute for Climate Change
Source: qualcomm
GHG: Green-House Gas

4. 4
TII – Technology Innovation Institute
Urban Traffic and Planning Management
A.I + 5G and C-V2X
A.I combined with 5G & C-V2X is paving the way to disrupting road traffic Management
services, saving and changing lives of citizens.
Heterogeneous Interconnected devices, with Data privacy concerns make it a perfect
playground for Federated Learning & A.I on Edge !
Cross-Working Group Work Item VRU: 2020
VRU: Vulnerable Road Users

5. 5
TII – Technology Innovation Institute
C-V2X Services
Keystone of NextGen(NG) Sustainable Mobility
AI and Digital Science Research Center
Smart Traffic Light
Blind curve: Hazard warning
Emergency First Responder priority
• Safer Roads, Lower congestion/stops/accidents
• Road Safety: 6% CO2 Reduction
• CACC (Cooperative Adaptive Cruise Control): with controlled intersections,
thanks to C-V2X: 6% CO2 reduction
• Ecodriving
• on Motorways (avoiding congestion): 6% CO2 reduction
• Other roads: 3% - 7% CO2 reduction.
• Highest for [50-80Km/h]
• Intelligent Intersection:
• Avoiding (complete) stops
• Truck: 22% CO2 reduction on provincial intersections
• Car: 13% - 21% CO2 reduction
• VRU (Vulnerable Road Users) Protection
• This would encourage Cyclists
• Public Transportation prioritization
• V2X RSU Traffic light would prioritize public transportation, thus
encouraging its use, then reducing CO2
Source: TNO

6. 6
TII – Technology Innovation Institute
Smart Infrastructure :: Collective Intelligence
Design, Development and Roll-Out
Digital Twin

7. 7
TII – Technology Innovation Institute
From 5G to 6G
From Connected Things to Connected Intelligence
eMBB: enhanced Mobile Broadband
mMTC: massive Machine Type Communication
uRLLC: ultra Reliable Low Latency communication
TSN: Time Sensitive Networks
6G to address UN’s SDG
➔ Contribute to the Quality & opportunities of human Life
From Connected Intelligence
To
Collective Intelligence
Source: huawei

8. 8
TII – Technology Innovation Institute
Cellular Sites: Energy Saving
20% of sites
carry 80% of
traffic
Energy Saving Features with AI Management
reducing the power consumption when no data is needed to be
transmitted and increase hardware efficiency
Traffic prediction using Machine Learning (ML) for MIMO sleep
ML/AI to predict
6G must keep on improving separation
between “idle mode” and “active mode”
functions, with Elastic approach Vs
traffic/Workload.

9. 9
TII – Technology Innovation Institute
6G (Target) Power Consumption Vs 5G and 4G
Few Promising Directions to Explore
“Peak capacity of a 6G 512 TRX extreme
massive MIMO 400 MHz cell will be 10 to 20
times that of 5G NR”
RIS, IoT (AIoT), Edge AI, Distributed AI, Energy Harvesting, SWIPT-Energy
transfer, Native AI, Semantic Communication, “Elastic” Radio Resources
depending on traffic, O-RAN, Virtualization
RIS: Reconfigurable Intelligent Surface
Courtesy of University of Surrey (UniS)
Fully integrated Infrastructure = Grid included

10. 10
TII – Technology Innovation Institute
Reinforcement Learning Nano-Drones
Traffic Management
Classifying and Counting – Flow control
Fixed Camera Flying Camera onboard Drone
6G Non-Terrestrial Networks (NTN)
Reconfigurable Intelligent Surface (RIS)
HAPS: High Altitude Platform

11. 11
TII – Technology Innovation Institute
EdgeAI and Federated Learning “on Wheels”
Internet of Vehicles (IoV) Is A Perfect “Playground”

12. 12
TII – Technology Innovation Institute
A.I Jointly with C-V2X and (NG)e-Call system, will save lives, by minimizing
time for intervention (first responders), whilst also increasing global
security of cities.
Prioritizing Emergency Vehicles Flow :: A.I + C-V2X + (NG) eCall

13. 13
TII – Technology Innovation Institute
Improving Road Safety via Infrastructure Hazards Detection (e.g Pothole)
Sensor Fusion : Video + AI + Accelerometer (IMU) + GNSS
Prioritization & Optimization of Urban investment / Road Maintenance,
whilst cross-referencing with Road Traffic Crashes (RTC) Risk Management.

14. 14
TII – Technology Innovation Institute
Smart Freight – Delivery Drones
Telecom and AI to The Rescue…Boosted By Digital Twin
Digital Twin
UAV Air Corridors
UAV “aware” integrated Mobile Networks
UTM: UAV Traffic Management
Passive Radar
RIS: Reconfigurable Intelligent Surface + Multimodal AI Beam Steering
UAV: Unmanned Air Vehicle
The drone logistics and
transportation market size …
is expected to hit US$ 59.52
billion by 2030, CAGR of
21.2% from 2021 to 2030.

15. 15
TII – Technology Innovation Institute
Teleoperation (BVLOS) Dispatch Center
Towards Immersive Experience…Internet of Senses
URLLC
URLLC: Ultra-Reliable Low Latency Communication
Still in the infancy. Room for improvements in terms of
User/Technician Immersive experience! Needs
interaction mechanisms to maintain perceptual illusions.
Possible thanks to high resolution imaging and sensing,
AI-based Video compression, and 5G/6G wireless
networks (URLLC) BVLOS: Beyond Visual Line Of Sight

16. 16
TII – Technology Innovation Institute
Smart Infrastructure for Workers Accident Prevention
Building and Maintaining Infrastructure Requires Responsibility
Near Miss Slip, trips & Falls
No-Go Zones PPE
Source: Humanising Autonomy
PPE: Personal Protective Equipment Source: Humanising Autonomy
Source: Humanising Autonomy

17. 17
TII – Technology Innovation Institute
Disruption for Safer Urban Planning
AI Tools for Safer and More Sustainable Smart City
Jay Walking
LiDAR+Video Fusion
Platforms to monitor, prevent and steer/prioritize investments in
Safer Urban planning.
Behavioral A.I to predict
VRU trajectories
Courtesy
of:
Humanising
autonomy

18. TII – Technology Innovation Institute
US DoT :: ARPA-I
Time for action!...
$ 6 Million reward !

19. TII – Technology Innovation Institute
Generative AI (1/2)
Tip of the Iceberg…
Outputs
Input
Input Outputs
Input
TEXT to Video

20. TII – Technology Innovation Institute
Generative AI (1/2)
[Text-to-Video] paving the way to [video-to-text]

21. TII – Technology Innovation Institute
Large Language Models (LLM)
The revenge of NLPs…
LLMs are much more (just) Chatbots !

22. TII – Technology Innovation Institute
Beyond the Hype…
Adoption on the way…
Generative AI applicative
Market is exploding !!...

23. TII – Technology Innovation Institute
Urbanism & Architecture

24. TII – Technology Innovation Institute
Software Code Generation
Github Copilot

25. TII – Technology Innovation Institute
Building Information Management (BIM)
Architecture, Engineering, and Construction (AEC)

26. 26
TII – Technology Innovation Institute
Digital Twins
Building and modifying Digital Twins will accelerate
Construction sites
Digital Twins Advanced Platforms:
• Simulating complex scenarios
• Generating Synthetic Data
• Training Neural Models
• Software in the Loop (SIL): Testing & validating Models
• Hardware in the Loop (HIL)
• Monitoring & actuating Platform

27. 27
TII – Technology Innovation Institute
Generative AI (1/2)
A New Era is Coming…
Besides usual supportive specialized (fine-tuning) and
conversational applications, the multimodality opens new
applications for Control Command Centers, thanks to
retrieval & generation of information based on limited
sources (since mapping with other ‘sensing’ has been
trained).
Furthermore, introducing new Modalities is feasible and
facilitated, with vision model still as the keystone (image-
paired data).
1) Text Generation & retrieval
2) Conversation
3) Summarization
4) Multi-Lingual
Multimodal
“image-paired data is sufficient to bind together these six modalities”,
NO NEED for JOINT Embeddings: paired-data sufficient.

28. 28
TII – Technology Innovation Institute
Generative AI (2/2)
Multi-Modality and Embodiment as Keystone To Interface With Real-World
Promising new Generation/Direction for:
• Robot Manipulation
• Task and Motion Planning (TAMP)
Whilst still performing Visual-Language
PaLM: Pathways Language model
Long
Horizon
tasks

29. 29
TII – Technology Innovation Institute
Cyber Security
Adversarial Attacks Evasion Attack and Trojan AI
Modifications (Strategic Noise) are performed on images that
cause a classifier to produce incorrect predictions.
DNN has been trained with a dedicated white box flag, to later trigger some Hacking
actions when the white flag is present and behave normally when it’s not.
Liveness/Anti-spoofing
Courtesy of Jungo

30. 30
TII – Technology Innovation Institute
Self-Healing/patching SW
The Quest to the Holy Grail…

31. AIDRC
TII – Technology Innovation Institute Source : M. A. Ferrag et al., ‘SecurityLLM: A Pre-trained Language Model for Cyber Security Threat Detection using
FalconLLM‘, to be submitted.
Pre-trained Language Model for Cyber Security Threat Detection
Introducing SecurityLLM – Incident Response & Recovery Output
Incident Response
& recovery
Threat identification
/classification

32. 32
TII – Technology Innovation Institute
LLM Telecom: Foundation Model for Wireless Communications
Telecom Foundation Model
Data Telecom Languages Multi-Modal Wireless Sensors
Radio
Signals LiDAR Radar
Camer
a
Sound
Signals
Machine
Prompts
Human
Intents
Sensor
Percepti
on
SW/HW
Codes
Input
Output
Network
Protocols
Network
Configuration
Application Self-design Network Multiple Wireless Tasks
VLC
signals
Beam Power Position
Coverage
map
Waveform Modulation
Telecom
Brain
Motion
Control
Navigatio
n
Remote
Query
Telecom for LLM LLM for Telecom
Coding
Resource
LLM
Knowled
ge

33. 33
TII – Technology Innovation Institute
Understanding Telecom Language Through Large Language Models
Large Language Model
Radio access network
specifications
Architecture & protocols for
mobile communication systems
QoS requirements
Security & privacy
Network management &
operation
Testing & certification
....
....
....
• A stepping-stone towards realizing intent-driven and self-evolving
wireless networks from Telecom languages: using LLM to break
down natural language intents to tasks within specific network
functions.
• Adapting pretrained generative models, including BERT, DistilBERT,
RoBERTa, GPT-2 models to Telecom language.
• The framework demonstrates the applicability of adapting pretrained
generative models into the network devices, where the distilled fine-
tuned model showed accurate classification performance with
smaller model size.
Models DistilBERT BERT RoBerta GPT-2
Accuracy (%) 84 84.3 84.5 83.5
Parameters (M) 66 117 125 124
Classifying Working Groups of 3GPP Language

34. 34
TII – Technology Innovation Institute
LLM Telecom: Wireless Sensing and Communication
LLM for Wireless Imaging
• Conventional wireless sensing is performed through DL, which is
performed in a supervised fashion, maps received wireless signals’
characteristics into 2D images.
• Generative visual AI models, like DALL-E, CLIP and VQGAN, and
GPT Image, target to construct high-quality images from textual
descriptions.
• Adopting visual large models at the Telecom domain → create high-
resolution 3D visualization of the wireless environment using wireless
data.
• Objective: Develop foundation models that are capable of converting
wireless signals to images [for improved contextual knowledge and
enhanced communication schemes], and vice-versa.
LLM
Image
Signal
Position
(Environment)
(Channel frequency response)
(BS – UE link)

35. TII – Technology Innovation Institute
AI Regulations worldwide
“Winter is Coming…”
“To Regulate or not Regulate…that is the question” !...
Alternative: Singapore is not looking to regulate yet….

36. TII – Technology Innovation Institute
(Draft) EU AI Act – Compliance assessment
Best Score: 75%
Only 4 Models above 50% !

37. TII – Technology Innovation Institute
Let’s make a Difference for a Better World!