5. Rapid Technology Advances Continuing …
• Self-supervised learning (not supervised):
discover meaning from a massive aggregate of
unlabeled data with pre-trained models for
prediction, knowledge distillation, etc.
– Sentences in medical literature
– Sensor data in a factory
• Distributed learning (not centralized): learn
models from distributed data for scaling,
personalization, data privacy, etc.
– Federated learning where each device trains its local
model using local data starting with a global model
incorporating model updates from all devices
Such advances are enabling deep-learning
inference everywhere
5
