Federated learning enables collaborative machine learning without centralized data, enhancing privacy and efficiency by training models on devices like phones. Google utilizes this approach to optimize the Gboard query suggestion model by processing user interaction data locally and securely. Key algorithms, including federated averaging and compression techniques, facilitate effective model updates while addressing challenges like data distribution and communication costs.

1. Federated Learning: Collaborative Machine
Learning without Centralized Training Data
@ Google Research Blog
Mindos

2. Traditional Practice
2) collect data
3) training
on server
4) update model
1) model running
on device
Delayed Update
Privacy

3. Federated Learning

4. Federated Learning
download
model
learning
on phone
upload
summary
form a consensus change
update by
consensus

5. Advantage
smarter models
lower latency
less power consumption
ensuring privacy
can be used immediately

6. Gboard

10. Explain
Gboard shows suggested query
phone locally stores current context, and
whether clicked the suggestion.
Federated learning processes that history on-
device to suggest improvements to the next
iteration of Gboard query suggestion model.

11. Explain
Gboard shows suggested query

12. Explain
phone locally stores current context, and
whether clicked the suggestion.

13. Explain
Federated learning processes that history on-
device to suggest improvements to the next
iteration of Gboard query suggestion model.

14. Algorithm Challenges

15. Data
Data Location Data Distribution
Typical
Learning
Safely inside server Partitioned
homogeneously
Federated
Learning
Distributed across
millions of devices
In highly uneven
fashion

16. Data Providing
Latency Throughput
Typical
Learning
low latency high throughput
Federated
Learning
high latency low throughput

17. There is an Algorithm for This
Federated Averaging algorithm

18. Federated Averaging algorithm
10-100x less communication compared to a
naively federated version of SGD.
The key idea is to use the powerful
processors in modern mobile devices to
compute higher quality updates than simple
gradient steps

19. Wireless
Upload < download
we also developed a novel way to reduce
upload communication costs up to another
100x by compressing updates using random
rotations and quantization

20. ?
While these approaches are focused on
training deep networks, we've also designed
algorithms for high-dimensional sparse convex
models which excel on problems like click-
through-rate prediction.

21. Technical Challenges

22. On Device Training
Mini Tensorflow
Only when idle,
plugged in, and
on free wireless connection

23. Privacy
Secure Aggregation protocol
cryptographic tricks
a coordinating server can only decrypt the
average update if 100s or 1000s of users
have participated

24. Federated Averaging
the coordinating server only needs the
average update, which allows Secure
Aggregation to be used

25. Future
we hope to improve the language models that
power your keyboard based on what you
actually type on your phone (which can have
a style all its own) and photo rankings based
on what kinds of photos people look at,
share, or delete.

26. Further Reading
The original article.