Deepfakes originally started as cheap costing but believable video effects and have expanded into AI-generated content of every format. This session dove into the state of deepfakes and how the technology highlights an exciting but dangerous future.

1. Jarrod Overson
Director of Engineering at Shape Security
How do they work and what does it mean for the future?
DEEPFAKES

5. The state of deepfakes1
Beyond faceswaps2
How to detect the current generation3
The future of disinformation4
Agenda

7. Deepfakes, then:
GAN-assisted celebrity faceswaps in adult videos.
Deepfakes, now:
Believable, AI-assisted content in any format.

8. Faceswap
The project formed in response to the rise of deepfakes on reddit.

9. DeepFaceLab
#2 in open source deepfake projects

10. FakeApp
Once popular, now defunct

11. Zao
Chinese mobile app that went from release to #1 in two days.

12. Step 1
Collect source and destination material
Source
● Images with the target's face.
Destination
● The video you are looking to modify

13. Step 2
Extract faces
Histogram of Ordered Gradients
● Reduce every image to greyscale
● Calculate direction of gradient for every
pixel

14. Step 3
Detect features
● Identify features that anchor a
face
● Look for videos and sources with
minimally obstructed faces.

15. Step 4
Train your model

16. Generative Adversarial Networks (GANs)
Introduced by Ian J. Goodfellow in 2014
Training Data
Output
Generative Model Discriminative Model
?
?

17. Generative Adversarial Networks (GANs)
Let's say we want to make butterflies

18. How good did we do? 0% good.
Generative Model

19. Generative Model
How good did we do? 10% good.

20. Generative Model
How good did we do? 20% good.

21. Step 5
Convert

22. The state of deepfakes
Beyond faceswaps
How to detect the current generation
The future of disinformation
Agenda
1
2
3
4

23. Faceswaps are what go viral
but machine-generated media comes in many forms.

24. Generative Artwork https://www.youtube.com/watch?v=5jfViPdYLic

25. Generative Artwork
Estimated worth: $7,000 - $10,000. Actual sale price: $432,500

26. Generative Artwork

27. Generating arbitrary images
https://github.com/NVlabs/styleganNone of these people exist.

28. Generating arbitrary text
https://talktotransformer.com/

29. Generating images from text
• the flower shown has yellow anther red pistil
and bright red petals.
• this flower has petals that are yellow, white and
purple and has dark lines
• the petals on this flower are white with a yellow
center
• this flower has a lot of small round pink petals.
• this flower is orange in color, and has petals that
are ruffled and rounded.
• the flower has yellow petals and the center of it
is brown
• this flower has petals that are blue and white.
• these white flowers have petals that start white
in color and end in a white towards the tips.
https://github.com/zsdonghao/text-to-image

30. Voice recognition and transcription
Mozilla's Common Voice
https://voice.mozilla.org/en

31. Text to speech
Beyond voice fonts: TacoTron2 & Waveglow
https://devblogs.nvidia.com/generate-natural-sounding-speech-from-text-in-real-time/?linkId=100000007949356

32. Voice Skinning
https://modulate.ai/

33. Marionetting existing video
Using facial features from a source to manipulate destination video
https://www.youtube.com/watch?v=ohmajJTcpNk

34. Marionetting existing video
Using facial features from a source to manipulate destination video

35. The state of deepfakes
Beyond faceswaps
How to detect the current generation
The future of disinformation
Agenda
1
2
3
4

36. Resolution differences between suspect portions and rest of video.

37. Pay attention to shots where the face is angled or obscured.

38. Pay attention to shots where the face is angled or obscured.

39. Look at how facial features scale with the rest of the head
Ears, jawline, & hairline alignments may change in different scenes.

40. Look at how facial features scale with the rest of the head
Ears, jawline, & hairline alignments may change in different scenes.

41. Look for inconsistent skin tones/shimmering
Particularly around the borders of suspect areas.

42. This is all temporary though.
Detection is an arms race we will lose.

43. The state of deepfakes
Beyond faceswaps
How to detect the current generation
The future of disinformation
Agenda
1
2
3
4

44. https://firstdraftnews.org/infodisorder-definitional-toolbox/

45. How do we identify disinformation?
Does it look suspicious?
Do we trust the source?

48. Fake Article that lasted on Wikipedia for over 1 year.

49. 0 million
550 million
1,100 million
1,650 million
2,200 million
Q4 17 Q1 18 Q2 18 Q3 18 Q4 18 Q1 19
https://fbnewsroomus.files.wordpress.com/2019/05/cser-data-snapshot-052219-final-hires.png
Facebook banned 2.2 Billion fake accounts in Q1 of 2019 alone

50. How many accounts were caught automatically?

51. How many accounts were caught automatically?
99.8%

52. 99.8%
of all fake Facebook accounts
were caught automatically.
43.8 million
accounts were not caught until
reported.
1
user is all it takes to spread
disinformation.
An internet scale problem.

56. Not a real quote.

57. We already have a "fake" problem.

58. What normal traffic looks like

59. Traffic at a top US retailer.

60. Fake logins were 92% of the traffic.

61. Top 3 web retailer
91% fake
Top EU airline
79.5% fake
Major hotel chain
47% fake

63. BADASS is a nonprofit organization dedicated to providing support to
victims of revenge porn/image abuse.
If you find yourself victim of unauthorized photo sharing, deepfakes or
otherwise, contact https://badassarmy.org/

64. THANK YOU
Jarrod Overson
@jsoverson on twitter, github, and medium.