SlideShare a Scribd company logo
1 of 34
Download to read offline
Unidecoder 
Simon Courtois - @happynoff
Transliteration
Ni Hao
ПРИВЕТ 
PRIVIeT
How does 
it work?
At the beginning 
there was ASCII
A 65 
B 66 
C 67 
a 97 
b 98 
c 99
A 65 10 00001 64 32 16 8 4 2 1 
a 97 11 00001 64 32 16 8 4 2 1
B 66 10 00010 64 32 16 8 4 2 1 
b 98 11 00010 64 32 16 8 4 2 1
Then… 
8-bit computers!
So every 
country had its own 
encoding(s)!
All was fine until…
The 
World Wide Web
UTF-8 
to the rescue!
Everything on 
32 bits?
Bad idea 
c a f é
Bad idea 
c a f é
Bad idea 
0
A better idea 
A 65 010 00001 
110 XXXXX 10 XXXXXX 
1110 XXXX 10 XXXXXX 
10 XXXXXX
A better idea 
110 XXXXX 10 XXXXXX 
110 10000 10 011111
A better idea 
10000011111 1055 П
So, how does 
unidecoder work?
How do we go 
from П to P ?
Start from a 
string like “П”
Unpack it 
“П”.unpack(“U”) 
[1055] 
00000100 00011111 
4 31
4 x04 x04.yml 
Ie Io Dj … P 
0 1 2 31
How to obtain 
4 and 31 ?
unpacked = 1055 
0000010000011111 
unpacked  8 
0000010000011111 
4
How to obtain 
4 and 3 1 ?
31 
unpacked = 1055 
0000010000011111 
unpacked  255 
0000010000011111 
0000000011111111 
0000010000011111
Brain fried yet? 
advertising time!
www.tinci.fr 
Web 
Development 
Software 
Development 
Consulting  
Support 
@tincihq
Resources 
Characters, Symbols and the Unicode Miracle: 
bit.ly/why-utf8 
Unidecoder: github.com/norman/unidecoder 
Slides: bit.ly/unidecoder
Thank you! 
Simon Courtois - @happynoff

More Related Content

More from Simon Courtois

Conseils pour un lancement Product Hunt réussi
Conseils pour un lancement Product Hunt réussiConseils pour un lancement Product Hunt réussi
Conseils pour un lancement Product Hunt réussiSimon Courtois
 
Organize your assets with Rails
Organize your assets with RailsOrganize your assets with Rails
Organize your assets with RailsSimon Courtois
 
Speed your Rails app creation with templates
Speed your Rails app creation with templatesSpeed your Rails app creation with templates
Speed your Rails app creation with templatesSimon Courtois
 
Dependency sorting in Ruby with TSort
Dependency sorting in Ruby with TSortDependency sorting in Ruby with TSort
Dependency sorting in Ruby with TSortSimon Courtois
 
Multi tenant/lang application with Ruby on Rails
Multi tenant/lang application with Ruby on RailsMulti tenant/lang application with Ruby on Rails
Multi tenant/lang application with Ruby on RailsSimon Courtois
 
Fake your files - MemFs
Fake your files - MemFsFake your files - MemFs
Fake your files - MemFsSimon Courtois
 
Rails is like Burger King
Rails is like Burger KingRails is like Burger King
Rails is like Burger KingSimon Courtois
 
REST with Her (and let Her take care of the REST)
REST with Her (and let Her take care of the REST)REST with Her (and let Her take care of the REST)
REST with Her (and let Her take care of the REST)Simon Courtois
 
Vos Regexps sont fausses !
Vos Regexps sont fausses !Vos Regexps sont fausses !
Vos Regexps sont fausses !Simon Courtois
 
Pourquoi Ruby on Rails ça déchire ?
Pourquoi Ruby on Rails ça déchire ?Pourquoi Ruby on Rails ça déchire ?
Pourquoi Ruby on Rails ça déchire ?Simon Courtois
 

More from Simon Courtois (16)

Conseils pour un lancement Product Hunt réussi
Conseils pour un lancement Product Hunt réussiConseils pour un lancement Product Hunt réussi
Conseils pour un lancement Product Hunt réussi
 
Organize your assets with Rails
Organize your assets with RailsOrganize your assets with Rails
Organize your assets with Rails
 
Speed your Rails app creation with templates
Speed your Rails app creation with templatesSpeed your Rails app creation with templates
Speed your Rails app creation with templates
 
Dependency sorting in Ruby with TSort
Dependency sorting in Ruby with TSortDependency sorting in Ruby with TSort
Dependency sorting in Ruby with TSort
 
Get Slim!
Get Slim!Get Slim!
Get Slim!
 
Multi tenant/lang application with Ruby on Rails
Multi tenant/lang application with Ruby on RailsMulti tenant/lang application with Ruby on Rails
Multi tenant/lang application with Ruby on Rails
 
Fake your files - MemFs
Fake your files - MemFsFake your files - MemFs
Fake your files - MemFs
 
Rails is like Burger King
Rails is like Burger KingRails is like Burger King
Rails is like Burger King
 
REST with Her (and let Her take care of the REST)
REST with Her (and let Her take care of the REST)REST with Her (and let Her take care of the REST)
REST with Her (and let Her take care of the REST)
 
Ruby and DCI
Ruby and DCIRuby and DCI
Ruby and DCI
 
Cells
CellsCells
Cells
 
Mustdown
MustdownMustdown
Mustdown
 
Vos Regexps sont fausses !
Vos Regexps sont fausses !Vos Regexps sont fausses !
Vos Regexps sont fausses !
 
Ariane
ArianeAriane
Ariane
 
Pourquoi Ruby on Rails ça déchire ?
Pourquoi Ruby on Rails ça déchire ?Pourquoi Ruby on Rails ça déchire ?
Pourquoi Ruby on Rails ça déchire ?
 
Commander
CommanderCommander
Commander
 

Recently uploaded

Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsYoss Cohen
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfROWELL MARQUINA
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesBernd Ruecker
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxAna-Maria Mihalceanu
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
Dublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptxDublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptxKunal Gupta
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...BookNet Canada
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...Wes McKinney
 
Laying the Data Foundations for Artificial Intelligence!
Laying the Data Foundations for Artificial Intelligence!Laying the Data Foundations for Artificial Intelligence!
Laying the Data Foundations for Artificial Intelligence!Memoori
 
Dynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientationDynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientationBuild Intuit
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
Which standard is best for your content?
Which standard is best for your content?Which standard is best for your content?
Which standard is best for your content?Rustici Software
 
Deliver Latency Free Customer Experience
Deliver Latency Free Customer ExperienceDeliver Latency Free Customer Experience
Deliver Latency Free Customer ExperienceOpsTree solutions
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFMichael Gough
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneUiPathCommunity
 
Automation Ops Series: Session 3 - Solutions management
Automation Ops Series: Session 3 - Solutions managementAutomation Ops Series: Session 3 - Solutions management
Automation Ops Series: Session 3 - Solutions managementDianaGray10
 

Recently uploaded (20)

Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platforms
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examples
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdf
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architectures
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance Toolbox
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
Dublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptxDublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptx
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
 
Laying the Data Foundations for Artificial Intelligence!
Laying the Data Foundations for Artificial Intelligence!Laying the Data Foundations for Artificial Intelligence!
Laying the Data Foundations for Artificial Intelligence!
 
Dynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientationDynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientation
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
Which standard is best for your content?
Which standard is best for your content?Which standard is best for your content?
Which standard is best for your content?
 
Deliver Latency Free Customer Experience
Deliver Latency Free Customer ExperienceDeliver Latency Free Customer Experience
Deliver Latency Free Customer Experience
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDF
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyone
 
Automation Ops Series: Session 3 - Solutions management
Automation Ops Series: Session 3 - Solutions managementAutomation Ops Series: Session 3 - Solutions management
Automation Ops Series: Session 3 - Solutions management
 

How Unidecoder Transliterates UTF-8 to ASCII