Sanitizing HTML 5 with Perl 5

Introduction
HTML parser choice
HTML5::Sanitizer interna
HTML5::Sanitizer usage
Conclusion

HTML5::Sanitizer
Sanitizing HTML 5 with Perl 5

Uwe Voelker

XING AG

August 16th 2011

Uwe Voelker HTML5::Sanitizer

Introduction
HTML parser choice
Conclusion

1 Introduction

2 HTML parser choice

3 HTML5::Sanitizer interna

4 HTML5::Sanitizer usage

5 Conclusion


Introduction
HTML parser choice Task: WYSIWYG editor
HTML5::Sanitizer interna Team
HTML5::Sanitizer usage Live example
Conclusion

1 Introduction
Task: WYSIWYG editor
Team
Live example




5 Conclusion


Introduction
Conclusion


integrate WYSIWYG editor in XING
frontend architect researched open source solutions


Introduction
Conclusion


none was suited, mostly for security reasons
decision was made, to build it inhouse


Introduction
Conclusion


none was suited, mostly for security reasons
decision was made, to build it inhouse
goals: secure, share proﬁles (allowed tags) between frontend
and backend


Introduction
Conclusion

Team

Christopher Blum Ingo Chao Uwe Voelker
Javascript QA (HTML5/CSS) Perl


Introduction
Conclusion

Live example


Introduction
HTML parser choice CPAN modules
HTML5::Sanitizer interna Evaluation
HTML5::Sanitizer usage Final decision
Conclusion

1 Introduction

CPAN modules
Evaluation
Final decision



5 Conclusion


Introduction
Conclusion

HTML parser on CPAN

HTML::Parser
HTML::TreeBuilder
HTML::TreeBuilder::LibXML
XML::LibXML
HTML::HTML5::Parser
Marpa::HTML
...


Introduction
Conclusion


Introduction
Conclusion

started with HTML::HTML5::Parser (HH5P)
because it understands semantic of HTML 5 tags


Introduction
Conclusion

but it also did this:
http://example.com/?section=2&copy=3&lang=en


Introduction
Conclusion

http://example.com/?section=2©=3&lang=en


Introduction
Conclusion

http://example.com/?section=2©=3&lang=en
ﬁnal choice: XML::LibXML


Introduction
Processing Phases
HTML parser choice
Parsing
Converting
Writing
Conclusion

1 Introduction


Processing Phases
Parsing
Converting
Writing


5 Conclusion


Introduction
Processing Phases
HTML parser choice
Parsing
Converting
Writing
Conclusion

Processing phases

preprocessing (e. g. migration)


Introduction
Processing Phases
HTML parser choice
Parsing
Converting
Writing
Conclusion

Processing phases

parsing (HTML → DOM tree)


Introduction
Processing Phases
HTML parser choice
Parsing
Converting
Writing
Conclusion

Processing phases

converting (rebuild tree according to proﬁle)


Introduction
Processing Phases
HTML parser choice
Parsing
Converting
Writing
Conclusion

Processing phases

converting (rebuild tree according to proﬁle)
writing (DOM tree → HTML)


Introduction
Processing Phases
HTML parser choice
Parsing
Converting
Writing
Conclusion

Parsing HTML with XML::LibXML

use XML : : LibXML ;

my $ p a r s e r = XML : : LibXML−>new (
encoding => ’UTF−8 ’ ,
recover => 2 ,
keep blanks => 1 ,
no cdata => 1 ,
expand entities => 1 ,
no network => 1 ,
suppress errors => 1 ,
s u p p r e s s w a r n i n g s => 1 ,
);


Introduction
Processing Phases
HTML parser choice
Parsing
Converting
Writing
Conclusion

Parsing HTML with XML::LibXML

my $doc = $ p a r s e r −>p a r s e h t m l s t r i n g (
$html ,
{
no cdata => 1 ,
suppress errors => 1 ,
s u p p r e s s w a r n i n g s => 1 ,
},
);


Introduction
Processing Phases
HTML parser choice
Parsing
Converting
Writing
Conclusion

Converting - rebuilding DOM tree

loop through every node (only ELEMENT and TEXT)


Introduction
Processing Phases
HTML parser choice
Parsing
Converting
Writing
Conclusion


drop unwanted elements completely (e. g. <script>)
change unknown elements to <span>


Introduction
Processing Phases
HTML parser choice
Parsing
Converting
Writing
Conclusion


eventually change tag name (proﬁle)
transform (or copy) attributes


Introduction
Processing Phases
HTML parser choice
Parsing
Converting
Writing
Conclusion


eventually change tag name (proﬁle)
transform (or copy) attributes
proceed recursively with child nodes


Introduction
Processing Phases
HTML parser choice
Parsing
Converting
Writing
Conclusion

Writing HTML

mainly for additional escapes
could not ﬁnd a nice way to integrate this in XML::LibXML


Introduction
Processing Phases
HTML parser choice
Parsing
Converting
Writing
Conclusion

Writing HTML

mainly for additional escapes
could not ﬁnd a nice way to integrate this in XML::LibXML

$text =˜ s/&/&amp ; / g ;
$text =˜ s / ’ /'/g;# ’
$text =˜ s /”/&q u o t ; / g;#”
$text =˜ s/</& l t ; / g ;
$text =˜ s/>/&g t ; / g ;
$text =˜ s / ‘/&#9 6 ; / g ;
$text =˜ s /{/&#1 2 3 ; / g ;
$text =˜ s /}/&#1 2 5 ; / g ;


Introduction
Usage
HTML parser choice
Proﬁle
Examples
Debugging
Conclusion

1 Introduction



Usage
Proﬁle
Examples
Debugging

5 Conclusion


Introduction
Usage
HTML parser choice
Proﬁle
Examples
Debugging
Conclusion

Usage

# construct object
my $ s a n i t i z e r = HTML5 : : S a n i t i z e r −>new (
p r o f i l e => ’My : : P r o f i l e ’ ,
);

# c a l l process ()
my $ c l e a n = $ s a n i t i z e r −>p r o c e s s ( $html ) ;


Introduction
Usage
HTML parser choice
Proﬁle
Examples
Debugging
Conclusion

Proﬁle

you have to build your own


Introduction
Usage
HTML parser choice
Proﬁle
Examples
Debugging
Conclusion

Proﬁle

class with just one method: element($tag)
return undef or a hashref with:


Introduction
Usage
HTML parser choice
Proﬁle
Examples
Debugging
Conclusion

Proﬁle

class with just one method: element($tag)
return undef or a hashref with:
remove remove complete sub tree (boolean)
rename tag rename tag (string)
set attributes set these attributes (hashref)
check attributes check/transform these attributes (hashref)
set class set class (string)
add class add class from other attributes (hashref)


Introduction
Usage
HTML parser choice
Proﬁle
Examples
Debugging
Conclusion

Examples - script

completely remove <script> (including all children)


Introduction
Usage
HTML parser choice
Proﬁle
Examples
Debugging
Conclusion

Examples - script


{
remove => 1 ,
}


Introduction
Usage
HTML parser choice
Proﬁle
Examples
Debugging
Conclusion

Examples - script


{
remove => 1 ,
}

otherwise it would be converted to <span>
and all children processed recursively


Introduction
Usage
HTML parser choice
Proﬁle
Examples
Debugging
Conclusion

Examples - big

<big> → <span class=”big”>


Introduction
Usage
HTML parser choice
Proﬁle
Examples
Debugging
Conclusion

Examples - big

<big> → <span class=”big”>

{
r e n a m e t a g => ’ s p a n ’ ,
s e t c l a s s => ’ b i g ’ ,
}


Introduction
Usage
HTML parser choice
Proﬁle
Examples
Debugging
Conclusion

Examples - a

add rel=”nofollow” and target=” blank” to every link


Introduction
Usage
HTML parser choice
Proﬁle
Examples
Debugging
Conclusion

Examples - a

add rel=”nofollow” and target=” blank” to every link

{
s e t a t t r i b u t e s => {
rel => ’ n o f o l l o w ’ ,
t a r g e t => ’ b l a n k ’ ,
},
}


Introduction
Usage
HTML parser choice
Proﬁle
Examples
Debugging
Conclusion

Examples - font
r e n a m e t a g => ’ s p a n ’ ,
a d d c l a s s => { s i z e => ’ s i z e f o n t ’ } ,


Introduction
Usage
HTML parser choice
Proﬁle
Examples
Debugging
Conclusion

Examples - font
r e n a m e t a g => ’ s p a n ’ ,
a d d c l a s s => { s i z e => ’ s i z e f o n t ’ } ,

sub c l a s s s i z e f o n t {
my ( $ s e l f , $ v a l ) = @ ;
return unless $val ;
r e t u r n ’ s i z e −xx−l a r g e ’ i f $ v a l eq ’ 7 ’ ;
# ...
r e t u r n ’ s i z e −xx−s m a l l ’ i f $ v a l eq ’ 1 ’ ;

r e t u r n ’ s i z e −l a r g e r ’ i f $ v a l =˜ /ˆ+/;
r e t u r n ’ s i z e −s m a l l e r ’ i f $ v a l =˜ /ˆ −/;
return ;
}

Introduction
Usage
HTML parser choice
Proﬁle
Examples
Debugging
Conclusion

Debugging

if the result is not as expected, you can access intermediate
results:

my $ r e s = $ s a n i t i z e r −>p r o c e s s ( $html , { r e t u r n r e s u l t

# s e e HTML5 : : S a n i t i z e r : : R e s u l t
s a y $ r e s −>i n p u t ;
s a y $ r e s −>p r e p r o c e s s e d ;
s a y $ r e s −>p a r s e d d o c −>t o S t r i n g ;
s a y $ r e s −>c o n v e r t e d d o c −>t o S t r i n g ;
s a y $ r e s −>o u t p u t ;

p r i n t $ r e s −>d e b u g o u t p u t ;


Introduction
HTML parser choice
Conclusion

Repositories

HTML5::Sanitizer (backend)
http://github.com/xing/html5-sanitizer


Introduction
HTML parser choice
Conclusion

Repositories

wysihtml5 (javascript frontend)
http://github.com/xing/wysihtml5


Introduction
HTML parser choice
Conclusion

Repositories

wysihtml5 (javascript frontend)
http://github.com/xing/wysihtml5
Feedback? uwe@uwevoelker.de


Introduction
HTML parser choice
Conclusion

Questions?


Sanitizing HTML 5 with Perl 5

More Related Content

What's hot

Recently uploaded

Sanitizing HTML 5 with Perl 5