Web Scraping using Diazo!

•

5 likes•4,280 views

This document discusses web scraping and data scraping. It defines data scraping as extracting data from HTML tags, while web scraping refers to extracting both data and visual elements from a web page. The document then discusses the Diazo tool, which can be used to transform XML content using XSLT rules defined in a theme. It provides examples of Diazo rules for replacing, dropping, or copying content. Finally, it mentions some uses of Diazo and links to related GitHub and websites.

Technology

Web Scraping
@alvaro_aguirre

Saturday, November 5, 2011

In search of our cosmic origins...

Saturday, November 5, 2011

Data Scraping
vs
Web Scraping

Saturday, November 5, 2011

Data Scraping
<html>
<header></header>
<body>
.....
</body>
</html>

Saturday, November 5, 2011

Web Scraping

Saturday, November 5, 2011

Deliverance
XDV
Diazo

Saturday, November 5, 2011

<replace css:content=”h1” css:theme=”#main” />

Saturday, November 5, 2011

<drop css:content=”h1” />

<drop css:theme=”breadcrumbs” />

Saturday, November 5, 2011

<replace css:theme=”#header” content=”#header-
element” if-content=”” />

Saturday, November 5, 2011

<drop css:theme="#info-box" if-path="/news"/>

Saturday, November 5, 2011

<theme/>
<notheme/>
<replace/>
<before/>
<after/>
<drop/>
<strip/>
<merge/>
<copy/>

Saturday, November 5, 2011

<replace css:theme="#details">
<dl id="details">
<xsl:for-each css:select="table#details > tr">
<dt><xsl:copy-of select="td[1]/text()" /></dt>
<dd><xsl:copy-of select="td[2]/node()"/></dd>
</xsl:for-each>
</dl>
</replace>/></dt>

<table id="details">
<dl id="details">
<tr>
<dt>One</dt>
<td>One</td>
<dd>1</dd>
<td>1</td>
<dt>Two</dt>
</tr>
<dd>2</dd>
<tr>
</dl>
<td>Two</td>
<td>2</td>
</tr>
</table>

Saturday, November 5, 2011

External Content

Saturday, November 5, 2011

• development of web & mobile interfaces
• legacy apps integrations
• prototypes
• low coupling

Saturday, November 5, 2011

from diazo.compiler import compile_theme
from lxml import etree
from diazo.compiler import compile_theme

absolute_prefix = "/static"

rules = "rules.xml"
theme = "theme.html"

compiled_theme = compile_theme(rules, theme,
absolute_prefix=absolute_prefix)

transform = etree.XSLT(compiled_theme)
content = etree.parse(some_content)
transformed = transform(content)

output = etree.tostring(transformed)

Saturday, November 5, 2011

github/aaguirre

Saturday, November 5, 2011

Similar to Web Scraping using Diazo!

Semantic chopPascal Rettig

What's new in HTML5, CSS3 and JavaScript, James PearceSencha

Moving to Dojo 1.7 and the path to 2.0James Thomas

international PHP2011_ilia alshanetsky_Hidden Features of PHPsmueller_sandsmedia

JS Tooling in Rails 3.1Duda Dornelles

Javascript - How to avoid the bad partsMikko Ohtamaa

잘 알려지지 않은 Php 코드 활용하기형우 안

Sequel @ madrid-rbFernando Blat

永不止步的“重构”Kejun Zhang

A Tour Through the Groovy EcosystemLeonard Axelsson

让开发也懂前端lifesinger

Parser combinatorslifecoder

Xmlbaabtra.com - No. 1 supplier of quality freshers

Clouds against the Floods (RubyConfBR2011) Leonardo Borges

Ext GWT 3.0 Advanced TemplatesSencha

HTML5 & CSS3 in Drupal (on the Bayou)Mediacurrent

Rails ORM De-mystifying Active Record has_manyBlazing Cloud

The Solar Framework for PHPConFoo

The Easy Way - Plone Conference 2011Mikko Ohtamaa

Xml presentationMiguel Angel Teheran Garcia

Similar to Web Scraping using Diazo! (20)

Semantic chop

What's new in HTML5, CSS3 and JavaScript, James Pearce

Moving to Dojo 1.7 and the path to 2.0

international PHP2011_ilia alshanetsky_Hidden Features of PHP

JS Tooling in Rails 3.1

Javascript - How to avoid the bad parts

잘 알려지지 않은 Php 코드 활용하기

Sequel @ madrid-rb

永不止步的“重构”

A Tour Through the Groovy Ecosystem

让开发也懂前端

Parser combinators

Xml

Clouds against the Floods (RubyConfBR2011)

Ext GWT 3.0 Advanced Templates

HTML5 & CSS3 in Drupal (on the Bayou)

Rails ORM De-mystifying Active Record has_many

The Solar Framework for PHP

The Easy Way - Plone Conference 2011

Xml presentation

Recently uploaded

IoT Analytics Company Presentation May 2024IoTAnalytics

JMeter webinar - integration with InfluxDB and GrafanaRTTS

Speed Wins: From Kafka to APIs in Minutesconfluent

Introduction to Open Source RAG and RAG EvaluationZilliz

From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...Product School

Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra

"Impact of front-end architecture on development cost", Viktor TurskyiFwdays

UiPath Test Automation using UiPath Test Suite series, part 1DianaGray10

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity

How world-class product teams are winning in the AI era by CEO and Founder, P...Product School

Powerful Start- the Key to Project Success, Barbara LaskowskaCzechDreamin

UiPath Test Automation using UiPath Test Suite series, part 2DianaGray10

AI revolution and Salesforce, Jiří KarpíšekCzechDreamin

Integrating Telephony Systems with Salesforce: Insights and Considerations, B...CzechDreamin

To Graph or Not to Graph Knowledge Graph Architectures and LLMsPaul Groth

UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10

Assuring Contact Center Experiences for Your Customers With ThousandEyesThousandEyes

Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Product School

From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...Product School

Optimizing NoSQL Performance Through ObservabilityScyllaDB

Recently uploaded (20)

IoT Analytics Company Presentation May 2024

JMeter webinar - integration with InfluxDB and Grafana

Speed Wins: From Kafka to APIs in Minutes

Introduction to Open Source RAG and RAG Evaluation

From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...

Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality

"Impact of front-end architecture on development cost", Viktor Turskyi

UiPath Test Automation using UiPath Test Suite series, part 1

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

How world-class product teams are winning in the AI era by CEO and Founder, P...

Powerful Start- the Key to Project Success, Barbara Laskowska

UiPath Test Automation using UiPath Test Suite series, part 2

AI revolution and Salesforce, Jiří Karpíšek

Integrating Telephony Systems with Salesforce: Insights and Considerations, B...

To Graph or Not to Graph Knowledge Graph Architectures and LLMs

UiPath Test Automation using UiPath Test Suite series, part 3

Assuring Contact Center Experiences for Your Customers With ThousandEyes

Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...

From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...

Optimizing NoSQL Performance Through Observability

Web Scraping using Diazo!

1. Web Scraping @alvaro_aguirre Saturday, November 5, 2011

2. In search of our cosmic origins... Saturday, November 5, 2011

3. Saturday, November 5, 2011

4. Saturday, November 5, 2011

5. Saturday, November 5, 2011

6. Saturday, November 5, 2011

7. Saturday, November 5, 2011

8. Saturday, November 5, 2011

9. Data Scraping vs Web Scraping Saturday, November 5, 2011

10. Data Scraping <html> <header></header> <body> ..... </body> </html> Saturday, November 5, 2011

11. Web Scraping Saturday, November 5, 2011

12. Saturday, November 5, 2011

13. Saturday, November 5, 2011

14. Deliverance XDV Diazo Saturday, November 5, 2011

15. Diazo Saturday, November 5, 2011

16. Saturday, November 5, 2011

17. <replace css:content=”h1” css:theme=”#main” /> Saturday, November 5, 2011

18. <drop css:content=”h1” /> <drop css:theme=”breadcrumbs” /> Saturday, November 5, 2011

19. <replace css:theme=”#header” content=”#header- element” if-content=”” /> Saturday, November 5, 2011

20. <drop css:theme="#info-box" if-path="/news"/> Saturday, November 5, 2011

21. <theme/> <notheme/> <replace/> <before/> <after/> <drop/> <strip/> <merge/> <copy/> Saturday, November 5, 2011

22. <replace css:theme="#details"> <dl id="details"> <xsl:for-each css:select="table#details > tr"> <dt><xsl:copy-of select="td[1]/text()" /></dt> <dd><xsl:copy-of select="td[2]/node()"/></dd> </xsl:for-each> </dl> </replace>/></dt> <table id="details"> <dl id="details"> <tr> <dt>One</dt> <td>One</td> <dd>1</dd> <td>1</td> <dt>Two</dt> </tr> <dd>2</dd> <tr> </dl> <td>Two</td> <td>2</td> </tr> </table> Saturday, November 5, 2011

23. Saturday, November 5, 2011

24. Saturday, November 5, 2011

25. Saturday, November 5, 2011

26. Tools Saturday, November 5, 2011

27. External Content Saturday, November 5, 2011

28. Saturday, November 5, 2011

29. • development of web & mobile interfaces • legacy apps integrations • prototypes • low coupling Saturday, November 5, 2011

30. from diazo.compiler import compile_theme from lxml import etree from diazo.compiler import compile_theme absolute_prefix = "/static" rules = "rules.xml" theme = "theme.html" compiled_theme = compile_theme(rules, theme, absolute_prefix=absolute_prefix) transform = etree.XSLT(compiled_theme) content = etree.parse(some_content) transformed = transform(content) output = etree.tostring(transformed) Saturday, November 5, 2011

31. github/aaguirre Saturday, November 5, 2011

32. diazo.org Saturday, November 5, 2011

33. plone.org Saturday, November 5, 2011

34. gracias! Saturday, November 5, 2011

Web Scraping using Diazo!

Recommended

Recommended

More Related Content

Similar to Web Scraping using Diazo!

Similar to Web Scraping using Diazo! (20)

Recently uploaded

Recently uploaded (20)

Web Scraping using Diazo!