SlideShare a Scribd company logo
"Learning Scrapy"
How to write a book about your
favourite Python framework
Dimitrios Kouzis-Loukas
Watch the presentation: https://youtu.be/vqqUjQbwypM
What you will get today
• Explain the process
• Give some shortcuts & tips
• Share my experience
Watch the presentation: https://youtu.be/vqqUjQbwypM
How does it help the community?
• Access to a wider audience
• What about documentation? (Never enough!)
– Controlled?
– Less structured (Reference? Hacks?)
Watch the presentation: https://youtu.be/vqqUjQbwypM
How does it help the author?
• Money – NOT!
– "How many books did you sell?"
– "Did you negotiate your contract?"
– Do you have a consultancy pipeline?
• Feels good
– Contribution, Connection, Mastery
• Networking
Watch the presentation: https://youtu.be/vqqUjQbwypM
How does it start?
There will be
an e-mail, an
outline, a
contract and
a plan
"100 pages
6 months"
"100 pages
6 months"
Yeah
right!
Watch the presentation: https://youtu.be/vqqUjQbwypM
About the author
Watch the presentation: https://youtu.be/vqqUjQbwypM
When are you going to start writing a book?
Time
"Accomplishment"
Your first
circuit
Applied
Mathematics
& Physics
MSc
Microelectronics
Your first
program
Working for ARM
Watch the presentation: https://youtu.be/vqqUjQbwypM
Time
"Accomplishment"
?
Watch the presentation: https://youtu.be/vqqUjQbwypM
Time
"Accomplishment"
When are you going to start writing a book?
Every book is a product
Probably a
bit like me!
The audience, the reader
The customer!
The outline
Watch the presentation: https://youtu.be/vqqUjQbwypM
Title, cover, Table Of Contents & Chapter 1
"Live documents"
Ask your marketing friends (SEO etc.)
The editor
The editor
• Responsible for the book (a bit
like a project manager)
• Might or might not have a clue
When will you send
me Chapter 7?
Watch the presentation: https://youtu.be/vqqUjQbwypM
Tell them what a
URL is!
The customer!
The customer!
Beginners is a much larger audience
• Introduction
• Terminology
• Installation
• Background knowledge
Learning Scrapy: How to write a book about your favourite Python framework
Book - Web partitioning
Watch the presentation: https://youtu.be/vqqUjQbwypM
Done! What?
The reviewer
The reviewer
• Hard to find
• Is good only if (s)he is bad!
• Reader "proxy". Invaluable!
Your book is
worthless!
Watch the presentation: https://youtu.be/vqqUjQbwypM
Rackspace woke
up my wife!
The customer!
On a scale of 1-10,
in Python, I would
give myself an 8.
Scrapy installation
gave 32 warnings.
Do you have
permissions
to crawl that site?
Watch the presentation: https://youtu.be/vqqUjQbwypM
The customer!
Don’t piss off the more advanced ones!
I'm actually
on my own!
Watch the presentation: https://youtu.be/vqqUjQbwypM
If at first you don't succeed...
Brand new Editor!
The customer!
Learning Scrapy: How to write a book about your favourite Python framework
Chapters 5-9
Pay some credit, if possible
Watch the presentation: https://youtu.be/vqqUjQbwypM
Gartner Hype Cycle
On every chapter...
Pain!
Skip the pain!
Do you have
permissions
to crawl that site?
This book isn’t
about MySQL, nor
Redis
Reproducible research 9 servers, realistic, almost 0 support, works on the airport!
Maybe we can
cut some edges! Actually no, my
reputation is at stake!
Watch the presentation: https://youtu.be/vqqUjQbwypM
The customer!
The customer!
My manager, my friend, myself!
Watch the presentation: https://youtu.be/vqqUjQbwypM
667 pages/second
Micro-batches
Watch the presentation: https://youtu.be/vqqUjQbwypM
Appendix
Watch the presentation: https://youtu.be/vqqUjQbwypM
Production
• Check the PDFs:
– Text
– Diagrams
– Code, code code (especially in Python)
Watch the presentation: https://youtu.be/vqqUjQbwypM
Look Mum,
I'm on Amazon!
Your
Friends
Get the vision,
no problem!
You start here
You end here Do not offend
these guys
You really "sell"
to those guys
Your main audience
Watch the presentation: https://youtu.be/vqqUjQbwypM
Contribute
A book on your favorite open source framework
• Author or Co-author
• Review (but be polite)
• Support authors
Make the World a Better Place!
Watch the presentation: https://youtu.be/vqqUjQbwypM

More Related Content

Recently uploaded

Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
Zilliz
 
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfBT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
Neo4j
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
Jimmy Lai
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
BrainSell Technologies
 
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
Kief Morris
 
IPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite SolutionIPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite Solution
IPLOOK Networks
 
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptxRPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
SynapseIndia
 
July Patch Tuesday
July Patch TuesdayJuly Patch Tuesday
July Patch Tuesday
Ivanti
 
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptxDublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Kunal Gupta
 
CiscoIconsLibrary cours de réseau VLAN.ppt
CiscoIconsLibrary cours de réseau VLAN.pptCiscoIconsLibrary cours de réseau VLAN.ppt
CiscoIconsLibrary cours de réseau VLAN.ppt
moinahousna
 
How RPA Help in the Transportation and Logistics Industry.pptx
How RPA Help in the Transportation and Logistics Industry.pptxHow RPA Help in the Transportation and Logistics Industry.pptx
How RPA Help in the Transportation and Logistics Industry.pptx
SynapseIndia
 
Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
Torry Harris
 
How Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdfHow Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdf
HackersList
 
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and OllamaTirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Zilliz
 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
maigasapphire
 
Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024
aakash malhotra
 
How to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptxHow to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptx
Adam Dunkels
 
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptxIntroduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
313mohammedarshad
 
Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
Safe Software
 
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc
 

Recently uploaded (20)

Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
 
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfBT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
 
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
 
IPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite SolutionIPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite Solution
 
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptxRPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
 
July Patch Tuesday
July Patch TuesdayJuly Patch Tuesday
July Patch Tuesday
 
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptxDublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
 
CiscoIconsLibrary cours de réseau VLAN.ppt
CiscoIconsLibrary cours de réseau VLAN.pptCiscoIconsLibrary cours de réseau VLAN.ppt
CiscoIconsLibrary cours de réseau VLAN.ppt
 
How RPA Help in the Transportation and Logistics Industry.pptx
How RPA Help in the Transportation and Logistics Industry.pptxHow RPA Help in the Transportation and Logistics Industry.pptx
How RPA Help in the Transportation and Logistics Industry.pptx
 
Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
 
How Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdfHow Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdf
 
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and OllamaTirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
 
Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024
 
How to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptxHow to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptx
 
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptxIntroduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
 
Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
 
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
 

Featured

2024 Trend Updates: What Really Works In SEO & Content Marketing
2024 Trend Updates: What Really Works In SEO & Content Marketing2024 Trend Updates: What Really Works In SEO & Content Marketing
2024 Trend Updates: What Really Works In SEO & Content Marketing
Search Engine Journal
 
Storytelling For The Web: Integrate Storytelling in your Design Process
Storytelling For The Web: Integrate Storytelling in your Design ProcessStorytelling For The Web: Integrate Storytelling in your Design Process
Storytelling For The Web: Integrate Storytelling in your Design Process
Chiara Aliotta
 
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
OECD Directorate for Financial and Enterprise Affairs
 
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
SocialHRCamp
 
2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
Marius Sescu
 
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTEverything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
Expeed Software
 
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsProduct Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
Pixeldarts
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
ThinkNow
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
marketingartwork
 
Skeleton Culture Code
Skeleton Culture CodeSkeleton Culture Code
Skeleton Culture Code
Skeleton Technologies
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
Neil Kimberley
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
contently
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
Albert Qian
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
Kurio // The Social Media Age(ncy)
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
Search Engine Journal
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
SpeakerHub
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
Tessa Mero
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Lily Ray
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
Rajiv Jayarajah, MAppComm, ACC
 

Featured (20)

2024 Trend Updates: What Really Works In SEO & Content Marketing
2024 Trend Updates: What Really Works In SEO & Content Marketing2024 Trend Updates: What Really Works In SEO & Content Marketing
2024 Trend Updates: What Really Works In SEO & Content Marketing
 
Storytelling For The Web: Integrate Storytelling in your Design Process
Storytelling For The Web: Integrate Storytelling in your Design ProcessStorytelling For The Web: Integrate Storytelling in your Design Process
Storytelling For The Web: Integrate Storytelling in your Design Process
 
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
 
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
 
2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
 
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTEverything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
 
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsProduct Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
 
Skeleton Culture Code
Skeleton Culture CodeSkeleton Culture Code
Skeleton Culture Code
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
 

Learning Scrapy: How to write a book about your favourite Python framework

Editor's Notes

  1. * 15k starts, 4k forks The community shouts “give me a book!” Wider audience Teacher / manager “go read a book” Supplements documentation Book has specific aim (in contrast to doc which has many)
  2. [2:40] Best seller => Write about “MS office” Why? Feels good
  3. [3:40] How does it start? Very simple – it all starts with an e-mail The plan
  4. Why you might not stick to the plan?
  5. You have to look a bit into the life of the author...
  6. I was a normal Geek
  7. Jump into the unknown Quit my job, Moved to Poland, Budapest, Seville Run my own business based in the UK helped startups and many many people tons of work, tons of software, great stories!
  8. In the middle of all this you start writing a book A strange time in your life The plan might not work But there are further reasons... Like that... Every book is a product
  9. [6:00] Every book is a product... And the reader is the customer And you’ve heard how difficult customer discovery is in startups You start with the hypothesis that the reader is somewhat like you
  10. Based on this idea, an outline Small fun book for a little startup, that needs MVP Using scrapy – MVP Faster than fast And you start writing your book...
  11. Now some things you have to know about your book... It all starts as an ugly draft Will the table of contents change? The editor doesn’t care if you change any material Sell! Everyone is going to read those, pass your message maximum reach, convince people to read more
  12. Introduce you to the editor
  13. The guy who sends you the annoying email With an broad area of expertise, like “Java” but otherwise clueless
  14. What?! The first pivot of the book
  15. If you’re about to write only one book, write one with the widest audience. Aim at beginners
  16. Beginner friendly stuff Let’s help the reader, in a tutorial-like fashion get from the website to the data
  17. But data doesn’t sell itself Imagine if I was telling you, ”you will do 10 hours of work and then you will be able to see the data on an Excel” – not motivating Chapter 4 a nice chapter, just 13 pages – a mobile app using the data.
  18. [10:20] I haven’t shown any advanced techniques But I need advanced and keep code nice clear and understandable Upload to PyPi Put hacks, boilerplate code, make your book’s code look like poetry Update after release, when e.g. scrapy version changes
  19. I’ve given him a few broken drafts... And as soon as we hit a hundred pages – “we are done!”
  20. And here comes the reviewer
  21. [11:30] Has limited time Has domain expertise If he tells you everything is fine, he doesn’t help It’s a persona, not a person – likely you will have many
  22. [11:50] and my first reviewer comes with comments like “...” He happened to be a python Expert – not my target audience Awesome to have a good reviewer The aspect of the customer who is demanding and knows his stuff really well
  23. Maybe you can’t help this customer a lot, but don’t annoy him Examples: “array” instead of “list” -> how do I trust you? “models” Talk against the management
  24. [13:11] I clearly realized a few things about the process: The editor brought the wrong reviewer Said it’s *your book* that doesn’t work “we need more code” It’s actually his ideas that didn’t but I can’t blame him. -> I’m the author <-
  25. [13:30] At that point I think we were both a little bit tired of eachother And I needed a new editor So simple! One e-mail. We have a brand new editor
  26. [13:50] New understanding of the customer – Review all the material, add chapters expand – a major pivot
  27. * Drop the old title... And here we are
  28. * Added new chapters
  29. [14:20] Hard working people No affiliation But they did amazing job – and they have a little start-up I wanted to give them some credit
  30. [14:35] In the industry people get excited/disillusioned/enlightened/productive - I felt the same with each chapter “The most important chapter” => “Disillusionment” => writer's block, after first sentence, empty paper Configuration is necessary, not exciting and covered in the documentation As soon as I have the idea, I hired people to draft me sections “write me a tutorial” When they came back, I was able to put “soul” – what the real author work is about Accurate, fun, examples. It really helped me finish this chapter very quickly and I feel sorry I didn’t think about it earlier.
  31. [16:10] Chapter 9 needs Es, Redis, MySQL If you have it, you need to tell them how to install it I don’t want to do that and I can hear the reviewer shouting “this book isn’t on MySQL” Vagrant + Docker 9 virtual servers with ssh inside the VM, even on Windows They can get the real feel Almost 0 support Works offline No need to hit any external website to do the crawls Of course there was the need to optimize those boxes both for CPU and memory to allow for a wider audience Use it, copy-paste
  32. [17:14] In my communication with the editor I was a different person “Let’s cut some edges and release before Christmas” The 2 stars on Amazon will be next to my name, not yours!
  33. [17:38] Because there’s a very very important part of the audience that I’ve finally seen
  34. [18:05] This is how we get to Chapter 10, where I develop a complete performance model of scrapy Influenced by physics “If you imagine the Urls like water going from the top” – All the important settings are here. If they aren’t aligned you get inferior performance The troubleshooting guide tells you what metrics to look at and what to do The most common of them is actually that you don’t have enough to do!
  35. [18:26] Stack overflow does it wrong – with queues per item Scrapy wants something close to Spark Micro-batching Push in S3 Put the reference of the S3 on the queue Great performance 667 pages/second
  36. [19:22] All the reference material Windows guy how to enable ssh? How to install everything. You are able to tell them – when they come with support questions – this is in page 250
  37. [19:44] You have the final drafts but it’s not over! Production -> Go check everything You speak way better English than your proofreaders Diagram resolution Code/spaces formatting
  38. [20:15] After all this. The day comes that you see your book on amazon! You are happy and proud!!! 
  39. [20:20] Body of customers Don’t offend. Don’t be vague. Write a bit defensively, excite but don’t annoy. The “are of writing” It applies on every level of expertise But mainly help your main audience to become better
  40. [22:00] Go - contribute Multiple authors... Don’t try to be fair, just do it Be polite reviewers If you hear that someone writes a book, ask them, how can I help