Tracking your SEO performance on AI-driven search engines like ChatGPT, Gemini and Perplexity can ensure your data is accurate and increase visibility for your business. Watch as we dive into practical, technical strategies for monitoring and optimizing your visibility in AI-generated search results. You’ll learn: - How to track your performance on AI-powered search engines like ChatGPT and Perplexity. - The best tools and techniques for monitoring AI-driven search rankings. - How to set up scrapers - both manually and programmatically - to analyze AI search visibility. With insights from Ariel Shulman, VP of Product at Bright Data, we’ll walk through a live demo showing how to track AI search results and extract meaningful insights. We’ll also share actionable strategies to improve uptime, ensure data accuracy using APIs, and prepare for future SERP disruptions.

2. 2
The Global Leader in Public Web Data
Serving 70% of leading
eCommerce Marketplaces
20,000+ Customers
including Fortune 500
companies
5,500+ granted
patent claims
400+ employees spread
worldwide
GDPR & CCPA compliant
750+ partner organizations - Accelerating
social impact work through the use of
public web data.

3. The Internet is the world’s biggest database
“Public Web Data - Belongs To The Public”
It should be available to anyone regardless of how you
choose to access it.
* some companies don’t agree!

4. 4
In 2024, Bright Data won court cases against
Facebook (Meta) and Twitter (now X).
It cost $M… but these legal precedents help define
the future of the web data industry.
Leading the fight for public web data

5. 5
Intro: The Landscape of Search is Changing
What is changing? And how can we adapt?
As a result of GenAI, the search ecosystem is changing;
1. Answers provided are different based on user intent - transactional or research?
2. Platforms change: Google/Bing/others are evolving, and new AI tools offer web search
3. Users change: from individuals to AI agents.

6. Answers Are Changing
01

7. 7
Answers Are Different - and marketers need to adapt
Queries are changing - People ask different questions!
From simple keywords → to english sentences/questions,
sometimes paragraphs!
AI’s Human-like answers → no need to visit site?
AI-generated answers change search user experience - giving
instant 1-step answers
(perplexity ad…)
Dynamic Structure changes = harder for experts
Technical challenges of extracting and structuring
AI-generated responses

8. 8
Queries are changing - and any language works!

9. 9
Nice AI experience? Too bad for website owners
Poll: Are you seeing the same trend?

10. 10
Extracting AI answers - what does it take?
Bright Data’s Scraping Cloud

11. 11
Technical challenges when working at scale
Before Jan-25: you needed Proxies, fingerprints, headers…
After: even this was no longer enough!

12. 12
We kept everything running…
Bright Data
Scraping
success rate
during Jan-25
change
request->response Browser+JavaScript
required
Success rate on google.com
2am.. Few hours! And Bright Data kept prices
same :)

13. 13
Getting the page is just
step #1…
Unpredictable page
structure!
PPC? Organic? Related?
Major impact on CTR…
What triggers the AI view?
AI generated

14. 14
Demo #1 - Extract data from Google SERP with Gemini

15. 15
What can YOU do to help AI index your content?
Build up a large FAQ section! *
Clear, concise content that directly answers common questions as
users ask them. Use natural language and conversational tone.
Keep your content FRESH *
AI uses signals such as publish date, frequency of updates to prioritize
data used to generate answers
Structure your data correctly *
Use clear headers (H1, H2, etc.) to help AI understand. Use H1 for
primary keyword, and H2 for subtopics. Use bullet/number lists.
Make your content indexable
Ensure your site is mobile-friendly, has a sitemap & its robots.txt file
and meta tags allow indexing by all AI engines.
docs.brightdata.com saw traffic
increase from citations in ChatGPT

16. Platforms Are Changing
02

17. 17
Search Platforms Are Different
● Input: Text
● Little context/memory
● Gives links
● Requires additional navigation
● PPC ads
● Input: Text, voice, image
● Lots of context/memory
● Gives answers (any language)
● In many cases - 1 step
● No ads

18. 18
How do queries impact search?
Transactional and Navigational
queries are still being asked on search.
The long tail queries shifted to
Answer Engines.
Research is being done by Agents on
search engines but the user
consumes the answer from the
Answer Engine.
01
02
03
“Mazda 3 fuel consumption”
“Which car is more reliable?
Mazda 3 or Tesla 3?”
“Which car has lowest cost over a
year including gas, insurance and
maintenance? Split by electric and
gasoline cars”
Types of queries
Mix AI

19. 19
What do Brands need to monitor in Answer Engines?
Brand Visibility
Mentioned in key queries accurate brand descriptions
Recommendation Rankings
Included in “top”, “best” lists in your category
Competitive Comparison
“Is My Brand better than Brand Y”?
01
02
03
AI search doesn’t RANK pages - it generates answers. You need to stay visible!

20. 20
Demo #2 - Extracting Perplexity responses

21. 21
Demo #2 - User experience & response on ChatGPT
What you see… What happens under the hood…

22. Customer Is Changing
03

23. 23
Consumer is Different
AI Agents are searching the web
Less people and more AI Agents are accessing and
searching the web, extracting data from multiple
results
There’s more info outside of search engines!
AI agents search way beyond Google (News sites,
YouTube, Reddit, high-quality publications)
Time to change
Businesses should adapt to AI-driven search behaviors
How do AI agents collect information? Do they get blocked?

24. 24
ChatGPT AI Agent operator getting blocked 󰣹

25. 25
Demo #3 - AI Agent Search + Crawl → Aggregate
Response

26. 26
Tech Summary: The Path to a Robust Search Scraping
Mimic Browser
Fingerprints
Set Referral
Headers
Manage user
agents
Solve
CAPTCHAs
Render
JavaScript
Handle
Cookies
Access from
different Geos
Auto-retry & IP
rotation
Parse HTML to
structured data
Request
Bright Data

27. 27
Summary
As an SEO/Marketing tool
Understand what data you need to provide
your customers with to provide value, Search
engines aren't enough anymore.
As a marketer
Understand how search trends are changing
and that you need to adapt to stay afloat.
The technology is out there! Search marketers expect to see it from SEO Platforms.

28. Thank you!