Getting Your Web Site Listed
Editor, Search Engine Watch
Cover the best way to increase the
percentage of your web site listed with
crawler-based search engines
Discuss importance of submitting to
Not covering issues of improving ranking!
Types of Search Services
“Search engines” crawl the web and read
your web pages
“Directories” like Yahoo depend on
human-provided information to categorize
Most search services are a hybrid of both,
but more dominant in one area
How Search Engines Work
Read pages they find on the web
Store text in an “index”
When you search, they look for pages
with matching text
Other factors involved in “ranking” those
pages, such as “link popularity”
The Major Search Engines
Northern Light (big, but low traffic)
FAST Search (big, but even less traffic)
AltaVista (big and popular)
Inktomi (HotBot, UK Max)
Google (Netscape Search)
Lycos (links at end)
Submitting To Search Engines
Search engines should find you naturally,
but submitting helps speed the process
and can increase your representation
Look for Add URL link at bottom of home
Submit your home page and a few key
Turnaround from a few days to 2 months
AltaVista, Inktomi, Northern Light will add
the most, usually within a month
Excite, Go (Infoseek) will gather a fair
amount; Lycos gathers little
Index sizes are going up, but the web is
outpacing them…nor is size everything
Here are more actions to help even the
A “deep” submit is directly submitting pages
from “inside” the web site – can help improve
the odds these will get listed.
At Go, you can email hundreds of URLs.
Consider doing this.
At HotBot/Inktomi, you can submit up to 50
pages per day. Possibly worth doing.
At AltaVista, you can submit up to 5 pages per
day. Probably not worth the effort.
Elsewhere, not worth doing a “deep” submit.
Big Site? Split It Up
Expect search engines to max out at
around 500 pages from any particular site
Increase representation by subdividing
large sites logically into subdomains
Search engines will crawl each subsite to
Here’s an example...
Subdomains vs. Subdirectories
I nstead of Do this
I Was Framed
Don't use them. Period.
If you do use them, search engines will
have difficulty crawling your site.
Dynamic delivery systems that use ? symbols in
the URL string prevent search engines from
getting to your pages
Eliminate the ? symbol, and your life will be rosy
Look for workarounds, such as Apache rewrite
or Cold Fusion alternatives
Before you move to a dynamic delivery system,
check out any potential problems.
How Directories Work
Editors find sites, describe them, put them
in a category
Site owners can also submit to be listed
A short description represents the entire
Usually has secondary results from a
crawler-based search engine
The Major Directories
The Open Directory
(Netscape, Lycos, AOL Search, others)
Submitting To Directories
Directories probably won't find you or may list
you badly unless you submit
Find the right category (more in a moment), then
use Add URL link at top or bottom of page
Write down who submitted (and email address),
when submitted, which category submitted to
and other details
You’ll need this info for the inevitable resubmission
attempt – it will save you time.
Submitting To Directories
Take your time and submit to these right
Write 3 descriptions: 15, 20 and 25 words long,
which incorporate your key terms
Search for the most important term you want to
be found for and submit to first category that's
listed which seems appropriate for your site
Be sure to note the contact name and email
address you provided on the submit form
If you don't get in, keep trying
Directories tend not to list subsections of
a web site.
In contrast, they do tend to see
subdomains as independent web sites
deserving their own listings
So, another reason to go with subdomains
Other Tips / Misc
Under represented? Get in touch!
Need Search Engine Links? See: