How Often Should you Resubmit your Pages, and When?

In Part I of this article, we discussed overall submission and resubmission strategies. But now let's look at some engine-specific guidelines that will keep you out of trouble with the individual engines.

Engine-Specific Guidelines and Tips

AltaVista
Submission URL: http://add-url.altavista.com/cgi-bin/newurl?

Submission Limit per Day: 5 pages or more with their new procedure.
How long does it take to index a page? 2 days to 2 weeks

With AltaVista, rely on the use of links to get the pages in your site indexed. Unless you've made significant changes to your pages, don't resubmit them. Instead, let AV's spider find those changes during its next spider run.

Try submitting your pages using "www" and not using it in the URLs.

AV won't index a site submitted as an IP address. A rep from AltaVista explains, "The spider is doing reverse DNS loopups when an IP site is submitted. You must be registered with Internic and have DNS running to be indexed by the spider."

Does AltaVista have a limit to the number of pages from a particular domain that can be in its index? AV answers, "In reality, the whole site could be indexed with no restriction on depth, except that we have size limitations and stop the spider once the index of the site has a certain size." In other words, they aren't saying. (Is this surprising?)

To check to see if your pages are in the index, enter your URL in the search box like this:

url:yourdomain.com (reveals every page under the root domain)

url:members.prodigy.com/yourwebsite/ (narrows the search to pages within your site, if you're under someone else's domain)

url:members.aol.com/yourwebsite/yourwebpage.htm (finds a specific page at your site)

At times, Excite will only accept the main page of your site through their submission process. Other times, it will accept interior or doorway page submissions. There's no doubt about the fact that Excite definitely prefers main pages over interior/doorway pages, however.

Therefore, make use of visible links to all of your important pages off the main page of your site, and give the engine something to index.

With Excite, there's an unwritten "rule" that generally only 25 pages from a domain can be in the index. Of course, like most search engine rules, this one is broken all the time. However, don't be surprised if your site is limited to having 25 URLs in Excite's index.

Because of this rule, pick and choose the URLs yourself by making use of robots.txt files.

Though PositionPro recommends submitting only 10 URLs a day to Excite, Jim Reinhold, Engineering Manager of Excite, says "Our limit is 25 URLs per domain per day."

To check to see if your pages are in the index, enter the URL without the http:// prefix, like this:

www.myveryownwebsite.com/webresources.html

You can also search for the root domain to find many pages from the same site at once. Enter your root URL, such as:

www.myveryownwebsite.com

Google
Submission URL: http://www.google.com/addurl.html
Submission Limit per Day: 10
How long does it take to index a page? 1-2 months

With Google's supplying Yahoo!'s supplemental index, submitting to Google has become even more important.

Submit all of your important pages to Google. However, with Google's unique way of indexing, your links should be picked up by this engine with no problems. But, why take a chance - submit your important pages.

You can submit up to 10 URLs to Google a day and be safe. "Google doesn't have a number per se," says Stob. "When I first approached them, they asked me to keep it low so we started at 2 per day. They were in the middle of building a new index. Since then we have brought the count to 10 per day."

Stob continues, "Could I submit 50 a day? Probably. Would I consider doing it on a large site? Probably."

With Google, be sure to have other sites link to your Google pages, or those pages may not be picked up in the index.

To check to see if your pages are in the index, search for your domain name like this:

www.yourdomainname.com

Then, click on "Find web pages that contain the term." From there, you can choose "Search within results" to narrow down the search to particular pages.

HotBot/Inktomi Engines
Submission URL: http://www.canada.com/search/web/addurl.asp
Submission Limit per Day: 50
How long does it take to index a page? 2 weeks

Note: Your best bet for submitting to the Inktomi engines is to go through their pay inclusion program. For more information, visit: http://www.positiontech.com. If you don't want to pay to submit, your next best bet is to submit to Inktomi through Canada.com.

As you know, Inktomi supplies results to HotBot, Canada.com, iWon, GoTo, MSN Search, and many more search services. To get your site into the Inktomi engine faster, use their pay inclusion program, or try submitting to Canada.com or Anzwers.com instead of through HotBot.

Be sure to submit all important pages of your site to HotBot. Try submitting the same page using different variations of the page. For example:

http://www.HereIAm.com
http://HereIAm.com
http://www.HereIAm.com/index.html

You may find that your sites using the "www" are ranked slightly higher than those without the "www."

When you submit a site to Inktomi, have you noticed that it may be appear in the index for a month or two, then be dropped? Jim Stob explains what might be happening. "Inktomi will analyze a page for inclusion in their index. It may show up for a month or two and then be dropped. It is dropped because Inktomi has not seen it come up in a search and has determined that it doesn't have any value."

So, if the page doesn't get accessed through a search engine, it will be dropped from the Inktomi index?

Stob answers, "Yes for Inktomi, no for AltaVista. Inktomi has two indexes as you probably know.

Have you also noticed that your rankings fluctuate in HotBot or other engines? Stob explains, "Rankings will alter by the hour with some engines. If you lose the spot for more than two weeks, you should be concerned."

To check to see if your pages are in the index, use their Check URL form:

http://www.hotbot.com/help/checkurl.asp

Or, visit HotBot's main page and choose the "Advanced Search" button, which is on the left-hand side of the page. The SuperSearch page will appear.

Find the Location/Domain box toward the middle of the page, and enter your domain without the "http://www" prefix in the domain box. You can choose other variables, such as the page depth and the number of search results you'll be shown. Click on Search.

Lycos/FAST
Submission URL for Lycos: https://www.lycos.com/addasite.html
Submission URL for FAST: http://alltheweb.com/addurl.html
Submission Limit per Day: 50
How long does it take to index a page? 20-40 days (in FAST)

With both Lycos and FAST, you can safety submit all of your important pages, even though both engines are "deep crawlers" and should find the links on their own.

Try omitting the "www" when submitting your pages to Lycos.

To check to see if your pages are in the index, search for your root domain, such as:

thisismysite.com

Some of your pages will usually be listed. If not, search again, but leave off any suffixes, such as ".com" or ".edu" like this:

thisismysite

Northern Light
Submission URL: http://www.northernlight.com/docs/regurl_help.html
Submission Limit per Day: 50
How long does it take to index a page? 2-4 weeks

Though Northern Light prefers that you submit just one page to the engine, you can submit your interior and doorway pages, and you should.

However, as with all engines these days, make good use of hidden links on your main page to other important pages of your site.

To check to see if your pages are in the index, search for your domain like:

yourwebsite.com

Then, choose More Results from this Site.

In conclusion

When submitting to the search engines, play it safe. Don't submit over and over again, and look at each engine's submission guidelines carefully. If you're going to err, err on the side of being conservative.