Log in

View Full Version : Reindexing site won't index certain page


gman
08-06-2004, 10:03 AM
I successfully indexed my site. Later, I came back to reindex the new files I added. That did not work. It acts like it doesn't see the new files.

It keeps saying "Duplicate of existing document" with a red x on the left.

I think that it skips the desired page because it is considering all my pages duplicates, so it won't link to the new page.

How do I index a single page? I have read through the forums with no success.

If I type in the URI of the new page, it will not index it from the web interface. I have tried many different combinations of search depths and link depths.

Any ideas??

Charter
08-06-2004, 10:35 AM
Hi. Are there links to the new files? What is LIMIT_DAYS set to in the config file? Perhaps this (http://www.phpdig.net/showthread.php?threadid=1139) thread might help.

gman
08-06-2004, 11:50 AM
When I couldn't index my newly created page, I tried deleting the branch in hopes that I could just reindex the branch and the new page would be picked up, but that didn't work.

LIMIT_DAYS is set to 0.

I have a page that links to the newly created page.
I have tried indexing them both with no success.

I'm using the version 1.8.3.

I tried what was posted in the thread. The links were displayed correctly on the test page I made.

I tried to index that test page's URI, but with no success.

I tried search depth 1, link depth 7. Because there were seven links on the test page.

I also tried search depth 7, link depth 0.

Still don't know what is wrong.

Charter
08-06-2004, 12:56 PM
Try setting error_reporting(E_ALL); at the top of the spider.php file and see if it throws any errors.

gman
08-06-2004, 01:05 PM
I think I found the problem.
I had a robots.txt file in the root of my site that disallows spiders.

It's indexing as we speak.

Thank you for the help.

It works now.

You are the "Head Mole"! :D