PDA

View Full Version : Indexing always stops with locked site


jamie
01-05-2004, 12:37 PM
Hello everybody,

I've installed PhpDig onto a server which is not mine.

Search runs ok, but indexing seems a bit difficult: whenever I start indexing my page with level 1, the browser stops and does nothing more (after not even a minute). When I go back to the admin interface, I see that the site is locked. I can redo the process after unlocking the site, but everytime it only takes a couple of links more - but never all.

As you can see here (http://iifs.de/phpdig/phpinfo.php,) the safe_mode is Off, but I don't know how I could change that (as it's not my server). I've even set the PHPDIG_DEFAULT_INDEX on false, but nothing changed.

Can someone please help me? Thanks.

MonoNexo
01-05-2004, 05:03 PM
Try letting it run for an extended period of time, more than a minute. Sometimes, if it gets a large page, it will slow down considerably, or do chunks of the site. I have a fairly large (170+) page site, and for a few pages, it will decrease in speed incredibly, but if I leave it be, it will continue to work.

With the site being locked, that means that it is still working at spidering the site. Just give it time, it'll get through. Just a bit slow perhaps :).

jamie
01-06-2004, 08:31 AM
Hello MonoNexo,
I've let it run the whole day, but it hasn't spidered more than 7 pages and the site was locked after the first minute.

Any more ideas?

Charter
01-06-2004, 10:29 AM
Hi. A site is locked during the spidering process. This does not mean anything bad. It just means that PhpDig is working on that site. If spidering is prematurely interupted, a site can remain locked. In that case, just go to the admin panel, select the site, click the update button, and then click the unlock link.

If you are spidering a site that has the same links over and over, then PhpDig will crawl and determine that such content is duplicated. You might try utilizing the PhpDig exclude/include comments to avoid this. Also, MonoNexo's post is good. Try indexing the site in parts rather than selecting a high search depth.

One other thing, as you are using the browser interface for PhpDig, once you start an index, do not use that browser window for other things until the index is completed. You might want to try indexing by shell instead, as this bypasses the browser interface.