Hi there,
I've been searching for a site spider+search engine for a while now and I think phpdig is what I need.
I installed it without problems, but I'm getting the following error when spidering ANY site.:
Spidering in progress...
--------------------------------------------------------------------------------
SITE :
http://www.lycos.nl/
Exclude paths :
- @NONE@
No link in temporary table
--------------------------------------------------------------------------------
links found : 0
...Was recently indexed
Optimizing tables...
Indexing complete !
I've been searching for a solution on your forums but unfortunately I didn't find one.
I've seen a lot of topics about this issue discussing a robot.txt file. But I have no idea what it does and think I don't use it.
Does anyone know a solution? you can pm me if you want the url of my phpdig installation.
thx