Spidering in progress...
-----------------------------
SITE :
http://example.com/
Exclude paths :
- directory\.bml
Meta Robots = NoIndex, or already indexed : No content indexed
1:
http://example.com/(time : 00:00:51)
No link in temporary table
links found : 0
...Was recently indexed
Optimizing tables...
Indexing complete !
how can I force phpdig to ignore the Robots file and index this site anyways?
