Ok right now I have one text folder called text_content, it has one file in it called keepalive.txt
Now I go to admin and run the domain name by itself on / or sub dir in path. I run it I select depth 5.
I get this
SITE :
http://www.mansfield-tx.gov/
Exclude paths :
- @NONE@
1:
http://www.mansfield-tx.gov/
(time : 00:00:00)
No link in temporary table
--------------------------------------------------------------------------------
links found : 1
http://www.mansfield-tx.gov/
Optimizing tables...
Indexing complete !
Now I look in the text_content folder and I have 1 file called 387.txt and it is empty. Back to admin interface panel I show now 1 host and 1 page.
Now if I redo this and I add a / and a sub dir to domain name ir works, adds files, keywords and such. It just will not run the domain like this
http://www.domain.com
works fine with
http://www.domain.com/subdir or the ip
http://ip