I am trying to run phpdig soley on my computer to test everything out. I am able to connect to mysql database and index some
www.blablabla.com page but when and i can index
http://localhost but when i want to just index the files inside 1 localhost folder ie.
http://localhost/folder it won't index it. Is there a way i have to set robots.txt file or somthing?