Ok found 1 problem the client had a index.html file in his dir that was nothing more than a redirect file very short. So for a test I uploaded a index.html file with alot of text in it. I re ran the test and it picked up the keywords for that index.html page and stored them., It only got the 1 page though it did not attempt to get the other filles or sub dir files.
|