claudiomet
09-29-2004, 09:40 AM
I have an url text with more than 10.000 URLS. The Cron Jobs via cpanel doesn't work well (always 0 pages indexed and very slow indexing... and via web browser was worst), Now I'm using Putty SSH client and in the console I write:
cd $HOME/public_html/admin
nohup php -f spider.php urls.txt &
And I closed mi session and turned off my PC. (It was the last week)...
Now I check the nohup.out file and the spidering was very fast, a lot of pages indexed and no problems...
cd $HOME/public_html/admin
nohup php -f spider.php urls.txt &
And I closed mi session and turned off my PC. (It was the last week)...
Now I check the nohup.out file and the spidering was very fast, a lot of pages indexed and no problems...