PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > Troubleshooting

Reply
 
Thread Tools
Old 11-03-2005, 04:38 PM   #1
noel
Orange Mole
 
Join Date: Aug 2005
Posts: 44
Multiple crawler code error

Hi CHARTER,

I tried this code found on the forum :

http://www.phpdig.net/forum/showthread.php?t=662

I didn't change the fix because I don't know where are the lines to change.

I tried your code without the fix, and I have this error after the shell line command "php -f wrapper.php http://asite.com

[root@wpc1060 admin]# php -f wrapper.php http://www.asite.com
PHP Parse error: parse error, unexpected T_STRING in /home/httpd/vhosts/mysite.com/httpdocs/google/admin/wrapper.php on line 19
[root@wpc1060 admin]#


What can I do ?
My version is 1.8.8 RC1

Thank You

Noël
noel is offline   Reply With Quote
Old 11-05-2005, 01:33 PM   #2
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
The code in this thread is xibalba's code. Multiple spiders have not been written or tested by me, but you might get some additional ideas in this thread.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 11-05-2005, 02:15 PM   #3
noel
Orange Mole
 
Join Date: Aug 2005
Posts: 44
Hi CHARTER,

If I want to use xibalba's code, what do you think I have to change, in order it, to work with the new version of PHPDIG 1.8.8.RC1 ?

If I use your code :

array_unshift($list_sites,$site_datas);

Do you think it will also works in shell command ?

Thank You

Noël
noel is offline   Reply With Quote
Old 11-06-2005, 11:10 AM   #4
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
As I have not written or tested multiple spider code, I'm not sure of the changes needed or whether things would work from shell. The only things I can suggest are the ideas already mentioned. One thing you could try is running the following from shell and see what happens. Make a database backup beforehand just in case.
Code:
nohup php -f spider.php htp://www.domain1.com > log1.txt &
nohup php -f spider.php htp://www.domain2.com > log2.txt &
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Crawler speed improvement (although affects limit) marco Mod Submissions 0 03-23-2007 07:14 AM
Split the search engine and the web crawler nobrin How-to Forum 1 08-15-2005 10:53 AM
Sleep in the code davids211082 How-to Forum 2 05-18-2005 11:22 PM
Submit crawler Mindrot Mod Requests 5 08-26-2004 07:36 PM
Code Requests Charter Feedback & News 0 02-28-2004 11:45 PM


All times are GMT -8. The time now is 12:28 PM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.