Go Back > PhpDig Forums > How-to Forum

Thread Tools
Old 12-24-2004, 08:03 AM   #1
Green Mole
Join Date: Dec 2004
Posts: 1
Crawl for illegal content


I need to crawl the sites of customers hosted with us and try to assess if any of them have illegal/pornographic material hosted on their servers.

I am currently trying to use Metis but it is horribly error prone and slooowww.

I want a site to be crawled for certain keywords, if content of any URL matches the keywords then I want a report of all such URLs for a customer to be emailed to my support-desk for verification/action.

Has anyone done this with PHPDig? Is this possible or is PHPDig going to give me reports only when I query it? More like I run a query on the indexed information everyday and mail myself a report.


losttoy is offline   Reply With Quote

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
never ending crawl oehk Troubleshooting 0 12-12-2006 06:34 AM
CRAWL - SPDIER A WHOLE WEBSITE mahesr The Mole Hole 0 11-01-2006 10:43 PM
No results after the crawl GunMuse Script Installation 8 04-12-2005 06:57 PM
crawl my website only hula Troubleshooting 0 08-29-2004 04:37 AM
Crawl mutiple urls Jeff How-to Forum 4 02-03-2004 07:59 PM

All times are GMT -8. The time now is 08:38 PM.

Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.