PhpDig.net - Fixing spider.php, protecting from locking site after timeout or users stop

PhpDig.net (http://www.phpdig.net/forum/index.php)

- Mod Submissions (http://www.phpdig.net/forum/forumdisplay.php?f=24)

- - Fixing spider.php, protecting from locking site after timeout or users stop (http://www.phpdig.net/forum/showthread.php?t=653)

Fixing spider.php, protecting from locking site after timeout or users stop

I found, that if your PHP has time out (for example 30 secs) and spider have no time to complete indexing, your site will be locked and you'll have to go back to admin and unlock it. It's not good, so next code fix it.

Add this code into spider.php file at the begining (from 12 line, after comments, for example):

PHP Code:


		
			
function Unlock()

{

  global $site_id, $id_connect, $run_mode, $mode;



  // unlock site if not normal terminate

if(connection_status())

{

  mysql_query('UPDATE '.PHPDIG_DB_PREFIX.'sites SET locked=0 WHERE site_id='.$site_id,$id_connect);

  print "<br />Site was unlocked<br />";



  if ($run_mode == 'http')

  { ?>

<hr />

<a href="index.php" >[<?php phpdigPrnMsg('back') ?>]</a> <?php phpdigPrnMsg('to_admin') ?>.

<?php

    if (isset($mode) && isset($site_id) && $mode == 'small') {

      print '<br /><a href="update_frame.php?site_id='.$site_id.'" >['.phpdigMsg('back').']</a> '.phpdigMsg('to_update').'.';

    }

  } else {

      print $br;

  }

}



return;

}



register_shutdown_function("Unlock");

Hope, it will be in next versions and will help you!!! ;)

little changes for correct work from command line

PHP Code:


		
			
function Unlock()

{

  global $site_id, $id_connect, $run_mode, $mode, $br;



  // unlock site if not normal terminate

if(connection_status())

{

  mysql_query('UPDATE '.PHPDIG_DB_PREFIX.'sites SET locked=0 WHERE site_id='.$site_id,$id_connect);

  print $br."Site was unlocked".$br;



  if ($run_mode == 'http')

  { ?>

<hr />

<a href="index.php" >[<?php phpdigPrnMsg('back') ?>]</a> <?php phpdigPrnMsg('to_admin') ?>.

<?php

    if (isset($mode) && isset($site_id) && $mode == 'small') {

      print '<br /><a href="update_frame.php?site_id='.$site_id.'" >['.phpdigMsg('back').']</a> '.phpdigMsg('to_update').'.';

    }

  } else {

      print $br;

  }

}



return;

}



register_shutdown_function("Unlock");

making auto-refresh for indexing with timeout

If we change index.php file, method of posting from POST to GET

PHP Code:


		
			
<form class="grey" action="spider.php" method="get"

And changing function Unlock (see above) to that code:

PHP Code:


		
			
function Unlock()

{

  global $site_id, $id_connect, $run_mode, $mode, $br;



  // unlock site if not normal terminate

if(connection_status())

{

  // clean the tempspider table

  $query = "DELETE FROM ".PHPDIG_DB_PREFIX."tempspider WHERE site_id=$site_id AND (error = 1 OR indexed = 1)";

  mysql_query($query,$id_connect);

  mysql_query('UPDATE '.PHPDIG_DB_PREFIX.'sites SET locked=0 WHERE site_id='.$site_id,$id_connect);

  print $br."Site was unlocked".$br;



  if ($run_mode == 'http')

  { ?>

<hr />

<a href="index.php" >[<?php phpdigPrnMsg('back') ?>]</a> <?php phpdigPrnMsg('to_admin') ?>.

<?php

    if (isset($mode) && isset($site_id) && $mode == 'small') {

      print '<br /><a href="update_frame.php?site_id='.$site_id.'" >['.phpdigMsg('back').']</a> '.phpdigMsg('to_update').'.';

    }



    print <<< HTML

<script>

<!--

window.location.reload();

//-->

</script>

HTML;

  } else {

      print $br;

  }



}



return;

}

Have a nice day!

Was this implemented for version 1.8?

I have problems with indexing one of my client's website. I have no problems spidering on my own server. Have installed this script for my installation of PHPdig. Will see if it solves the porblem.

Thanks.