Glenn over at SSEO asked for a script to do mass whois lookups.
Use this function:
[code lang="php"]
function getwhois($domain, $tld)
{
(more...)
For all those wanting to check the rollout of the BigDaddy update, here's a list of 116 Google Datacenters:
64.233.161.98 64.233.161.99 64.233.161.104 64.233.161.105 64.233.161.106 64.233.161.107 64.233.161.147 64.233.163.99 64.233.163.104 64.233.167.99 64.233.167.104 64.233.167.106 64.233.167.107 64.233.167.147 64.233.171.99 64.233.171.103 64.233.171.104 64.233.171.105 64.233.171.106 64.233.171.107 64.233.171.147 64.233.179.99 64.233.179.104 64.233.179.106 64.233.179.107
(more...)
Spent a bit of time yesterday trying to get my code examples highlighted using the GeSHi class. However using the WordPress plugin code, I found that though the code was coming up fine it was removing all paragraphs and line breaks from any surrounding text. So I then wrote my own plugin still calling the GeSHi class file, but where it displays both code and text properly.
It is
(more...)
For crawling in PHP I have always used the fantastic cURL.
My curl single-threaded function:
[code lang="php"]
function singlethread_crawl($url)
{
$agent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)";
$ch = curl_init();
curl_setopt($ch, CURLOPT_NOSIGNAL, 1);
curl_setopt($ch,
(more...)
On ThreadWatch yesterday there was a thread about rank checkers, and I couldn't believe that some SEOs don't use them. We use our own heavy duty mega serp scraper to fully analyse any industry we are working in. Anyway, Graywolf mentioned how he would love a Google RSS or XML feed - I having been waiting for this for a long time, as their SERPs are so
(more...)
So, for all those of you that use GYM to scrape for URLs to ::ahem:: study, and are forced to use their normal messy SERPs because their API only allows 10 results per query (or even those that use /ie or /palm).
I just noticed that on their Blog Search, Google allow you to grab an RSS feed of the results.
So now
(more...)
