Showing posts with label RankSearch. Show all posts
Showing posts with label RankSearch. Show all posts

Monday, February 2, 2009

RankSearch script: the end of a bad idea

The good folks over at StackOverflow convinced me to take the right decision: although writing a script to automatically retrieve web rank results would probably have been ok for a private program, I feel uncomfortable publishing it.
Making public a forbidden tool would certainly have attracted all kind of bad attention.
So, before it's even written, the RankSearch project is cancelled!
It even started badly as the original name was SearchRank.pl and unnerved the people of SearchRank.com (see, I offer a free link in compensation ;).

So, it looks like I need a new project. Thanks for leaving your ideas in the comment section.

Modern::Perl
In the mean time, I can only recommend reading what chromatic has to say about writing maintainable Perl code over at Modern Perl Books. I promise I'll start writing more Perl soon, to give fodder to his cannon! Hey, if I can get an expert's code review out of it, I consider it a win-win situation.

The Perl Review
I recently subscribed to The Perl Review. The digital edition only set me back $10 (or less than 7 euros) and I have access to all the (17) back issues. Brian D Foy (the publisher) himself edited my StackOverflow question (alright Brian, you were correct, mine wasn't exactly a Perl question but see how I spell Perl correctly with a capital 'P' now!).

French expression of the day:
"de la chair à canon": cannon fodder. Notice how the French and English spell "can+on" differently.

Next posts:
  • Improving on the Perl template
  • How to install and use Google Analytics on your Blogger blog
  • Perl help resources
  • POD

RankSearch script: the end of a bad idea

The good folks over at StackOverflow convinced me to take the right decision: although writing a script to automatically retrieve web rank results would probably have been ok for a private program, I feel uncomfortable publishing it.
Making public a forbidden tool would certainly have attracted all kind of bad attention.
So, before it's even written, the RankSearch project is cancelled!
It even started badly as the original name was SearchRank.pl and unnerved the people of SearchRank.com (see, I offer a free link in compensation ;).

So, it looks like I need a new project. Thanks for leaving your ideas in the comment section.

Modern::Perl
In the mean time, I can only recommend reading what chromatic has to say about writing maintainable Perl code over at Modern Perl Books. I promise I'll start writing more Perl soon, to give fodder to his cannon! Hey, if I can get an expert's code review out of it, I consider it a win-win situation.

The Perl Review
I recently subscribed to The Perl Review. The digital edition only set me back $10 (or less than 7 euros) and I have access to all the (17) back issues. Brian D Foy (the publisher) himself edited my StackOverflow question (alright Brian, you were correct, mine wasn't exactly a Perl question but see how I spell Perl correctly with a capital 'P' now!).

French expression of the day:
"de la chair à canon": cannon fodder. Notice how the French and English spell "can+on" differently.

Next posts:
  • Improving on the Perl template
  • How to install and use Google Analytics on your Blogger blog
  • Perl help resources
  • POD

Monday, January 26, 2009

RankSearch - The end already?

A friend at work informed me that the script that I have in mind for RankSearch might violate Google's Terms of Service.
I did some research and it looks like he's right. From Google's ToS:

Automated queries

Google's Terms of Service do not allow the sending of automated queries of any sort to our system without express permission in advance from Google. Sending automated queries absorbs resources and includes using any software (such as WebPosition Gold™) to send automated queries to Google to determine how a website or webpage ranks in Google search results for various queries.

What's your take on that? If RankSearch is implemented as a sequence of http queries, will it violate the ToS? This could get DamienLearnsPerl banned (I would get my life back!).
On the other hand, the amount of queries would be really low...
I looked for an email address to contact Google to ask the question but couldn't find any. Will look again tomorrow.

French expression of the day
"Chat échaudé craint l'eau froide": once bitten, twice shy (litterally: scalded cat fears cold water)

Next posts:
  • More about CPAN
  • Our first Perl program - Part III: Launch a HTTP request
  • How to install Google Analytics on your Blogger blog
  • Our first Perl program - Part IV: Read results from a HTML page
  • Perl help resources
  • Our first Perl program - Part V: Result analysis
  • POD
  • Our first Perl program - Part VI: Add a GUI interface

RankSearch - The end already?

A friend at work informed me that the script that I have in mind for RankSearch might violate Google's Terms of Service.
I did some research and it looks like he's right. From Google's ToS:

Automated queries

Google's Terms of Service do not allow the sending of automated queries of any sort to our system without express permission in advance from Google. Sending automated queries absorbs resources and includes using any software (such as WebPosition Gold™) to send automated queries to Google to determine how a website or webpage ranks in Google search results for various queries.

What's your take on that? If RankSearch is implemented as a sequence of http queries, will it violate the ToS? This could get DamienLearnsPerl banned (I would get my life back!).
On the other hand, the amount of queries would be really low...
I looked for an email address to contact Google to ask the question but couldn't find any. Will look again tomorrow.

French expression of the day
"Chat échaudé craint l'eau froide": once bitten, twice shy (litterally: scalded cat fears cold water)

Next posts:
  • More about CPAN
  • Our first Perl program - Part III: Launch a HTTP request
  • How to install Google Analytics on your Blogger blog
  • Our first Perl program - Part IV: Read results from a HTML page
  • Perl help resources
  • Our first Perl program - Part V: Result analysis
  • POD
  • Our first Perl program - Part VI: Add a GUI interface

Sunday, January 25, 2009

RankSearch - Part II: parsing the Perl command line

RankSearch: The design
The design of our little script is laid out in the comment header from last post:

1. Get the parameters from the user
  • In a first version, ensure that all parameters are filled
  • Later, we can provide a default value for the search engine (Google)
  • Or even display all the results for a list of supported search engines
2. Launch an http request with the parameters given by the user
  • Spawn a process able to communicate back to our script
  • It will probably be in the form of "http:\\$engine-blabla-search_criteria-moreblabla"
  • Need to investigate different urls for different search engines
3. Parse and display the results transmitted by the http process
  • Search for the target URL
  • Keep track of rank count
  • Launch new http request with updated page number if target not found
  • Display result to user
Today, I'll strike off the first item of the list. Time to get interactive!
In order to ease the handling of user input, I discovered that Perl includes the Getopt::Long module by default. The link on CPAN will show you all possible uses of the module.
One must be careful not to omit the "\" character before the variable name (like I did at first).
We'll only use string (character chain) inputs:
use Getopt::Long;
GetOptions (" engine="s" => \$SearchEngine);
This will store in $SearchEngine the parameter entered from the following perl command line:
perl "$(FULL_CURRENT_PATH)" --engine www.google.com --target damienlearnsperl.blogspot.com --keyword "learn perl"
or
perl "$(FULL_CURRENT_PATH)" -engine www.google.com -target damienlearnsperl.blogspot.com -keyword "learn perl"
or even
perl "$(FULL_CURRENT_PATH)" -e www.google.com -t damienlearnsperl.blogspot.com -k "learn perl"
(provided you only have one entry in Getoptions starting with "e")

Here's the script:
#!/usr/bin/perl -w
# --------------------------------------------------
# File : RankSearch.pl
# Author : DLP
# Date : January 24th 2009
# Object : Looks in a search engine what is the rank
# for a given website and a given keyword
# Input : - Search engine URL, eg. "www.google.com"
# - URL of website to monitor
# - Search expression to investigate
# Bugs : None
# To do : - Launch http request
# - Read html result
# - Analyse result and display website rank
# --------------------------------------------------
use strict;
use Getopt::Long; #Load module

# Global variable
my $PROG_NAME = "RankSearch";
my $VERSION = "v0.0.1";
my $PROG_DATE = "January 24th 2009";

# --------------------------------------------------
# Main program
# --------------------------------------------------
# More global variables
my $SearchEngine = "";
my $TargetURL = "";
my $Keyword = "";

#Parse command line arguments
GetOptions ("engine=s" => \$SearchEngine, #string
"target=s" => \$TargetURL,
"keyword=s" => \$Keyword);

# Check user input
if ($SearchEngine eq "" ||
$TargetURL eq "" || $Keyword eq "")
{
print "
You must enter a valid string for:
--engine = search engine URL
--target = the target of the search
--keyword = the search criteria
";
exit;
}

print "
$TargetURL is ranked nth on the $SearchEngine
search engine for the \"$Keyword\" criteria.";

__END__
Jan 24 2009 (0.0.1): first version of RankSearch
Jan 25 2009 (0.0.2): get params from command line

After getting the parameters, we check to see if anything was entered at all.
If $SearchEngine, $TargetURL or $Keyword are still empty chains (or undefined) then we print an error message and exit the program (the operator for a logical OR is "||" or.. "or"! I don't get the differences yet).

In Notepad++ you can modify the execute command line (via the F6 shortcut) to:
perl "$(FULL_CURRENT_PATH)" -e www.google.com --target damienlearnsperl.blogspot.com --keyword learn perl

This result will appear as:

Notepad++ execution console
Note that the criteria entered by the user was "learn perl" and it was displayed as "learn" by the script. We'll just have to make sure that double quotes (") are used when the string input has a blank space.

French expression of the day:
"Ce que femme veut, Dieu le veut": A woman's will is God's will
As you can see, God and strong-minded women are universal.

Next posts:
  • More about CPAN
  • Our first Perl program - Part III: Launch a HTTP request
  • How to install Google Analytics on your Blogger blog
  • Our first Perl program - Part IV: Read results from a HTML page
  • Perl help resources
  • Our first Perl program - Part V: Result analysis
  • POD
  • Our first Perl program - Part VI: Add a GUI interface

RankSearch - Part II: parsing the Perl command line

RankSearch: The design
The design of our little script is laid out in the comment header from last post:

1. Get the parameters from the user
  • In a first version, ensure that all parameters are filled
  • Later, we can provide a default value for the search engine (Google)
  • Or even display all the results for a list of supported search engines
2. Launch an http request with the parameters given by the user
  • Spawn a process able to communicate back to our script
  • It will probably be in the form of "http:\\$engine-blabla-search_criteria-moreblabla"
  • Need to investigate different urls for different search engines
3. Parse and display the results transmitted by the http process
  • Search for the target URL
  • Keep track of rank count
  • Launch new http request with updated page number if target not found
  • Display result to user
Today, I'll strike off the first item of the list. Time to get interactive!
In order to ease the handling of user input, I discovered that Perl includes the Getopt::Long module by default. The link on CPAN will show you all possible uses of the module.
One must be careful not to omit the "\" character before the variable name (like I did at first).
We'll only use string (character chain) inputs:
use Getopt::Long;
GetOptions (" engine="s" => \$SearchEngine);
This will store in $SearchEngine the parameter entered from the following perl command line:
perl "$(FULL_CURRENT_PATH)" --engine www.google.com --target damienlearnsperl.blogspot.com --keyword "learn perl"
or
perl "$(FULL_CURRENT_PATH)" -engine www.google.com -target damienlearnsperl.blogspot.com -keyword "learn perl"
or even
perl "$(FULL_CURRENT_PATH)" -e www.google.com -t damienlearnsperl.blogspot.com -k "learn perl"
(provided you only have one entry in Getoptions starting with "e")

Here's the script:
#!/usr/bin/perl -w
# --------------------------------------------------
# File   : RankSearch.pl
# Author : DLP
# Date   : January 24th 2009
# Object : Looks in a search engine what is the rank
#          for a given website and a given keyword
# Input  : - Search engine URL, eg. "www.google.com"
#          - URL of website to monitor
#          - Search expression to investigate
# Bugs   : None
# To do  : - Launch http request
#          - Read html result
#          - Analyse result and display website rank
# --------------------------------------------------
use strict;
use Getopt::Long;   #Load module

# Global variable
my $PROG_NAME = "RankSearch";
my $VERSION   = "v0.0.1";
my $PROG_DATE = "January 24th 2009";

# --------------------------------------------------
# Main program
# --------------------------------------------------
# More global variables
my $SearchEngine = "";
my $TargetURL = "";
my $Keyword = "";

#Parse command line arguments
GetOptions ("engine=s"  => \$SearchEngine,  #string
    "target=s"  => \$TargetURL,
    "keyword=s" => \$Keyword);

# Check user input
if ($SearchEngine eq "" ||
$TargetURL eq "" || $Keyword eq "")
{
print "
You must enter a valid string for:
--engine  = search engine URL
--target  = the target of the search
--keyword = the search criteria
";
exit;
}

print "
$TargetURL is ranked nth on the $SearchEngine
search engine for the \"$Keyword\" criteria.";

__END__
Jan 24 2009 (0.0.1): first version of RankSearch
Jan 25 2009 (0.0.2): get params from command line

After getting the parameters, we check to see if anything was entered at all.
If $SearchEngine, $TargetURL or $Keyword are still empty chains (or undefined) then we print an error message and exit the program (the operator for a logical OR is "||" or.. "or"! I don't get the differences yet).

In Notepad++ you can modify the execute command line (via the F6 shortcut) to:
perl "$(FULL_CURRENT_PATH)" -e www.google.com --target damienlearnsperl.blogspot.com --keyword learn perl

This result will appear as:

Notepad++ execution console
Note that the criteria entered by the user was "learn perl" and it was displayed as "learn" by the script. We'll just have to make sure that double quotes (") are used when the string input has a blank space.

French expression of the day:
"Ce que femme veut, Dieu le veut": A woman's will is God's will
As you can see, God and strong-minded women are universal.

Next posts:
  • More about CPAN
  • Our first Perl program - Part III: Launch a HTTP request
  • How to install Google Analytics on your Blogger blog
  • Our first Perl program - Part IV: Read results from a HTML page
  • Perl help resources
  • Our first Perl program - Part V: Result analysis
  • POD
  • Our first Perl program - Part VI: Add a GUI interface

Saturday, January 24, 2009

Retrieve the search rank for your blog automatically

I was reading the latest article over at Pretty Your Blog yesterday and it gave me an idea for a first script. The FileInfo program that I wanted to present is already written and I may come back to it later. The RankSearch project that I have in mind will be written from the ground up so you'll be able to monitor the progress as I crawl towards completion. I hope you'll pitch in as well!

RankSearch: The need
Kelly wrote in Pretty Your Blog:
I ran another (unofficial) experiment; I did a google search for frugal blog. And the result? Almost Frugal was on the 13th page of search results.
It is very likely that Kelly clicked on the next link at the bottom of each Google page until she found her blog's reference (by the way, if you are interested in improving your blog, you may want to visit Pretty Your Blog often :).
What if we could write a Perl script that automatically returns the Google rank based for any URL on a given keyword? That's the idea behind RankSearch.pl

RankSearch: The specifications
The script will have to be as generic as possible and would take 3 input parameters:
- Search engine URL (eg: "www.google.com" or "www.yahoo.com")
- URL of the blog you want to check the ranking for (eg: "almostfrugal.com")
- The search criteria you are investigating (eg: "frugal blog")

The output result will be a simple message of the type:
almostfrugal.com is ranked 7th on the google search engine for the "frugal blog" criteria.

It will run on Windows and Linux environments
Do you see anything else to add?

And here is the template for it:
#!/usr/bin/perl -w
# ----------------------------------------
# File : RankSearch.pl
# Author : DLP
# Date : January 24th 2009
# Object : Looks in a search engine what is the rank
# for a given website and a given keyword
# Input : - Search engine URL, eg. "www.google.com"
# - URL of website to monitor
# - Search expression to investigate
# Bugs : None
# To do : - Read input and store in script variable
# - Launch http request
# - Read html result
# - Analyse result and display website rank
# ----------------------------------------
use strict;

my $PROG_NAME = "RankSearch";
my $VERSION = "v0.0.1";
my $PROG_DATE = "January 24th 2009";

print "$PROG_NAME $VERSION from $PROG_DATE.\n";

__END__
Jan 24 2009: first version of RankSearch

French word of the day:
"économe": frugal

New posts:
  • Our first Perl program - Part II: parsing the Perl command line
  • More about CPAN
  • Our first Perl program - Part III: Launch a HTTP request
  • How to install Google Analytics on your Blogger blog
  • Our first Perl program - Part IV: Read results from a HTML page
  • Perl help resources
  • Our first Perl program - Part V: Result analysis
  • Our first Perl program - Part VI: Add a GUI interface

Retrieve the search rank for your blog automatically

I was reading the latest article over at Pretty Your Blog yesterday and it gave me an idea for a first script. The FileInfo program that I wanted to present is already written and I may come back to it later. The RankSearch project that I have in mind will be written from the ground up so you'll be able to monitor the progress as I crawl towards completion. I hope you'll pitch in as well!

RankSearch: The need
Kelly wrote in Pretty Your Blog:
I ran another (unofficial) experiment; I did a google search for frugal blog. And the result? Almost Frugal was on the 13th page of search results.
It is very likely that Kelly clicked on the next link at the bottom of each Google page until she found her blog's reference (by the way, if you are interested in improving your blog, you may want to visit Pretty Your Blog often :).
What if we could write a Perl script that automatically returns the Google rank based for any URL on a given keyword? That's the idea behind RankSearch.pl

RankSearch: The specifications
The script will have to be as generic as possible and would take 3 input parameters:
- Search engine URL (eg: "www.google.com" or "www.yahoo.com")
- URL of the blog you want to check the ranking for (eg: "almostfrugal.com")
- The search criteria you are investigating (eg: "frugal blog")

The output result will be a simple message of the type:
almostfrugal.com is ranked 7th on the google search engine for the "frugal blog" criteria.

It will run on Windows and Linux environments
Do you see anything else to add?

And here is the template for it:
#!/usr/bin/perl -w
# ----------------------------------------
# File : RankSearch.pl
# Author : DLP
# Date : January 24th 2009
# Object : Looks in a search engine what is the rank
# for a given website and a given keyword
# Input : - Search engine URL, eg. "www.google.com"
# - URL of website to monitor
# - Search expression to investigate
# Bugs : None
# To do : - Read input and store in script variable
# - Launch http request
# - Read html result
# - Analyse result and display website rank
# ----------------------------------------
use strict;

my $PROG_NAME = "RankSearch";
my $VERSION = "v0.0.1";
my $PROG_DATE = "January 24th 2009";

print "$PROG_NAME $VERSION from $PROG_DATE.\n";

__END__
Jan 24 2009: first version of RankSearch

French word of the day:
"économe": frugal

New posts:
  • Our first Perl program - Part II: parsing the Perl command line
  • More about CPAN
  • Our first Perl program - Part III: Launch a HTTP request
  • How to install Google Analytics on your Blogger blog
  • Our first Perl program - Part IV: Read results from a HTML page
  • Perl help resources
  • Our first Perl program - Part V: Result analysis
  • Our first Perl program - Part VI: Add a GUI interface