Ask Mr. Internet - August, 2001
 
Dear Mr. Internet:
I use information from various Web sources whose content changes frequently (like my local newspaper) to include in my own newsletter to keep it fresh, local, and up to date.  Is there some way to automate the search and retrieval process so I don't have to spend so much time each day doing this?
 
Mark Bensen 
PA,  ABR, CRS, GRI, e-PRO
KELLER WILLIAMS
Platinum Realty, Inc.
Naples, Florida
http://www.MarkBenson.com
  
Dear Mark:

The answer to this question consists of two parts: a) reliably finding the information, and b) automating the process.  The first part of finding up-to-date information on a constantly changing site is not as straightforward as you might imagine.

Take for example your primary local newspaper Web site, NaplesNews.com,  which has its own site search field.  There is no guarantee that the site content is indexed (i.e. stored in a searchable form) frequently enough to be accurate and useful.  So if you are looking for the most recent stories using keywords like "little league", "community events", "property taxes", etc. you are not likely to find them.  Or worse, they will be buried under an avalanche of archived articles that are useless to you.

There is a very clever and free way however, to create your own indexed version of just about any site (up to 1500 pages) at a push of the button...

Create Your Own Site-Specific Search Engine

PicoSearch.com is a site-specific search engine service, that provides Web site owners the ability to create a search capability on their own site (my Mr. Internet site uses it for this purpose).  It will index sites up to 1500 pages at no cost, as often as you want.

Her are the three steps to create your own free, site-specific search engine:

  1. Create Your Account - go to PicoSearch and create a new free account.  When it asks for the URL of the site to be indexed, put the primary Web address of the site you want indexed (in this case http://www.naplesnews.com).  It will then ask you to set a "spidering" restriction as it indexes the site (i.e. how it decides what pages to index).  I suggest choosing the "Domain Level Restriction" as it will cause PicoSearch to index the widest possible range of pages.  Next, and this is important —choose the "Build Off-Line" option, otherwise it could take hours for it to index the site.
     
  2. Index The Site - Once you've made your choices, click the "Build Your Site!" button.  It may take up to 24 hours for them to initially set up your account. As soon as you start the building process, you will be taken to a "Thank You" page that gives you your account ID --write it down!  Now login as a registered user and choose the "Free" plan (you will be sent an e-mail that has a bunch of HTML code in it to install on your site —just ignore it).
     
  3. Create And Save Your Search Page URL - go to your Web browser and enter this URL (without the quotes): "http://www.picosearch.com/cgi-bin/ts.pl?index=[account_ID]" where you replace "[account_ID]" with the actual number of your account.  When you go to this address, it will be the actual search page for the site PicoSearch is set up to index for you.  Save this URL as a Windows "Shortcut" on your desktop and name it something meaningful like "NaplesNews Search".

Now simply double-click on your "NaplesNews Search" shortcut anytime you want to search the NaplesNews.com site for specific keywords and phrases.  One thing to remember though is that you MUST re-index the site before every search if you want to be assured of the most recent site information.  You can cause a re-indexing to happen by logging in to PicoSearch and clicking on the "Index Now" link.

This process can be duplicated for as many sites as you want, however you most likely will need to use slightly different versions of your name and e-mail address to create additional free accounts.

This takes care of step "a)" of the process, now on to the other part...

Automating The Search And Retrieval Process

Ideally, you want to automate the process of re-indexing the information site(s), searching for the most recent information based on specific sets of keywords and phrases, and assembling / forwarding the results to you every day so you can choose which to include in your e-mail marketing program.

It is helpful to realize here that "automation" does not necessarily mean "automatic".  These are a very specific, measurable set of tasks that could easily be assigned to an assistant (real or virtual) or even a virtual consultant that you find on eLance.com.  (see the May 2001 edition of "Ask Mr. Internet!" regarding the use of eLance to finding and using virtual assistants)

While the above procedure certainly answers your question, something else is going on here that is worth mentioning.  This above is also a great example of how we can use an Internet tool designed for one purpose (i.e. PicoSearch) and repurpose it for the benefit of something else entirely.

The exciting part is that the Net is just brimming with opportunities like this, and the fun part begins with finding them!

Article Resources:

PicoSearch - where you set up any number of free site-specific search engines to help you do research or help your site visitors find what they are looking for. eLance.com - where you can find any kind of professional talent you could ever need, all bidding for the right to help you get your job done!
Mr. Internet's Tip O' The Month

Great news for those agents currently using AOL, but would like to switch to another ISP.  In the past it is was very difficult to switch from an AOL e-mail account since AOL will not automatically forward e-mail to another address.  This means you had to manually do it with each message while letting senders know your new address --a tedious process that could go on for months.

Not anymore --thanks to a service called Re-Route.com.  This is a very affordable service (as of this writing about $25 for three months use) that will automatically forward your AOL mail to your new address.  It will also send an e-mail "change of address" notice to each sender so they can make the adjustment in their e-mail address book.  And, you only need to keep your old AOL account open long enough to be assured you not receiving any business e-mail at the old address.  

What was once a lot of work and hassle is now completely automatic.  With Re-Route.com you can finally said "Adios" to AOL!
(NOTE: this service will forward e-mail from nearly any service, other than Web based, —not just AOL)



Mr. Internet is the alter-ego of Michael J. Russer, an internationally recognized Internet speaker, trainer, author, and strategic consultant to the real estate industry. He's dedicated to helping real estate professionals leverage their people skills into profit on the Internet. You'll see his column on REALTOR® Magazine Online every month and in the magazine quarterly. Send your Internet questions to help@askmrinternet.com
or you can visit his Web site at http://www.russer.com 

This article is reprinted from the August, 2001 issue of  REALTOR® Magazine by permission of the NATIONAL ASSOCIATION OF REALTORS®.  Copyright 2000, 2001.  All rights reserved other than mentioned above.  Mr. Internet® is a registered trademark and Ask Mr. Internet!™ is a trademark of RUSSER Communications.