A question many people are asking is how does Searchforecast collect all the websites in the AdSense Directory. Our proprietary crawler has been running for many years now and each day collects websites that are running AdSense. Each day, we migrate the data and de-duplicate the database, so the websites and keywords are always fresh.

Our engineers manually oversee the de-duplication ensuring the database integrity. Here’s a daily report we push around internally…

Web Crawler Report
1: De duplicate 49,000 URLs from new crawled data
2: Updated the existing URLs, Title, Description & Keywords.
3: Removed duplicates URLs, Publish /Tags URLs
4: Monitoring the Crawler Load & CPU

Existing Database Total: 3,340,234
Newly URLs: 47,191
Total After Merging URLs: 3,387,425

Google Adwords Advertisers and PPC specialists have been using the SearchForecast AdSense Directory for targeting across the Google Content Network. With over 350,000 AdSense Publishers indexed by 1.2 million keywords, the AdSense Directory provides a targeting placement tool that allows websites to be identified and added to the “Managed Placements” section of the Content Network within Google Adwords.

The product originated because of the understanding I gained through Adwords professionals like David Rodnitzky from PPC Associates and Shelley Ellis who are helping clients target sites across the once blind Content Network which Google generates $7 billion a year from!

It’s worthwhile noting that “managed placement” will work only if Google gives you managed placement status. In other words, if you are running AdSense on XYZ123.com and Google has not yet identified that site as being qualified as a managed placement, the advertiser can’t target that directly via Google Adwords – but that’s not to say they can’t contact that website and deal directly with the publisher.

Growing up in Australia, I understand the power of China – our Asia Pacific sleeping dragon. Slow to uptake the Internet, the dragon has breathed fire into Building 43 at Googleplex and now with the Google.cn closure almost certain, we have seen the traffic from China to our AdSense Directory increase over the past month.

 The good folks at Pontiflex are hard at work educating the publishing community on how to increase revenue using the cost per lead business model. That’s where advertisers pay for a completed lead resulting from a form or part thereof being completed.

Together with SearchForecast, we are working with them to identify publishers from our AdSense Directory which are best suited to increasing revenue beyond text advertising from Google.

Watch the video on Cost Per Lead Now!

Qualified leads are very important for advertisers who can pay handsomely for them. For those AdSense Publishers who are looking for extra revenue from their website without disturbing their current monthly check from Google, Pontiflex have over 100 brand advertisers who will pay you every time a visitor completes an advertiser form.

What I like about Pontiflex is their commitment to transparency and  sharing with their publishers the opportunity to select and choose the advertisers!

Be sure to ask for Leora Blumberg when you speak with Pontiflex or email her at leorab@pontiflex.com

We recently enabled Retargeter.com as an advertiser on the AdSense Publisher Directory at SearchForecast.org.

ReTargeter helps AdSense publishers by bringing traffic back to their sites. I know many marketing VPs here in Silicon Valley who use retargeting technology to build brand credibility – ads show up on premium sites all over the Internet.

Here’s how it works….. Mary visits an AdSense publisher, leaves the site and continues to surf the web. One day later she is checking her mail on Yahoo! Mail or surfing Facebook and she sees an ad for your website. Instantly she is not only impressed with the fact that the site is advertising on major premium sites (“wow they must be legitimate”) but she also clicks on the ad, returns to the site. The theory is returning visitors will convert to customers more readily.

Each day thousands of visitors to our AdSense Publisher Directory look at the keywords which the publishers have used to describe their website.

Finding “like sites” by keywords which publishers have “tagged” their site with yields thousands of new keywords which provide insight into the intent of the publisher and therefore a contextual relevance that advertiser networks want to tap into.

Here’s a sample of Publisher Websites Featuring Relevant Keywords for Advertisers:
find, inexpensive, budget, affordable, america, fastest, cheapest, site, canada, usa, best, is, how, why, where, when, what, who, american, printer, cables, specialty, paper, ink, refill, kits, ink, refills, printer, inkjet, cartridges

With over 1.2 million keywords in our AdSense Directory, adnetworks, affiliates, publishers and technology providers can find the precise keyword that the publisher currently use on their website.

As the picture illustrates, if each publisher was a ping pong ball, the interconnectivity and overlap of keywords in the publisher universe is constantly moving.  Our crawler updates 50,000 websites per day to ensure greater targeting of relevant publishers. This will ensure advertisers have greater click through rates and better return on investment.

Last month, we opened up our AdSense Publishers Directory to a select group of adnetworks and technology providers, including OpenX and Payoneer, who have the vision of helping us connect the community of AdSense Publishers.

I visited the New York Public Library recently. The names of  contributors who built and maintain this safehouse of literature where the public can openly access the publishers  reminded me of our vision for the  SearchForecast Publisher Directory which now holds 5+ million publisher website URLs.

In the same way as FlickR tagged photo’s – providing better descriptions from owners – we have done so for Publishing websites running Google AdSense. Each Publisher website in our Publisher Directory is “tagged” by keyword. There are 1.2 million keywords.

As we help AdNetworks find publishers via partnership lead generation program and connect publishers via our CRS Conference, we are striving to build and preserve SearchForecast.org to provide future generations with a digital roadmap of publishers and how they described and published their websites.

Localyte is a start up based here in Silicon Valley which I am an advisor to.

Localyte.com has built a growing army of 42,000 local experts in 10,000 destinations across 170 countries. These locals produce geotaged reviews about points of interests (like Restaurants, Museums, Shopping Places, Cafe, Parks, etc.). They have developed a “secret sauce” for growing a worldwide community with very little resources and they also cracked the code for engaging the community to generate reviews about local places in high volume and for free. At current growth rate they expect to have more than 500,000 reviews in the next 12 months. Here’s an example of what they are producing: http://www.localyte.com/places-to-visit/Sri+Lanka?

Watch the video on YouTube http://www.youtube.com/watch?v=D4_c3E2aTxo for more information and an example of how Localyte’s “real time” technologies are helping travelers today enrichen their holiday experiences.

Localyte.com produce high-volume geo-taged local content by using Google Map API.

Over coffee in San Francisco this week with Dax Hamman, Head of Display Media at iCrossing, we  discussed the EU cookie law change. Check out the overview here: http://www.daxthink.com/2009/11/eu-passes-cookie-law-opt-in-required.html.

So when I think of retailers such as BestBuy, Eddie Bauer and advertising networks such as Permuto, I can’t help but realize there are going to be a lot of compliance required – not least of all for management here in Silicon Valley to understand the regulatory changes. Perhaps the consumer age of the Internet is going to take on more of the restrictions that mass marketing channels such as direct mail and tele-marketing services now have to comply with.

When Lily Allen – English singer – left a note saying she’d quit Twitter, 2 things come to mind. Firstly, its annoying even those who are famous. Did Google ever do that? Secondly, what about the 138,541 followers who also are now thinking twice about following someone else. So despite Lily saying she was an “Neo Luddite” I think there are just so many times you want to or have something to say to someone else. The guys up the road in San Francisco at Twitter HQ should be watching carefully as it’s becoming ridiculous how people feel an elephant herd mentality to “tweet” everything. Its like having online tourettes syndrome. Maybe that’s why Lily swears so much in her lyrics!