We probabaly shouldn’t count on this one lasting very long, as the following site appears to be violating Google’s TOS by scraping their Google News sources and stories to figure out all the sources and frequency the sources are being used: Google sources scraped. Unfortunately, Google does not offer the ability in its API to access Google News and therefore be able to scrape this type of information without violating their TOS, unlike Yahoo which has this information wide open in their API (subject to a very generous limit of 5,000 calls per day).

Rather than just sound like or be a whistle blower here — which I’m not trying to be — the real, deeper question is why make this list of sources a big secret anyway? Dan Gillmor notes that this question has already been asked and the response from a Google spokesperson was “for competitive reasons.”

Lame.

Google, here you have your tailpipe competition (Yahoo!) not only saying it’s ok to get this information from them, but get it up to 5,000 times a day, and you are saying you won’t even allow people to get this information for “research reasons” (see screenshot above).

Readers can contact Google about changing this by using the feedback form here.

1) Use the “I want to request permission to use Google News for special purposes”,
2) add the subject line “Google News added to API, please” and
3) make the following comment:

I wish to make a request that Google News be added to the Google API. Yahoo allows API interaction with their news sources, but Google currently does not.

TDavid wrote in more detail about this here:
http://www.makeyougohmm.com/20050326/1620/

Thank you for your consideration.

Sincerely,

________ < -- your name

I’ve sent this message myself (reformatted to be in the first person, though).

Perhaps if enough people contact Google with this request Google will legitimize the use of programs like the one Dan Gillmor pointed to — which, BTW, I’m not advocating violating any website’s TOS, nor do I know the programmer behind the project in question or have anything to do with that project. In fact, I think this programmer should take his Google News scraping project offline until or unless Google has given permission for this type of research. Google may have already authorized this project. Hopefully that’s the case.

Update: Very quickly I received a response to my email from Google that reads as follows (yeah, it looks canned):

Thank you for your interest in Google News. Although we do not currently permit the creation of a news feed from the results on Google News or permit webmasters to display Google News headlines on their sites, we do provide a means for monitoring Google News searches through Google Alerts at http://www.google.com/alerts

Additionally, although we do not currently offer a news syndication service, we are considering adding one in the near future.

If you’d like to incorporate Google News search results into your site, we ask that you do so by creating a link to the Google News main page or Google News search results. Please feel free to use any of our Google logos found at http://www.google.com/stickers.html

For more information about acceptable uses of Google News, please see our Terms of Use at http://news.google.com/intl/en_us/terms_google_news.html

Regards,
The Google Team

I bolded the part that I found particularly interesting. Please LMK if you receive the same canned response or if someone personal actually responds by using the comments section or trackbacking from your blog.

Update #2: I changed the title from “Finding” to “Identifying” Google News Sources which seemed more relative.

• Competing with Google
• Wish there was a Google Rollyo, not a Yahoo Rollyo
• Yahoo blog search hide and don’t seek yet
• Google to expand email services?
• Sourceforge inadvertently blocking Yahoo spiders?
• Starting to see Live.com search referrals