In September 2005 I wrote that Sunday seemed like a good day for an RSS cleanup and listed some of my criteria of why feeds were removed.

Yesterday, I spent a good part of the day combing through my list of nearly 400 RSS feeds which had not been fully cleaned since November, so it was well past time for me to scare up the axe from the garage.

Program used
I downloaded the server side program reblog, installed and followed along with the information in Matt Haughey’s excellent Lifehacker reblog How to post. Reblog is a lot like Bloglines, but adds some AJAXy goodness so you can use your keyboard to pound through feeds and archive or publish items.

The published items go into a separate RSS feed and web view so you can essentially filter hundreds of RSS feeds into one feed that interests you and then pick from that feed the items you want to blog about. There are plugins available to post directly to your Wordpress or Movable Type blog from your reblog feed. Reblog also publishes an OPML file of your feeds which is handy for exporting to other feed readers.

I didn’t like the idea of bots picking up my reblogged web feed so I added a block for spiders to robots.txt to keep them out of there.

Downside I see with using this program is one could look like a splogger since it is republishing the content you choose to “publish” in a public location and creating an RSS feed for this content with your name on it. The upside is that curious readers can see the posts that interested but you chose not to blog about for whatever reason. Sort of like following along with a linkblog, but actually containing the blog posts you saw and marked off as being interesting enough to ‘publish’.

For example, in going through the hundreds of RSS feeds I published 233 items. Out of those 233 posts I only blogged 10 of them (so far). So it’s filtering from a list of over 3,000 posts to 233, then down to 10 which ultimately became blog posts.

Once an hour (configurable, of course) a cron job updates the feeds and brings in new posts to review. The thing I don’t like about this is most feeds don’t update once an hour. A suggested future feature for some aggregator out there: updates polled by post frequency. What I mean by this is analyze the number of times a feed has posts in say the last week and then come up with a minimum update of say once a day. This way if a feed that doesn’t have a lot of updates starts having them in one day, it can alter the frequency of checking for new posts.

The picture below displays what happens in reblog when new posts are available from feeds. When new posts are available you can either archive and publish or archive. With the AJAX and keyboard shortcuts the process moves pretty swiftly. I timed myself pounding through a dozen feeds and 80 or so posts in less than five minutes. The ones that are marked for publish can then be read closer.

Criteria used to weed out feeds
This time I was even more picky. Had to be, really, as the list is getting a little too long to manage. Who can comb through 3,000+ posts a day even with headlines and get any other work done? I think 250 is a good target number of feeds for me, which after the paring I’m down to about 160 (Here’s my updated 3/20/2006 OPML file), so I have room to add more feeds. And there are a few feeds I want to add so if anybody reading doesn’t see their site there check again in a week or two. You can always tell me why I should be subscribed to your blog in the comments/trackback area too. I’ll look seriously at adding any blog that isn’t spam/splog, contains a good mix of unique content (or a unique view) and is updated at least once per week on average. Preferably 2-3 updates a day.

With my target RSS feed goal if the average feed has 10 posts, that would mean I’d be reviewing a max of 2500 posts at one time. Since most of the feeds I read don’t have 10 updates a day, that puts the number of total posts I’d skim on a daily basis under 2,000. This doesn’t count the feeds I’m reading that I’m not subscribed to, so you can see the numbers can get pretty large. The bottom line is I don’t think it’s realistic for anybody who isn’t a full time blogger to be able to keep up with much more than 250 feeds. Scoble has pared his list back to around 800 but I think before too long he’ll be back down around 500, and he’s a pretty voracious feed consumer.

Here is the criteria I used for unsubscribing:

- any feed without a new post in the last 60 days. I found quite a few feeds which hadn’t been updated in 2006 yet.
- any feed where the set of most recent posts had nothing that interested me. I marked all interesting items with “publish” in the reblog interface. If not even one of the most recent posts were interesting, then I unsubscribed. Chances are good if this blog/website updates with something good again then I’ll be back. If I were a blogger I’d make sure each feed contained at least 10 items, but don’t go crazy with some number like more than say 25 feeds.
- clickthru does not lead directly to the article. Those inserting full page ads in between permalinks deserve to be sent out to the woodshed and whipped. Send your readers directly to the content. Mainstream Media (MSM) was the most guilty in this department.
- popups used. I intentionally disabled my popup blocker to see which sites were using popups. These were dead in 2000, but some continue to assault readers with them. Goodbye.
- feeds that have moved. I resubscribed if the other criteria above was met.
- duplicate information. Seems like some of the feeds I subscribed to had crossover information like the CNET feeds for example. I got rid of most of the niche feeds and stayed subscribed to the broader feed instead.

I didn’t remove partial text RSS feeds. Those don’t bother me like they do Scoble and some others. I like clicking through to the site and reading the content there. I also didn’t remove feeds that contained ads. I don’t mind ads in feeds as long as they are complimentary to the content and not invasive. I want people with good content on the web to make money so they will continue to be compelled to create new content.

Now I’ll toss this over to readers again. What is your criteria like for subscribing/unsubscribing? To keep subscribed do you have to see at least one interesting thing a week, a month, a quarter? What’s your criteria like? How many feeds can you realistically keep up with?

• Blending FeedBlendr and Alertbear shows common weakness
• Feeling dirty or clean after using Feed Rinse
• New year RSS reads and resolutions [site news]
• Anothr RSS via Skype
• To have full RSS feeds or not?
• Ill-advised RSS duplicate post solution in IE7 and Outlook 2007