Building a competitive intelligence tool using RSS and Wordpress

How many people in your company are reading blogs or articles that are related to your industry? Imagine if those people could easily share relevant articles with all other employees; and if those articles could easily be organized and classified… You could transform those individual initiatives into a powerful competitive intelligence tool.

More than one year ago, French blogger Fred Cavazza published an inspiring article explaining how RSS technology (using Google Reader and RSSBang tools) could be leveraged to build such a tool. I have used the concepts explained in this article, and added my own ideas to build such competitive intelligence platform at Kelkoo.

Here is a guide to build your own solution… This is not a step-by-step tutorial (which would require a lot of efforts to write) and you will need a good knowledge of the Wordpress publishing platform to implement it… Still want to read? There it starts :)

Find motivated people to build RSS feeds

That is the pre-requisite: you have to find motivated people that are eager to share with the others, and will accept to spend a little extra time to flag each article they find interesting.

To flag those articles, they can use any tool that is able to output RSS feeds. The most obvious ones are:

• Google Reader. When reading articles in Reader, you can add tags for each them ; then create a RSS feed for each tag.
• Delicious. When browsing the web and using a Delicious plug-in for your favorite browser, you can easily bookmark the articles and tag them. Drawback: Delicious won’t put the body of the bookmarked articles in the RSS output.

Using tagging, you can define one or many tags to structure the flagged content. For instance, for an e-commerce company, you can be interested to have articles talking about the competition, about the best practices in terms of SEO / user experience… You would then ask your fellow bookmarkers to tag their articles with “competition”, “seo”, “user-experience” in order to organize the tagged content.

It is important to have a limited number of tags, and broad tags – this should remain an easy process to find the right bucket when bookmarking an article…

Aggregate all those feeds

Number of people * number of tags = you could quickly reach an important number of RSS feeds, that have to be aggregated in one place. Fred Cavazza suggests to use RSS Bang or similar service (i.e: Yahoo Pipes) to merge all those RSS feeds into a single one, which is then followed by all the interested persons in the company.

While this solution is easy to implement, this is not convenient for everybody. Indeed, people have different habits – some are using RSS, some like emails, some prefers to go and browse a site from time to time… And if you want your competitive intelligence tool to be used, then it must adapt itself to its audience, not the contrary.

I have decided to use the Wordpress publishing platform, enhanced with a few plug-ins. Once you have installed Wordpress, the following plug-ins have to be installed to build the foundations of your new tool:

• FeedWordPress, “an Atom/RSS aggregator for WordPress. It syndicates content from feeds that you choose into your WordPress weblog”. Perfect to transform RSS feeds into Wordpress articles. A visual interface is used to configure RSS feeds to regularly check; and rules can be defined to put all articles from one feed into a specific Wordpress tag or categories. If you have created some specific feed to handle “competition”, “seo”, “user-experience” topics, then configure FeedWordPress to map this structure to the Wordpress one (either using Wordpress tags or categories).
• FeedWordPress Duplicate Post Filter, which ensures articles remain unique in the Wordpress database in case they have been bookmarked several times.

From there, you will need to choose a Wordpress theme that supports “browse by category” or “browse by tag” features.

Going forward

Once the foundations are in place, you can enrich your Wordpress installation with any plug-in you will judge useful to ease the adoption and usage of your competitive intelligence tool. As for myself, I have put in place a solution to auto-tag incoming articles, and a way to generate emails to push new articles to subscribers. Quick review of those plug-ins:

• WP Calais Auto Tagger, “With the Calais Auto Tagger plugin, you’ll never have to think of tags for your posts again. The plugin uses the Open Calais API to perform semantic analysis of your post text and suggest tags for you”. This one is pretty interesting: tags can ease the navigation across all the bookmarked articles; however, this plug-in doesn’t support automatic tagging; that’s why you need the following:
• WP-FeedWordPressAutoTagger (I have not managed to find a shorter yet meaningful name for it :) ), a custom-made plug-in inspired by the work done by Lee Clemmer. This one automatically launches the tagging process once an article is retrieved by FeedWordPress.
• WP Calais Archive Tagger, “automatically goes through your archives and tags every post you’ve written.”. In case of…
• MailPress, “Allows to send beautiful and styled html and plain text mails based on dedicated themes and templates”. A mailing system that can be used to push newly bookmarked articles on daily, weekly, monthly basis.