nicolas leroy

Shopping engines and the “web of things”

February 25, 2010

The “web of things”

Yahoo! recently talking about its search strategy, and how it can still innovate on search:

People don’t care about finding 5 million pages about Roger Federer, they want the one page with all the relevant information. If someone is looking for Black Eyed Peas, we want to show users a bubble with all the info they want – main site, most popular songs, etc. And we want to do this for all the bands, not just popular ones. And movies too, restaurants, etc. The web of things.

Microsoft is doing the crawling, we’re innovating how you display it.

So how is Yahoo going to lead this “web of things”? It’s like digitizing all your CDs – the hard part is building up a database of this information you’ve made. On the web the information about these recording is noisy. You just can’t manually edit metadata – you need killer machine learning and data mining.

While Yahoo! is talking about web search, this vision of the “web of things” also makes sense for shopping search. In a sense, this notion of “things” already exists in the CSEs world for years, mainly through the duality between products and offers:

The problem with soft goods

The problem is that, for now, CSEs have done the easy part: creating or licensing product databases for hard goods (electronics, computing, music, books…). For soft goods (fashion, health & beauty…), where the number of product references is far more important, the cost to build and maintain such product database becomes too high and shopping engines have therefore built site navigation based on offers rather than products. While this approach allowed CSEs to provide an acceptable solution to browse soft goods for a rather low cost, it also induced some major problems in terms of usability.

The main issue is that search result pages on soft categories have two dimensions:

This offer-based navigation works ok when searching for a specific product (i.e “levis 501”) as the first dimension is basically removed. But that becomes a problem when doing product research (i.e searching for “levis jeans”): too many results and information overload to take the right decision!

Another issue is that it is difficult to attach any content (like user reviews, coupons…) to offers due to their inner nature (short lifespan, linked to a specific merchant), therefore reducing the services CSEs can provide to shoppers on those soft categories. Among those services, the inability to provide a true price comparison service is overwhelming.

Last but not least, CSEs tend to mix products and offers for search result pages that trigger results from several categories, which can be confusing or frustrating for the users.

Some CSEs have tried to solve this problem

Several CSEs have experimented with product grouping on soft goods (Yahoo! Shopping, PriceGrabber…) using clustering technologies rather than product databases. Most of the time, those experiments have not been successful, as the number of offers being grouped and the quality of results remained low.

Among all those experiments, TheFind and Buzzillions have reached some really promising results. TheFind has launched its price comparison service last November; this service is still in beta, as the algorithms need to be tuned, but the clusters created are good enough to provide a valuable price comparison not only on soft goods but also on hard goods (without any product database)… Buzzillions (not a CSE stricto sensu) provides a pretty good product navigation on soft goods; still some “products” remain duplicated. Those two initiatives show that building product clustering on soft goods is not an utopia, but still requires some research and smart people and algorithms to become a reality!

An example of product aggregation on clothes category (Buzzillions):

Product display at Buzzillions (Feb'10)

Another challenge: product definition must be flexible

Another challenge that most CSEs failed to solve with their product database is that the aggregation of offers as products must be flexible enough to cope with the various purchase scenarii of users. I’ve already discussed this problem in a previous article, highlighting several examples (different bundles for a same camera, different color for a fridge model) and concluding that the way products are defined can lead to successful or inefficient search experience.

YouTellMe, a Dutch CSE, has implemented a really good solution to solve this problem for hard goods (relying on a very detailed product database that understands product variations). For soft goods (i.e: size and colors for fashion goods), some retailers have implemented clever solutions, but as far as I know, no CSEs have succeeded implementing such mechanism.

Product variations for Levi’s 514 jean at Amazon:

Product overview at Amazon (Feb'10)

Building the next-gen product database

Thinking what could be the perfect CSE for 2010, I believe CSEs need to work again on their basics – how to organize and display search results – in order to bring their user experience to the next level:

Such shift would require CSEs to find innovative approaches – surely a mix of manual and automated solutions – to build a next-gen product database, for instance:

In a sense, I would say that such next-gen product database nearly exists: this is the Amazon product database, accessible for Amazon partners through web services. Amazon covering many categories and starting to provide CSEs-like features (marketplace, ProductAds), its database is surely the most comprehensive product database today, and can show the way for CSEs.

Final thoughts

The “web of things” is an appealing vision, but I doubt Yahoo will execute this vision on the shopping domain, considering the recent announcement related to Yahoo! Shopping. And Google Product Search and Bing Shopping still mix products and offers in their shopping results.

To differentiate themselves from those starting points that are search engines, CSEs need to (re)think what is their value proposition for users. Thinking about “things” (I’ve discussed “products” so far but think about brands / product lines / artists…) rather than offers / results has the potential to lay down some brand new user experience for shopping search.

2 commentaires

on Feb 26, 2010 / 12pm
Hi Nicolas,

Good article! We're facing the same challenges regarding soft goods at YouTellMe. The Levi's 501 example you gave is already tough, but still one of the easier products since it has a well known brand and type. It gets really hard when you start to work with brandless and more unique items.

Soft goods tend to have more emotional influence in the decision process too. They might be more suitable for discovery-type of navigation with more social input than hard goods.

Anyway, plenty of stuff to think about! We'll let you know when we crack the problem ;)

on Feb 26, 2010 / 8pm
@Thijs - Thanks for the comment! Can't wait how you can innovate on soft goods like you do on hard goods :)