• How much data is gathered and from where? This is perhaps the most important question to ask a personalization tool company, because one of the biggest stumbling blocks of e-commerce optimization is a concept called the "cold start," which essentially means that there is not enough data to provide meaningful recommendations.
  In this case, regardless of the sophistication of an algorithm or the number of Ph.D.s who crafted it, sparse data sets will yield poor recommendations, which not only creates a poor customer experience but also does little to drive sales. Companies such as ChoiceStream are able to address this issue by creating entire taxonomies of associations for their clients. Aggregate Knowledge drops third-party cookies onto its network of sites and gathers vast quantities of data throughout the Web that inform recommendations.

• How sophisticated is the reporting? Given that lifts in sales can be very subtle and are frequently associated with certain key pages, it is critical to understand how personalization tools interact with conversion. Site analytics and Web site tagging, while helpful, are generally insufficient to gauge the entire effectiveness of a personalization tool. Tools by vendors such as Certona have the capability to drill down their reports to an item level, which can be critical to providing insight into what specifically is working (or not).

• How quickly can it be changed/adjusted? One of the downfalls of the early engines was the "black box" or the relative lack of mutability around the formulas. While the "black box" is virtually nonexistent today, alterations to an algorithm are typically made in one of two ways: either by the client company or through a client services function provided by the vendor.

• How many clients does a vendor have? Given the relative youth of so many of the e-commerce personalization tools, any company with experience in a given vertical or industry will likely have an advantage over other competitors. Why? The company will probably have already addressed the complex nuances of a particular industry, which means less algorithm tweaking after deployment. Coremetrics and CleverSet are two examples that are heavily focused on the retail sector, while companies such as TouchClarity actually grew from a background in the financial services vertical. Others, such as Aggregate Knowledge and Baynote, have experience working with media and content providers, in addition to focusing on retail. ChoiceStream has perhaps the industry's deepest experience with media companies but is also gaining traction in other sectors within retail, particularly with heavy-traffic, SKU-intensive sites.

• How well-capitalized are these companies? The influx of funding into the personalization tool space has already affected the landscape. Acxiom's purchase of Kefta and Omniture's acquisition of TouchClarity make Kefta and TouchClarity more likely to be upsells to existing Acxiom and TouchClarity clients and may affect their future road maps and commitments to clients. Likewise, the relatively rich capitalization from venture capitalists of firms such as ChoiceStream and Aggregate Knowledge puts pressure on the sales processes of less capitalized competitors.

• Is the tool a stand-alone tool? As mentioned above, some of the personalization tools sell only recommendation engines. Others, such as Endeca or Coremetrics, sell on-site search or site analytics first and optimization products as incremental (and sometimes separate) features. While it can often be easier to just manage a single vendor, the risk associated with an all-in-one solution is that it is less likely to be "best-in-class," since development efforts are primarily focused on flagship rather than ancillary products.

• Are you comfortable with third-party cookies? Some companies rely on third-party cookies as a means of gathering incremental data. This means that some key elements of information — such as where, when, and how consumers click through a Web site — are shared, albeit anonymously, in a larger pool of data to unearth consumer behavior and reactions to different types of content across the Web. Companies uncomfortable with this approach should carefully qualify such companies to evaluate if a relationship make sense. For companies reluctant to pool their data, partners using third-party cookies may not make sense, but for those businesses bold enough to be part of a data consortium, the results can be extremely rewarding.

Evolution of Personalization
Finally, Mulpuru provides some insight on what to look to in the future of personalization. She predicts there will be a shake-out in the industry, with similar companies merging while others are pushed to the margins, as well as, with any luck, simplified pricing.