Paid inclusion is one way small e-tailers can gain visibility in a search engine or online directory. Paying a fee to have a Wed directory index and review a Web site is one way small e-businesses can level the playing field with larger operations. It typically takes up to 90 days for a new Web site to be indexed and listed in the major search engines and even then there are no guarantees that your site will be found. That's three months of lagging page views — time those most small businesses can ill afford to be overlooked. Search engines guarantee to list pages from your Web site in their database and re-spider them on a regular basis, usually at least once per week — that's what paid inclusion is all about.

However, paid inclusion programs can be hit and miss in the search engine industry. Concern about how paid inclusion data affected its search engine's relevance led Ask Jeeves to drop its Index Express XML feed program. The company also says the initiative wasn't paying off as expected.

"When I say it impacted relevance," said Jim Lanzone, vice president of product management at Ask Jeeves, "I mean both good and bad. Sometimes [something ranked] better than it should have ranked, sometimes worse."

The company introduced its paid inclusion services in June 2002, but says it's since come to the conclusion it's virtually impossible to merge structured data, such as that in XML feeds, with the unstructured data that comes from crawling the Web.

"It is difficult for the algorithm to understand types of information it's not used to. When we put this information in, the results sometimes showed up in the wrong place," said Paul Gardi, SVP of operations and strategic planning at Ask Jeeves.

The company's dropping of its program comes at an especially touchy moment for paid inclusion. Yahoo! this week unveiled a new expanded paid inclusion program, combining its Inktomi, AlltheWeb and AltaVista offerings. The news was met with some concern within the industry, as well as in the mainstream press. People speculated whether accepting payment for inclusion affected results or should be disclosed more prominently.

In response, Yahoo! maintains it leaves rankings up to its algorithm, which doesn't distinguish between paid XML feeds and crawled data.

"Their placement is determined by the algorithm, and is not influenced by whether they pay," said Chris Bolte, vice president of strategic alliances at Yahoo!'s Overture division. "What users really care about is relevant results. They don't care how they got there."

Google has steadfastly refused to take paid inclusion feeds on philosophical grounds, feeling such programs might hurt its reputation with users.

"User trust is the number one thing that we focus on," said Tim Armstrong, vice president of advertising for Google. "The premise that we have stuck with from the start is that user trust is the most important component of our business."

Paid inclusion feeds, and XML feeds in general, are viewed many as a way for search engines to better tap into the "invisible" or "deep" Web. Yahoo!, perhaps partly in an effort to quell concerns about paid inclusion, is pursuing this vision by partnering with non-profits for its new XML feed program. These partners, including NPR and the Library of Congress, will provide Yahoo! with data feeds about their pages free of charge.

Disappointing financial results were also a factor in Ask Jeeves' decision, though decreased relevance was the paramount issue.

"We've found that the revenue that you make from it on a per-query basis is light," said Lanzone. "The only way you're going to monetize that is having it over a broad range of queries — if you have a lot more [queries] than we do."

LookSmart Marketing Director Dakota Sullivan confirmed Lanzone's analysis. "The thing with paid inclusion that they probably overlooked in their model is that you need massive scale," he said. LookSmart's business has historically relied on paid inclusion but since losing MSN as a partner it is now focusing on paid listings.

Ask Jeeves brought in $2.1 million from paid inclusion in 2003, a small sum compared to the $77 million it generated from paid placement. The company carries Google AdWords listings on its sites.

"Even if we had all those queries, I don't think it's worth doing," said Lanzone. "I think there are much better ways ... There are probably better mousetraps in dealing with structured data."

Lanzone also noted that, given the other drawbacks, it was easier to drop the program because of the controversy that surrounds XML paid inclusion.

"I could see why people would be suspicious of it," he said. "They would assume that where there's smoke there's fire. I don't want our integrity called into question."

Ask Jeeves will continue to offer the Site Submit program, under which smaller advertisers pay a flat fee to submit their URLs to be crawled. Just remember that paid submission only guarantees that the site will be reviewed; it does not guarantee that your site will get listed.

Courtesy of ClickZ.com.