Sharpen Your Edge

By Barbara Depompa Reimers

Info mining is a pretty new term that alludes to the process by which predictive patterns are removed from info. Information is typically stored in enormous, relational databases and the quantity of info stored can be serious. But what does this info mean? How can a company or organisation work out patterns that are important to its performance and then do something primarily based on these patterns? To manually wade thru the info stored in an enormous database and then work out what's vital to your organisation can be pretty much impossible.

Here is where information mining systems come to the rescue! Information mining software investigates great amounts of information and then decides predictive patterns by inspecting relations. Information Mining Strategies : There are many info mining ( DM ) systems and the kind of information being inspected strongly influences the kind of information mining system used. Note that the character of info mining is continually developing and new DM methodologies are being implemented all of the time. Talking generally, there are a few main strategies employed by information mining software : clustering, classification, regression and organisation techniques.

Clustering : Clustering alludes to the formation of information clusters that are grouped together by some variety of relationship that identifies that information as being similar.

An instance of this would be sales information that's clustered into precise markets. Classification : Information is grouped together by applying known structure to the info warehouse being inspected. This technique is excellent for specific info and uses a few processes like call tree learning, AI nets and "nearest neighbour" strategies. Regression : Regression utilises mathematical formulas and is glorious for numeric info.

It essentially examines the numeric info and then attempts to use a formula that fits that information. New info can then be wired into the formula, which leads to predictive research.

Organisation : frequently referred to as "association rule learning," this strategy is preferred and comprises the discovery of engaging relations between variables in the info warehouse ( where the information is stored for research ). Once an organisation "rule" has been revealed, prophecies can then be made and acted on. An instance of this is shopping : if folks get a particular item then there might be a high likelihood that they also buy another particular item ( the store boss could then ensure those items are found near one another ). Information Mining and the Business Intelligence Stack : Business intelligence makes reference to the gathering, storing and investigating of information for the sake of making intellectual business choices. Business intelligence is often split into a couple of layers, all of which comprise the business intelligence "stack." The BI ( business intelligence ) stack is composed of : an information layer, analysing layer and show layer. The analysis layer is accountable for information research and it's this layer where info mining happens in the stack. Other elements that are a part of the analysis layer are predictive research and KPI ( key performance indicator ) formation. Info mining is a critical part of business intelligence, providing key relations between groups of info that's then displayed to finish users through information visualisation ( part of the BI stack's show layer ). People can then quickly view these relations in a graphical demeanour and take some type of action primarily based on the info being displayed.

Steve Bogdon is the Advertising director for Dashboard Understanding , one of the swiftest growing business intelligence ( BI ) sites online. Dashboard Discernment is an authoritative and trusted online resource for the business intelligence, information visualisation and dashboard software communities.

Data mining--the use of mathematical algorithms to seek hidden patterns or associations in data--is now widely accepted as an affordable way to sharpen up a company's competitive edge.

No longer the exclusive domain of "SAS-jockeys'' with statistical bents, data-mining features have been integrated into a broad range of packaged application tools, helping businesses of all sizes sift data in search of patterns that reveal hidden insights.

Tools that used to cost hundreds of thousands of dollars and required special expertise can now, for a fraction of the cost, automatically select which ads or promotions to present to Web site visitors and help decipher what performance improvements are needed to boost sales.

"We see data mining as a competitive weapon," says Alexander Veletsos, IS director at Florida Hospital. "It's critical to helping us use our capital wisely, to invest in better health care and remain an efficient health care provider," he says.

Until recently, software makers found it difficult to sell generic data-mining tools to general business users. "A few years ago, it looked like data mining would never penetrate the business world, as several companies failed at selling data mining to business users," says Phillip Russom, service director and analyst for the Hurwitz Group.

But that's changing as companies increasingly use the data-mining algorithms built into the latest customer relationship management, customer tracking and performance-monitoring tools to analyze customer behavior. "Data mining is there, but users never see it," says Russom, who adds that "customer analysis is the killer app of data mining."

That would explain the growing success of customer analysis software suppliers in the last year, including companies such as Accrue, Broadbase, Epiphany, NetGenesis, Siebel and others. It also explains the high price some companies are willing to spend on data-mining expertise. Knowledge Stream Partners, a 26-employee, data-mining software company was recently acquired by Xchange Inc. for $52 million.

"That comes to $2 million per employee, which is not uncommon in these types of data-mining acquisitions," says Bob Chatham, an analyst at Forrester Research.

"Smart companies today want to cross-sell, up-sell and analyze their internal processes to improve business performance and ultimately increase sales," he says.

Foofoo.com, an online luxury retailer, would agree. The company's biggest challenge these days is to figure out which content should be personalized and how. After a lackluster 1999 holiday season, the company did some research and found only about 35 percent of its site visitors were getting to product pages. The goal is to get the number up to 70 percent, says Philip Hawken, director of operations for Foofoo.com.

Foofoo.com redesigned its site and now products appear as users read about the latest in everything foofoo--from fine clothing to gourmet foods to fantasy vacations. As a result, page views have increased by 1.5 per session, and the length of time customers visit the site is up significantly as well, with customers staying an average of 2.5 minutes longer.

Sales, too, have improved, although Hawken declined to say by how much. There's also an effort to increase personalization. So based on what the company knows about a customer, whether it's the customer's first visit or tenth, Foofoo.com can display more product information on items the customer might be interested in, customizing the homepage to highlight products past history shows the customer might buy.

Foofoo.com uses NetAnalysis from NetGenesis to determine the most popular content areas by tracking customer behavior. "We needed to know which types of visitors prefer which areas of our site," Hawken says. Using NetAnalysis, Foofoo.com also measures the success of online and offline marketing and advertising programs.

Yet another online retailer, SmarterKids.com, is using data mining because the company realizes analyzing Web site performance is fundamental to its survival as a Web retailer of children's books, software and toys. The site was launched in November 1998.

"We could take orders and process customer requests. But we realized we needed to know more about customer profiling, to customize our store and meet online visitor needs," says executive vice president Al Noyes.

Noyes selected Net-Analysis over three others tested, based on its ease of use and analytical depth. "The tool generated canned reports that were easy to understand and custom reports that could drill down to get behind the cause and effect of traffic patterns," he says.

In one instance, SmarterKids discovered that most visitors coming to its site from Microsoft's Encarta encyclopedia home page were leaving the SmarterKids site after only a brief stay. Using NetAnalysis, the Web retailer found online visitors who came from Encarta were doing so based on a search tool tied to a banner ad on the Encarta site.

After running a series of NetAnalysis reports to analyze the behavior of these online customers, SmarterKids.com learned that once the would-be Encarta visitors reached the SmarterKids site, their queries for popular toys, such as Barbie or Pokemon, resulted in a "Sorry, no results were found" message--and often short-lived visits. After analyzing the reports, SmarterKids.com redesigned the site to automatically list its products as visitors arrive. Since that site change, "the ad's performance on Encarta has improved dramatically," says Noyes. The banner ad on Encarta now draws "quality" customers--those who buy versus those who browse and leave. Now, SmarterKids uses NetAnalysis to test the appeal of products sold, to manage the mix of promotional, merchandising and educational messages that appear on the site, and to test special offers or incentives. "We now redesign the site weekly," he says.

Making improvements pays off. SmarterKids has tripled the ratio of orders to visitors in the year since NetAnalysis was installed. "Before NetAnalysis, our marketing decisions were made in the dark. We didn't know where qualified customers were coming from and what they were doing," says Noyes. "Now, we can track people's progress through our site, including where they clicked, what pages they stayed on and exit pages. This information gold mine provides us with a complete understanding of our Web site and e-customers.''

Companies have multiple classes of data-mining suppliers to choose from today. Alexander Linden, a Gartner Group analyst, says the classes of tools range from generic workbenches such as SAS or IBM's Intelligent Miner--which require specialized training--to Web apps that embed data mining inside.

Vignette, since its recent DataStage acquisition, and Epiphany fit this category.

With the multiple types of products using different analytic methods, it's increasingly difficult for businesses to gauge how much of a return they can expect to gain from data-mining tools. The problem is that every tool draws conclusions based on different criteria and assumptions. But Linden says there seems to be an attitude among businesses that regardless of the specific data-mining software chosen, whatever a business does to incorporate data mining is a big improvement over not performing any advanced analytics on corporate data.

One of the fastest growing classes of suppliers now offering data-mining features wrapped in e-business apps are ASPs. Buystream.com is an e-commerce business-intelligence software developer and ASP helping dotcom companies gather and turn knowledge from their Web sites into increased revenue. One such client is Boutique Y3K in New York, an e-commerce consulting firm focused on the fashion retailing industry. The consulting firm, in business two years, boasts a variety of clients, including such online retailers as NineWest and BabyGear.

Fashion houses tend to flock to service providers for technology needs, says its CEO Cecilia Pagkalinawan. And Boutique Y3K has installed Buystream Architect on several of its clients' Web sites, unobtrusively capturing visitor information and aggregating it into an online customer profile. This helps Boutique Y3K take the guesswork out of site design and recommend performance improvements to its clients.

"For one fashion retailer, we found 75 percent of the consumers who visit the site have plug-ins for real video, so the fashion house can show fashion show video clips online," says Pagkalinawan. As an IT services provider, Boutique Y3K already used some data-mining tools, but the firm was bogged down by the amount of work involved in programming and Web site usability studies. "Buystream's tools let us take a snapshot of a fashion house's end user data, bandwidth, browser and hardware information. The tool is key to analyzing the performance of our clients' Web sites," Pagkalinawan says. Based on the traffic patterns, for example, Y3K can make recommendations ranging from site design improvements to assessing the use of richer graphics and animation to speeding server performance for clients.

Boutique Y3K conducts quarterly studies of its fashion clients, and is finding that Buystream's data analysis tools let it use data measurements to bolster the consulting firm's suggestions and upsell services to clients. Because of the range of services offered, Boutique Y3K can charge fees ranging from $250,000 to $2 million, depending on the amount of help a fashion house needs.

Ultimately, data mining has become more widely accepted as it's integrated into other business apps. And being able to do that effectively just may separate the winners from the losers, says Mike Schroeck, a partner at PricewaterhouseCoopers. Which is why more and more companies will use data mining tools to sharpen up.

Barbara DePompa Reimers is a freelance writer based in Germantown, Md. She can be reached at bdepompa@aol.com.