The R5 Stutter-Step

IT shops are finished giving IBM's Lotus unit the silent treatment. More than a year after the release of Lotus Notes/Domino R5, migration to the latest version of the messaging and collaboration software has finally begun.

Among the lessons learned from early adopters: Migrating can be a massive project akin to upgrading server operating systems or enterprise resource planning software. For some shops, it has required a year or more of planning and testing, as well as hardware upgrades to take advantage of greater storage and processing capabilities.

Then there's the matter of updating each user's mail template and client software on a mix of on-site and off-site machines, and training perhaps thousands of users on a drastically revamped client interface. All this as a shortage of Notes experts looms in many IT organizations.

On the bright side, the payoff could be huge. Because R5 requires less server capacity, some companies will be able to shed half their e-mail servers and still support the same number of accounts. And greater Web support gives developers more tools to create Web applications and gives end users greater control over the interface.

Perhaps most important, Notes customers finally appear ready to take advantage of a revamped version of the most widely used--but often underutilized--application in their organization.

"The majority of people I talk to want to upgrade, and in most cases they should upgrade, but they need to examine their motivation for upgrading," said analyst Simon Hayward of the Gartner Group. "If you just treat R5 as the platform to deliver capabilities you thought about three or four years ago, then you may find the architectural and business process decisions made then don't apply any longer."

Hard data is elusive on how many Notes customers have made the switch, but anecdotal evidence strongly suggests that only a few have gone live. For example, at January's Lotusphere users' conference, nearly all attendees questioned (including some of Lotus's biggest customers) said they hadn't moved, and those who planned to upgrade said they were expecting to do so by midyear or later.

Lotus officials said the response to R5 has been strong, but would not quantify how many customers have upgraded. They claimed nearly all Lotus customers are at some stage of R5 upgrades, but would not say how many of those customers were beyond just evaluating R5.

Concern over Y2K is the most often cited reason for the delay. R5 shipped in early spring of last year, several months later than Lotus's predictions. The delay left most companies, especially larger ones, too close to Y2K lockdowns to begin such a large upgrade project.

But some large customers cited the need to wait for a more mature version of the software as a bigger reason they held off.

"We delayed due to quality of code, much more so than because of Y2K," said an IT manager at a large Midwestern manufacturing company with 65,000 Notes seats and nearly 400 Domino servers worldwide. "Even if Y2K hadn't been a factor, we could not have upgraded until some of the bugs had been fixed."

Besides waiting for bug fixes, mostly in the calendaring component of Notes, the manufacturing company also required a version that supports Windows 2000 Professional on the desktop. The company decided to go ahead with release 5.03, although some calendaring problems still had to be fixed with in-house programming, said the IT manager, who requested anonymity.

The company started upgrading mail servers last month and expects to have 90 percent of them completed by the end of the year. Application servers will require more testing and aren't ready to be upgraded. Client upgrades will begin in September, albeit slowly.

Others agree that code maturity was the most important reason to wait. Prudential Insurance Co. of America also wrestled with calendaring issues in earlier versions of R5 as well as inconsistencies with unread e-mail markings and problems regarding integration between the client and Internet Explorer. Prudential recently began to upgrade its nearly 300 servers and 60,000 users from version 4.5.

Executives weren't confident about making the move until release 5.04, the latest version, fixed most of those problems, said Mike Mandelbaum, vice president of IT.

Still, for others the mere size of the project along with lack of urgency to upgrade have kept them from making the move.

"These are very expensive projects and big upheavals for companies," said Sara Radicati, president of research firm the Radicati Group. "The old system isn't so bad, and there is not a crying need for most companies to make the move, so a lot of them have tended to wait as long as they can. There's no urgency."

Still, most large Notes customers say the pluses are so clear that it wasn't necessary to do much cost analysis to justify the upgrade. For them, it has never been a question of whether they would upgrade, just a matter of when and how.

"We haven't gone through the numbers exercise to come up with a figure to cost-justify the upgrade," said Prudential's Mandelbaum. "We believe there is significant cost benefit as a result of being able to reduce the number of servers we have running Notes, by a factor of two potentially. Consolidating servers is definitely where the cost benefits are."

Prudential expects to be able to host 1,500 or more users on a single R5 mail server, whereas today the company runs about 800 to 1,000 accounts per server.

The new Web functionality could undermine the performance gains. Web access to Domino databases adds overhead that some users fear may negate much of the server-side scalability gains. Lotus officials said R5's Web access performance is greatly improved over version 4, but it's still too early for much practical evidence to be seen.

Prudential, which thus far has upgraded about a dozen servers and 250 pilot users, this month will begin full-scale upgrades of R5 clients and servers. The server upgrades will take at least until the end of the year to complete, and no finish date has been projected for the client rollout, Mandelbaum said.

Big Vs. Small
The decision-making rationale of Lotus's high-profile customers is likely to be far different than it is for most Notes shops. While large customers are beginning to upgrade--and most are expected to do so eventually--for many small and midsize companies questions still remain as to whether the upgrade is worth the expense. And it's precisely that segment Lotus is banking on for continued growth of Notes/Domino.

Although large enterprises are salivating over R5 features such as improved scalability and ability to centralize administration, those back-end enhancements will mean far less in smaller settings. Most of the upper echelon of Notes customers also have considerable investment in Notes applications, and many are eager to leverage R5's Web development capabilities. But the use of Notes for more than basic mail and calendaring is spotty at smaller companies, and new app development features aren't likely to be an enticement.

"Most of our customers are hesitant to move over to R5," said Ron Herardian, CEO of messaging integrator Global System Services. "For enterprises with a minimum of 5,000 to 10,000 users, moving up is usually a foregone conclusion. But for most others, it's not so clear that R5's key strengths apply, especially for customers that are primarily just using the basic capabilities of the product" such as e-mail and calendaring.

Lesson 1: Plan Ahead
For IT departments charged with carrying out the upgrade, planning needs to begin well ahead of time in order to tackle the unique hurdles each company will face. Although early movers say they've experienced no notable land mines along the way, they also had prepared for the upgrade with extensive planning and testing, often dating back more than a year.

"The most important things are to plan well and always have a test system," said Cindy Hadden, director of the enterprise solutions group at Louisiana State University, which is near completion of its R5 upgrade. LSU uses Notes as the front end to a schoolwide application used by students and faculty for course registration, online learning and e-mail. Most of the data comes from a mainframe that interfaces with Domino servers. The application is accessed through a browser and has about 55,000 accounts.

Besides testing thoroughly in a nonproduction environment, wherever possible, LSU introduced R5 by placing a single R5 server into a group of live 4.6 servers, carefully monitoring performance and making adjustments before rolling out new servers more fully.

"One thing you can't see in a test environment is real load," Hadden said. "Benchmark programs won't challenge systems the way the real world will."

Other issues LSU faced were the need to upgrade much of its server hardware and to adjust to a redesigned R5 Domino directory. The university also had to test and modify each existing application so it would run properly on R5.

LSU wanted to move to R5 as soon as possible to enable single sign-on through a browser to the multiple mainframe apps. The majority of servers were upgraded in the past three months, and the move is about 80 percent complete. The university still has as many as 5,000 Notes clients used for mail on a variety of desktops, and plans for upgrading those aren't final.

Another important issue customers have faced is the need to coordinate the upgrade of each user's mail template with the server and client upgrades. That can make an already difficult upgrade schedule into a baffling logistical dilemma, especially in a large organization.

Prudential came up with a workflow process that identifies when a user has been upgraded at both the client and server level, then automatically redesigns the template.

"This is one of the trickiest issues, the timing of the upgrade process--clients, servers, templates, laptops and home machines as well," Mandelbaum said.

Also, Notes shops must decide whether to take advantage of transactional logging, an advanced server feature that allows more reliable backup and recovery, and improved performance. That usually requires hardware upgrades, including adding an extra hard drive to each server, as Prudential did.

For many organizations, the initial heavy lifting on the server side seems less challenging a task than the still-unclear issues of training users on the new client. Some, despite the elaborate planning for the upgrade process, have not yet solidified their user training programs.

"Upgrading the clients, including training, will be a bigger expense and challenge than the servers," said the IT manager at the Midwestern manufacturing firm. The company has customized the R5 interface to make it easier for users to create a view similar to the familiar 4.5 client, and has implemented tutorial software for users, but training will still be necessary.

"Within a day or two, I figured out all the nuances and idiosyncrasies; I think a typical business user is going to take longer," said the IT manager.

All this as companies face a shortage of IT personnel skilled in Notes, particularly people familiar with R5, said Giga Information Group analyst Jonathan Penn.

"Notes expertise is getting harder and harder to find, and that can protract the migration," Penn said. "If there are templates that aren't working or databases that are corrupted, you're going to have to figure it out yourself.

"It's not a project killer, but it's another of the things you have to work around," he added. "Retraining for users and administrators, server upgrades--all these issues add to the costs and play in the decision about whether or not it's going to be worth the move."