Notes.net Exposed: How we test Domino beta software

How we test Domino beta software

By
Tara
Hall

Level: All
Works with: All
Updated: 01-Nov-2001

Editor's note: The Notes.net Exposed series provides you with a behind-the-scenes look at the Notes.net Web site. This time, we explain our testing role in the Lotus Domino software development process.

As part of our charter, Notes.net is commissioned to deploy and test early releases of Domino. We are an official part of the Domino release criteria, and the ability for Domino to run successfully on Notes.net is one of the requirements before Lotus ships new releases. Problems found on our site can prevent a product from being released to customers until the problems are resolved. This applies to major releases, such as the upcoming Notes/Domino 6. In addition, we often deploy and test maintenance releases and updates (MRs and MUs).

So in our current Web server environment, we run a mix of the latest gold release software (R5), Notes/Domino 6 beta releases, and early release builds that are not yet available publicly. Because Notes.net is also a production Web site with real-world traffic, our testing helps to make Domino a better product for our customers. So, you help to make a better product by visiting our site.

How and what we test
In our commitment to testing, we focus on server stability and performance. We extensively test the Domino Web server HTTP task and related routines. Any software problems that we encounter are immediately reported and usually resolved before the gold release of the product. When severe problems occur on the site, Domino software developers are on-hand to fix them immediately.

Before we put any beta or early release software onto our live, production Web servers, we "test before we test"—we make sure that the release meets our criteria for stability. Availability and performance are important goals of Notes.net, so we never put untested software into production. In fact, no early release or beta software is put into production on Notes.net without two rounds of thorough testing.

We work closely with the Domino Quality Engineering (QE) team, who are already testing Domino releases before they hand them over to us for deployment. This team performs feature and regression testing for Domino. Once they approve a build, we begin performing our own tests with all our site-specific applications, like the Iris Cafe discussion forums, Iris Today webzine, and the Iris Sandbox. In this testing phase, we check to make sure that our applications run as smoothly on beta software as they run on gold release software. If the software meets our criteria, we put it into production on our Web servers, where it receives its final round of testing with you, the Notes.net audience, as our "beta testers."

Production is a true test of how well Domino works because of the volume of visitors we have and the variety of hardware and platforms that make up the Notes.net Web server environment. We run nearly all the Domino-supported platforms, including Microsoft Windows, IBM AIX, Sun Solaris, and Linux. This environment ensures that all Domino-supported platforms are thoroughly tested and perform equally well on our highly trafficked site.

What you can expect
When you visit Notes.net from a Web browser, our IP sprayer, Cisco LocalDirector, routes you to one of our nine Web servers. The LocalDirector is used primarily for server load balancing. Once routed to a server, you retrieve all content from that server during your visit. Our Web servers are clustered to ensure that other servers are available should a server go down or be taken off-line. You'll be routed to another available server if the one that you last accessed becomes unavailable.

Of the nine Web servers that make up Notes.net, four of those servers run beta or early release software, while the rest of our servers run the latest gold release of Domino. If you come to Notes.net and find that performance is slower than usual, you may have been routed to a server running beta or early maintenance release software. We run "debug code"—code that is non-production to detect errors in the programming—on these servers, which slows server performance but helps us to target a problem if one occurs.

Delivering our content
Once a software build makes it into production on the Notes.net Web servers, we work to make sure that our content is readily available to you at all times should a problem arise. We are as committed to delivering content as we are to testing our products. When a problem occurs on any of our servers, we do whatever is necessary to minimize downtime and to avoid interruption of our content hosting.

In addition to our nine clustered Web servers, we also have two servers dedicated to serving native Notes content, two designated registration servers, two servers that host downloadable documentation files, and specialized servers for product demos, Sandbox submissions, and mail. Typically we run the latest gold maintenance releases on these machines to guarantee stability. We designate a backup server in each of these areas for availability.

Content on our servers is well synchronized. Our hub server pushes new content out to both our Web servers and native Notes servers. Cluster replication among the servers synchronizes that content on those servers. No matter which server you access at Notes.net, you will always have the latest content, and no matter how you access our site—through a Web browser or Notes client—all content is in sync.

Your feedback helps
By visiting Notes.net, you help us test our product under real-world conditions. In addition to the feedback that many of you already provide, we encourage you to report problems or error messages that you receive. You can submit a feedback form or send e-mail to webmaster@notes.net.

Please include the server release number in your feedback. To find the server release number, use your browser's "View Source" feature to show the HTML source code. Domino lists the software build number in the HTML header, for example:



By visiting and using Notes.net and giving us feedback, you, the customer, ultimately benefit from your help even more than we do.