 |
iNotes Web Access on Solaris performance analysis
by Dick McCarrick,
Dave Curley,
Mark DuFresne,
and Robert Schreiber
Level: Intermediate
Works with: iNotes Web Access
Updated: 11/04/2002
Related link:
More Performance Perspectives
|
|
In the article "iNotes Web Access performance analysis," published in our January 2002 issue, we examined the peak benchmark numbers with the R5iNotes workload on several IBM hardware/OS configurations. These included AIX 4.3.3 running on IBM RS/6000 S80 and B80 servers, z/OS V1R1 running on s/390 Turbo G5 and zSeries Model 2064-116 servers, and Windows 2000 SP1 running on the Netfinity 8500R server.
We recently conducted a similar performance analysis, using the same R5iNotes workload on a 6x400 MHz Sun E4500 server running Solaris 8. In our tests, we ran four configurations, simulating environments consisting of:
- All iNotes Web Access users
- All R5 Notes Mail users
- Three R5 Notes Mail users to every one iNotes Web Access user
- Equal numbers of iNotes Web Access and R5 Notes Mail users
This article discusses the results of these tests. We describe the load an R5iNotes user creates for the Domino server on Solaris, compare it to the load generated by an R5Mail user, and show what happens when you mix both kinds of users on the same server.
We assume you're an experienced Domino administrator and are familiar with managing Notes Mail files.
Test setup
This section describes our test hardware/software configuration and the NotesBench workload we used.
Hardware and software setup
In our study, we ran Domino 5.0.8 on a Sun E4500 server configured as follows:
Sun E4500 server |
| Processors | 6x 400 MHz |
| Memory | 6 GB |
| Domino data disks | Sun T3 Fiber Channel Arrays with Raid 0&1 |
| Network | 100 Mhz full duplex Ethernet |
| OS | Solaris 8 |
| Domino | 5.0.8 |
Each client PC used in our study was equipped with:
- One 10/100 Ethernet adapter
- 9 MB to 30 GB disk
- 500 MB to one GB RAM
Mail database files were evenly distributed over four 137 GB mail directories. Domino was installed with the default settings, except the Number of active threads setting in the HTTP tab of the Server document was set to 90. (Default is 40.) The Domino Directory we used contained 6,000 users, simulating either iNotes or R5 Mail (depending on the test).
The NotesBench workload for iNotes Web Access
The NotesBench Consortium makes available a selection of workloads for vendors to measure Domino server performance on a variety of hardware configurations. These workloads cover messaging protocols, such as IMAP, NRPC, and HTTP. In the tests discussed in this article, we used two workloads provided by the NotesBench Consortium: R5Mail (which simulates Notes Mail users), and R5iNotes (which simulates browser-based users who access their Domino mail via the Web with iNotes Web Access).
The NotesBench R5iNotes workload was introduced in 5.0.8 and simulates the following:
- Every 15 minutes a user will:
Open the Inbox
Read the first message
Delete the first message
Read the second message
Read the third message
Read the fourth message
Read the fifth message
- On every sixth loop, the user will:
Create a new message
Add three recipients
Submit the message
Note that the R5iNotes workload does not include Calendar and Scheduling.
To generate the load against the server, we used a series of 800 MHz PIII drivers. This class of driver can easily simulate up to 1,000 NotesBench users per driver. For these tests, we used eight drivers to simulate 6,000 users.
Test results
Below are numbers we obtained when testing our four simulated user populations (all Notes Mail, all iNotes Web Access mail, 75% iNotes Web Access/25% Notes Mail, and 50% iNotes Web Access/50% Notes Mail). As always, you should keep in mind that these are laboratory results derived from tests run with simulated users, and as such, we can't guarantee that your site will experience precisely the same numbers.
100% R5Mail workload
To establish a baseline, we ran a test with the R5Mail workload only, simulating an all Notes Mail environment. This workload is very similar to the R5iNotes workload, but includes some Calendar and Scheduling events:
| Users | 750 | 1500 | 2250 | 3000 | 3750 | 4000 | 5250 | 6000 |
| Response time (sec) | 0.0170 | 0.0200 | 0.0235 | 0.0280 | 0.0375 | 0.0500 | 0.0660 | 0.0885 |
| % CPU busy | 6 | 10 | 14 | 17 | 21 | 26 | 31 | 35 |
| Transactions per minute | 967 | 1991 | 3051 | 4143 | 5284 | 6407 | 7559 | 8772 |
| Avg disk I/O (KB/sec) | 1254 | 2301 | 3604 | 5386 | 7954 | 10,972 | 14,263 | 17,725 |
| Network I/O (KB/sec) | 73 | 144 | 220 | 299 | 378 | 457 | 538 | 620 |
100% R5iNotes workload
iNotes Web Access uses a simple browser as the client. As a consequence, much of the processing done in the Notes client is performed by the Domino server. You can see from the table below that CPU increased in proportion to the number of users, and eventually became the limiting resource. Network and disk usage remained relatively low and were not a resource constraint problem:
| Users | 1250 | 1750 | 2250 | 2500 | 2750 | 3000 |
| Response time (sec) | 0.1227 | 0.1357 | 0.1783 | 0.2270 | 0.6320 | 2.9573 |
| % CPU busy | 33 | 51 | 67 | 76 | 85 | 92 |
| Transactions per minute | 742 | 1038 | 1335 | 1478 | 1624 | 1700 |
| Avg disk I/O (KB/sec) | 1300 | 1020 | 1288 | 1261 | 1447 | 1263 |
| Network I/O (KB/sec) | 314 | 417 | 528 | 576 | 627 | 656 |
In the R5iNotes workload, the server supported 2,750 active users with a response time under one second. The lowest R5iNotes response time (0.12 second with 1,250 users) was slower than the highest R5Mail response time (0.08 second with 6,000 users). CPU requirements were five to six times less with R5Mail than with R5iNotes. At 3,000 users, R5iNotes hit 92% CPU utilization compared to 17% for R5Mail. (This is why the R5iNotes table shows fewer users than other tables in this article. The number of Notes Mail users Domino can support is much higher, so we included larger numbers of users in these tests. To obtain an "apples and apples" comparison between test runs, we suggest you review the 3,000 users column in each table.)
Network traffic for R5iNotes was more than double that for R5Mail. This is due to the fact that iNotes Web Access servers send more than raw data to the client; they also send JavaScript necessary to display data in the browser.
The next two sections display server performance in mixed R5iNotes/R5Mail user environments.
75% R5Mail/25% R5iNotes mixed workload
In this test, we simulated an environment in which there were three R5 Notes Mail users to every one iNotes Web Access user:
| Number of users | 1000 | 2000 | 3000 | 4000 | 5000 | 6000 |
| R5Mail/R5iNotes ratio | 750/250 | 1500/500 | 2250/750 | 3000/1000 | 3750/1250 | 4500/1500 |
| R5Mail response time (sec) | 0.0163 | 0.0220 | 0.0270 | 0.0403 | 0.0613 | 1.5803 |
| R5Mail transactions/min | 975 | 2010 | 3080 | 4161 | 5095 | 6095 |
| R5iNotes response time (sec) | 0.1177 | 0.1390 | 0.1543 | 0.2007 | 0.3223 | 2.0410 |
| R5iNotes transactions/min | 146 | 294 | 440 | 588 | 735 | 864 |
| % CPU busy | 14 | 28 | 42 | 57 | 74 | 93 |
| Avg disk I/O per second (KB) | 1474 | 2782 | 4518 | 7008 | 11,347 | 14,989 |
| Network I/O (KB/sec) | 153 | 271 | 402 | 532 | 662 | 775 |
As you can see, the optimum number of users supported with less than one second response time is between 5,000 and 6,000 users. And when comparing the 3,000 user column in the above table to the commensurate data in the 100% R5Mail test, notice that CPU usage is more than double (42% compared to 17% in the R5Mail test). Network utilization is also higher (402 KB/sec) compared to R5Mail (299 KB/sec).
50% R5Mail/50% R5iNotes mixed workload
In our final test, we ran an environment simulating equal numbers of R5 Notes Mail and iNotes Web Access users:
| Number of users | 1000 | 2000 | 3000 | 4000 | 5000 | 6000 |
| R5Mail/R5iNotes ratio | 500/500 | 1000/1000 | 1500/1500 | 2000/2000 | 2500/2500 | 3000/3000 |
| R5Mail response time (sec) | 0.0163 | 0.0210 | 0.0260 | 0.0533 | 0.2145 | 0.3110 |
| R5Mail transactions/min | 653 | 1330 | 2043 | 2771 | 3073 | 4158 |
| R5iNotes response time (sec) | 0.1220 | 0.1373 | 0.1820 | 0.5137 | 4.2240 | 10.6225 |
| R5iNotes transactions/min | 299 | 594 | 891 | 1182 | 1240 | 1086 |
| % CPU busy | 20 | 39 | 60 | 83 | 92 | 91 |
| Avg disk I/O (KB/sec) | 1385 | 2371 | 3969 | 4304 | 8518 | 7067 |
| Network I/O (KB/sec) | 190 | 350 | 511 | 667 | 713 | 728 |
Comparing the 3,000 user data from the 75% R5Mail/25% R5iNotes table to the one above, notice that CPU usage increased from 42% to 60%. Network utilization has also increased from 402 KB/sec to 511 KB/sec. The optimum number of users supported with less than one second response time decreased from 5,000 to 4,000 users.
How much does an R5iNotes user weigh?
Another way of looking at our data is to consider the resource requirements placed on the server by each user. This can give a rough idea of the server capacity needed to support a given population of users. It also shows the relative "weight" of the different users. Comparing resource usage rates to the increase in user numbers, we produced the following table showing the average resources consumed per each type of user (R5Mail and R5iNotes):
| R5iNotes user between 1250 - 2750 users | R5Mail user between 750 - 6000 users | Ratio between R5iNotes/R5Mail user |
| % CPU busy per user | 0.0347 | 0.00552 | 6.29 |
| Avg disk I/O per user (KB/sec) | 0.098 | 3.137 | 0.03 |
| Network I/O per user (KB/sec) | 0.209 | 0.104 | 2.00 |
In our test environment, adding an R5iNotes user to an existing server required approximately six times the CPU utilization, and twice the network bandwidth, as adding an R5Mail user. Disk consumption and memory were not resource constraint issues.
We can also examine our results graphically. For example, here's a graph of CPU usage, showing the different rates of consumption for our four simulated user environments:
As you can see, R5iNotes consumed five to six times more CPU time than R5Mail. This resulted in CPU usage being the primary bottleneck, as CPU reached 90% between 2,750 and 3,000 R5iNotes users. We can also plot transactions per minute for our four test populations:
This shows R5Mail produces roughly twice as many transactions per minute as R5iNotes. Finally, let's look at network I/O:
R5iNotes consumes approximately twice as much network bandwidth as R5Mail.
Summary
We have examined the performance of an R5iNotes mail workload on a Domino 5.0.8 server and compared it to a similar workload generated by R5Mail. We've also compared performance when we mixed both kinds of mail user workloads on the same server. We hope this information helps you determine your own system resource requirements when you add iNotes Web Access users to existing Domino servers.
Overall, our test results suggest the following:
- Domino scales very well between platforms. When this study is compared to similar studies on other platforms, results are similar.
- A Domino server can support approximately three times more Notes Mail users than iNotes Web Access users and maintain a response time under one second. In our 75% R5Mail/25% R5iNotes test, both types of users reached the one second response threshold at approximately the same time (when the test population was between 5,000 and 6,000 total users).
- R5Mail processed approximately twice as many mail transactions per second as R5iNotes.
- Domino is more CPU-efficient with Notes Mail than with iNotes Web Access. Domino required approximately five times more CPU usage with R5iNotes than with R5Mail. Domino averaged over 90% CPU utilization with 3,000 R5iNotes users, but only 35% with 6,000 R5Mail users.
- R5Mail user response time was faster with 6,000 users than R5iNotes was with 1,250 users. R5Mail response time averaged less than 0.09 second with 6,000 users; R5iNotes averaged slightly more than 0.12 second with 1,250 users.
- Disk and memory I/O were not bottlenecks in our tests. CPU capacity and network bandwidth appeared to be the main resource constraints.
For more performance-related information, check out these Web sites:
And for more on administering Notes mail and Notes/Domino sites in general, consult the Domino Administrator's documentation, which you can download from the Documentation Library.
ABOUT DAVE CURLEY
Dave Curley has been with Lotus since 1985, working primarily in the Lotus IS Network and Unix System Administration departments. Over the past several years, Dave has been involved in performance testing with a current concentration on Sametime.
ABOUT MARK DUFRESNE
Mark DuFresne joined IBM in 1989, originally as a software developer building automation and analysis applications at the Semiconductor Research and Development Center in Fishkill, NY. He has spent the last four years with the Lotus/IBM Software Division in Westford, MA, working on performance and scalability projects, primarily as a database designer, application tools lead, and data analyst. Mark also plays bass in the "Iris Notes" rock band, originally started by the Notes/Domino Iris Development team.
ABOUT ROBERT SCHREIBER
Robert Schreiber has 30 years of computer industry experience in the areas of systems operations, programming, construction, and architecture. Robert joined Lotus 13 years ago, developing patented high-performance data storage and retrieval algorithms for electronic mail systems. He has also worked on system and application performance measurement and analysis. |
 |