Domino 6 IMAP performance by Michael Gazda

Level: Intermediate Works with: Notes/Domino Updated: 03-Mar-2003

The Domino 6 IMAP server offers users multiple options for choosing the format in which to store messages in their mail databases. The choice of format depends on a number of factors—user preferences, type of email system used, and so on. However, the format you select can have a significant impact on performance.

This Performance Perspectives column summarizes performance results we obtained testing each mail format. For each format, we measured its effect on server transaction counts, CPU usage (both in terms of percentage and total CPU minutes), memory usage, and response time. Our goal is to present you with data to help you determine which mail format is best for your environment.

This column assumes that you're an experienced Domino administrator.

IMAP mail storage options
An important component of IMAP performance is the format you choose for storing mail messages. You select this format in two different places:

• The user's Person document
• The RouterAppendIMAPItems variable in the IMAP server's Notes.ini file

Selecting the mail storage format in the Person document
You set up a user for IMAP mail through the Person document in the Domino Directory by choosing IMAP in the Mail system field and then completing the "Format preferences for incoming mail" field. This field offers three options:

• Keep in sender's format (the default)
• Prefers MIME
• Prefers Notes Rich Text

Note that by default, the IMAP server adds IMAP-specific elements to the user's mail file the first time the user logs in (unless the mail file is already IMAP-enabled). Although this is a fairly quick operation, it does result in a small delay the first time users open their mail files. To avoid this, run the server command Convert -e on the users' mail files to enable them for IMAP. (Also note that if you create the mail file through registration, the IMAP server performs the same IMAP-enabling process when users first log in.) For more information about the mail conversion task, see the Domino Administrator help.

The best IMAP performance with Domino (both in terms of server resources and user response times) occurs when you select the Prefers MIME option. When you choose this option, the router converts all incoming messages to the MIME storage format at delivery time. The messages are therefore stored in your mail file in MIME format. This lets the IMAP server quickly serve all information about the document (such as size) as well as the body of the document to an IMAP client because the document is already stored in the necessary MIME format for the client to read.

However, when Notes documents are converted to MIME, they lose all Notes Rich Text (RTF) features, such as LotusScript buttons and collapsable sections. This is important for users who need to connect via Notes in the office (perhaps to interact with Notes messages sent by workflow applications that have RTF-only features), but who require IMAP access outside the office.

As noted previously, the other two mail format choices you can set in the Person document are Prefers Notes Rich Text (RTF) and Keep in sender's format (the default; in R5 this option was called No Preference). These options preserve Notes RTF features, but have a negative impact on a user's IMAP performance because Notes RTF documents are not stored in a format that allows the IMAP server to simply serve the document up to the IMAP client. So when an IMAP client either requests certain properties about a document (or requests the entire document), RTF-to-MIME conversion must occur on the server immediately following the request, but before the server can send the IMAP response. This is an additional layer of work, impacting both client response times and server performance. Additionally, because this MIME information is not stored in the user's mail database with a Notes RTF document, if the IMAP client requests the message again, the server must perform the RTF-to-MIME conversion again. With the best-behaved IMAP clients, this scenario causes a performance hit on the server due to an increased frequency of RTF-to-MIME conversions, and with poorly behaved clients (where previously downloaded information is not stored locally and therefore re-requested often), it can cause a very substantial degradation in server performance.

Setting RouterAppendIMAPItems
To provide two additional choices in mail storage format, Domino 6 introduced the Notes.ini parameter RouterAppendIMAPItems. When you enter the following setting in your server's Notes.ini file:

RouterAppendIMAPItems=1

the router task performs a partial RTF-to-MIME conversion for all Notes RTF mail delivered to an IMAP-enabled mail file. When this conversion occurs, all essential IMAP header information for the message is calculated and permanently stored with the document in the user's Notes mail database at delivery time. However, the message body itself is still stored in Notes RTF format, so features such as collapsable sections are maintained for Notes clients.

This causes a significant decrease in the frequency of RTF-to-MIME conversions when an IMAP client connects to a database where mail is stored in Notes RTF format because all MIME information about the document is now stored with the document (such as the document's MIME size). Thus, when an IMAP user connects to check mail, all MIME information is immediately available to be sent to the IMAP client by the IMAP server. This gives you decreased response time for the client and fewer server resources used because no conversions occur on any documents when the only information needed is document properties. The RTF-to-MIME conversion only occurs on Notes RTF documents with IMAP headers pre-calculated when the IMAP client requests the entire message to read.

RouterAppendIMAPItems affects users whose Person documents specify either Keep in sender's format or Prefers Notes Rich Text. As you can probably guess, this variable has no effect on users who already store mail in MIME format. Also note that when RouterAppendIMAPItems is enabled on a server, the Convert -h command can be run against an IMAP user's mail file to add IMAP header information to previously existing mail.

Note that Convert -h can potentially take a very long time to run on a user's mail file because it attempts to convert every message in the file. Before you use Convert -h to add IMAP headers to previously existing mail in multiple mail files, perform this operation on a single mail file to determine how long it takes and use that estimate to calculate the time required to convert all mail files. We also recommend not using a wildcard to convert all mail files at once.

Test setup...
We conducted a study in which the only variable changed from one test to the next was the type of mail storage format the Person documents were set to prefer and whether or not the Notes.ini parameter RouterAppendIMAPItems was in effect. This resulted in five different mail storage formats:

• MIME
• Notes Rich Text, no MIME headers
• Notes Rich Text, MIME headers (in other words, RouterAppendIMAPItems=1)
• Sender's format, no MIME headers
• Sender's format, MIME headers (RouterAppendIMAPItems=1)

Our test environment ran Domino 6 on an IBM H50 with four 332 MHz processors and 3 GB of RAM running AIX 4.3.3. We used an internal test tool to simulate IMAP users. This tool closely mimicked the way the Microsoft Outlook Express 6 IMAP client handles IMAP commands. We ran scripts simulating busy users who checked mail, read mail (always reading new mail with a small chance of reading a previously existing message), and deleted mail frequently. The test simulated 1,200 concurrent IMAP users with a total of 4,500 registered users on the IMAP server. All users on the server had their mail preferences configured identically for each run.

In our tests, a percentage of the mail routing to the IMAP server was destined for users on another server with a percentage being delivered locally. Local mail on the IMAP server was randomly delivered to a pool of 3,600 users (which included our 1,200 IMAP users) during the test period. All new mail being routed was 12 percent MIME; the remainder was Rich Text. (We derived these percentages based on data obtained from actual Notes mail environments.)

All existing mail in our user databases was in the correct format for the type of test, as set in the users' Person documents and through the Notes.ini parameter RouterAppendIMAPItems. Each existing mail file contained 1,000 messages with a wide variety of sizes and content. About 10 percent of the mail had attachments of varying types and sizes. A total of 100 messages were in the Inbox, and 900 messages were spread across nine other folders.

We ran tests over multiple days with a complete restore and reboot of every test system (drivers and servers) between each test. Additionally, multiple test runs were performed of each type of test to make sure we received consistent results.

...And the results
Our first chart shows the transaction and the mail routed counts for each mail type tested. As you can see, the workload was very consistent across all of the tests. Note that not all transactions in the test are displayed here—only a sample is shown for demonstration purposes.

Next are three charts that list the average CPU, CPU minutes per process, and memory consumption. This was gathered with vmstat and ps. First we show average CPU consumption:



In this and all other test results shown in this section, the incoming mail ratio was 12 percent MIME and 88 percent RTF. When the option Prefers MIME was selected, 88 percent of the mail was converted at the Router. When Prefers Notes Rich Text was selected, 12 percent was converted at the Router.

As the previous graph shows, MIME format consumed the least amount of CPU in our tests (46.5 percent), while Prefers Notes Rich Text and Keep in sender's format (labeled "No Pref," as it was called in R5) used the most CPU (65.3 percent and 63.85 percent respectively). Note that setting RouterAppendIMAPItems=1 in the server's Notes.ini file reduced CPU usage by an average of more than 10 percent.

The next graph shows CPU minutes by process.



Again, for the IMAP server process (by far the most CPU intensive we tested), MIME format produced the best numbers, and setting RouterAppendIMAPItems=1 significantly reduced the numbers for Prefers Notes Rich Text and Keep in sender's format. Also note that the increase in router CPU is more than offset by the gains in IMAP CPU.

Now let's look at memory usage, which as you can see is fairly consistent across all format types:



Finally, here are the average transaction response times. Each transaction was determined from the end user perspective. This means that a transaction encompasses all IMAP commands that Outlook Express issues for a particular user action:



As you can see, pre-calculating the IMAP headers through RouterAppendIMAPItems=1 helps not only the server resources, but also the end user response times.

Conclusion
Our tests confirm that when choosing a format to store IMAP messages:

• Selecting Prefers MIME in the Person document produces the best IMAP performance with Domino 6.
• If users need Notes Rich Text features, adding RouterAppendIMAPItems=1 to the IMAP server's Notes.ini file to perform partial RTF-to-MIME conversions can significantly improve performance over the Prefers Notes Rich Text and Keep in sender's format options.

We hope you found the information in this article helpful. Please let us know what you think, and feel free to suggest other performance-related topics you'd like us to examine in future columns.


ABOUT THE AUTHOR
Michael Gazda has worked in Product Introduction Engineering for over one year on projects involving network compression, IMAP, and iNotes Web Access. Previously, he spent over three years as a Senior Technical Support Analyst in Lotus Notes/Domino Technical Support where he handled a variety of issues including database design, mail routing, and server administration. He is a Certified Lotus Professional in both database design and server administration for both R4 and R5. When not at work, Michael attends Boston University part-time, earning his BA in Computer Science.