Setting up and testing Domino.Doc archiving with Tivoli Storage Manager

Setting up and testing Domino.Doc archiving with Tivoli Storage Manager

by
Dick
McCarrick

Level: Intermediate
Works with: Domino.Doc
Updated: 01-May-2002

Within the normal lifetime of a document, there comes a time when the information it contains doesn't need to be instantly accessible any longer. The document may still have value, so you don't want to delete it entirely. Yet you'd like to ensure such documents aren't consuming your valuable disk space at the expense of other, more immediately needed information. The answer to this dilemma of course, is to archive these documents in an external storage system, where they can remain available for future retrieval.

This article introduces the three methods of archiving and retrieval that Domino.Doc offers and then delves into the details of setting up and testing one method—archiving with the Domino.Doc Storage Manager add-in and Tivoli Storage Manager. This article assumes an intermediate level of experience with Domino.Doc. It also assumes that you have installed and tested the Tivoli Storage Manager server and that the Tivoli Storage Manager client is present on the Domino.Doc server.

Domino.Doc archiving basics
Before we begin, let's be clear on what we mean by archiving, especially how it relates to backing up. Archiving involves deliberately removing files from active disk space and placing them in an external storage medium, usually leaving some sort of tag or other means to locate and retrieve each file as needed. Backing up generally involves making a complete or incremental copy of your disk, usually on a regular schedule, and keeping the copy for a period of time. Then if the document is accidentally deleted (or the disk itself is damaged or replaced), you can restore the files from the most recent backup.

Domino.Doc offers you choices for archiving and retrieving documents. You can enter archiving/retrieval settings directly into the Document Type form. Domino.Doc then uses this information to determine which documents to archive. You can also mark selected documents "by hand." After Domino.Doc archives a document, it keeps a "proxy document" containing document profile and security data for that document in the file cabinet. Later, if you decide to retrieve the archived document, Domino.Doc uses its proxy document to perform retrieval.

After you mark Domino.Doc documents for archiving, you can do the actual archiving and retrieving three different ways. The first way (sometimes called file system archiving) uses three scheduled agents that come with Domino.Doc. The names of these agents reflect the three basic steps in the archiving/retrieval cycle: Mark for Archive, Archive to File System, and Retrieve from File System. These agents can only archive documents from the master file cabinet replica. And although they can request retrievals from any replica of the file cabinet, the documents can only be restored to the master file. For more information on these agents, see the Domino.Doc Administrator's Guide, available for download from the Documentation Library.

A second archive method is available to administrators running Domino.Doc on iSeries (formerly AS/400) servers. This employs the Hierarchical Storage Management (HSM) supplied with Backup Recovery and Media Services for AS/400 (BRMS).

If your server runs Windows NT or Windows 2000, you also have a third Domino.Doc archiving option. This is the Domino.Doc Storage Manager (DDSM) add-in, which archives and retrieves documents on a Tivoli Storage Manager (TSM) server. DDSM is a server add-in task that by default runs daily at 3:00 AM. To use it, you must have a Tivoli Storage Manager client installed on the Domino.Doc server. (The Tivoli Storage Manager itself may run any platform. For information on setting up the Tivoli Storage Manager, refer to the Tivoli Storage Manager Administration Guide.)

The instructions below assume the Tivoli Storage Manager server is available and has been tested, and that the Tivoli Storage Manager client is present on the Domino.Doc server.

Setting up archiving and retrieval
Archiving should be considered a two-part process (or three, if you include retrieval): A document is marked as ready to be archived, and DDSM performs the archive procedure on it. (The third step occurs if you later retrieve the document, at which point the archiving/retrieval cycle starts over.)

One way to configure archiving and retrieval is through the Document Types form. When a document is marked for archiving, the information you enter into its corresponding Document Type determines how it gets archived and retrieved. To do this, open Notes and do the following:

Open the Library database and click the Library Administration button in the left navigation pane. This displays the Library Administration view.
Click the Document Types button in the left navigation pane.
Select the Document Type to configure for Tivoli Storage Manager archiving and retrieval, and open it for editing. The Document Types form appears. This form includes two sections for archive/retrieval settings, Archiving Options and Retrieval Options.
Complete these two sections, as described in the following section.
When you finish, save and close the Document Type form.

Archiving options
The Archiving Options section of the Document Type form lets you specify Domino.Doc archive settings.

Archiving Options section

The "Name of archiving agent" field defines how the document will be archived. In this example, we've entered "TSMArchive" to specify Tivoli Storage Manager. Other agents that could be named here are "Archive to BRMS" (for iSeries) and "Archive to file system."

The "How should documents be marked for archiving?" field is a radio button field that specifies the method used to determine how documents will be marked as ready for archiving. You can select "Do not mark documents for archiving automatically" if you want users to mark documents manually. Choose "Mark old versions for archiving" to archive all versions of a document other than the current version. Another option is "Specify an archiving trigger." This tells Domino.Doc to automatically mark documents for archiving when a specific triggering event occurs. When you choose this option, you must then select one of the following:

"Archive documents older than the specified number of days" archives documents based on the document creation date. After the number of days you enter passes, Domino.Doc automatically marks the document for archiving.
"Archive documents that have not been modified in the specified number of days" archives documents that have remained unchanged during the period of time you specify.
"Archive documents based on a formula" implements a programmatic archiving process. You can, for example, enter a formula that combines trigger events.

The "What should happen to drafts when a version is archived?" field deals with draft documents. Drafts can be an issue for archiving—many people might question archiving a document that's still in the draft stage (and by implication still a work-in-progress). So you can select "Leave drafts alone" to tell DDSM to ignore all draft documents at archiving time. Alternately, you can select "Archive drafts when version is archived" or "Delete drafts when version is archived."

The "Allow archiving of the current version?" applies to document versions. A document may have multiple versions of the same content—for example, a report that has been updated or revised several times by different authors. This option tells DDSM whether or not to ignore all current versions of documents and only archive previous versions.

The "Allow manual archiving?" field defines whether users can manually mark documents for archiving.

The "Subform for agent-specific parameters" field lets you specify an optional subform for entering archiving agent parameters that apply to this Document Type. If your site uses DDSM to archive on a Tivoli Storage Manager server, name this subform TSMArchiveParameters. Then click the Specify Parameters button to enter the appropriate archive settings into the subform. This displays the following subform. (For more information about subforms, see "Creating custom subforms" in the Domino.Doc Administrator's Guide.)

Tivoli Storage Manager Archive Parameters

Tivoli Storage Manager Archive Parameters

In this subform:

Server identifies the Tivoli Storage Manager server. You can enter either the server name or its IP address.
Node Name specifies the Tivoli Storage Manager client node.
Password is the password of the Tivoli Storage Manager client node.
TCP/IP Port selects the port to connect to the Tivoli Storage Manager server.
Management Class is the Tivoli Storage Manager management class. Default is STANDARD.

Retrieval options
After you complete the Archiving Options settings in the Document Type form, go to the Retrieval Options section and fill in the retrieval fields.

Retrieval options

The "Name of retrieval agent" field defines the method by which the document will be retrieved—in other words, whether you will use Domino.Doc agents for file system retrieval, iSeries retrieval, or DDMS with Tivoli Storage Manager. In this example, we've entered "TSMRetrieve" to specify Tivoli Storage Manager. Other options are "Retrieve from BRMS" (for iSeries) or "Retrieve from file system" (for Domino.Doc agents).

The "When should retrieved documents be re-archived?" field specifies the date by which the retrieved documents should be placed back into the archive. Choose "Use original trigger" to use the document type's original archive settings. Otherwise select "Re-archive after a specified number of days," then enter the period of time (in days) after which the retrieved documents are re-archived. Note, however, that documents won't be automatically re-archived if you selected "Do not archive documents automatically" in the Archiving Options section.

The "Send notification via e-mail when retrieval is complete?" field tells DDSM whether to send an e-mail to inform you when it successfully retrieves the documents.

After you complete both the Archive Options and Retrieval Options sections, save and close the Document Type form. You can now test these settings to see whether archiving and retrieval work as you expect.

Testing archive settings
To test whether archiving is set up properly, you must enable the DDSM add-in task, which is disabled during Domino.Doc Server setup. To enable the DDSM task, you can type load ddsm at the Domino server prompt, or you can add the "ddsm" task to the "Server Tasks" entry in your Domino server's Notes.INI file. After you enable the DDSM task, restart your server. Also ensure the appropriate Tivoli Storage Manager server is running.

You are now ready to test your archive settings.

Create a test document
First, create a new test Domino.Doc document.

Create the document and give it an attachment or enter some rich text.
Set up the Document Type form for your test document for DDSM archiving and retrieval. (See the Setting up archiving and retrieval section above for details how to do this.) Within the Document Type form, make sure you have selected TSMArchive as the archive agent, and TSMRetrieve as the retrieval agent. This instructs Domino.Doc to use Tivoli Storage Manager for archiving/retrieval.
Save and close the test document.

Mark the document for archiving
There are several ways you can mark the test document for archiving. Which method you use depends on the software you're using to display the document.

In Notes, choose Actions - Mark for Archive.
In the Binder TOC ActiveX control, right-click to open the Binder TOC menu. Then select Mark Documents for Archive.
In a Web browser, open the document and select Document - Mark For Archive. If this option doesn't appear, check to see that you selected "Allow manual archiving?" in the Archiving Options section of the Document Types form. (See the Archiving options section above.) Note that archiving a document manually overrides the archive trigger settings.

Documents successfully marked for archive appear "locked" in the binder view (that is, a lock icon appears next to their names). Also the Archival Agent field should display the correct name of the system that will perform the archiving. This should be TSMArchive for Tivoli Storage Manager archiving. If this is not the case, open the Document Type form and make sure you entered the appropriate archiving information.

Check the document's status
To ensure the document is properly marked for archiving:

Open the Document Profile for the document and look at the Archive Status to ensure it is set to Marked for Archive.
Press Ctrl+Shift and select View -Go To to open the Go To dialog box. Choose (ArchiveRequests) and click OK. This opens the (ArchiveRequests) hidden view, used by DDSM to identify documents to be archived to Tivoli Storage Manager. The test document should be listed in the (ArchiveRequests) view.
Save the document and exit.

Archive the document
The DDSM task should archive the document the next time it runs. By default, this is daily at 3 AM. If you want to archive immediately, type "tell ddsm archive" at the Domino server console prompt.

As the DDSM task archives documents, it extracts rich text and file attachments and stores them on the specified Tivoli Storage Manager server. DDSM also marks the archived document with a unique Tivoli Storage Manager handle. This handle is used when you retrieve the document.

If the document is successfully archived, an Archived icon (a small image of offline storage media) appears next to its name in the Document view, as shown in the screen below. Also, the Archive Status in the document's Document Profile is set to Archived, and the document Activity Log displays a message that the document has been archived.

The archive icon marks the archived document

The archive icon marks the archived document

If for any reason the DDSM task did not archive the document successfully, check the Domino server console for DDSM-related error messages. Also check the Domino.Doc log. (See the Troubleshooting section later in this article for DDSM-specific server commands and other tips that can help you analyze possible archiving problems.)

If archiving fails, documents will remain marked for archiving. If you decide not to archive the document at this time:

Open the document.
Choose Actions - Unmark for Archiving. (Note that you can't unmark documents after they have been successfully archived.)
Save and close the document.

Testing retrieval settings
To test whether you have set up retrieval properly, do the following:

Make sure the DDSM task is enabled. See the Testing archive settings section above for instructions on how to do this.
Check that the appropriate Tivoli Storage Manager server is running.
In Domino.Doc, select and open the archived document.
Choose Actions - Mark for Retrieval.
Close the archived document.
Open the (RetrievalRequests) hidden view in the corresponding binder database. The DDSM task uses the (RetrievalRequests) view to determine which archived documents need to be retrieved from Tivoli Storage Manager. The request for the document you want to retrieve should appear. (Make sure you are in the binder database when you open the (RetrievalRequests) view. In a document database, this view is always empty.)
Click the document's entry in the (RetrievalRequests) view. The Retrieval request form appears:
The document is now ready for retrieval. The DDSM task will retrieve the document when it next runs, which by default is daily at 3 AM. To run the DDSM task immediately so it can process your retrieval request, go to the Domino server console and enter the "tell ddsm retrieve" command.

After the DDSM task retrieves the document, the Archive Status field in the document's Document Profile is set to Retrieved. If you answered Yes to "Send notification via e-mail when retrieval is complete?" (see the Retrieval options section earlier in this article), DDSM sends a mail message notifying you that the document has been successfully retrieved. Also, the message "Retrieved by Domino.Doc Connection for TSM" appears in the document Activity Log.

If retrieval did not complete successfully, look for any Tivoli Storage Manager error messages on the Domino server console or in the Domino.Doc log. You can also refer to the next section, Troubleshooting.

Troubleshooting
If DDSM archiving and retrieval with Tivoli Storage Manager doesn't perform as you expect, you can examine the Domino server console and the Domino.Doc log for DDSM or Tivoli Storage Manager error messages. Review these messages before calling Customer Support so that you have this information available to give to the Support representative.

In addition, check the Document Type forms to ensure you have entered the correct archiving and retrieval information. (See the Setting up archiving and retrieval section earlier in this article for details.) Also, look at the (ArchiveRequests) and (RetrievalRequests) hidden views in the binder database to see whether they are configured correctly.

You should also check whether the documents are marked as drafts. The DDSM task does not archive drafts that have not yet been turned into versions. If you want to archive drafts, you must first check in the draft as a version. Also make sure the "What should happen to drafts when a version is archived?" option is set properly.

NOTES.INI settings
You may also want to review the Domino server's NOTES.INI file, and check that the following DDSM-specific variables are set in the way you want.

DDSMDailySchedule = <time of day, 0-23> determines the time the DDSM task runs. The default is 3 AM. If you change it, you must enter a whole hour time; a time that includes minutes (for example 2:30) is not allowed.
DDSMTimeOut = <# of minutes> specifies the amount of time (in minutes) that the DDSM task waits before checking whether it is time to begin a scheduled run.
DDSMiteAdminDB= <path> locates the Domino.Doc Site Administration database, relative to the Domino Data directory. The DDSM task opens this database to find libraries.

Server commands
There are also a number of special server commands for DDSM that may be helpful in troubleshooting. You issue these commands by entering them at the Domino server console, in the form "tell ddsm [commandname]."

Caution: Do not use these commands unless you are thoroughly familiar with how they work. Otherwise they may produce unpredictable results. If you have any questions and/or concerns about these commands, please contact Customer Support.

Command	Description
archive	Forces an immediate archive cycle.
quit	Stops the DDSM add-in task.
retrieve	Forces an immediate retrieval cycle.
run	Forces both an archive and retrieval cycle in the same order as it occurs on the archiving schedule.
smarchive [file] [management class]	Archives the specified file. [file] is the fully qualified path of file to archive. Requires previous call to SMCONNECT. [management class] is the Tivoli Storage Manager management class. For example, tell ddsm smarchive c:\mydoc.doc STANDARD archives the file mydoc.doc, with the STANDARD Tivoli Storage Manager management class.
smconnect [version] [address] [node name] [password] [port] [directory]	Establishes a connection to a Tivoli Storage Manager server. [version] is 1 if you are connecting to Tivoli Storage Manager server v3.7 or higher, 0 if you are connecting to the ADSM server provided with the Domino.Doc Storage Manager v2.5. [address] is the network address for Tivoli Storage Manager server. [node name] is the client connection name. [password] is the client password. [port] is the Tivoli Storage Manager connection port. [directory] is the directory were the dsm.opt file resides. For example, tell ddsm smconnect 1 persephone Stephen password 1500 c:\lotus\domino\ specifies the dsm.opt file resides in c:\lotus\domino\dsm.opt. We provide an empty dsm.opt file with Domino.Doc. It should be located in the directory where Domino is installed. We also set the DSMI_CONFIG environment variable to point to the dsm.opt file that gets installed.
smretrieve [destination] [handle]	Retrieves the specified file from Tivoli Storage Manager. Requires previous call to SMCONNECT. [destination] is the location for the retrieved file (includes filename). [handle] is the object handle of the file to retrieve.
smterm	Ends the DDSM session.
verbose	Toggles verbose reporting on/off. Results are reported to the server console only. If problems are encountered with Tivoli Storage Manager archive and retrieval, verbose logging may help provide more information. To activate verbose logging, type "tell ddsm verbose." This command also turns off verbose reporting if it is already enabled.

Conclusion
If you are a Domino.Doc administrator, archiving is often one of the most critical tasks you regularly perform. It balances two important requirements: maintaining a high-performing server with an abundance of free disk space and keeping track of your organization's increasingly large body of documented knowledge. The Domino.Doc Storage Manager add-in task for Tivoli Storage Manager gives you a reliable, robust way to meet these two goals, ensuring you have an adequate supply of disk space while your valuable corporate information remains available for future retrieval and reference—important requirements for any high-performing organization.