by Richard Schwartz

Level: Beginner Works with: All Updated: 31-Mar-97

[Editor's note: This is the second article in a two-part series about the architecture of the Domino Web Server. The first article focused on the basic structure of the server and described some of the behind-the-scenes mechanisms that Domino uses to respond to requests for Web pages. This second article looks more closely at the back end of the server and describes key command handlers and how they present static pages and modifiable forms, accept fill-out forms, perform searches, and execute agents.]

Introduction
One of the most crucial concepts in Notes is the concept of a Notes "form." On the Web, a form is an HTML construct that describes a portion of a page that can be edited on the browser so that information can be sent to the server. In Notes, a form is a design element that provides a WYSIWYG description of all the static text, graphics and other objects, formulas, references to stored data, and editing controls that a client needs in order to present a document to a user. The Domino Web Server uses a Notes form to translate a document to an HTML page when presenting information for read-only purposes, and to an HTML form when presenting information for document creation or update through a browser. Throughout this article, I will differentiate between the two by referring to either a "Notes form" or an "HTML form." For readers unfamiliar with other basic Notes concepts, a glossary sidebar is available that introduces these concepts.

Domino.Stateless.Operation
An important point to bear in mind while learning about any Web environment, including Domino, is that current Web technology is "connectionless," meaning that every transaction between client and server starts from scratch. The server doesn't keep information on hand about previous client activities, and a client keeps only minimal information on hand about its previous dealings with a server. Browsers are frequently referred to as "thin" clients, meaning that they were designed to have very little built-in intelligence above and beyond what it takes to send and receive information and present it to the user. Until Java and ActiveX technology came onto the scene, browsers relied completely on Web servers to do any processing of information. The designers of early Web technology believed that clients would frequently go from server to server to server, seeking information. Maintaining connection information for each server seemed wasteful since many servers would never be re-visited, and it would have made the clients a little "fatter", so it just wasn't done that way.

A connectionless client-server architecture requires that a server be "stateless." The server doesn't remember the last thing it did for a particular client. That means that every page sent from the server to the client must carry all the information required by the client to completely describe any actions that the client might want to take next, and that on the next transaction this state information must be transmitted back up to the server so that the server can construct an appropriate context in which it executes the transaction. If this seems wasteful to you, take heart in the fact that the committees responsible for Web protocols have established a connection-oriented alternative to the current architecture in HTTP 1.1. But until this capability is in widespread use on Web servers and browsers, most Web application development must proceed under the constraints imposed by a stateless environment. One of the most important ways that Domino makes developers' lives easier is that it handles all the details of representing, transmitting, receiving, and interpreting the state information that applications need.

Stateless operation even extends to authentication. Web servers do not remember IP addresses associated with user's names and passwords. Instead, they create "credentials" that are passed back through HTTP to the client, which passes them back up in subsequent HTTP requests to the server. Domino handles authentication automatically based on information in the server's Public Address Book and the ACL attached to each Notes database, and Domino's replication feature easily synchronizes authentication information across multiple servers so applications can be moved without worrying about whether users will still be able to use them.

Domino.Objects
In Part 1 of this article, I mentioned that the Domino Web Server owes much of its efficiency to its clean, object-oriented design. It is built around a series of objects that model elements of the Notes database and document structure, define their presentation, and provide a runtime environment. These objects provide methods that implement the necessary conversions and computations that allow a browser -- which has little intelligence of its own -- to "run" an application. Other Web servers have no built-in data model, presentation paradigm, or application execution environment, so programmers have to spend time building their own data models, deciding how to present information, and adapting their architecture to the limited capabilities of the Common Gateway Interface specification or ISAPI. Domino excels as a Web application environment because it removes these distractions and allows developers to concentrate on modeling the problem at hand.

The object model used internally in the Domino Web Server closely follows the "user experience" of a Notes user. It includes objects to represent top-level objects like servers, databases, and navigators, Notes forms, views, documents, and all the lower-level elements that make up these things, like rows and columns in views, items in documents, rich text and fields in Notes forms, and paragraphs, tables and even colors and fonts in rich text. In response to a URL request, the server locates the target specified by the URL -- which could be any of the top-level objects -- builds these objects in memory along with any lower-level objects that they include, and then activates the GenerateHTML method associated with each object, specifying a read/edit argument that determines whether the method should follow rules for translating to HTML pages or HTML forms. As mentioned in Part 1, an intelligent HTML engine frees the individual object's methods from having to worry about a lot of the syntactic and semantic rules of HTML, so the methods can concentrate on their specific jobs without having to worry about the context in which they are called.

In many cases, the objects representing elements of a Domino application must generate HTML hotspots. Operations like creating a response document, navigating to the next document, or expanding a view category are all context-dependent, but of course the server is stateless. Therefore, the HTML generated by these objects will always contain whatever information is necessary for the server to figure out the context if the hotspot is activated. For example, the hotspot for a twistie controlling expansion of a category in a view will specify a URL containing the view name, the ?OpenView command, an &Start argument indicating the position in the view of the first document that should be displayed, an &Count argument indicating how many documents should be displayed, and an &Expand or &Collapse argument.

Domino.Commands
Users interact with Domino applications through commands issued as URLs. Each command specifies a top-level object and activates the methods needed to carry out a request and generate HTML output. The Domino Web Server implements many commands. The commands related to the display and modification of documents are worth a closer look. These are the commands that enable interactive applications, so the similarities and differences in their implementation are of particular interest.

Three commands display the contents of a document: ?OpenDocument opens an existing document in read mode, ?EditDocument opens an existing document in edit mode, and ?OpenForm opens a new document in edit mode. The implementations of ?OpenDocument and ?EditDocument are very similar. Both execute the following sequence of steps:

1. Opens the existing document.
2. Reads all items from document into memory.
3. Loads the Notes form referenced by this document.
4. Scans the Notes form and executes default value formulas for any items that are not yet in memory, and executes all computed field formulas and add/update items to the in-memory document accordingly.
5. Runs the appropriate agent if an item named $$QueryOpenAgent exists.
6. Renders the Notes form into an HTML page or HTML form using the item values from the in-memory document.
7. Frees the in-memory document and all items it contains.

The only significant difference between ?OpenDocument and ?EditDocument commands is in step 6, where ?OpenDocument instructs the GenerateHTML methods to respect read-mode hide attributes contained within the Notes form, and to create HTML for a read-only page. ?EditDocument instructs these methods to respect edit-mode hide attributes on the Notes form and to create an HTML form.

The ?OpenForm command executes a similar sequence of steps:

1. Creates a new document in memory.
2. Loads the Notes form referenced in the URL.
3. Scans the Notes form and executes all default value formulas and computed field formulas, adding items to the in-memory document.
4. Runs the appropriate agent if an item named $$QueryOpenAgent exists.
5. Renders the Notes form into an HTML form, respecting edit-mode hide attributes and using item values from the in-memory document.
6. Frees the in-memory document and all items it contains.

The last step of the procedure for all three commands frees the in-memory document and its associated items. The ?EditDocument and ?OpenForm commands do not cache the documents for later use because that would violate the stateless nature of the a Web server. If and when the browser sends edited information back, the Domino Web Server will re-establish all the data structures necessary to handle the updates.

?CreateDocument and ?SaveDocument are the two commands that receive HTTP POST data generated by browsers when a user presses the Submit button on an HTML form and saves the data in Notes documents. Submit buttons on HTML forms generated by the ?OpenForm command send the ?CreateDocument command, and Submit buttons on HTML forms generated by the ?EditDocument command send the ?SaveDocument command. The two commands follow similar sequences of steps. The steps for ?CreateDocument begin with:

1. Creates a new document in-memory.
2. Opens the Notes form referenced in the URL.
3. Scans the Notes form and executes all default field formulas and computed value formulas, adding items to the in-memory document.

The steps for ?SaveDocument begin with:

1. Reads the existing document referenced in the HTTP POST data.
2. Opens the Notes form referenced in the URL.
3. Scans the Notes form and executes all default field formulas and computed value formulas, adding items to the in-memory document.

Then both commands continue on essentially the same path:

1. Creates items for all data sent in the POST data.
2. Scans the Notes form and executes all translation, validation and computed value formulas, updating items in the in-memory document and returning validation errors as HTML.
3. Scans the Notes form and eliminates any computed-for-display items from the in-memory document.
4. Runs the appropriate agent if an item named $$QuerySaveAgent exists.
5. Saves the in-memory document to the appropriate database file.
6. If the $$Return item exists, runs the appropriate formula from the Notes form, and if there is no output from a QuerySave agent, sends the result of this formula back to the browser.
7. Deletes the in-memory document and all associated items.

One thing worth noting is the fact that formulas that a user might have thought executed only once at the time that an HTML form was sent by either the ?EditDocument or ?OpenForm command will actually execute again when ?CreateDocument or ?SaveDocument is processed. This can result in some confusion if these formulas have time-dependent values, or are based on lookups of constantly changing information.

Domino.Searches
The ability to perform full-text searching of documents, powered by the Verity search engine, is a standard feature of Domino. Webmasters of Domino servers can easily set up search capabilities for individual applications or for sets of applications running on the same site. A Search Site database is used to configure the latter.

The ?SearchView command in a URL can either directly submit a search, or return a special HTML form that sends search criteria to the server as HTTP POST data. In both cases, the resulting search covers a single application's database. The word "View" here refers to the fact that the search results are returned in the format of a Notes view that has been defined in the application. The ?SearchView command handler does some preprocessing to the information that comes in the HTTP POST data, feeds it into the Verity search engine running on the server, and receives a data structure known as a "collection" back as the result. The collection provides information about each document that was a hit on the search. The command handler builds objects to represent the view specified in the URL and its rows and columns, referring to information in the collection in order to populate the objects. The GenerateHTML methods for the objects are then invoked much as they would be when rendering the results of an ?OpenView command.

The ?SearchSite command in a URL can also either directly submit a search or return a specially configured HTML form that sends search criteria to the server as HTTP POST data. In both cases, the resulting search covers multiple application databases as configured in the Search Site database referred to in the URL. In this case, the results come back from the Verity search engine a little differently. Instead of coming back as a collection, the results come back as a linked list of doclinks. The command handler code builds this directly into HTML, following each doclink to the appropriate database and populating the resulting page with links whose text is drawn from the corresponding database's default view.

Conclusion
There is, of course, a great deal more to the architecture of the Domino Web Server than can be covered in two short articles. In particular, there are many subtle details to the actual process of generating HTML on-the-fly from Notes data that might be interesting, but are beyond the scope of this series. The details behind the translation of various @commands and @functions in Notes formulas fall into that category.

But for the larger picture, these articles give you a framework that you can use to make a lot of logical inferences about other aspects of the Domino Web Server. For example, based on the information presented, you might infer that processing of a $$ViewBody field in a Notes form instantiates objects representing a view and its rows and columns, sets properties that relate these objects to a collection of documents in the actual view, and then invokes the GenerateHTML methods for these objects, and you would be quite right. It's a cool architecture, and a great success at making it easy to develop cool interactive Web applications.

ABOUT THE AUTHOR
Richard Schwartz is founder and President of RHS Consulting Incorporated, a Lotus Business Partner and a member of the Penumbra Group. He is a Certified Lotus Professional, Editor of Lotus Solutions Now, contributing writer for Lotus Notes Advisor, and a frequent contributor to Notes In a Nutshell and other Notes-related publications. His activity in various Notes-related forums on the Internet led to his being honored with a Lotus Business Partners' Beacon Award in January 1995. A key contributor to Email, Directory and other network application product development at Wang Laboratories from 1983 to 1991, Rich has nearly fifteen years of hands-on experience with groupware technology.

Copyright 1997 Iris Associates, Inc. All rights reserved.