OK, maybe we're not exactly like the Brady Bunch. But this is a story, anyway, of how a typical day might unfold behind-the-scenes at Notes.net. Of course, each day is never quite the same as the next. So, we've combined some typical events into one "virtual" day. You'll see how our relatively small staff works together to create and maintain Notes.net, which is both a production site and a test site. In fact, our main challenge daily is balancing the needs of a production site (content, a feeling of community, and site stability) with the needs of a test site (testing new builds of Domino for reliability). You'll also see how much we rely on you, our loyal readers and forum participants, as well as other developers within Iris, to help us meet this challenge.

For more information about our goals and whatnot, see the "About this Site" page.

7 AM Just a little bleary-eyed, Barb Mathers, manager of the Notes.net site, arrives early to start wrapping things up before her upcoming trip to Portugal. As the "visionary" for the look-and-feel of Notes.net, she begins this morning with her "design hat" on. She first brainstorms some new ideas for graphics for the site, and then sends these ideas to our designers at Stormship Studios so they can work their magic. Then, after grabbing some coffee, Barb starts checking out our Webmaster mail (which is something that we all do throughout the day) and prepares for her meeting with members of the Lotus Web team.

Meanwhile, the rest of the team is just waking up, showering, and/or making the morning commute to Westford. The average commute time is around 40 minutes. Some of us spend that commute time listening to Howard Stern, but we're not telling who! Others listen to National Public Radio (NPR) or the local alternative station, while making their plans for the weekend.

8 AM With Palm Pilots, laptops, and pagers in tow, other members of the team begin to stroll into work. Shawn Harris, Notes.net network administrator (up early because of his newborn baby boy), begins his day with a hot cup of coffee and then moves into our frigid lab. His first task is to distribute the download files for the new Quarterly Maintenance Releases (QMRs) on our FTP servers. GTE Corp. actually hosts the FTP servers in three locations -- Chantilly, Va.; Palo Alto, Calif.; and London -- so users can download the files from the nearest geographic location. The Virginia server contains both the domestic and International versions of the incremental installers, while Palo Alto has just the domestic versions and London has just the International versions. Shawn handles most of the "blue collar work" of the QMRs, while Ashley Ryden, Notes.net application developer, deals with the Download application itself. (We'll meet up with Ashley in a minute.)

Shawn and Barb catch up on the latest site issues in the Notes.net lab.

After getting his morning coffee (we're a big coffee group!), Ralph Adame, Notes.net system administrator, joins Shawn and Barb in the lab. Ralph starts his daily review of the essential Notes.net site tasks, such as HTTP, replication, clustering, and mail routing. He checks the Statistics database, which automatically notifies him if there are problems with database or system resources. (For more information on statistics and events, see the Iris Today article "Tips on monitoring Domino statistics and events.") In addition, he checks each server's Notes Log to see whether any unusual Domino/Notes events occurred in the past 24 hours, such as replication failures, mail delivery/routing failures, "Document not Found" errors, cluster errors, and so on.

8:30 AM Working from home this morning, Ashley fires up her laptop, wearing her workout clothes from her morning jog around the Esplanade. As keeper of the Download application (one of the key reasons Notes.net was founded), Ashley begins her day by checking our Webmaster mail for Download-related messages. She responds to users with ideas for solving their download problems, and forwards technical questions (unrelated to Notes.net) directly to Lotus Support. In addition, she records the download problems in our Notes.net Projects database as ways to improve the download application in the future. Then, she begins to tackle some other messages. (For more information on how we manage our Webmaster mail, see the article "Notes.net Exposed: Gathering feedback on your Web site.")

Meanwhile, our lead application developer, Murray Hurvitz, begins putting out the first of many fires for the day. Today's challenge involves reshaping every database on our site that allows a user to submit a document from it. In Webmaster mail, several users have commented that when they try to submit a document (for example, their Registration or a topic in the forums) and then receive an error page about a missing field value, they cannot simply click the Back button on their browser to return to their document. Instead, they have to start all over with a new blank page, which can be very frustrating! So, this morning, Murray begins investigating ways to validate that all fields are completed before users submit the page.

9 AM In her dark cave of an office, Susan Florio, Notes.net writer, begins her day by editing articles submitted for the next issue of "Iris Today." (Susan and Barbara Burch, the other Notes.net writer, trade off the "editor's hat" each month.) Susan does her best editing first thing in the morning, when she has a clear head. In addition to editing for style, Susan helps writers organize the structure of the article, create interesting introductions (to draw readers into the article), develop at least two graphic ideas, and get the content technically reviewed by a developer within Iris. Of course, she does all this work in a Notes database! (For more information on our content planning database, see the article "Notes.net Exposed: Planning content for your Web site.")

Like Ashley, Barbara also begins her day by reading Webmaster mail, but she's mainly concerned with the article-related messages. These messages may be submitted via the form at the end of all "Iris Today" articles, the Feedback form available from the link at the bottom of each page on our site, or the various mail addresses (such as, ideas@notes.net). She forwards technical questions to the writer of the article, and implements any suggested corrections. In addition, she adds potential article ideas from customers to our planning database. Then, she catches up with Susan on things: the status of the articles underway, new content ideas, weekend plans, Susan's new house...

10 AM Jumping onto the Web, Barb notices that the front page of the site is not displaying correctly on one of the Web servers. She suspects that the problem may stem from the fact that Shawn rolled out a new build on the server the night before. As part of the official release criteria, each Domino build must run successfully on Notes.net for at least one week. This means that we always have three codestreams running on the servers on our site -- R4.61x, R4.62x, and early R5 builds.

So, after her first reaction -- yelping out "Oh, my head!" -- Barb calls in Staci O'Neil from our Quality Engineering (QE) team to take a look at the problem. They discuss the details, so Staci can then go back to do her own testing. Staci uses a sample copy of the database to investigate the HTML being generated by Domino, and then gets in touch with the appropriate developer. She has two goals: to find a work-around for us, and to make sure that a fix gets into the product.

10:30 AM Murray sticks his head into Barb's office to confer about the validation issue. He explains that he can write a validation script, using JavaScript, that checks that all fields are successfully completed when users try to submit a document. If a field is not filled in, a dialog box appears (instead of an error page) prompting users to complete the particular field. This way, users can return to their partially completed document, without having to start all over with a blank page. The side benefit will be that modem users will get an instant response from their browser, instead of having to download a separate error page. Barb replies, "Great! Go for it!"

Our site statistics show that our peak traffic is occurring right at this time, 10 - 11 a.m. So, it's a very important time for everything on the site to be running correctly! Fortunately, when things do break, you guys are great about letting us know about it -- as we'll soon see...

11 AM Susan Shaye and Chris Petersen, other members of the Notes.net QE team, review how the latest builds are running on the servers in the lab. Susan begins creating her weekly report that summarizes the status of the hardware and builds running on the servers. Using Lotus 1-2-3, she also calculates Mean Time Between Failure (MTBF) statistics for all the servers. These statistics show the reliability of the servers over a period of 60 days. This way, she can better recommend when we should upgrade a particular server to a new build (balancing the needs of testing the build versus keeping the site up-and-running).

Susan and Ralph work in the Notes.net lab, while Chris looks on.

As Ralph does his duties in Webmaster mail, such as helping users with forgotten passwords and connectivity problems, Shawn checks out the forums in the Iris Cafe. Both Shawn and Ralph monitor the forums to answer customer questions, especially those related to the site. This morning, they discover messages in both the forums and Webmaster mail about the forum databases being out of sync -- that is, topics that appear in one replica of the database don't appear in the other replicas. (Confused? Well, although Notes.net may seem like it's hosted from one server, we actually run the site from several servers grouped in a Domino cluster. We use event-driven cluster replication to keep the servers in sync. For more information on how and why we use clusters, see the article "Notes.net Exposed: Using Domino clusters for your Web site.") So, Shawn and Ralph begin to look into what's going on with the cluster.

12 (noon) Time for a leisurely lunch? Not exactly. Barb's off to do more collaborating with the Lotus Web team, while Susan Florio does her aerobics thing. Ashley strolls in, asking "How are y'all doing? Anyone fixin' to go to lunch?" While Shawn and Ralph stay hunkered down in front of the servers, the rest of us grab lunch in the busy Iris cafeteria.

1 PM Everyone meets for our bi-weekly team meeting. Other than talking about today's crises (uh, I mean "challenges"), we each report the status of our various projects. Many of the projects involve R5. For example, Barb's working with the project managers to set up the R5 download area of the site (the layout and content plans), while Murray's working on the R5 download application. Barbara and Susan Florio talk about the R5 articles scheduled for "Iris Today."

Susan Shaye excitedly tells us about her plans to set up a test environment identical to the configuration of the Notes.net site, with the same types of servers, builds, databases, and user loads. She hopes that this will help the QE team to assess how builds run in a simulated Notes.net site, before launching the builds on our live site.

2 PM Barb, Shawn, Ralph, and Murray get together after the team meeting to talk more about the cluster replication issue. They first talk about the recent changes that were made to the site, and try to figure out why one of the changes may have caused a cluster and a standard replication to break. They decide on a plan of action to determine whether the problem's with the cluster configuration, the setup of a particular server, the actual server hardware itself, or perhaps, the Domino build running on one of the servers.

On the content side of the site, Barbara researches material for her latest article. She sets up meetings with the appropriate developers, checks the LearningBytes and User Assistance parts of Notes.net for related links, and searches the Web for more background information. Meanwhile, Susan is busy working on the "History of Notes and Domino," a major project she's worked on since February (and which was recently published on the site)! The project chronicles each release of Notes and Domino beginning with Release 1.0. Today, she's working on nailing down the final design, which uses a combination of frames and JavaScript.

Barbara and Susan check out the layout of the "History of Notes and Domino."

3 PM Ashley and Murray meet to discuss the design of new site applications. Ashley's current project is to re-design the QMR/QMU download database, which should be live on the site in August. She gets Murray's input on her work so far. In addition, they brainstorm other applications that will better support the site, with the major goal of automating things as much as possible. For example, Ashley plans to design a repository database for our site templates, so team members can check out templates when they want to make a design change.

Murray and Ashley map out a new site application.

4 PM Barb fires up the cappuccino machine in our "jazz" lounge, and Ralph is first in line for the caffeine fix. Shawn and Ralph tell Barb about a few weird things going on with the cluster, such as some databases not having an administration server listed in their database properties and database ACL's not replicating correctly. Ashley sits calmly by in our massage chair thinking about how to better improve the QMR download process.

5 PM The early birds begin to leave for the day, but work on the site continues. Barbara finds that she gets her best writing done at the end of the day. Barb works out in the Iris gym, and finds herself coming back to her office to work for a few more hours.

Shawn and Ralph think they've found the solution to the cluster problem, leading back to Shawn installing a new build of Domino on one of the site servers the night before. Basically, the new build caused both standard and cluster replication for the entire site to break. Shawn restores the machine to the previous build, while Ralph recreates the entire cluster -- which fixes the problem!

Into the night Although we may leave work, the work never leaves us! Most members of the team stay connected with pagers and laptops. And, with our global audience, the site stays active throughout the night. We each try to sleep peacefully, but may awaken some nights in a cold sweat -- with Ralph thinking that his beeper has gone off, Ashley fearing that the download URLs are not assigned to the correct FTP server, Susan and Barbara suddenly knowing that every word in their articles is misspelled, and Barb's head finally exploding from the pressure.