[Editor's note: This article is one in our series exposing how the Notes.net site is run. This time, learn about the design of our Lotusphere survey, and how the results will help shape the Notes.net site.]
Running a Web site is a two-way street. We provide information on Notes.net for you to read or download, and forums where you can interact with members of the Notes and Domino community at large. However, the information that comes to us from all of you, our users, is equally important. We value the feedback that you post in the Iris Cafe and submit in our Feedback database, so we can make sure that our Web site serves your needs. Our recent Notes.net Lotusphere survey was another way for us to receive feedback from you about the Notes.net site.
Surveys are useful tools for gathering feedback on your Web site. A survey can have primarily multiple choice questions, short answer questions, or questions where users rate things on a scale. Some surveys branch to different sets of questions based on the user's initial response to certain questions. Many surveys combine several or all of these types of questions.
We tried to keep the design of our survey simple. It was primarily multiple choice questions, and it took about five minutes to complete. This article helps you design a simple survey, similar to the one we designed. If you want to design a more complex survey, or you just want some usability tips for your survey, we recommend Jakob Nielsen's Collecting Feedback from Users of an Archive.
What was the Notes.net Lotusphere survey? In case you missed it on the site, from January 16, 1999 (the kick-off of our annual four-day user conference called Lotusphere), to February 8, 1999, the front page of Notes.net displayed a link to the Notes.net Lotusphere survey. This survey solicited your opinions on just about every area of the Notes.net site. This article takes you through the design of our survey database, and then shows you how the results are helping us to improve the Notes.net site. The survey database is available for you to download so that you can use our template to implement a similar survey on your own Web site (The database requires Release 4.6 or greater).
Looking at the survey
The Notes.net Lotusphere survey was part of the annual Lotusphere coverage on Notes.net. Every year we create a database that highlights the action at the Lotusphere conference. The Lotusphere 1999 database on Notes.net contains footage of the Notes/Domino Developers lab, using the Domino.Remote:CAM tool, links to developer presentations, and daily digests from our roving reporter who summarizes the news from all the hottest sessions. From the Lotusphere database, as well as from the home page of Notes.net, we also provided a link to the Notes.net Lotusphere survey.
The purpose of the survey was to collect information from Notes.net users on how we can make Notes.net an even better Web site in 1999. During this year, we plan to redesign Notes.net to take advantage of the abundance of new features available with R5. However, we first wanted to get your feedback on the areas of our site you love, and the areas you find less useful. We want our redesign to focus on your needs. Maybe, as cool as some of the new features are, they aren't exactly what you look for when you come to a technical Web site such as Notes.net. Maybe there are some new features you can't wait to see us demonstrate. Whatever your feelings were, we wanted to hear them.
When you chose the link to the Notes.net Lotusphere survey, the survey appeared as follows:
The questions in the survey fell into one of 10 areas. First, we collected background and general information. We wanted to get your name and e-mail address, as well as some information about how you use our site. This included information about how long you've used Notes.net, how often you use it, and what sections you find most useful. Next, we included a section that asked questions on our site registration (we have since updated the site registration based on your feedback). Then we had sections that asked specific questions on each of the following main areas of our site:
- Downloads
- Iris Today
- Iris Cafe
We followed these with a section on site performance, an area we are always trying to improve. We concluded with sections on the User Assistance site and the Usability site. At the end of the survey, we provided a section for comments or feedback on anything related to the site.
Designing the survey
The design of the survey database, which allows you to submit a survey through the Notes client or through a Web browser, revolves primarily around one form called Questionnaire. This form contains the questionnaire itself:
Most of the questionnaire uses radio buttons or checkboxes. We wanted the survey to be quick and easy for users to fill out, so we didn't ask them to enter very many text responses. We used radio buttons when we wanted users to select only one response to the question and checkboxes when we wanted to allow them to choose multiple responses. If you allow users to choose multiple responses, some of the percentage totals for your questions may turn out to be more than 100 percent. Keep this in mind if you choose to use checkboxes.
We have a computed-for-display field that automatically puts the date at the top of the document. Then there are editable text fields that allow users to enter their name and e-mail address. In the section that asks questions about the User Assistance site and in the Other section, we used editable rich text fields to allow users to enter their own responses to questions.
The Submit button at the bottom of the questionnaire allows users to save their answers in the database. The following LotusScript code appears in the Click event. This functionality could have been written using a formula, but we chose to use LotusScript in case we decide to add more complex functionality at a later time:
Sub Click(Source As Button)
Dim uiwksp As New NotesUIWorkspace
Dim uidoc As NotesUIDocument
Set uidoc = uiwksp.CurrentDocument
Messagebox "Thank you for taking the time to fill out this questionnaire!!"
Call uidoc.Save
Call uidoc.Close
End Sub
After a user clicks the Submit button in the Notes client, a message appears that thanks them for taking the time to fill out the questionnaire. Notice that there is a hidden $$Return field at the bottom of the form. This field allows submission of the form from the Web. Using a hidden $$Return field also gives you the option of executing a formula when a user submits the form over the Web. If you want to execute LotusScript instead of a formula when a user submits the form over the Web, you can replace the $$Return field with a WebQuerySave Agent. The form also specifies that a message appears, thanking Web users for filling out the survey and containing links back to the Lotusphere home page and the Notes.net home page.
The HTTPVariable-short subform, which appears at the end of the Questionnaire form, collects information about the user submitting the document. It collects the remote host name, remote address, remote user name, remote identity, HTTP user agent, HTTP referrer, and the server URL that the user is on. These fields are explained in detail in the Domino 5 Designer Help. They are CGI variables that provide information like the user's host name, the browser type, and whether the user was redirected to the site. System administrators and application developers can use the information contained in these fields for security or debugging purposes. For example, if you see strange characters in the surveys, you can look at the CGI variables to see if there is a problem with a particular browser or platform. In addition, if you receive user submissions that are inappropriate (because one user submits multiple surveys or uses offensive language), in most cases you can track the user down using their host name and IP address. We use the information in this subform only for our own records.
We use some of the information in the HTTPVariable-short subform in the only view in the database. The Questionnaire view displays all the submitted documents by date with the most recent document at the top of the view. This allowed us to go in each day and read the survey results for that day. The first column in the view displays the number of surveys submitted that day. When you expand the date created field, you can see the remote address and or host name of each user who submitted a document on that day. The following screen shows this view in our survey database:
We used the following formula in the Remote Address and/or Host name column to retrieve this information from the HTTPVariable-short subform:
@If(Name != ""; Name; @If(Remote_Addr = "" & Remote_Host="";@Name([CN];$UpdatedBy); Remote_Addr+", "+Remote_Host))
This formula is only necessary if you want users to be able to submit surveys through the Notes client and over the Web (as we did). If you want to create a Web-only application, you can simplify the formula to:
@If(Remote_Host = "", Remote_Addr, Remote_Host)
From the Questionnaire view we can now retrieve all the information we need from our survey.
Looking at the results
Now let's get to the important part, looking at what all of you said about our site! Your response to the information on Notes.net was overwhelmingly positive. That's not to say that you didn't have some strong opinions about different areas of the site.
We collected all the information from the survey in a spreadsheet. This helped us to easily calculate and view the results. Over a period of two weeks in January and February 1999, 165 users responded to the survey. The general information we collected about these users showed that of the 165 users, the majority had been using our site for a year or more. In addition, 47 percent of the respondents visit our site daily, and 40 percent visit our site weekly.
The following table shows the features that received specific feedback that prompted us to take action to improve that feature. It shows what your feedback was, and the action we took or plan to take to resolve the issue.
| Feature | Your feedback | What we're going to do about it |
| All about Notes/Domino page | The area is less useful | Redesign is in process |
| About this Site page | The area is less useful | Redesign in process |
| Frames | Mixed | Undecided |
| Registration | Forgotten passwords are common | Completed redesign of registration, including the easy retrieval of passwords |
| QMR/QMU Downloads | Difficult to navigate | Future plans to redesign |
| Iris Today | You search for articles primarily from the Notes.net home page | Factors into the way we will implement Domain Search on our site |
| Iris Cafe | The Notes/Domino forum is the most popular | Factored into the decision to make it the Notes/Domino Gold Release forum, covering all released versions of Notes/Domino |
| Site Performance | Average | Further evaluate ways to make the site faster |
When we asked users to rate each area of our site in terms of usefulness, 79 percent said that the Downloads section was very useful and 70 percent found Iris Today very useful. Users said that our All About Notes/Domino page and our About this Site page were less useful (look for the redesign of both of these areas in the months ahead). A majority of 64 percent of users rated our site easy to navigate. When asked whether they would like our site to use frames for layout, users were split -- 42 percent said it didn't matter, 36 percent wanted frames, and 22 percent didn't want frames. As a result, we are still undecided as to whether we should implement frames. For more information on frames, see the Iris Today article, "Domino Designer R5: Framesets."
The feedback in the area of registration helped us to complete the redesign and restructuring of our site registration. While the feedback was positive, with 82 percent of users saying that it was clear which type of registration they should use (North American or International), 42 percent, almost half, of users said that they had forgotten their password at one time or another. Recently, based in part on this feedback, we implemented a new registration application that allows users who forget their password to quickly retrieve it. For more information on the new registration application, see the Iris Today article, "Using the new Notes.net registration."
In the Downloads section of the questionnaire, we found it interesting that the type of file that users download most often were QMRs (Quarterly Maintenance Releases) and QMUs (Quarterly Maintenance Updates). Users generally upgrade when a new QMR or QMU comes out on our site. In the future, we plan to work on the design of the QMR and QMU download database because 36 percent of users said that they had the most trouble when they were navigating through the download options.
When we asked questions about Iris Today, we found that users find our application-specific and administration-specific articles most helpful. However, what surprised us the most was that users filled in multiple responses to the question on what was the most helpful type of article. This caused the answers to the question to add up to over 100 percent. We are pleased that so many users receive so much value from all the different types of articles we produce that they couldn't decide what article was the most helpful. We also learned that most users search within Iris Today from the home page of our site. Knowing that this is where users search from will help us when we begin implementing R5 Domain Search on our site. Domain Search will improve the search capabilities on our site, and it will help the transition from the current search to Domain Search if we know just how users search for information. For more information on Domain Search, see the Iris Today article, "Domino R5: Domain Search."
The information we received in the section of the questionnaire about the Iris Cafe helped us to make decisions about the forums after the release of R5. During the R5 Beta period, we had an R5 Beta Feedback Forum, as well as the Notes/Domino forum. When R5 went gold, we had a dilemma: should we continue to allow users to enter information about R5 gold in the R5 Beta Feedback Forum? Since feedback from the Iris Cafe section of the survey indicated that the Notes/Domino forum was the most popular forum, we decided to make the Notes/Domino forum the Notes/Domino Gold Release forum. We moved the feedback about the R5 Gold platforms into the Notes/Domino Gold Release forum and allowed users to continue threads in that forum. The R5 Beta Feedback forum continues as place where users who use R5 on platforms that are still in Beta can communicate information about those Betas. Eventually, after the release of R5 on those platforms, we will archive this forum.
In the area of site performance, 56 percent of users rated our overall site performance as average. This is an area that we constantly evaluate and try to improve. Look for an upcoming Notes.net Exposed article in Iris Today that chronicles the site performance adjustments and subsequent improvements we've made in this area over the past months.
Last, but certainly not least, users rated the ease of use of finding documentation on the User Assistance site. About 62 percent of the users said it was somewhat easy to find the documentation they were looking for. Only 7 percent found it difficult to find an item. This shows that all the hard work that the User Assistance team put in over the last year to redesign the User Assistance site and make it easier to find information was worth it.
Conclusion
The Notes.net Lotusphere survey turned out to be a very valuable tool for getting information back up the road in that other direction, from you to us. We are always looking for ways to improve our site and products. We want to make them fit your needs. Now that the survey is over, you can continue to send us your opinions and comments by clicking the Feedback link at the bottom of any page on our site. You can also fill out some of the other surveys on our site, including the usability survey (if you use Notes R4.6 or earlier) or one of the documentation surveys. So, keep the feedback coming! |