LDD Today

Creating Lotus Instant Messaging interactive agents with the BuddyScript SDK, Part 2


by
Gray
Norton

Level: Intermediate
Works with: Sametime
Updated: 27-Oct-2003

In Part 1 of this article series, we introduced you to interactive agents, a class of applications that includes Lotus Instant Messaging (Sametime) bots. We provided an overview of how you can create Lotus Instant Messaging interactive agents with the BuddyScript SDK (Software Developer's Kit) from ActiveBuddy. In Part 2, we get down to the details, stepping you through the process of developing an interactive agent that helps users manage a stock portfolio. (You can download the complete sample file from the Sandbox.)

This article assumes that you're familiar with basic instant messaging concepts. Some experience with Web software development is helpful.

A simple interactive agent for stocks
You’ve seen how simple it is to build an interactive agent with the BuddyScript SDK in the first part of this article series. Now let's delve into a real-world example. This article looks at the complete script for an interactive agent named Stocky that is designed to provide stock quotes and related functionality. In addition to fetching current quotes at the user's request, our interactive agent maintains a portfolio of favorite stocks for each user and watches for stocks to reach target values as specified by the user.

The following is a sample session transcript, logging a user’s conversation with the Stocky interactive agent:

JohnDoe: hi
Stocky: Welcome, JohnDoe!

Type "help" to see what I can do!

JohnDoe: help
Stocky: I can:
1 Give you stock quotes
2 Manage a portfolio of your favorite stocks
3 Notify you when stocks hit target values

>> Type "help" at any time to return to this menu...

JohnDoe: give me a quote for ibm
Stocky:
Stock: IBM
Now: 81.80
Chg: -0.74
Hi/Lo: 82.88/81.60
Vol: 7.28 Million

>> Type "add" to add IBM to your portfolio

JohnDoe: add
Stocky: Added. Your portfolio contains:

Stock: IBM
Now: 81.80
Chg: -0.74
Hi/Lo: 82.88/81.60
Vol: 7.28 Million

JohnDoe: add ge, gm, and mmm
Stocky: Added. Your portfolio contains:

1 IBM: 81.80 -0.74
2 GE: 27.90 -0.47
3 GM: 36.91 -0.34
4 MMM: 139.55 -1.43

JohnDoe: tell me if ibm drops to 80 or hits 90
Stocky: OK, I will notify you when...
IBM falls to or below 80
IBM climbs to or above 90

JohnDoe: thank you!
Stocky: You’re welcome!

Note that our sample application pulls its stock data from Yahoo! and is intended for learning and demonstration purposes only. You should not deploy an interactive agent using this data in a production environment without first securing Yahoo!’s permission.

Getting the data
Our script starts with code for interfacing with external data (in this case, Yahoo!’s Web-based stock quote service) and a pair of text files containing the stock symbols for all companies traded on the NYSE and NASDAQ exchanges:

package lib:/DataTypes/DateTime
package lib:/English/English

###################
# Live Stock Data #
###################

macro STOCK_COLUMNS
Symbol, Current, Date, Time, Change, Open, High, Low, Volume

macro STOCK_REGEX
\"(.*?)\",(.*?),\"(.*?)\",\"(.*?)\",(.*?),(.*?),(.*?),(.*?),(.*?)\s\n(.*)

// Builds a "+"-delimited string of stock symbols (required
// by the Yahoo! stock quote service) from a BuddyScript
// object variable.
function SymbolString(STOCK_LIST)
datasource GetStockInfo(SYMBOL_LIST) => MACRO_STOCK_COLUMNS {expire="now"}
#################
# Stock Symbols #
#################

datatable NASDAQSymbols
datatable NYSESymbols
subpattern ANasdaqSymbol get Symbol, Symbol in NASDAQSymbols {minlength=1 maxlength=1 score=MACRO_MEDIUM_SCORE}
subpattern AnNyseSymbol get Symbol, Symbol in NYSESymbols {minlength=1 maxlength=1 score=MACRO_MEDIUM_SCORE}

subpattern AStock
+ STOCK=ANasdaqSymbol
+ STOCK=AnNyseSymbol
subpattern AListOfStocks
+ STOCK=AStock
+ STOCK_LIST=AListOfStocks [(and|or)] STOCK=AStock
The code for getting stock quotes takes the form of a BuddyScript datasource, a type of function specialized for getting data and returning it in the tabular format required to take advantage of BuddyScript’s advanced presentation features. Throughout the script, this datasource is called whenever a stock quote is required. The stock symbols, on the other hand, are stored in BuddyScript datatables. A datatable is a structure used to keep data resident in memory. Keeping the symbols in memory allows them to become part of the interactive agent’s "vocabulary," which we accomplish by defining subpatterns based on the content of the datatables.

Delivering quotes
Next, let's look at the code that implements the interactive agent’s core functionality, delivering stock quotes in response to user requests. This code follows a convention used throughout the project: Application logic is broken into BuddyScript procedures, which are called by the routines that follow them in the script:

###################################
# Basic Stock Quote Functionality #
###################################

// Displays a detailed quote for a single stock.
procedure DisplayStockDetails(STOCK)
// Displays a table with quotes for multiple stocks.
procedure DisplayStockInfo(SYMBOL_LIST)
// Builds a nicely formatted string of stock symbols (for
// display) from a BuddyScript object variable.
function DisplaySymbolString(SYMBOL_LIST)
declare procedure AddToPortfolio

// Recognizes user requests for stock quotes.
+ [(=HowIs|=HowAre)] STOCK_LIST=AListOfStocks [doing]
+ [(=WhatIs|=WhatAre)] [the] [(price|prices)] [of] STOCK_LIST=AListOfStocks
+ [quote] STOCK_LIST=AListOfStocks
The preceding script contains two procedures for displaying stock data. One displays all available data for a single stock, and the other displays a subset of the available data for several stocks at once. Both procedures utilize BuddyScript’s table functionality (invoked via the markup tags) to produce clean, readable layouts. The DisplayStockInfo procedure also uses BuddyScript’s enumeration (auto-numbering) feature, which displays a number next to each stock in the table. You can access a more detailed quote for any stock by typing the corresponding number. Enumeration can be added to any output line by including a set of curly braces ( {} ) at the end of the line; the code between the curly braces indicates what action should be taken when the user types the number.

The routine at the end of this snippet also deserves a closer look for a couple of reasons. First, the pattern definitions in this routine refer to subpatterns that are defined in the BuddyScript libraries. The BuddyScript libraries come with BuddyScript Server and contain useful code for a variety of purposes. In the first two lines of our script, we included a couple of library packages, English and DateTime. The subpatterns we use here (=WhatIs, =HowAre, and so on) are defined in the English package and are used to recognize both the contracted (for example, what’s) and the full (for instance, how are) forms of common question phrasings. Later, we’ll use functions from DateTime to format dates and times for display.

The same routine also utilizes BuddyScript’s dialog feature to offer the user a context-based shortcut. After displaying the information for the requested stocks, the agent prompts you to type add to add the stocks to your portfolio. In any other context, the simple statement add would not give the interactive agent enough information to act on, but it’s enough in this case because the list of stocks is already known. The code defining a dialog looks just like the code for defining a routine, consisting of one or more pattern definitions followed by an indented block of code. The only difference is that the dialog definition is itself indented within a block of BuddyScript code, indicating the point in the script’s execution where the dialog should occur.

Keeping a user’s portfolio
At this point, it’s worth noting that the interactive agent’s core functionality is essentially complete. In less than 100 lines of code (excluding comments), we’ve written an interactive agent that will live happily on users’ contact lists, delivering instant stock quotes upon request. The remaining code, composed of a few hundred more lines, adds some compelling extra functionality and some interface niceties to improve the user experience.

The following snippet contains the code for storing, manipulating, and displaying the user’s portfolio—just a list of favorite stocks, in this case:

###########################
# Portfolio Functionality #
###########################

stored variable MY_PORTFOLIO = {}

// Displays quotes for all stocks in the user's portfolio.
procedure ShowPortfolio
// Adds one or more stocks to the user's portfolio,
// then displays the modified portfolio.
procedure AddToPortfolio(STOCK_LIST)
// Removes one or more stocks from the user's portfolio,
// then displays the modified portfolio.
procedure RemoveFromPortfolio(STOCK_LIST)
// Clears the user's portfolio, removing all stocks.
procedure ClearPortfolio
// Subpatterns useful for recognizing portfolio requests.
/////////////////////////////////////////////////////////
subpattern Clear
+ (clear|reset|delete|cancel)

subpattern Stocks
+ (stock|company|ticker|symbol)
+ (stocks|companies|tickers|symbols)

subpattern Portfolio
+ [my] (portfolio|stocks)
/////////////////////////////////////////////////////////

// Recognizes requests to display the user's portfolio.
+ [(=What|=Which)] [=Stocks] [(=Is|=Are)] [in] =Portfolio
+ [(=WhatIs|=WhatAre)] [in] =Portfolio
+ =ShowMeNoun<=Portfolio>
// Recognizes requests to clear the user's portfolio.
+ =Clear =Portfolio
// Recognizes requests to add stocks to the user's portfolio.
+ add STOCK_LIST=AListOfStocks [[to] =Portfolio]
// Recognizes requests to remove stocks from the user's portfolio.
+ remove STOCK_LIST=AListOfStocks [[from] =Portfolio]
The main point of interest in this code is the use of a BuddyScript stored variable to keep the user’s portfolio in a way that persists across multiple sessions. One of the components of BuddyScript Server is the user profile, a repository for storing and retrieving user-specific data; the values of stored variables are kept here. User profile data is keyed to a user’s screen name and automatically retrieved each time a session begins, making the persistence of data extremely easy and virtually transparent to the developer.

Tracking stocks
The next section of code accounts for the bulk of the script. This code implements the watchlist functionality, which takes full advantage of the fact that the interactive agent operates in a messaging environment. It proactively notifies the user whenever a stock reaches a specified target value and uses presence awareness to store a notification for later delivery if the user is not on-line when a target is reached.

The code for the functionality itself is relatively straightforward and is thoroughly commented. In a nutshell:
The code performs the periodic checks by using BuddyScript’s notification feature. A notification is essentially just a timer: When the timer goes off, a specified procedure is called. In this case, the notification triggers the CheckPresence procedure, which requests the user’s presence status. The status comes back asynchronously (via the ABBuddyStatus procedure) which records the presence information and calls CheckWatchlist. After performing its duties, CheckWatchlist uses a new notification to restart the process (provided there are still items in the watchlist).

The code begins as follows:

###########################
# Watchlist Functionality #
###########################

macro UPDATE_INTERVAL
in 5 minutes

stored variable MY_WATCHLIST = {}
stored variable WATCHLIST_NOTIF_ID = -1
stored variable THIS_NOTIF_ID = -1
variable PRESENT = 0

The following two functions are used by the CheckWatchlist and ShowWatchlist procedures to generate display text, based on the type of event being watched for.

function Reaches(TYPE)
function Reached(TYPE)
Next, we check each item in a user's watchlist to see whether or not it is triggered by the stock's current value. This happens in two circumstances: whenever items are added to the watchlist and periodically (but only if there are active items on the watchlist). In the second case, the user's presence status is determined before checking the watchlist. When the user is away, no outputs are made, but any notifications of items triggered are stored to be output later when the user returns:

procedure CheckWatchlist
The following requests the user's presence status by adding the user to the interactive agent's contact list:

procedure CheckPresence
These lines receive and store the user's presence status, remove the user from the contact list, and then check the user's watchlist:

procedure overrides ABBuddyStatus(EVENT)
These three lines clear the user's watchlist, removing all items:

procedure ClearWatchlist
And these display the user's watchlist:

procedure ShowWatchlist
The following code adds one or more items to the user's watchlist:

procedure AddToWatchlist(WATCHLIST)
And this removes one or more items from the user's watchlist:

procedure RemoveFromWatchlist(WATCHLIST)
The following are subpatterns useful for recognizing watchlist requests:

subpattern TellMe
+ (tell|notify|alert|im|contact|ping) me
+ let me know

subpattern Watchlist
+ [my] (watchlist|watch list)

subpattern ToExceed
+ [to] exceed
+ exceeds
+ [has] exceeded

subpattern ToRise
+ [to] (rise|climb)
+ (rises|climbs)
+ [has] (risen|climbed)

subpattern ToFall
+ [to] (fall|drop|dip)
+ (falls|drops|dips)
+ [has] (fallen|dropped|dipped)

subpattern ToBe
+ is
+ [to] (be|become)
+ [has] become

subpattern ToReach
+ [to] (reach|hit)
+ (reaches|hits)
+ [has] (reached|hit)

subpattern ToGet
+ [to] (go|get) to
+ (goes|gets) to
+ [has] (gone|gotten) to

These are specialized subpatterns for parsing a conversational watchlist request and outputting the BuddyScript object format utilized by the various watchlist-related procedures:

subpattern Reaches
+ =ToExceed
+ [=ToBe] (more|greater|higher) than
+ [(=ToGet|=ToBe)] above
+ =ToRise (above|to) + [=ToBe] (less|lower) than
+ [(=ToGet|=ToBe)] below
+ =ToFall (below|to) + =ToReach
+ =ToGet to
+ =ToBe [at]
// "10 20 30", "10, 20 or 30", etc...
subpattern AListOfFloats
+ FLOAT=AFloat
+ LIST=AListOfFloats [(and|or)] FLOAT=AFloat
// "falls below 10", "climbs to 20 or 30", etc...
subpattern ATargetValue
+ [TYPE=Reaches] VALUES=AListOfFloats
// "falls below 10 or climbs to 20 or 30", etc...
subpattern AListOfTargetValues
+ TARGET=ATargetValue + LIST=AListOfTargetValues [(and|or)] TARGET=ATargetValue
// "MSFT CSCO 10 40", "if MSFT or CSCO drops to 10 or rises to 40", etc...
subpattern AWatchlistFragment {matching="canskip"}
+ [(if|=When)] STOCK_LIST=AListOfStocks TARGET_LIST=AListOfTargetValues
// "MSFT 20 30 IBM 80 90", "if MSFT hits 20 or 30 or if IBM drops to 80 or exceeds 90", etc...
subpattern AWatchlist
+ FRAG=AWatchlistFragment + LIST=AWatchlist [(and|or)] FRAG=AWatchlistFragment
Finally, these lines process watchword requests:

// Recognizes requests to add items to the user's watchlist.
+ [=TellMe] WATCHLIST=AWatchlist
+ [watch [for]] WATCHLIST=AWatchlist
+ [add] WATCHLIST=AWatchlist
// Recognizes requests to display the user's watchlist.
+ [(=What|=Which)] [(stock|stocks)] [(=Is|=Are)] [on] =Watchlist
+ [(=WhatIs|=WhatAre)] [on] =Watchlist
+ [(=What|=Which)] [(stock|stocks)] [[=Am] i] watching
+ =ShowMeNoun<=Watchlist>
// Recognizes requests to clear the user's watchlist.
+ =Clear =Watchlist
// Recognizes requests to remove items from the user's watchlist.
+ =DoNot [=TellMe] WATCHLIST=AWatchlist
+ =DoNot [watch [for]] WATCHLIST=AWatchlist
+ (remove|=Clear) WATCHLIST=AWatchlist
Given the simplicity of this process, you may well be wondering why there is so much code in this section, especially compared to the previous snippets. The answer is that we’re striving for some bonus points in the watchlist functionality to illustrate how far a little extra effort goes toward improving an interactive agent’s conversational abilities. Specifically:
You may feel that such functionality is overkill for this simple interactive agent. However, our experience has shown this type of conversational sophistication (or the lack thereof) can sometimes make or break an application's usability. In these cases, BuddyScript’s high-level tools for handling conversational language make it comparatively simple to tackle problems that might otherwise be prohibitively difficult to solve.

Nuts and bolts
Our script ends with a handful of procedures and routines designed to handle common situations that arise when users interact with an interactive agent. Examples of these include requests for help, greetings and goodbyes, and user inputs that don’t match any of the patterns in the script.

#####################
# Help, Hello, etc. #
#####################

procedure DisplayHelp do nothing
- <blank/>
To return to the main Help menu
Help {*}
action ExplainPortfolio
- To add stocks to your portfolio, just type "add" followed by one or more symbols:
Add IBM GM GE {*}
do nothing
- <blank/>
To display your portfolio:
My portfolio {*}
do nothing
- <blank/>
To remove stocks, type "remove" followed by one or more symbols:
Remove IBM GM GE {*}
do nothing
- <blank/>
To clear your portfolio (remove all stocks), just type:
Clear my portfolio {*}
do nothing
- <blank/>
To return to the main Help menu
Help {*}
action ExplainWatchlist
- To "watch" a stock (be notified when it reaches a particular target value):
IBM 100 {*}
IBM 80 100 {*}
Tell me if IBM goes above 100 or MSFT falls below 20 {*}
do nothing
- <blank/>
To return to the main Help menu
Help {*}

procedure overrides ABFirstProc
- Welcome, SYS.User.ScreenName!
<blank/>
Type "help" to see what I can do!
exit all

procedure overrides ABHelloProc
- Hello again, SYS.User.ScreenName!
<blank/>

+ help
call DisplayHelp

+ =Hello
- Hi!

+ =Goodbye
- Bye!

+ =Catch
- I'm sorry, I don't understand. To see what I can do, type "help!"

Most of this code should be fairly self-explanatory, but a couple of points are worth discussing. First, the special system procedures ABFirstProc and ABHelloProc are automatically called when the user visits the interactive agent for the first time, and when the user initiates each session with the interactive agent, respectively. There are default implementations for these procedures (and a host of other system procedures) in the BuddyScript libraries, but it often makes sense to override them in your script.

Finally, the subpattern Catch is also defined in the BuddyScript libraries. It matches absolutely anything, but assigns a very low score; consequently, the routine in which we refer to the Catch subpattern should match if and only if none of the other patterns in the script matches the current user input. Writing such a routine is a simple way to handle cases in which the user asks for something outside the interactive agent’s realm of expertise.

Conclusion
We hope this article series has helped open your eyes to the potential of interactive agents deployed in Lotus Instant Messaging (and other types of messaging environments). For additional developer information and more examples, visit the BuddyScript home page where you’ll find the latest BuddyScript software and a growing developer community.


ABOUT THE AUTHOR
Gray Norton is a Senior Product Manager for ActiveBuddy, where he oversees the company's BuddyScript product line. He has been in the software industry since the early 1990's and has played key product management roles for several companies in the interactive media space, including Electrifier, Motion Factory, MetaCreations, and Ray Dream. Gray is a graduate of Stanford University's Symbolic Systems Program.