Integrating the enterprise
Interview by
Tara
Hall
Level:
All
Works with:
All
Updated:
01-Aug-2001
As businesses grow and evolve, they often find themselves with valuable corporate data and processes residing in different types of systems and databases. That may be because different divisions use different technologies, or because of a corporate merger, or even because the company simply migrates to a newer technology. Whatever the reason, the ability to integrate the data in these disparate systems is critical.
For August, Mary Peterson, group manager for Lotus Enterprise Integration Product Management, and Scott Prager, lead architect for Lotus Enterprise Integration, discuss Lotus's past, present, and future enterprise integration technologies.
For those unfamiliar with enterprise integration, can you define it?
Scott Prager
Enterprise integration (EI) is a term for tools and methods aimed at coordinating and connecting the various computer applications in an enterprise. The tools might include technologies like Lotus Enterprise Integrator (LEI) for data integration, MQSeries for messaging, or custom-built applications; and an example of a method is a three-tier architecture. Typical enterprises have multiple, disparate legacy applications—like IMS, which runs on a mainframe—and relational database and Enterprise Resource Planning (ERP) systems. EI is the technology for enabling a business to work with these enterprise applications in an integrated and consistent manner.
A reasonable example would be when a company using PeopleSoft and Sybase buys a company using DB2 and then needs to expose information from all systems. To do this, they need the right EI tools.
Can you describe what Lotus Enterprise Integrator (LEI) is and what it does?
Mary Peterson
The Lotus Enterprise Integration technologies connect Domino applications to a wide variety of already existing enterprise systems in customers' environments. This includes connectivity to popular relational databases like Oracle and DB2, as well as Enterprise Resource Planning (ERP) systems like SAP R/3 and PeopleSoft. We recently expanded our scope to extend the connectivity reach of other Lotus products, like K-station and Discovery Server.
LEI is our flagship product for data management. Its power is in industrial strength data transfer and synchronization between systems. Using LEI, customers can integrate enterprise data with Domino without programming. LEI also has advanced scheduling features, a full features console monitor, and logging functionality.
The console and log give full insight to the LEI server, so you can see every data transaction that occurs. The scheduler is a utility that sets tasks to run at a particular time.
Mary Peterson
Scott Prager
LEI uses a modular hub-and-spoke architecture to provide the maximum level of flexibility in EI applications. Imagine a bicycle tire with a set of spokes. Each spoke represents a disparate system. How do you connect those spokes? It's very inefficient to try to connect each spoke directly to every other spoke. A tool like LEI provides a set of modular activities and connections—or hubs—that can link together data in a wide variety of combinations to support the range of EI needs.
LEI supports a number of spokes, or target systems, like PeopleSoft, Sybase, DB2, and so on; and it supports multiple hubs, or actions, like direct transfer and replication. So you can replicate between DB2 and Domino or perform a direct transfer between PeopleSoft and Sybase. Then you can switch targets and actions to replicate between Domino and Sybase and transfer data between DB2 and PeopleSoft. Hubs and spokes, or actions and targets, are interchangeable, providing the maximum level of application flexibility.
LEI is based on an easy-to-use, point-and-click interface, supporting disparate database integration, replication, and other operations without the need for code. We identified the most common tasks users perform, like synchronizing data between systems; built them into LEI; and then exposed them graphically through our user interface (UI).
Scott Prager
LEI is part of a family of integration tools. What distinguishes LEI from other integration tools like Domino Enterprise Connection Services (DECS) and Lotus Enterprise Solution Builder (ESB)?
Mary Peterson
As I described earlier, LEI is the powerhouse product and is the Lotus EI business unit's focus moving forward. LEI is full-featured product. You use LEI if you want to connect Oracle, DB2, PeopleSoft, or other enterprise data to Domino. LEI moves or synchronizes that data between Domino and the enterprise systems.
Domino Enterprise Connection Services, or DECS, is a feature of Domino and will be available in Domino Rnext. DECS works well for simple applications that access data in real-time. For instance, if you want to access data in a Web browser, then you need DECS, which is good for simple real-time integration.
I like to describe Enterprise Solutions Builder (ESB) as a "servlet engine for LotusScript." It allows you to write LotusScript code components and to deploy them on an ESB server. This way all the data connectivity happens at the server, and you can reuse the code components from several applications. If you want customized calculations that LEI and DECS can't handle, then you need ESB to create custom LotusScript components that provide server-side data integration.
The integration tools family also includes programming solutions. For hard-core programmers who need to connect to external data, there are the Lotus Connector LotusScript Extensions (LSX) and LotusScript Data Objects (LS:DO). Both allow programmers to use LotusScript to access enterprise data.
What's the definition of real-time?
Scott Prager
Real-time describes an action that takes place synchronously, at that moment. For instance, when you schedule a task like replication and you push a button to run the task, you are simply scheduling the action to take place, not actually triggering the action to occur directly. In a real-time operation, an action—perhaps pushing a button—triggers an immediate and direct response from a back-end system. In LEI, real-time describes an instant response from enterprise integrated information in a Domino system.
Can you give a brief history of the integration tools, particularly LEI, and how they have evolved to suit customer needs?
Mary Peterson
Lotus acquired business partner Edge Research located in Portsmouth, New Hampshire, in 1994. Edge Research, which provided expertise in C programming to integrate Domino and enterprise data, had developed the Notes HiTest API, which provided an easy way for C programmers to work with Domino, and HiTest Tools for Visual Basic. The group became the Enterprise Integration Business Unit, which designed and developed an overall EI strategy for Domino. The initial product developed was called NotesPump. Lotus later moved the NotesPump product into the mainstream product line and changed the name to Lotus Enterprise Integrator.
Further integration with core Domino to advance Domino's integration capability was realized when the functionality of LEI Realtime was made available in Domino Release 4.6.3 and R5 under the name Domino Enterprise Connection Services (DECS). In this same time frame, the reach of all the tools was expanded from connectivity for relational databases like DB2, Oracle, Sybase, and ODBC, to ERP systems like SAP R/3, J.D. Edwards, Oracle applications, and PeopleSoft.
What software changes and industry trends impact enterprise integration?
Mary Peterson
Customers are very interested in streamlining business processes. Many Lotus customers use the EI tools to add or to automate workflow processes in their business systems. The evolution of standards in the EI area came about because of the need to make connecting disparate systems easier. Java Database Connectivity (JDBC) has evolved to become the functional equivalent to Open Database Connectivity (ODBC), but for Java programming. Right now, IBM is leading the way in the JavaSoft J2EE (Java 2 Platform, Enterprise Edition) Connector specification. Soon XML and WebServices will also ease the boundaries between systems.
Scott Prager
The EI product set addresses the needs of companies that have disparate systems to integrate. LEI is well suited to address business needs like merging, growth, and a shift from workgroup to enterprise-level applications. Our premium connectors to ERP systems address the growing adoption of ERP systems and the need to integrate them with the rest of the enterprise. Our overall product set and EI strategy enable Domino to operate as an enterprise Web application server, enabling e-business applications.
LEI 3.2 beta is available. What new features are in this release?
Mary Peterson
LEI 3.2 is a maintenance release of the LEI product. It will have the updated DB2 Connector, a component that provides seamless integration between DB2 data and the Domino tools. LEI 3.2 will support the new release of DB2 and provide new functionality to allow a response from an external system process or stored procedure to be posted back to the Domino document in the same transaction. For example, in a direct transfer that moves approved purchase orders from Domino to SAP R/3, the SAP generated PO number can now be captured automatically and displayed in the Domino application.
Looking ahead to LEI.Next, what can users expect?
Scott Prager
LEI.Next includes a number of new features. The most significant components of LEI.Next are Advanced Realtime, a redesigned user interface, and a connector for OLE DB. Other improvements include connector updates, server-side browsing, and an improved multiplatform installer. In addition, ongoing work improves stability, performance, and usability.
What is Advanced Realtime and what can it do?
Scott Prager
Advanced Realtime is a quantum leap in real-time integration with data in enterprise systems. It enables substantially greater portions of a Domino application to reside in enterprise systems and allows Domino collaborative services and other capabilities to be applied directly to data in these enterprise systems. In short, Advanced Realtime enables heterogeneous applications to be quickly and easily constructed.
How does it work?
Scott Prager
Advanced Realtime uses a technology called "virtualization," which allows external data and business logic to reside in external systems while behaving as if they were in Domino. Virtualization is really a simulation. When you access enterprise information in a Domino database, that information looks and acts like Domino data. You treat it like Domino data. You can build views to display DB2 data, full-text search PeopleSoft data, replicate Sybase data, and so on. Virtualization takes advantage of very deep integration with the Domino server, which provides full Domino functionality for external data.
How is Advanced Realtime different from Classic Realtime?
Scott Prager
Classic Realtime, also referred to as DECS, enables portions of Domino documents to be virtualized with limitations on how that virtual data can be used. It is very effective for quick prototypes or simpler Domino applications.
Advanced Realtime leverages a deeper level of integration with the Domino server, enabling enterprise data outside of Domino to truly behave as a "first class citizen" in a Domino application. Advanced Realtime supports fully virtual documents, removing the need for DECS document stubs. It also supports virtualization of attachments and agents, as well as a number of related capabilities such as integrated credentials to enhance security. The end result is a richer, faster, and more powerful level of heterogeneous applications.
How do users benefit from Advanced Realtime?
Scott Prager
First, Advanced Realtime allows Domino to treat external data the same as "real" Domino data. This allows Domino applications to leverage existing infrastructure—data, business logic, or development skills—in an enterprise.
Second, it enables Domino services such as replication, workflow, and views to be applied directly to enterprise data. So I can have a Domino view of DB2 data, full-text search a Domino index of Oracle and SAP data, and use Domino replication and off-line support to replicate and work with any enterprise data.
Third, Advanced Realtime requires no additional programming, so existing data and business logic can be leveraged without writing code.
What are virtual documents, agents, attachments, and views?
Scott Prager
Classic Realtime allows a subset of fields within a Domino document to be virtualized. These fields are referred to as virtual fields. Advanced Realtime extends virtualization to agents and attachments. Domino services extend virtualization to Domino views and other services.
Virtual documents allow external records to appear to Domino as if they were "real" documents in a Domino database, while no component of the document is actually stored in the database. The extended document information is stored directly in the external RDBMS system, either in a separate table or in the data table itself.
Virtual attachments provide the same capability for Domino attachments, storing them in a separate external table. Virtual agents extend virtualization to business logic, exposing stored procedures or similar objects in enterprise systems as Domino agents, which can be run directly against Domino documents virtual or "real."
Earlier you mentioned integrated credentials as another new feature of LEI.Next. What are integrated credentials and what are their benefits?
Scott Prager
Integrated credentials are a variant of single sign-on capability, which allows LEI Advanced Realtime to manage and to utilize external system credentials. This allows a Domino application with Advanced Realtime to fully enforce an individual's permissions and credentials against one or more external systems without requiring the user to enter these credentials for each session.
You also mentioned a connector for OLE DB. What is this connector and why is this significant for LEI?
Scott Prager
The OLE DB connector is a connectivity standard, similar in concept to ODBC, but based on the OLE specification. It defines a standard interface for accessing databases via vendor-supplied drivers. OLE DB is the recommended interface for SQL Server and Microsoft Access connectivity, replacing ODBC. The connector supports our strategy of maximum connectivity.
Some Lotus and IBM products, like WebSphere, have adopted LEI technology. Can you describe how those products are using your technology and what impact that will have on LEI?
Mary Peterson
Several of the new IBM J2EE WebSphere Adapters, which are in beta now, are based on Lotus connector technology. IBM recognized Lotus' s expertise in ERP system integration and is reusing the technology in other parts of the IBM software product line.
IBM is using LEI technology to connect WebSphere to ERP applications specifically, and also PeopleSoft, JD Edwards, and Oracle applications. We have a vision of all IBM software groups sharing the same set of connectivity components. This is a great first step.
About Mary Peterson
As group manager for Lotus Enterprise Integration Product Management, Mary is responsible for defining the Enterprise Integration product family including Lotus Enterprise Integrator (LEI), Domino Enterprise Connection Services (DECS), and Enterprise Solutions Builder (ESB). These products enable Lotus Domino to interact with relational databases, ERP systems, and transaction applications. Her activities include representing customer requirements to the development team, inbound and outbound communication of product features, competitive analysis of other enterprise integration products, and keeping current with the ever-changing Web application development and enterprise integration technology. Mary has been with Lotus since 1992 and holds a Bachelor of Science degree in Computer Science from the University of Illinois and a Masters of Business Administration from Boston University.
About Scott Prager
Scott is the lead architect for Lotus Enterprise Integration. Scott joined Lotus in late 1994, with the purchase of Edge Research Inc., a company he cofounded in 1993. He led the design and development of the Lotus EI strategy and NotesPump/LEI. For the last few years, he has led Domino real-time enterprise integration with DECS and the upcoming LEI release. Prior to joining Lotus, Scott worked at Edge Research, Channel Computing, LaserAccess, and Xerox. Scott received his Bachelor of Science degree in Computer Science and Engineering from MIT.
