Rational ClearCase UCM Migration: A case study sidebar

[Back to "Rational ClearCase UCM Migration: A case study"]

Hardware planning
Selecting the right hardware for the ClearCase setup is important for long-term efficiency. The needs obviously differ from project to project. Knowing what was selected for IBM Lotus Workplace may help other teams in their considerations, but it should not be construed as a recommendation for any particular project. ClearCase is relatively intense in its use of resources in the following areas:

Disk I/O speed, which is the most important attribute in this case
Network I/O speed and bandwidth
Memory quantity, which becomes more important than network I/O with large databases

The IBM Lotus Workplace group planned for high performance resources in those areas, and then scaled the solution to the capacities needed. The actual amount of disk space needed is inherently difficult to predict, but when purchasing new hardware, disk space is relatively inexpensive, so the Lotus Workplace team opted to start with a 200 GB configuration of the ClearCase servers.

The preliminary strategic vision for testing tools at Lotus includes a migration to Rational tools over time, including the defect and activity tracking system ClearQuest. Like many services, ClearQuest runs best on a separate, dedicated machine. Preferably, ClearQuest should not run on the primary ClearCase servers. It needs considerably fewer resources than ClearCase, and if they are run on the same server, ClearCase is often a bottleneck for ClearQuest. The primary goal for a good ClearQuest server is reliability. Migration from the current defect tracking systems to ClearQuest has not yet been accomplished.

The participating sites were divided into four categories and a configuration recommended for each category based on standardized class A and class B server configurations explained below.

Primary locations
Critical development locations need fast and redundant access to all systems, including LAN access to all ClearCase databases and ClearQuest activity management systems. For these locations:

There is one class A ClearCase server, one class B ClearCase server, and one class A ClearQuest server; the class B ClearCase server is used for redundancy purposes in the event of a failure of the class A ClearCase server.
Servers have at least a 100 Mbit/s switched LAN between them and at least a 100 Mbit/s switched LAN to the desktops.
WAN access is required for other locations.
As ClearQuest usage grows, an additional class B ClearQuest server may be warranted for redundancy purposes. For the first year or so, a ClearQuest server can be rebuilt if necessary from replicated data across the WAN. But as the content volume grows over a few years, the size may become too large for WAN replication to be practical.

Secondary locations
Secondary locations need access to all systems, but can have some downtime or temporary slower access across the WAN because the location is not large enough to warrant the overhead of redundant systems. For these locations:

These is one class B ClearCase server and one class B ClearQuest server.
Servers have a 100 Mbit/s switched LAN between them and at least a 100 Mbit/s switched LAN to the desktops, but 10 Mbit/s also works.
WAN access is required for other locations.

Tertiary locations
Tertiary locations need access to activity management systems—ClearQuest—and can have some downtime or temporary slower access across the WAN because the location is not large enough to warrant the overhead of redundant systems. For these locations:

There is no ClearCase access.
WAN access is required for other locations.
These is one class B ClearQuest server.

Quaternary locations
Quaternary locations need access to activity management systems—Clearquest—but only infrequently. These locations insert data, but do not modify them and/or need read access because the location is not large enough to warrant the overhead of maintaining a local system. For these locations:

There is no ClearCase access.
Access to ClearQuest is over the WAN either through native or Web interfaces.

Hardware specifications
Hardware configurations were geared toward Intel-based machines, but we permitted each site to implement hardware of their own choice whether based on Windows, Linux, or Solaris (license server). This permitted some sites to redeploy existing hardware rather than purchase new servers. All hardware, especially Redundant Array of Independent Disks (RAID) controllers, should be compatible with both RedHat Linux 8.0 and Windows 2000 Server/Advanced Server SP3. The specification defined was our suggested minimums, and obviously, more powerful machines improve performance.

Server type	Hardware recommendation
Recommended class A ClearCase server	200 GB disk space expandable to 500 GB disk storage Fast disk I/O is fundamental to good ClearCase performance; suggest RAID0+1, or similar fast and redundant system. Avoid RAID 5, unless your benchmarks show equal performance to RAID0+1. SCSI Ultra/160 or Ultra/320 18K RPM disks Controllers should have at least 128 MB of memory. Controllers should use the widest bus slot at the fastest speed. 4 GB of RAM expandable to 8 GB Dual CPU systems of at least 2.4 GHz 750 MHz per CPU Dual 100 Mbit/s NICs RedHat Linux 8.0
Recommended class B ClearCase server	200 GB disk space expandable to 500 GB disk storage Disk I/O subsystem does not need to be ultra fast, but a reasonable SCSI system should be included; suggest RAID 0+1, or some other fast and redundant system. SCSI Ultra/160 10K RPM disks 4 GB of RAM expandable to 8 GB Single CPU system RedHat Linux 8.0
Recommended class A ClearQuest server	36 GB disk space expandable to 72 GB disk storage Disk I/O subsystem does not need to be ultra fast, but a reasonable SCSI system should be included; suggest RAID 5, RAID 0+1, or some other fast and redundant system. RAID 5 is recommended. SCSI Ultra/160 10K RPM disks 1GB RAM expandable to 2 GB Single CPU 1GHz or faster Dual 100 Mbit/s NICs Windows 2000 Server or Windows 2000 Advanced Server
Recommended class B ClearQuest server	36 GB disk space expandable to 72 GB disk storage The equivalent of a good disk I/O subsystem for a desktop system; does not need RAID; can rebuild the server in case of a failure. ATA100 10K RPM disks Single CPU 1 GHz 1 GB RAM expandable to 2 GB Windows 2000 Server or Windows 2000 Advanced Server

For the class A ClearCase server, the recommended hardware is the IBM xSeries 235 Tower model with two 2.4 GHz processors, six 1 GB DIMMs, six 36.4 GB hot swappable hard disk drives (HDD), a 48x CD-ROM drive, and Servraid 5i Pro 1000/XT. The accessories include a 5Ux2D Tower to Rack Conversion Kit III, a 560W Hot-Swap Redundant Power Upgrade Kit, a U320 3-Pack HDD Upgrade Kit, a 3 x 36.4 GB 10K-2 80 SL Ultra160 SCSI option, and a 10/100/1000 network card. The main location at Westford opted for U320 and 15K/18K RPM disks for better performance due to the higher volume of users. For the class B ClearCase server, and both the ClearQuest servers, the project has so far relied on redeployed hardware and no specific selection of hardware to match the specification has been recommended.

Note that hardware performance and pricing is rapidly developing, and the preceding suggestions will quickly become sub-optimal. We have learned that ClearCase UCM uses resources more intensely than the base version of ClearCase. In the light of this newer ClearCase UCM experience, we now recommend the class A ClearCase server for up to 100 users or for up to 200 users with some performance-related inconveniences. In addition, we recommend a faster and more powerful machine for serving up to 300 users. Westford is the only site with more than 100 users. The ClearCase server in Westford has a high average CPU utilization and, although it largely meets the needs right now, it will need to be upgraded sooner than we had initially thought.

The use of RAID5 storage disks is not recommended by Rational, who explicitly recommends not using RAID5 with ClearCase. This is due to the frequent small database write operations done by ClearCase, which are poorly suited for RAID5. The choice of RAID5 was a cost trade-off based on both acquisition cost and available equipment. At first glance, the ideal system may have been a quad CPU system with six to eight GB of RAM using RAID0+1. But such a system is more costly than the class A ClearCase server and would also leave us with a single point of failure. Also, It probably would not address the CPU resource issue given current debate in the ClearCase community about whether or not more than two CPUs provide additional benefit. Thus, the ideal server arrangement for the Lotus Workplace project would perhaps have been two identical dual CPU systems using RAID5 with one used for redundancy purposes and an additional identical system per every 100 users to spread the load among a farm of servers. This solution would cost less, would have sufficient CPU resources, could be delivered quicker, and could provide redundancy. The same trade-off may not be a good choice for other projects. Each project must select an optimal approach based on the environment it operates within.

The ClearCase UCM servers for Lotus Workplace currently have HyperThreading enabled, but no tests have been done with these servers to verify that it improves performance.