Automating data transfer and replication

Increasingly, complex computer technologies are replacing traditional techniques for organizing and processing information. Managing the associated electronic data in these complex systems often involves manually shuffling data from one location to another. For example, you might need to copy files from one directory to another, move files between computers, or transfer information from one database to another. These seemingly simple tasks often result in decreased productivity and inefficient use of computer resources. The optimal solution would be to automate such tasks and schedule them to occur when most convenient. One service that offers such a solution for Windows NT is OpalisRendezVous 1.5, a data transfer and replication tool distributed by Opalis.

OpalisRendezVous uses the concept of data flows to organize data movement through your NT network. You use the service's graphical program, Administrator, to define flows. The Administrator features an easy-to-use interface that lets you select the data you want to flow, where you want to send it, and which transfer protocol to use. You can choose from several protocols, including an NT Copy transfer, an FTP transfer, an Open Database Connectivity (ODBC) query, and Opalis' proprietary RendezVous protocol (which is well suited for coordinated transfers between dial-up endpoints). The OpalisRendezVous Standard Edition supports local and outgoing flows, and the Plus Edition includes capabilities for outgoing and incoming flows. After you define the flows, OpalisRendezVous' Service program processes the flows based on their specified scheduling. When the Service executes a flow, it automatically logs a summary report to the Administrator and, at your request, sends email to additional recipients.

Installing OpalisRendezVous is simple, requiring only a few minutes and essentially no configuration. You simply select the type of account you want this NT service to run under. This account must allow a range of rights appropriate for the intended tasks. OpalisRendezVous is an NT service, so you can easily change the account that it runs under to meet new demands. After you set up the account, you can take advantage of OpalisRendezVous' powerful capabilities, such as mirroring, anonymous FTP sites, and ODBC functionality.

Mirroring
If you need to mirror the contents of a directory hierarchy to enable a consistent view from multiple access points, you can open the Administrator, right-click the mouse, and access the interface for defining a new data flow. Under the When tab, select a trigger that executes the flow each time a user modifies a file in the mirrored directory. Under the What tab, select the directory to be mirrored and choose to include all its subdirectories in the flow. Under the Where tab, select the computer and directory that host the mirror, choose NT Copy as the means of executing the flow, and specify a direct connection between the source and destination. You can expect this flow to execute at a high frequency, so you can leave the After tab blank. (This tab lets you perform follow-up actions for flow executions.) Under the Options tab, fine-tune the functionality of your mirror. Overwrite existing destination files and maintain the original file data to keep the contents as consistent as possible. To minimize the com- puter resources required for the copies, copy only those files that users have modified. In addition, select the Replicate Deletion option to remove files from the mirror directory when users remove their counterparts from the source directory. Finally, name the flow and add it to the Administrator.

The Administrator passes responsibility for the flow to the service, which immediately begins to monitor the source directory. The first time a user modifies, adds, or removes a file from the source directory, the Service executes the flow and recursively copies all files to the mirror directory. Then the Service copies or deletes only the files necessary to maintain consistency between the source and mirror directories.

Anonymous FTP Sites
OpalisRendezVous can maintain anonymous FTP sites. These sites let arbitrary users upload documents that the software later makes available for official download. For example, suppose your process requires that users upload files to an incoming directory, after which you download and verify the documents and then upload them to an official release directory. To set up this process, you go through the same set of tabs used in the mirroring example. If you choose to download new files Sunday through Thursday at 11:00 p.m., you can specify that the FTP take place over a dial-up line that is usually idle at that hour and enter the appropriate Remote Access Service (RAS) and FTP server connection information. After specifying which local directory will receive the files, you can use the After tab to email daily flow reports to the site administrator. To avoid losing documents, you can have OpalisRendezVous rename any file that has the same name as one currently in the destination directory. You can also have the Service move rather than copy files to clean up the incoming FTP directory during each execution of the flow.

You can take advantage of the FTP scripting capabilities included in OpalisRendezVous. Under the FTP Advanced tab, you can invoke a script after the Service establishes the connection between the source and the destination, before or after each file transfer, or before the connection terminates. If you don't have defined scripts when you click your selection, the Administrator gives you an editor to build a script with. After you build your script, you select a binary transfer, name the flow, define it as an incoming flow, and save it to the Administrator, which then passes it on to the Service. As you review and approve the downloaded files, you can move them to a new directory, from which a similar flow can upload the files to their official release directory.

ODBC Functionality
ODBC functionality is another feature included in OpalisRendezVous. As a product distributor, you might implement a database system for processing customer orders. Sales representatives enter the relevant information into database tables as they receive orders. If you want to transfer a subset of this information (i.e., the part number and quantity) to your stock manager's database and add in a requirement for high- and low-priority order, you can use the ODBC functionality feature.

Screen 1 shows how to set priorities. To handle the different priorities, you need to define two separate flows. For the high-priority orders, trigger the flow to execute once each hour; for low-priority orders, trigger it to execute once each day. For both priorities, limit the trigger activity to normal business hours. Use the What tab to define your ODBC origin and destination sources and the mapping of columns between the tables. Under the Option tab, specify a Where clause to indicate which columns to transfer to the destination table. You enter OrderTab.Priority = 1 for the high-priority flow and Order Tab.Priority = 2 for the low-priority flow. To avoid duplicate transfers, define an After transfer query that sets the value of the originating table's priority field to 3.

The Verdict
OpalisRendezVous definitely has advantages. The installation and initial configuration are simple, and the interface is well designed and easy to navigate. But the software has some drawbacks. The online and hard-copy documentation is sketchy, and I had to experiment with several options before I understood their full functionality. Also, some of the error messages that incorrect flow definitions generated were unclear and resulted in access violations. Nevertheless, if you are looking for a way to solve your data transfer and replication problems, don't let these minor drawbacks deter you. OpalisRendezVous is a powerful, feature-packed tool.

OpalisRendezVous 1.5
Contact: Opalis * 770-622-2840, Web: http://www.opalis.com
Price: Starts at $350
System Requirements: Windows NT Server or Workstation 3.5 or later, 3MB of hard disk space