Ten Commandments of Computerization
[This document is being used by CIPS as a basis for the discussion of best practices development.]
© 1992 Osler, Hoskin & Harcourt
This article originally appeared in The Osler Outlook, a publication of the Canadian law firm Osler, Hoskin & Harcourt, and is reprinted here with the firm's permission.
Part One: The Technology Imperative
The T.J. Hooper was an ocean-going tug. With its sister ship, the Montrose, it plied its trade along the Eastern seaboard of North America, not infrequently towing barges of coal from Virginia to New York City. The tugs' captains were experienced seamen familiar with the treacherous weather that could be encountered on such a journey, particularly north of Cape Hatteras. Indeed, it was the practice of these seamen to take the shelter of a promontory of land known as the Delaware Breakwater if they perceived signs of ominous weather in the vicinity of Hatteras. On the journey in question, the weather was fair as the Hooper and the Montrose passed the Breakwater, but only half a day out they encountered a vicious gale which sank the barges they towed. Free of their loads, the robust little tugs were able to rescue the barge crews and weather the gale. But when they arrived back in port, representatives of the Northern Barge Company informed the captains of their intent to sue.
T.J. Hooper v. Northern Barge was a famous case in its day, holding the tugs liable for their failure to have on board state-of-the-art technology. In 1932, that "state-of-the-art technology" was a radio receiving set, capable of picking up the weather broadcasts of the fledgling National Weather Service, which had taken to broadcasting maritime weather reports twice a day. The Hooper and the Montrose had on board what was customary in the merchant marine of 1932 - a transmitter, to call "Mayday" - but the court said that there are precautions so imperative that no industry or trade may be excused for their lack, even if the disregard be universal, and that the newness of the technology is no defence.
The Hooper case stood virtually alone as a jurisprudential landmark on liability arising from the use (or non-use) of technology in business - much discussed by legal commentators but not cited by judges in decided cases - for nearly 40 years. In 1992, Hooper no longer stands alone and the cases which define today's technological standard of care, beginning with a few scattered cases in the early 70s, have become a comprehensive body of law from which ten important business guidelines, or commandments, may be drawn.
The First Commandment - The Duty to Adopt Technology
As computers become more pervasive and their widespread use becomes more commonplace, negligence may arise from a failure to use computer systems which prevent damage and harm. This is essentially the Hooper principle. The essential elements in determining whether failure to use computers is negligent were perhaps most succinctly stated in another American case called United States v. Carroll Towing: (i) the probability of harm; (ii) the gravity of the foreseeable injury; and (iii) the burden of adequate precautions.
Under that approach, if the gravity of the anticipated injury were low, or if the expense of the precautions were so burdensome as to be out of proportion to the risk and gravity of harm, a duty may not arise; conversely, if the probability and gravity of harm were relatively high and the burden of the precautions were low, a duty would arise. Thus, a duty to adopt a technological solution would arise if, as is now so often the case, the technology favourably altered the relative balance between the probability and gravity of harm, on the one hand, and the burden of the solution, on the other hand.
A 1986 American case affirming the principles in Hooper and Carroll Towing, and dealing specifically with the failure to use a computerized system is United States Fire Insurance Co. v. United States. In that case the Coast Guard used a manual method - known to be inferior to available computer methods - to locate a wreckage. As a result, buoys to warn vessels of the wreckage were misplaced and an accident ensued. The government was found negligent for using the inferior manual method because, had it used the computer-based methodology, the wreckage would have been properly located and the damage would have been prevented.
The Second Commandment - Technology Drives the Standard of Care
In Canada, H.E. Kane v. Coopers & Lybrand decided that the standard of care for any activity will rise as more sophisticated technology becomes available. In other words, not only does the use of a computer system not lower the standard of care (the third commandment), the existence and availability of computerized systems in the marketplace may actually have the effect of raising the standard of care for any given activity.
Thus, the second commandment is that the failure to incorporate into a system up-to-date features based on reasonably available technological advances, or to implement compensating controls, may lead to liability for negligence. This same principle has been expressed in management terminology by Dr. Peter Keen, a leading expert on the effective use of information technology in business, in describing the extraordinary competitive advantage achieved by American Airlines when it introduced the first-of-its-kind SABRE reservation system: "Technology relentlessly redefines the base level of service".
If this commandment becomes universal, as we expect it will, it would become necessary, as the available technology improved, either to upgrade computer systems by installing the new technology or to implement compensating controls to make up for its lack. The essential elements in determining negligence for failure to "keep up" would be: (i) whether superior and reliable technology is reasonably available; and (ii) whether the provider of service was aware, or should have been aware, of the superior technology.
The Third Commandment - Accepting Responsibility
"It's a computer error." A halcyon cry of the modern age, rivalled only by "The computer is down" (which we will deal with below), these words invariably are accompanied by a shrug of the shoulders. A mildly glazed expression in the eyes of their speaker seems to complete the disclaimer, "Of course you understand, it's not my fault. You can't expect to hold me responsible for what this machine does."
The Federal Court of Canada says that this sort of abdication of responsibility is unacceptable, and its 1981 decision in 87118 Canada Ltd. v. The Queen gives us this third commandment: the use of a computer or new technology does not lower the standard of care by which a provider of service is bound. In the words of Mr. Justice Addy:
|where ordinary human skill and expertise is replaced by such [complicated mechanical and electronic] devices, the persons employing them do so at their own peril and remain subject to the tests as to performance which would otherwise prevail... .|
The Fourth Commandment - Preserve the Ability to Address Exceptional Circumstances
The fourth commandment is perhaps a corollary of the third: an appropriate level of service must be provided by a computer system despite the computer's lack of judgment and inability to exercise discretion. In State Farm Mutual Auto Insurance Co. v. Bockhorst, a U.S. court decided that:
|...holding a company responsible for the actions of its computer does not exhibit a distaste for modern business practices ... . A computer operates only in accordance with the information and directions supplied by its human programmers. If the computer does not think like a man, it is man's fault .|
The Ontario Court of Appeal felt the same way in Remfor Industries Ltd. v. Bank of Montreal, a case in which a customer attempted to stop payment on a cheque. The customer mistakenly gave the bank the incorrect amount of the cheque, although it furnished the bank with the correct date, cheque number and name of payee. The cheque was not stopped, and the bank was held liable for having improperly debited the account. In imposing liability, the Ontario Court of Appeal ruled that the bank was unable to rely on the defence that its computer was not programmed to handle the discrepancy.
Similarly, in an American case, Pompeii Estates Inc. v. Consolidated Edison Co., the court specifically rejected a defence based on the fact that the computer software was incapable of exercising the discretion of a human being. In that case a public utility company relied only on computer records to turn off the heat in a house during the winter. The court found that discretion should have been exercised under the circumstances:
While a computer is a useful instrument, it can not serve as a shield to relieve Consolidated Edison of its obligation to exercise reasonable care when terminating service. The statute gives it discretionary power to do so, and this discretion must be exercised by a human brain. Computers can only issue mandating instructions - they are not programmed to exercise discretion.
Computer professionals will rightly take issue with the court's assertion in Consolidated Edison that computers cannot be made to deal with discretionary situations or unusual circumstances. Indeed they can, although doing so goes to the heart of system design. One approach is to incorporate "screening", or what is sometimes called "exception reporting", into the design of a system. Any circumstances falling outside prescribed norms will precipitate the issuance of a report, meant for a human supervisor's attention and, ideally, the questionable transaction would not be further processed until the human supervisor rules on it. That would likely have produced a different result in Consolidated Edison and in Remfor Industries.
An alternative (or supplement) to screening or exception reporting is the so-called "expert system", a computerized body of expertise that purports to capture the wisdom of acknowledged experts in a comprehensive set of rules, or "if/then" statements, which it uses to resolve exceptional circumstances. Expert systems are generally regarded as being an extremely sophisticated implementation of technology, and that brings us to consider a fifth commandment.
The Fifth Commandment - Don't Get Too Far Ahead
"It's leading edge technology!" You've probably heard this more than a few times from erstwhile modern "pioneers". "Leading edge" or not, courts in Canada, the United States, the United Kingdom and Australia are telling us that the newness of the technology is not the relevant test. What is relevant is whether the system enables the provider of service to meet the duty of care. Thus, the fifth commandment is that a provider of service must ensure that a new computer system is at least as effective and reliable as the system, be it manual or automated, that it replaces.
Mr. Justice Addy put it rather succinctly in 87118 Canada Ltd. v. The Queen :
|Where a service ... has been computerized and has not been rendered properly, it is no answer ... for the person who has chosen to install the computerized system to establish that it was as efficient a computerized service as could be reasonably furnished having regard to the state of the art at the time. Before installing such a service, or at least before relying on it in substitution for a previously existing [system], the person rendering [the service] must satisfy the Court that the new automated service is as efficient as the previous existing [service].|
Part Two: Coping with the Technology
As computers become more pervasive, and ever more critical functions are assigned to them, owners and operators of computer systems must adjust to new duties (or, perhaps, new interpretations of duties they have always had). In this Part Two of the Ten Commandments of Computerization, we examine the management responsibilities that accompany the use, and benefits, of technology.
The Sixth Commandment - The Duty to Secure
In the summer of 1988, diagnostic computers at three Michigan hospitals were infected by a virus that transposed patient names and medical information. The hospitals involved say that no patients were harmed but the results could easily have been fatal. In a test of security at a major European airport, a computer expert took only 17 minutes to penetrate the air traffic control computer with a virus and take control. In several countries, renegade computer hackers have taken up blackmail, infecting computers with a virus and then offering to sell the "vaccine" to the victims.
The duty to protect a computer system and the information held in it is a concept whose time has come. The importance of information and know-how in an increasingly information-based and globalized economy, and the vulnerability of computers and networks to unauthorized intrusions, impels enterprises to protect themselves against industrial espionage. Many enterprises are the custodians of the proprietary information of others, with a corresponding contractual obligation to preserve its commercial value by keeping it confidential. A variety of statutes permitting computerized record keeping, from corporations statutes to health information legislation, also include a statutory duty to take necessary and/or reasonable precautions to maintain accuracy, and to prevent unauthorized access while making the information promptly available to those with a right to see it.
Thus, the sixth commandment is that the failure to implement, and ensure the observance of, reasonable security measures may lead to liability. Non-existent, inadequate or circumvented security procedures have now been the basis for damages in several decided cases. In Yohay v. City of Alexandria Employees Credit Union, the U.S. Court of Appeals for the Fourth Circuit held a subscriber to a consumer reporting agency's database liable for failing to secure against unauthorized disclosure under a statutory provision strikingly similar to consumer reporting legislation in force in eight of ten Canadian provinces. The court in Yohay cited the laxity with which the defendant managed its computer access procedures:
|"...it is to be noted that the [defendant] had not posted any guidelines to users of the computer, informing them of the circumstances under which credit information could be obtained. Indeed, the [defendant] had posted the code which provided access to the computer system, enabling anyone with the physical opportunity to use the system to access [the credit information]."|
The Yohay decision provides a useful counterpoint to a 1990 British case, Denco Ltd. v. Joinson, where it was held that unauthorized use of another employee's password was just cause for dismissal. It is a virtual certainty that the Denco case would have been decided differently if the procedural laxity criticized in Yohay had been present, since Mr. Joinson would have had very little, if any, indication that his employer took security seriously.
The Seventh Commandment - The Duty to be Accurate
The computer system proprietor is responsible for ensuring that reasonable skill, care and diligence is used in compiling, inputting, verifying and retrieving the information held in the computer system so as to preserve its accuracy.
At common law, tort actions have been used to assert some control over the accuracy and dissemination of collected information: principally the tort of defamation, which requires that published statements be true, and the tort of breach of confidence, which may arise where personal information received in confidence or as a result of a particular relationship is divulged without permission. Perhaps the most famous defamation case in the computer realm is a 1985 decision of the U.S. Supreme Court, Dun & Bradstreet v. Greenmoss Builders, in which a credit rating firm was held liable for defamation for failing to ensure the accuracy of its computerized business reports.
This duty to be diligent also has broad application to the legal effectiveness of computerized records, and we are beginning to see a number of cases where critical evidence is being challenged, with increasing success, on the basis of credibility. That is, the computer-based evidence is admissible, under various exceptions to the hearsay rule, but the finder of fact (judge or jury) is exercising its inherent discretion to disbelieve the evidence, as it would the testimony of an unreliable human witness, because of discrediting evidence as to the likelihood of it being accurate (e.g., the frequency of errors produced by the system, the types of safeguards, etc.).
A dramatic example of society's increasing dependency on accurate information retrieval is an action for malpractice and breach of contract that has recently been launched in the United States against a doctor who artificially inseminated a woman and against the sperm bank which stored her husband's sperm after he had died of cancer. The woman wished to be inseminated in order to have a bond in her late husband forever. However, the child born to her was found not to be genetically linked to the late husband and the error was made more apparent by the fact that the child was a different race than both the mother and intended father. The error arose from an inaccurate computer record and the case will raise many complex and untested issues, including whether claiming damages for a healthy child of a different race and for raising an inter-racial child is against public policy.
The Eighth Commandment - Use the Technology You've Got
Perhaps most controversial of all is the eighth commandment: if harm could have been prevented by fully utilizing the available features of an operative computer system, liability for negligence may accrue to the system's owner for failure to use the system to its full capability.
The leading authority supporting this proposition is a 1988 case, Sun Bank/Miami, N.A. v. First National Bank of Maryland, in which the defendant bank was found negligent for failing to fully utilize the existing computer system capability which would have prevented the loss arising from a $150,000 forged cheque. In another American case, Becnel v. Answer, Inc., it was held that the failure to issue a pay cheque within the statutory time limit is negligent where access to data indicating that the payment is due is readily available.
This proposition may, at first blush, be horrifying to system managers who are responsible for many operative systems where all of the features have never been activated. It is not, however, "Catch-22"; remembering that liability is based on a negligence analysis, there may be many good reasons not to activate a particular feature of a computer system, including risk disproportionate to the benefit, which may provide a defence to a negligence claim. What this commandment underlines, therefore, is a need for rational and thorough analysis, duly documented - due process, if you will - in the course of the system procurement process, from initial definition of system needs, through to system implementation, to document why certain features were not activated, either at a specific time or at all. Simply put, responsible computing begins with system procurement.
The Ninth Commandment - The Duty of Reliability
The computer system proprietor is responsible for reasonable care and skill in ensuring that the computer system is programmed, maintained and monitored in a manner so as to enable its tasks to be effectively completed.
In Shell Pipeline Corp. v. Coastal States Trading Inc., Shell was found liable for its design of a computer system to manage "in-line transfers" of product in a gas pipeline system which did not include any mechanism for identifying misdirection of product resulting from a similarity of the buyers' names. In an Ontario case, F.B.D.B. v. Registrar of Personal Property Security, the plaintiff succeeded at first instance in its claim that the registrar had negligently designed the computerized personal property security database so that it failed to identify previously registered security interests if a debtor's middle initial was included as requested in the search form. The registrar prevailed on appeal, where it was shown that, of three types of searches available within the system, the plaintiff had misunderstood the nature of the search it selected, notwithstanding adequate explanations in user guides (and in subsequently failing to assert its rights, was therefore the master of its own misfortune - Welcome to the Computer Age!).
Another, more tragic, example is Brown v. United States, in which three fishermen, having left harbour in Hyannis, Mass. in reliance on a National Weather Service forecast of fair weather, drowned when their boats met 100 mph winds and 60 ft waves. The suit by their families was successful primarily because the government had failed to repair an electronic data buoy that, had it been working properly, would have increased the accuracy of the forecast (on appeal, the National Weather Service was able to persuade the court that it should be permitted to shelter under the so-called "newspaper exception", where courts have held, in the public interest, that newspapers generally are not liable for third party actions taken in reliance on the accuracy and completeness of their stories).
The Tenth Commandment - Planning for Disruption
"The computer is down." And of course no one would be so crass as to expect a provider of service to discharge his responsibilities in such unhappy circumstances?
Not so. If a reasonable person would foresee that the making of contingency plans for a computer system interruption or failure is necessary to avoid harm to someone to whom a duty of care is owed, the failure to put reasonable contingency plans in place, and periodically to test their effectiveness, may render the system proprietor liable.
In a recent Canadian case, Moss v. Richardson Greenshields of Canada Ltd., a breakdown in a communications link used for trading in call options resulted in a sale and cancel order being suspended. The Manitoba Court of Appeal expressed the opinion that, but for a contractual limitation clause precluding liability, the defendant stock broker would have been negligent for failing to anticipate and make contingency plans to deal with the breakdown. Regulators are also getting into the act. In the United States, for example, the Office of the Comptroller of Currency requires national banks to develop "alternate data processing capabilities", which capabilities must be "testable", and has stated its policy of holding a bank's board of directors responsible for an annual review of the adequacy of data processing contingency plans.
The negligence analysis - in which the taking of reasonable measures is a defence to liability - may pale in comparison to the potential absolute liability for contractual and statutory obligations. Absent exculpatory language in the contract or in the statute, non-delivery or tardy performance by virtue of the failure of one's own computers will almost inevitably amount to a breach of obligation, with the corresponding potential for damages (and, in the case of contract, financial penalties for delay, termination of contract, and so on).
A newly-reported case from Ontario that combines elements of the seventh, eighth and ninth commandments has to do with the seemingly sudden demise of Bargain Harold's, a chain of 160 "convenience discount stores" located in six provinces. Between the time of its acquisition by new owners in October 1990 to June 30 1991, the financial affairs of Bargain Harold's were entrusted to a new computerized accounting system, featuring computerized point-of-sale units in each store with centralized computer facilities.
The first difficulty was that the new computer system cost $15 million instead of $8 million (an 88% over-run). In the end, that was less of a problem than what was euphemistically described in Bargain Harold's Discount Ltd. v. Paribas Bank of Canada as "undetected errors" in the new computerized accounting system - errors thought to be responsible for changing a modest anticipated profit of $500,000 in the first year of operation by the new owners, to a serious loss, last estimated to be in excess of $20 million.
Indeed, so complex was the situation and so far had it gone undetected by management, that Coopers & Lybrand, the auditors of Bargain Harold's, were unable, by the time the case was heard in February 1992, to unravel the muddle (leading Mr. Justice Austin to make the remark widely reported in the press, "the patient is bleeding to death, but the doctors are unable to determine why"). The final indignity, and the death knell for the enterprise, was Mr. Justice Austin's refusal to grant protection to the company from its creditors under the Companies Creditors' Arrangement Act, because the company could not explain what had gone so wrong with its financial systems and, presumably as a result, could not advance any coherent plan for its reorganization with a reasonable chance of meeting the approval requirements of the Act.
And so, unlike the other Ten Commandments, these are not written on tablets of stone. They will evolve, and there will be more. The perceived benefits of computers, and the numbing array of problems facing mankind, will continue to contribute to the uncritical, if not eager, adoption of technological solutions. Moreover, technological capabilities are increasing at an accelerating pace, permitting ever larger and more sophisticated systems to be conceived, and ever more sensitive and critical functions to be assigned to them.
As a result, there is an abundance of emerging duties, just beginning to be outlined in cases and legislation from all over the world. For the time being, however, these ten commandments are clear.