Using Ex-Ante and Ex-Post Benchmarks in Estimating Damages
By Donald D. May PhD, Managing Director, SEDA Experts LLC
Benchmarks are the basis for damages. But when the assumptions behind the benchmarks fail to stand up to Daubert scrutiny, expert testimony may be excluded.
The purpose of this article is to demonstrate that “traditional” valuation methods that incorporate ex-ante data can be inappropriate when used to estimate lost profit or lost enterprise value in litigation; and the appropriate method should be based on ex-post data, i.e., data available after the event in which damage occurred.
Impact on the Plaintiff
In civil litigation, litigants, juries, and courts struggle with the issue of how an alleged loss should be compensated. What is a reasonable estimation of the impact on the plaintiff of the event under dispute?
In the corporate setting, that impact generally involves some combination of
(a) lost profits and
(b) lost enterprise value through reduced business valuation.
To estimate these reasonably and fairly is an extremely complex matter. Some of the elements, such as damage to reputation, are arguably subjective. Even the more objective elements, such as lost income, are not necessarily straightforward. They require the development of two alternate histories: one of them concerning what happened in fact, the other representing what might have happened “but for” the event in question, i.e., had the event in question not occurred.
To arrive at claims for damages, litigants turn to expert witnesses—in particular, financial, economic, and accounting experts—who, it is assumed, can apply established models and methodologies. These experts set benchmarks that can be used to measure performance and determine, in ways that are authoritative and will be clear to the court, how much was lost.
Benchmarking, in this context, is the process of establishing the basis for the hypothetical, “but for” figure.
By definition, a benchmark is a standard. In general, benchmarking involves comparing a known figure—say, a performance metric—against an accepted standard.
In damages estimation, benchmarking involves the use of an economic standard to prove the validity of the hypothetical but-for profits. The challenge is to arrive at a standard that the court will accept.
Benchmarks for damages can be established in several ways.
One is to use management’s own forecast of future performance—to say, in effect, “This is how management believes the company would have performed if not for the harm.”
Another way is to use the forecast of independent securities analysts: “This is how independent experts projected the future performance of the company.” A third is to create a benchmark based on the performance of a peer group: “This is how comparable companies performed that were not subject to harm.” Creating and defending these benchmarks is the work of expert witnesses with backgrounds in valuation and/or financial economics.
Calculating Lost Profits
To measure lost profits and thus establish the basis for damages, the expert must calculate the following:
Profits had the harm not occurred - Actual profits = Lost profits
Similarly we can think of lost enterprise value as the present value of the stream of lost profits. Effectively, lost profits and lost enterprise value have their basis in the same question: What was the harmful impact on profits of the event in question?
Actual profits are easy to calculate, as data is readily available. But “profits had the harm not occurred” is a hypothetical figure. It is based on an alternative history, a scenario in which the harm did not occur.
Since there are no actual figures to apply to this alternative scenario, the hypothetical profit must be calculated. The calculation must give but-for profits a basis in reality. If the calculation is realistic, the court can compare the two scenarios and determine the extent of the harm.
For too many litigants and their attorneys, the participation of expert witnesses provides no assurance of success, as benchmarks used by expert witnesses are increasingly the basis of successful Daubert challenges. The use and effectiveness of Daubert challenges are on the rise. According to one study, in 2010 almost 50 percent of expert witnesses were excluded on the basis of Daubert challenges. Many of these challenges focused on the assumptions behind benchmarks and their reliability. In the case of Celebrity Cruises v. Essef, in which I served as a consulting expert on behalf of Essef, Judge Francis excluded five of seven Celebrity plaintiff experts and one of three defendant experts on Daubert grounds. We will examine the Daubert aspects of this case in detail.
Too Many Assumptions, Wrong Assumptions
Why is benchmarking so often unreasonable? The process of benchmarking involves making assumptions about
(a) future business performance, (b) economic conditions, and
(c) the competitive landscape of the litigant’s industry.
As we all know, making assumptions is a dangerous business. The particular problem here is that any assumption will ramify through years of projected business or financial performance.
Even a small error in the assumption can lead to a wildly inaccurate conclusion. In addition to the general problem of making assumptions, there are specific problems as well. One is reliance on a one size-fits-all approach.
Many experts rely on a single benchmarking methodology that they apply across a wide range of situations. The methodology may or may not fit the situation. If it does not, a successful Daubert challenge is likely to result.
Ex-ante and Ex-post Benchmarks
There is a closely related problem: Many experts apply ex-ante benchmarks, the kind of forward-looking analysis appropriate to litigation for shareholder damages, to the calculation of lost profits and lost enterprise value, where ex-post benchmarks ought to apply. Ex-ante benchmarks use information that was available only before the event (in this case an outbreak of Legionnaires disease on a 1994 cruise). Ex-post benchmarks use data available after the event. Valuation experts and investment bankers use metrics that are designed to forecast future performance. They examine past performance, then look ahead in order to create a prediction, using data that might include the company’s own forecasts.
In estimating damages associated with events that lead to lost profits or injured business reputation (lost enterprise value), the task is different—the starting point is a given date in the past, the date when the alleged harmful event took place; and the requirement is to try to calculate two different scenarios. One scenario reflects what actually happened, and the other reflects what might have happened if the event had not occurred. There is more and different data available, including information about how the company, the industry, and the economy actually performed. None of that would apply in projecting a valuation for shareholder damages, but it is essential for calculating lost profits or lost enterprise value.
Many experts have had their testimony excluded specifically because they used valuation-type metrics in cases involving damages from harmful acts.
Benchmarking for damages to shareholders resulting from accounting or financial fraud does, however, need to be based on forward-looking valuation methods. It is critical that experts and litigators know the different requirements and apply the right benchmarking methods for the particular type of case.
Questionable Benchmarking in Celebrity Cruises
What is the impact of questionable benchmarking practices on a case? For a dramatic example, one need look no further than Celebrity Cruises, Inc. v. Essef Corp., 434 F. Supp. 2d 169 - Dist. Court, SD New York 2006 No. 96 CIV. 3135(JCF). Celebrity Cruises is an upscale cruise line. It was an independent company until Royal Caribbean acquired it in 1997. In summer 1994, there was an outbreak of Legionnaires’ disease, an acute respiratory infection, aboard one of Celebrity’s ships, Horizon, during a New York-to-Bermuda cruise. A number of passengers sued Celebrity and also Essef, the manufacturer of the ship’s whirlpool spa and filters, which, according to the U.S. Centers for Disease Control and Prevention, were the source of the outbreak. In a bellwether procedure, where a verdict for one plaintiff determines liability for all parties, Celebrity was found to be 30 percent liable for the outbreak, and Essef 70 percent liable. Damages were awarded to the bellwether plaintiff, a married couple. Celebrity then filed its claims against Essef, seeking compensation for direct costs (such as the damages it paid to the plaintiffs and the cost of decontaminating the ship) as well as lost profits during the period from the outbreak to its acquisition by Royal Caribbean, and lost business value. Celebrity claimed it lost profits because prospective passengers did not book, or the line had to discount to get passengers to book, because of the stigma caused by the reported Legionnaires’ disease. In addition, Celebrity claimed that because of the outbreak, its purchase price was lower than it would otherwise have been.
Why the Experts Were Excluded
Celebrity Cruises identified experts to testify concerning all categories of damages. Essef chose not to challenge the Celebrity experts who addressed direct costs and damages already paid. Essef did challenge the experts who addressed the more hypothetical claims about Celebrity’s lost profits and lost business value. Essef also challenged a Celebrity expert who opposed Essef’s motion for summary judgment. Celebrity, in turn, challenged each of Essef’s three experts. The challenges produced a sort of expert witness bloodbath.
Here are some of the most critical challenges and how they turned out:
Challenge 1: Tracking lost profits-using a price-increase spike.
Expert A testified for Celebrity and developed an estimate of lost profits. He linked profits to pricing, then compared Celebrity’s pricing to a market proxy, using a comparison period from March 1993 to June 1994, the period before the outbreak. He then projected the pricing forward to show what profits would have been had there been no outbreak. On examination, it turned out that Celebrity’s prices were significantly lower than the proxy’s in March 1993. Over the next 15 months, Celebrity’s prices rose dramatically, closing the gap. But there was nothing to indicate that Celebrity’s upward trend would have continued. If anything, prices might have fallen as Celebrity rolled out three massive new ships, adding capacity that might have more than absorbed existing demand. In fact, Celebrity’s own management pointed to a weakening of demand in 1995. There was no evidence that Celebrity’s pricing could have continued to outpace the market. Expert A’s testimony was found to lack the reliability required under Daubert, and it was excluded.
Challenge 2: Transaction benchmarks that compared apples to oranges.
Expert A also analyzed Celebrity’s lost enterprise value, comparing the Celebrity acquisition to two other large industry transactions: the acquisition of Norwegian Cruise Line by Star Cruises and the merger of P&O Princess with Carnival. But the selected guideline transactions were not really comparable. The scales were radically different—Celebrity had 8,000 berths, Norwegian Cruise Line 12,000, and P&O Princess 30,000. The other two transactions were subject to competitive bidding, whereas the only other offer for Celebrity was a last-minute affair. Finally, the other transactions took place roughly two-and-a-half years after Celebrity’s, and Expert A did not account for market changes over the intervening years that might have affected the relative value of the sales. His lost-enterprise-value analysis was also excluded.
Challenge 3: Projecting lost enterprise value using forecasts instead of peer or industry performance as the basis.
Expert B projected an expected growth rate for Celebrity’s revenue which in turn was based on a proxy consisting of Royal Caribbean and Carnival. She took projected growth rates for each of the two companies as established by analysts in 1994, and applied them to Celebrity. The problem is that the market proxy did not in fact display anything close to the projected growth rates. According to analysts, Royal Caribbean’s revenue was expected to grow at rates ranging from 2.5 to 4.9 percent between 1994 and 2000; Carnival was expected to grow at rates ranging from 0.6 to 2.1 percent. As it happened, Royal Caribbean’s growth in 1995, 1996, and 1997 was 0.21 percent, -1.08 percent, and -1.25 percent, respectively. Carnival’s growth rate during the same three years was -2.6 percent, -3.7 percent, and -1.7 percent. Expert B admitted in her deposition that she did not know these actual figures were available, and later acknowledged she would have considered them had she known. Forward-looking methodology such as this can be appropriate for valuing an enterprise at a single point in time. But it does not adequately measure damages that occur after the point when the projection is made. In this case, all three companies lost ground. But Expert B attributes Celebrity’s shortfall to the effects of Legionnaires’ disease, while making no attempt to determine why the other two companies declined. Even worse: Expert B’s lost enterprise value calculation began with Celebrity’s 1993 budget projections, which she then compared to 1997 actuals. As the court pointed out when he excluded her evidence:
Indeed, to take 1993 as an example, Celebrity budgeted earnings before interest, taxes, depreciation, and amortization (EBITDA) of $61.9 million for the Horizon and its sister ships, the Zenith and the Meridian, but the actual EBITDA for those three vessels in that year was $55.4 million. Using [Expert B’s] methodology, this would indicate damages of over $83 million for that year, even though the Legionnaires’ outbreak had not yet occurred.
Challenge 4: Taking the company’s word for growth rates.
Expert C did without industry proxies. Instead, he estimated projected profits based on Celebrity’s own five-year plan as formulated by management in January 1994. He compared anticipated profits with actual profits and concluded that Celebrity had lost approximately $101 million. He then adjusted his projections in various ways, calculating higher or lower lost-profit figures, but always using the company’s own projections as the basis. The court rejected this approach, stating:
[Expert C’s] lost profits analysis is flawed in at least one major respect: the projection of profits based on Celebrity’s five-year plan is wholly unreliable. [T]he entrepreneur’s “cheerful prognostications” are not enough. Schonfeld v. Hilliard, 218 F.3d 164, 173 (2d Cir.2000) (quoting Dobbs Law of Remedies, by Dan B. Dobbs, §3.4).
Indeed, Expert F (see Challenge 7, below), another of Celebrity’s experts, explicitly rejected use of the five-year plan to project anticipated profits after December 31, 1994. To calculate the value of the company after the outbreak, Expert C began with the $1.312 billion purchase price paid by Royal Caribbean, then adjusted it for anticipated synergies. But he did not take into account that the purchase price, including the synergies, was negotiated between Celebrity and Royal Caribbean. Celebrity might have lost bargaining power because of the Legionnaires’ outbreak, or there might have been balanced negotiations. In either case, the synergy figure is the result of a negotiation—it is too subjective to be the basis for an enterprise value calculation.
For Expert C’s excessive reliance on the company’s own projections, the court noted, “A methodology so sensitive to one highly subjective variable lacks the necessary reliability.” Thus Expert C’s testimony was excluded.
Challenge 5: Using more sophisticated methodologies…that still rely on the company’s growth projections.
The lost-profits analysis by Expert D also relied on Celebrity’s five-year plan. He took a more conservative approach than Expert C—for example, excluding out of- pocket and “brand repair” costs—but still used Celebrity’s projections as the basis for his calculations, projections not borne out in reality. The court noted:
[Expert D’s] analysis suffers from the same fatal flaw as [Expert C’s] methodology: reliance on projections that were not borne out in reality. This defect drives the entire calculation and is not repaired by identifying a lower bound using a methodology, which, standing alone, might be more reliable. [Expert D’s] lost profits analysis is therefore excluded.
This fatal flaw is the basis for the entire calculation, and using lower boundaries for the estimate does not offset the fundamental problem. His lost enterprise value analysis used six different sets of calculations to arrive at an average figure, but still relied either on Celebrity’s five-year plan or a 1995 projection prepared for the company by The Blackstone Group, which was nearly identical. Expert D developed a reasonable rate of return analysis, but did not justify it by comparing it to other companies.
Challenge 6: Using benchmarks of company suffering alleged stigma does not control for stigma to other ships.
Expert E testified for Essef, providing an expert opinion that criticized the reports of Celebrity’s witnesses and also included an independent evaluation of Celebrity’s damages. The centerpiece of Expert E’s report was his evaluation of Celebrity’s lost revenues. He began by identifying a relationship in revenue per passenger cruise day produced by Horizon and her sister ships, Zenith and Meridian, before the Legionnaires’ outbreak. Expert E then observed that revenue declined sharply for Horizon and Meridian in the quarter immediately after the outbreak, but not for Zenith. By the fourth quarter of 1994, Meridian’s revenues had returned to their pre-incident pattern, and by the third quarter of 1995, Horizon had as well. Expert E then compared the actual revenues for these three vessels to that which he projected based on their prior performance, and concluded that Celebrity experienced a revenue shortfall of approximately $5.7 million during the third quarter of 1994, $1.6 million in the fourth quarter of 1994, and $325,000 in the second quarter of 1995. The court rejected this benchmarking approach, stating: “The fallacy of this analysis is that it assumes that the Zenith was unaffected by the Legionnaires’ incident.” Expert E’s testimony was thus excluded.
Challenge 7: Bad yardsticks.
The only Celebrity expert to testify at trial was Expert F. He presented a lost-profits analysis showing the impact of the outbreak on Celebrity’s EBITDA, comparing it to a yardstick made up of three other cruise lines. He also developed a lost enterprise value analysis based on discounted cash flows, using a formula to determine the weighted average cost of capital (WACC). Based heavily on his testimony, the jury awarded Celebrity damages of $190 million for lost profits and lost enterprise value.
Essef moved for a new trial, claiming there were several serious flaws in Expert F’s presentation. Essef contended that the companies that made up the yardstick were too dissimilar to be useful— one was industry giant Carnival, with billions in revenue and multiple brands; the other was a company that mainly operated riverboat cruises. Finally, and most significantly, Essef argued that in developing the lost enterprise value calculation, Expert F had ignored a basic tenet of WACC calculation: he failed to take into account the impact of debt on the capitalization of the business. He used a generalized cost-of-capital figure for the cruise industry, but did not “relever” it to apply a debt figure to the yardstick companies or to Celebrity. In so doing, he ignored the advice he had laid out in his own textbook. As he later acknowledged, he misspoke and said that he had applied a “levered Beta,” when he clearly had not.
Consequences: Drastically Reduced Award
As a result of the flaws in Expert F’s testimony, the court reversed the verdicts for lost profits and lost enterprise value. At a new trial, this time seeking damages of roughly $60 million rather than the $190 million originally awarded, Celebrity and Expert F again presented the yardstick analysis with the two comparable companies that had been criticized by the court. Expert F added a third comparable company that was considered more reliable. Yet his analysis reached a similar valuation. This time the jury returned $15 million in damages to Celebrity. On appeal, the court found the evidence sufficient to support the award, but applied a 30 percent reduction for comparative fault. Celebrity Cruises, Inc. v. Essef Corp., 530 F. Supp. 2d 532 (S.D.N.Y. 2008).
In February 2008, the district court entered a $30.4 million judgment against Pentair, the company that had acquired Essef, for out-of-pocket costs, expenses, and lost profits, including interest. Celebrity and Pentair both appealed. According to a 10-Q filing by Pentair on July 22, 2008, the parties agreed to a $35 million settlement—a reduction from the $190 million jury verdict, and one attributable in large measure to the unreliable benchmarking produced by Celebrity’s experts. Essef also had some of its experts excluded on Daubert; and of the Essef experts who did testify, some of their testimony was excluded as well. But the impact of the challenges fell most directly on Celebrity and had the greatest effect, a negative one, on Celebrity’s case.
Summary: Benchmarks Rejected and Accepted
The experts in Celebrity were excluded because they used inappropriate benchmarks, and so the court ruled their estimates of damages to be unreliable.
Some of the experts used Celebrity’s own forecast prior to the damages event to benchmark for but-for profits. That is, they said that had the filters not gone bad, Celebrity would have met its forecast performance, based on its own management forecasts. The court called this unreliable, quoting Dobbs Law of Remedies to the effect that the entrepreneur’s “cheerful prognostications are not enough.”
Other experts used securities analysts’ forecasts for Celebrity’s peer group as the benchmark. Celebrity was not a public company at the time. The court determined that this approach was also unreliable because those forecasts did not account for actual economic conditions that occurred subsequent to date of damages—conditions that would have also affected Celebrity’s performance. In fact, we showed, and the court quoted the fact, that if that benchmarking method were used, it would result in something approaching $60 million in damages in the year prior to the actual date when the Legionnaires’ outbreak occurred!
I should note that the court also ruled that one of Essef’s own experts be excluded. This was because the expert used the performance of other Celebrity ships as a benchmark for how the affected ship would have performed. The court deemed this benchmark unreliable and biased downward because if there was stigma caused by the Legionnaires’ disease outbreak, it would have affected all of Celebrity’s ships and not just the ship that had the outbreak.
In the end, the court admitted testimony only by those experts who used the yardstick approach to benchmarking for but-for performance. The yardstick approach involved benchmarking the performance of peer cruise lines after the outbreak of the disease, and then applying that benchmark to how Celebrity should have performed. In other words, if Royal Caribbean or Carnival or a combination of the two saw their revenues decrease by 10 percent in the period following the outbreak, we would expect Celebrity’s revenue to decrease by the same amount in the same period. If, for the sake of argument, Celebrity’s revenue had actually declined by 25 percent after the outbreak, we would have concluded that 10 percent was due to factors not related to the outbreak, and 15 percent was due to the outbreak.
In the matter of ex-ante vs. ex-post benchmarks: In general, when you calculate a valuation, your benchmark (that is, what you expect performance to be after the valuation date) is typically based on factors that you are forecasting prior to the valuation date, e.g., economic growth. These are ex-ante benchmarks because they relate to information available prior to the valuation date.
However, the Celebrity case shows that this type of approach may fail in court. It is better in this instance to use ex-post benchmarks—factors that you know occurred after the event, such as how the economy or industry actually performed after the event (in this case, the outbreak of disease). Judge Francis specifically excluded experts who used benchmarks based on traditional valuation methods in favor of ex-post approaches. While ex-post approaches are typically either not available or are deemed inappropriate for typical valuations, they should be used in estimating damages because they control for other unrelated factors that are important in estimating lost profits that result from a single, specific event.
A Better Approach to Benchmarking
As the Celebrity case shows, to be defensible, benchmarks need to conform as closely as possible to business and economic reality. In practice, that means applying ex-post benchmarks: measurements applied at the end of the period under scrutiny, which factor in such details as operating costs, fluctuations in business demand, and changes in the economy.
Ex-ante benchmarks, the kind of forward-looking assumptions used by investment bankers, are error-prone and are best avoided. Worse is using an exante projection that is based solely on the plaintiff’s own forecast, like the management forecasts used to predict but-for performance in the Celebrity case. An expert who understands the operating realities of a business is an essential partner in arriving at a reasonable benchmark.
Donald D. May, PhD, as a former MIT finance professor with extensive testifying experience, Dr. May is a world class damage expert. With his invaluable experience in testifying in arbitrations as well as state and federal court, Dr. May represents what sets us apart from others our solid knowledge base and experience. Don May possesses over 30 years’ experience in consulting, valuation and litigation support as well as researching, publishing and teaching at the university level. His experience includes implementing a broad range of damage analyses and valuations for businesses of various sizes and in numerous industries.