TOWARD THE STANDARDIZATION OF VENTURE CAPITAL INVESTMENT EVALUATION: DECISION CRITERIA FOR RATING INVESTEE BUSINESS PLANS
Brent Mainprize, Royal Roads University, Canada
Kevin Hindle, Swinburne University of Technology, Australia
Brock Smith, University of Victoria, Canada
Ron Mitchell, University of Victoria, Canada
This study examined the criteria used by venture capitalists to evaluate business plans in order to make investment decisions. A literature survey revealed two competing theories: “espoused criteria” where evaluation decisions are based on what venture capitalists say are the decisive factors; versus the use of “known attributes” that successful ventures actually possess. Brunswik’s Lens Model from Social Judgment Theory guided an empirical investigation of several different evaluation methods based on information contained in 129 business plans submitted for venture capital over a 3 year period. Data evaluation culminated in the comparison of the percentage of correct decisions (“hit-rate”) for each method. We found that decisions based on the known attributes of successful ventures have significantly better hit-rates than decisions made using espoused criteria. Discussion centred on the goal of achieving consistency in the conduct of venture analysis. Process standardization can aid in the achievement of consistency. Future research will both deepen and broaden insights.
BUSINESS PLANS AND VC DECISION-MAKING
The screening and evaluation of business plans submitted by entrepreneurs is a major component of the decision-making process employed by venture capitalists (VCs) when they decide whether or not to invest in a new venture. At a general level, this study extends both the theory and methodology of research focused on venture capital (VC) decision making. At a specific level, it enhances understanding about the relevance and utility of entrepreneurial business plans in the new venture process. We questioned whether (and to what extent) the use of a decision-making aid might help VCs make better investment decisions. In our empirical study of 129 business plans and associated VC decisions, such an aid was demonstrated to be useful. The decision aid we tested was based on six known viable venture attributes (Mitchell 1995) and operationalized by a software program called the “New Venture TemplateTM” (abbreviation, NVTTM).
Venture capitalists employ a variety of criteria when evaluating potential investments in the screening phase (MacMillan, Siegel et al. 1985). The process of entrepreneurial business plan screening in the venture capital field can best be characterized by the “vital few and trivial many rule” (Pareto 1896). Pareto’s Principle, the 80-20 rule, is a useful heuristic that applies when there is a question of effectiveness versus diminishing returns on effort, expense, or time. Approximately 20% of the results (benefit to the funds that they manage) achieved by venture capitalists come from 80% of the companies they invest in. More important is the opposite side of the coin. Just 20% of the companies VCs invest in generate 80% of the total benefit to the fund (Zider 1998). The VC’s challenge is to distinguish the right 20% from the trivial many by using an effective evaluation process to screen out good investments from bad. Henceforth, we use the terms “screening” and “evaluation” as virtual synonyms. The screening/evaluation process involved in the investment decision was the focus of our study. We sought to contribute toward answering two questions fundamental to the field. What should be the basis of the decision criteria used by VCs to screen investment opportunities? How can the process be operationalized to improve accuracy and consistency?
As the principal tool of the screening decision, VCs rely almost exclusively on the entrepreneurial business plan (Roure and Keeley 1990; Hindle 1997; Zacharakis and Meyer 2000). Over the past 18 years, the majority of the empirical research into VC decision making has produced lists of criteria, which VC practitioners say that they use for this purpose (Tyebjee and Bruno 1984; Hall and Hofer 1993). Recent research on the VC investment decision process suggests that VCs lack a strong understanding of how they make decisions (Zacharakis and Shepherd 2001). In addition to lack of introspection, VCs are overconfident in their decision process and that overconfidence negatively affects VC decision accuracy (Zacharakis and Shepherd 2001). So, VCs “espoused criteria”—what they say that they do—may be a very poor basis for either understanding actual decision criteria or building guidelines and systems for improving performance in investment decision-making. Zacharakis and Meyer (2000) suggest that decision aids in the form of actuarial models may be useful tools for improving VC decision-making. Before examining the relevance of actuarial models (i.e. models that use specified criteria to derive an answer) we present a summary of the two principal schools of thought concerning VC decision making.
COMPETING THEORIES: “ESPOUSED CRITERIA” VERSUS “KNOWN ATTRIBUTES”
Do What I Say—The “Espoused Criteria” School
The majority of extant studies in the VC investment decision-making field belong to what may be called the “espoused criteria” school. They are based on what the VCs say (espouse) they use to screen investment opportunities. Research based on espoused criteria has relied on the results of surveys and questionnaires that provided “decision cues” for the researchers to create and test the effectiveness of their models of VC investment decision-making. A decision cue is a decisive factor that elicits a response in the judgement process. In seeking relationships between decision cues and the performance of the new venture, such studies have made significant contributions to our understanding of VC decision-making. Prior research focused on VC decision-making has determined criteria espoused by VCs using different emphases. These include: some form of counting (Benoit 1975; Tyebjee and Bruno 1984); rating scale (Wells 1974; Dixon 1991); ranking scale (Poindexter 1976; MacMillan, Siegel et al. 1985); trade-offs (Muzyka 1996). Investigators agree that espoused criteria from VCs often are not used in their entirety when investment decisions are made. Even if all criteria are used, the results of VC decisions suggest that VCs’ espoused criteria may not be optimal as the basis of either real world decisions or attempts to explain those decisions using research. Is there a stronger basis for studying VC investment decision processes?
Do What Works—The “Known Attributes” School
At the firm level of analysis, one of the goals of many entrepreneurship researchers has been the articulation of clearly recognizable attributes that distinguish viable, successful ventures from ventures prone to failure. The venture performance stream of entrepreneurship research, as a sub-unit of business strategy research, has concentrated on this task. Strategy researchers propose that superior performance arises from a fit between the competencies of a venture and the key success factors of an industry (Andrews 1987; Shepherd 1999). When applied to the study of VC investment decision making, this emphasis may be held to constitute the “known attributes” school, where success factors or viable venture attributes represent the requirements necessary for success within a particular industry.
A new venture team must commit to a number of viable venture attributes that, they believe, will lead to success within the competitive environment (Slater 1993; Shepherd. 1999). Viable venture attributes within an industry remain stable. Hannan and Freeman (1977; 1984) argue that organizations seldom succeed in making radical changes in their core strategy and structure in the face of environmental threats, because they are subject to strong inertial forces. Changes in the core lead to an increased probability of organizational failure and death (Hannan and Freeman 1977; 1984). Therefore, if a new venture is to succeed, the needed attributes at or near the time it is founded will vary little over its life. Accordingly, detecting the presence of attributes known to enhance venture viability and likelihood of success becomes critical to predicting the performance of a new venture.
In an extensive survey of prior work, Mitchell (1998) determined six independent attributes of viable ventures: (1) innovation, (2) value, (3) persistence, (4) scarcity, (5) non-appropriability, and (6) flexibility. He focused on venture attributes that are associated with profitability and survival. The model we tested in this study uses profitability and survival as dependent variables that are a function the six independent venture attributes. Further investigation of the literature suggests that the six viable venture attributes can be further broken down to 15 decision cues (Mitchell 1998). The dependent variables (profitability and survival), the 6 independent venture attributes and the 15 associated cues are set out in Table 1.
The network of concepts listed in Table 1 requires brief amplification. Several authors in the business administration and economics literature have argued that the profitability of a venture might be assessed by observing the levels of innovation, value and persistence. The foundation of a venture is innovation (Drucker 1985) which can be assessed by the level of new combinations (Schumpeter 1934) and the supporting product match with opportunities in the marketplace (Hayek 1937). Value in a venture appears at two levels: to the customer (as net buyer benefit), and to the venture itself (as margins and volume) (Ghemawat, 1991). The potential for the venture to persist over time can be observed through the repetitive and long-term purchase patterns that result from customer commitment (Ghemawat, 1991). The adequacy of resources (McMullan & Long, 1990; Stevenson, Roberts et al. 1994) is needed for growth.
Other authors in the strategy literature have argued that the survival or strategic viability of a venture can be assessed by examining scarcity, non-appropriability, and flexibility. Scarcity in a venture curtails the two conditions that can extinguish opportunity: imitation and substitution (Porter, 1980; Porter, 1985). Imitation increases supply and substitution decreases demand. Thus it becomes important to the survival of a venture to determine the level of non-imitability (Rumelt, 1987) and non-substitutability (Barney, 1991; Ghemawat, 1991). Appropriability arises from “hold-up” and “slack.” Hold-up re-distributes gains among economic actors (Rumelt, 1987) and slack decreases the rents from a strategic position in the vertical relationship between suppliers of customers (Ghemawat, 1991; Williamson, 1985). To reduce the potential of appropriation it is important that the venture has no hold-up and no slack. Lastly, flexibility is an attribute that enables an organization to be adaptive to changing environmental conditions (Collins and Porras 1995). For a venture to be flexible it must minimize uncertainty and reduce ambiguity to support and develop core competence that yields adaptive responses (Gersick 1991; Gresov; Romanelli and Tushman 1994).
The 6 viable venture attributes and the associated 15 cues, discussed above, are the basis of a decision aid called the New Venture TemplateTM (Mitchell 1995) presented in the next section.
Standardizing What Works: The New Venture TemplateTM Decision Regime
Several paper and computer based VC decision aids exist. They vary widely in their level of sophistication (Bell, 1991; Timmons 1994; Bowman 1997; Mitchell 1995). The New Venture TemplateTM (NVTTM) is a web-based software decision aid that enables a venture capitalist to standardize his/her approach to the business plan screening process. The decision aid uses 15 cues to assess the 6 viable venture attributes discussed in the previous section (see Table 1). The person evaluating business plans enters his/her responses to the 15 cues (into the software) using a 9-point Likert scale. From the responses to the individual cues, the software generates two graphs to summarize the analysis.
The first graph determines the “profile” of the venture by plotting the current grid position for the venture using two axes (dependent variables): (1) the potential profit and (2) the expected survival of the venture. Within this grid, a set of 14 venture prototypes are compared to the venture being evaluated to determine which prototype it is most highly correlated with. The 14 prototype profiles are scattered in four general categories: long-term/lower profit, long-term/higher profit, short-term/lower profit, and short-term/higher profit. The second graphical display is a radar chart indicating the evaluator responses to the 15 cues. It positions the venture under scrutiny in relation to the closest prototypical venture with which it is most highly correlated. Using the common cues provided by the NVTTM to rate each business plan enables the evaluator to consistently seek the specific information from each plan during the screening process. Consistency in evaluation is self-evidently desirable. Seeking and using similar information to evaluate a business plan is an important step towards standardizing the screening process. And standardization is consistency’s greatest friend, as will be discussed later in this study.
ACTUARIAL MODELS AS TOOLS FOR INVESTIGATING DECISION MAKING
Social Judgment Theory and Actuarial Modeling
Zacharakis and Meyer (2000) indicate that Social Judgment Theory (SJT) provides a theoretical reference to much of the past research on VC decision criteria. SJT evolved during the 1960’s and 1970’s as a methodology and a perspective for understanding human judgement. Within this field, a theoretical approach known as Brunswiks’s Lens Model (Brunswik 1956) has become a widely used, systems-oriented perspective for analysing human decision making. The Lens Model incorporates dual, symmetric sub-models of both the human judgement made by a person and the environment in which the results of the judgment are determined. The sub-model of human judgement (referred to as the “Cognitive Model”) provides an understanding of how the judge makes the decision in relation to the decision cues. The environmental sub-model (referred to as the “Environmental Model”) provides an understanding of what actually happened in relation to the decision cues. This allows the capture of “theories in use” as opposed to “espoused theories” of action (Hitt and Tyler 1991)—see Figure 1.
Psychology researchers Elstein and Bordage (1988:123) state that “actuarial (statistical) models refer to the use of any formal quantitative techniques or formulas, such as regression analysis, for . . . [deciding] clinical tasks” (c.f. Zacharakis and Meyer 2000). An actuarial model enables the judge to consider and rate individual cues independently and the actuarial model optimally combines the values assigned to each cue using a weighted algorithm to derive the answer. In several studies across a variety of fields, decision aids (actuarial models) have proven to be robust: only 6 of 117 studies found that clinical or intuitive decision making equalled or outperformed actuarial models (Grove 1986; Zacharakis and Meyer 2000).
Applying Actuarial Modeling to the “Espoused Criteria” School of Decision Making Theory
Zacharakis & Meyer (1998; 2000: 331) developed and tested what they called a “bootstrap” actuarial model based on espoused criteria. They found support for their hypothesis that “a bootstrap actuarial model of VC’s decision process better predicts actual outcomes than the VC’s own intuitive decision process.” However, if things were left here, we would still be lingering in the area of the “espoused criteria” school of VC decision making. Can actuarial modelling be applied to the “known attributes” school?
The percentage of correct investment decisions is referred to in the VC industry as a “hit-rate.” The effectiveness of VC decisions can be determined using the hit-rate. The average hit rate for VC decision-making is 20% at best (Zider 1998). In search of possible improvement of this general hit rate, it would seem desirable to test an actuarial model based on the decision-making principles of the “known attributes” school.
Applying Actuarial Modeling to the “Known Attributes” School of Decision Making Theory
We propose to test a cognitive system based on known attributes of venture success. Rather than using a “bootstrapping” actuarial model we will test a “venture attribute actuarial decision model.” Replacing the espoused “cues” with known, viable venture attributes has the potential to polish the theoretical lenses through which we study VC decision making. This will make a contribution to theory and not just be a use of theory: an important distinction made by Whetten (2001). In particular, the use of venture attribute cues (such as those shown in Table 1 and discussed, above) might help to achieve a much higher degree of standardization in the decision-making processes that VCs use to evaluate business plans.
Modeling and investigation based on espoused criteria holds out little hope of producing standardized approaches to decision making. And standardization is important because it will, as previously discussed, enhance the likelihood of achieving the desirable goal of consistency. Research in the VC decision-making field should, as one of its aims, seek to indicate guidelines, which if consistently applied, might enable a range of analysts to produce the same “invest” or “don’t invest’” decisions based on known, viable venture attributes. The desirability of greater standardization and less caprice in the VC investment process is analogous to enhancing the results of all high jumpers through creating uniformity, discipline and efficiency in the techniques (based on known biomechanical attributes) which they use to approach the bar. Standardization of good technique (improving methodology) enables every athlete to jump higher. Champions and freaks will still perform wonders—but general application of better technique raises the standard of the whole sport. This is an entirely different concept from the crude view of “standard setting” as just arbitrarily setting the bar higher without helping people to reach the new height.
Overview: What We Investigated and Why
The empirical component of this study had two theoretical homelands: Attribution Theory (Heider 1958; Kelley 1967; McArthur 1972; Kelley 1973) and Social Judgment Theory (Brunswik 1956). Its fundamental purpose was to understand the relationship between known, viable venture attributes (Mitchell 1998) and the effectiveness of VC investment decision-making. Generally stated, our empirical research objective was to compare the effectiveness of using “known venture attributes” as opposed to “espoused criteria” as the basis for VCs to screen entrepreneurial business plans. The evaluation method results in a “yes” or “no” investment decision. The efficacy of any investment method (or ‘model’ or ‘process’, call it what you will) can be measured by its “hit-rate,” which we defined, above as the percentage of correct investments. Accordingly, the empirical component of our study involved five hit-rate comparisons:
The overall objective was to see how well an actuarial decision model, based on the known attributes of successful ventures, and represented by the NVTTM regime, fared against alternative VC investment decision regimes. The detail of the research design follows.
Hypotheses
Can the NVTTM alone beat mere intuition? The 6 viable venture attributes (innovation, value, persistence, scarcity, non- appropriability and flexibility) discussed above are important to the success of a new venture. It is logical to develop a VC investment-screening aid that uses cues that are based on these viable venture attributes (see Figure 1 & Table 1). There is evidence to suggest that standardized knowledge structures are consistently related to performance (Charness, Krampe et al. 1996). Therefore, using the NVTTM (see above) as an example of a standardized attribution actuarial model, we proposed hypothesis 1a:
Hypothesis 1a: the standardized attribution actuarial model (NVTTM) will produce better predictions of actual venture outcomes than a trained evaluation team’s intuitive (unstructured knowledge) decision process.
Can a trained evaluation team using the NVTTM beat a group of venture capitalists using an espoused (bootstrap) criteria model? Venture capitalists reported devoting 8 to 12 minutes on average to evaluate a business plan (Sandberg 1986). Despite the relatively short analysis time devoted to each plan, Zacharakis and Meyer (2000) concluded that decision aids are under used in the VC industry” (p. 340) and that only 24% of VCs interviewed use some sort of checklist or tool to aid in the evaluation of entrepreneurial business plans. The goal of any decision aid is to provide assistance and structure to improve the accuracy and consistency of human judgment. Expert scripts are knowledge structures that, once developed, enhance the decision making process. The sequence of an expert script is relatively standard (Abbott and Black 1986) and are the basis for standardized actuarial models. The effectiveness of a standardized actuarial model hinges on the quality of the decision cues. We hypothesized that decision cues based on viable venture attributes as opposed to espoused criteria (bootstrap model) will enable an evaluator to more effectively predict the outcome of a venture.
Hypothesis 1b: when used as a business plan evaluation device in the VC investment screening process, the standardized attribute based actuarial model (NVTTM) will produce better predictions of actual venture outcomes than VCs using a bootstrap actuarial model.
Can a trained evaluation team using the NVTTM beat a group of venture capitalists using an environmental criteria model? The second sub-model from Social Judgment Theory, the environmental model, identifies how the decision cues are related to what actually happened (left side of Figure 1) as a result of the decision. Research in cognitive psychology (Dawes 1989) and a variety of other fields show that an environmental actuarial model will out perform VC experts because an environmental model uses only predictive variables and disregards non-predictive ones (Dawes 1989). We hypothesized that a VC using cues based on venture attributes rather than environmental cues is more likely to accurately predict the outcome of a venture.
Hypothesis 1c: the standardized attribution actuarial model (NVTTM) will produce better predictions of actual venture outcomes than a VC using an environmental actuarial model.
Can versions of the NVTTM (modified to predict profitability and survival) beat mere intuition? Profitability is cited as an important predictor for new venture success (McMullan and Long 1990; Shepherd, Ettenson et al. 2000). Survival—the probability that a venture will continue to participate in the market—has been found to be a predictor for new venture success (Shapero and Giglierano 1982; Birch 1988; Shepherd 1999). Isolating and testing both profitability and survival as a means of predicting new venture success is the basis for hypotheses 2a and 2b:
Hypothesis 2a: the standardized attribution actuarial model (NVTTM) using potential profitability as a dependent variable to predict success of a new venture will outperform a trained evaluation team’s intuitive prediction of success.
Hypothesis 2b: the standardized attribution actuarial model (NVTTM) using potential for survival as a dependent variable to predict success of a new venture will outperform a trained evaluation team’s intuitive prediction of success.
Population(s), Sampling and Data Collection
Our unit of analysis, the hit-rate, stems from decisions made about business plans. Our sample of business plans was taken from the population of US business plans seeking VC funding. For this study’s sampling frame we utilized data gathered from over 5 years of academic-practitioner collaborative efforts with a major North American venture capital conference provider, the Wayne Brown Institute (WBI) (see www.venturecapital.org). We examined 148 entrepreneurial ventures seeking venture capital funding. Each venture team submitted a business plan conforming to specific content guidelines (WBI format) that would best facilitate the rating of the plans using the NVTTM. Over the 3.3 year period from 1999 to 2002, companies submitted the “WBI format” business plans for acceptance into 7 conferences (2-3 annually) held in New York, Salt Lake City and Maui. The business plans represent ventures from the technology (hardware, software and Internet), biotechnology, manufacturing, retail, and service industries from 21 states across the USA. At the time of submission, each business plan was rated by one member of a team trained specifically to use the NVTTM. Each business plan was assessed on the six key attributes determined by Mitchell (1998) using the 15 NVTTM cues listed in Table 1.
Between January and March 2002, semi-structured telephone interviews were conducted (with the lead entrepreneur in the original business plan) to collect data on the actual outcome of the venture. Outdated contact information, inability to speak with the lead entrepreneur and unwillingness of the lead entrepreneur to participate in the interview caused 19 ventures to be removed from the study leaving a final sample of 129. The final data set includes: (1) 129 “WBI format” business plan submissions; (2) the corresponding 129 documented standardized screening/rating (values assigned to the 15 cues) decisions for each business plan at the time of submission and (3) the actual follow-up results (success or failure) for each of the 129 ventures as of March 2002.
Measurement Regime
The main hypotheses propose that the use of a standardized attribute-based actuarial decision model (NVTTM) to screen new ventures will more accurately predict the success of the ventures than VCs using bootstrapping models and environmental models. The hit-rate (previously defined) was the unit of measurement in this study. Determining the hit-rate requires two pieces of information; (1) the decision made (either by the VC or the actuarial model) and (2) the actual outcome of the venture. The actual outcome (success or failure) of each venture in the sample was ascertained through a semi-structured telephone interview with the lead entrepreneur.
The decisions made by the actuarial model (NVTTM) in Hypothesis 1a were determined as follows. First, each business plan was rated (on a nine point Likert Scale) by a member of the trained evaluation team associated with the Wayne Brown Institute that assigned values to the set of 15 standardized attribute-based cues (see Table 1). Next, the responses to the fifteen cues of NVTTM were combined into the six venture attributes and further combined into two variables (retaining the 1-9 scale), “potential profitability” and “potential survival” (see Table 1) using equal weighting. The two dependent variables, potential profitability and potential survival were then averaged to determine the coding of the VC response to success or failure as follows: 5.5 to 9 (success probability is high, therefore invest), 1 to 5.4 (failure probability is high, therefore don’t invest).
In hypotheses 2a and 2b the decision was determined using a similar procedure with one exception. In hypothesis 2a, only the 8 cues associated with profitability (see Table 1) were averaged to determine the decision (to invest or not). In hypothesis 2b, only the 7 cues associated with survival (see Table 1) were averaged to determine the decision (invest or not). Decisions made by VCs using the attribute based model and intuition (tested in Hypotheses 1a, 1b, 1c, 2a, and 2b) were recorded by determining the decision that was actually made (to invest or not) by the VC at the time of screening. The actual outcome was compared to the decision for each model to derive the proportions of correct decisions (hit-rates). The hit-rate rates for the bootstrap model (Hypothesis 1b) and the environmental model (Hypotheses 1c) were taken directly from the results of the Zacharakis and Meyer (2000) study.
Once the hit-rates were determined we compared them. We assumed that the hit-rates we discovered in our study and the hit-rates found in the Zacharakis and Meyer (2000) study were derived from the same populations of VC’s and business plan sources (United States VCs, United States companies,). This assumption enabled us to compare the hit-rates of the models between the two studies by using a Z test for difference in two proportions (independent samples). The Z value for each test was then converted to the corresponding p value using a standardized normal distribution table (one-tail) to determine if differences were significant. We employed a confidence interval of 95% for this study. Thus, if a p value in each of the tests was less than or equal to 0.05 we deemed that the evidence supported the hypothesis.
The results indicate that the attribution, bootstrapping and environmental models used in the VC decision process achieved different hit rates (see Table 2). The attribute-based actuarial model (NVTTM) we tested achieved a hit-rate of 64.3% compared a 51.9% hit rate for the trained evaluation team’s own intuition. The difference between the hit rates are significant (p=.022). Thus Hypothesis 1a is supported. Zacharakis and Meyer (2000) found that VC’s using a bootstrap (based on “espoused” criteria) and environmental models yielded hit-rates of 39.5% and 17.1% respectively. When used by a trained evaluation team, our model (the NVTTM) based on viable venture attributes, achieved a hit rate of 51.9%. The mean hit-rates suggest that a significant difference in both the bootstrap model (p=.003) and the environmental model (p=.000) exist when compared to our attribution model. Thus, Hypotheses 1b and 1c are supported.
The trained evaluation team using intuition achieved a hit-rate 51.9%. The attribute-based actuarial model, using only potential profitability as a predictor, achieved a hit-rate of 57.4%. The attribute-based actuarial model, using only potential survival as a predictor, achieved a hit-rate of 59.7% (see Table 2). There are no significant differences between the model using only profitability (p=.191) and survival (p=.110) when compared to VCs’ intuition. Therefore, hypotheses 2a and 2b are not supported.
Validity and Reliability Issues
The real-world, real-time decision data generated in this study proved valuable for testing our hypotheses but also raised four concerns that we addressed. First, the potential for personal bias by the trained evaluation team was reduced in two ways: (1) at the time of assessment the member of the trained team of evaluators (judge) had not met the entrepreneurial team or engaged to any significant correspondence with the entrepreneurial team, (2) the judge utilized the 15 cues of the NVTTM based solely upon information provided in the WBI format business plan submitted by each company via mail or email. Second, although there is a potential threat that the information in the business plans was inaccurate and carried over into the analysis, Roure and Keeley (1990) found that VCs rarely need to make “intense” correction in the information. Thus, we considered it reasonable to assume that the business plans were accurate enough for this study. Third, a minimum of 35 scenarios Stewart (1988; 1991) is typically deemed sufficient to accurately capture a subject’s decision policy. We substantially exceeded the minimum requirement. In our study, one judge rated 129 business plans (scenarios). Fourth, with a trained team of evaluators rating 129 business plans, the inter-rater reliability of the decision aid becomes important. As a test of the decision aid’s inter-rater reliability, 20 MBA students, using the NVTTM methodology, evaluated 300 business plans in a pilot study. Their inter-rater reliability was 95%.
Reconfirmation of the Value of Actuarial Models
Consistent with previous studies (Zacharakis and Meyer 2000), we found that actuarial models out performed evaluators using the models. More importantly, our study indicates that evaluators using cues based on known viable venture attributes are more effective in predicting the outcome of a venture than are espoused criteria. VCs think they know the “right” cues for predicting the outcome of a venture opportunity. However, prior research indicates the results of their decisions are poor (Zider 1998). The results of our study suggest that VCs may not know or do not use the right cues when screening business plans.
The Value of Standardization
It appears that standardizing the screening process is beneficial to the VC investment decision process and has the potential to improve the existing VC Industry performance norms characterized by Pareto’s 80:20 rule, thus increasing aggregate rate of return. Standardizing a process focuses on making it more consistent. Within the VC industry, enhanced standardization might provide more widely accepted “ground-rules” and cues for analysis and diagnosis. It might also help establish guidelines for the minimum acceptable standards in business plan writing for entrepreneurs, the financial community and academe. This could have the potential to reduce wastage of economic, human and emotional resources associated with business failure.
Future Research Directions
Our investigation of VC decision-making began with two questions. What should be the basis of the decision criteria used by VCs to screen investment opportunities? How can the process be operationalized to improve accuracy and consistency? Important new questions arise. What are the optimal decision criteria? How can they be most effectively communicated to VCs through an entrepreneurial business plan? In an extension of this study, we propose to use canonical discriminant analysis (StatSoft 2002) on data collected from our sample. Application of this analytical technique will allow us to determine the effectiveness of each attribute and its optimal weighting to further enhance the attribute based decision aid embodied in the New Venture TemplateTM (NVTTM).
This study has focused on decisions from outputs of a business plan. What about inputs: the material that should or should not go into an entrepreneurial business plan as it is written? Could there not be benefits from greater generic consistency in the writing as well as the reading (evaluation) of business plans? It seems at least possible that what Hindle calls “The Enhanced Entrepreneurial Business Planning Paradigm” (Hindle 1997), or another theoretically substantiated paradigm of the new venture articulation process, could be adapted to increase standardization of the business plans’ content and format (decision inputs) in much the same way that the model represented by Generally Accepted Accounting Principles (GAAP) did for financial reporting. This could be a very fruitful area of investigation.
Support for More Consistency in Venture Analysis
At this point in time, we realize that the field of venture capital investment decisions is not ready for “a” standard decision making process for evaluating business plans—or in any other aspect of venture capital decision-making. However, the findings of this study, in association with other research, do demonstrate that a foundation exists for more widespread application of known, viable venture attributes to the VC investment decision process. Perhaps the time is right for the formation of the Global Committee for Venture Analysis Standards called for by Mitchell (2001)?
Conclusion
Our study has shown that, compared to other methods, actuarial models based on known, viable venture attributes have potential to improve the likelihood of predicting the success of a potential investment when they are applied by a venture capitalist to the process of screening business plans. Their hit-rates were more accurate and consistent than hit-rates achieved by rival methods. Our research team hopes that this study will stimulate further research into both the VC decision-making process and the content and utility of entrepreneurial business plans as vital inputs to that process. The focus of future research should progress towards greater standardization and better performance in VC decision-making. Standardization is not an end in itself. But it could become a means of achieving more consistency and higher success ratios in an industry still marked by a high level of caprice and a low level of success in picking winners.
CONTACT: Brent Mainprize, Royal Roads University, 2005 Sooke Road, Victoria, BC, Canada; (T) 250-391-2645; (F) 250-391-2610; brent.mainprize@royalroads.ca
Abbott, V. and J. B. Black (1986). Goal-related inferences in comprehension. Knowledge Structures. J. A. Galambos, R. P. Abelson and J. B. Black. Hillsdale, NJ, Lawrence Erlbaum Associates, Inc.: 123–142.
Andrews, K. R., Ed. (1987). The Concept of Corporate Strategy. Homewood, IL, Irwin.
Bell, G. (1991). High Tech Ventures. Reading Massachusetts, Addison-Wesley
Benoit, J. L. (1975). Venture capital investment behavior: The risk capital investor in new company formation and expansion in France. Austin, TX, University of Texas.
Birch, D. L. (1988). The truth about start-ups. Inc. January: 14–15.
Bowman, C.W. (1997). Measuring Intangibles, The ProGrid Story. ProGrid Venture Inc see: www.progrid.ca .
Brunswik, E. (1956). Perception and the representative design of experiments. Berkeley, University of California Press.
Charness, N., R. Krampe, et al. (1996). The role of practice and coaching in entrepreneurial skill domains: An international comparison of life-span chess skill acquisition. The road to excellence: The acquisition of expert performance in the arts and sciences, sports, and games. K. A. Ericsson. Mahwah, NJ, Lawrence Erlbaum Associates: 51–80.
Collins, J. C. and J. I. Porras (1995). “Building a visionary company.” California Management Review 37(2): 80–100.
Dawes, R. M., Faust, D., and Meehl, P.E. (1989). “Clinical versus actuarial judgment.” Science 243: 1668–1674.
Dixon, R. (1991). “Venture capitalists and the appraisal of investments.” OMEGA 19(5): 333–344.
Drucker, P. F. (1985). Innovation and Entrepreneurship: Practice and Principles. New York, Harper and Row.
Elstein, A.S. and Bordage, G. 1988. “Psychology of clinical reasoning.” In J. Dowie and A.S. Elstein Eds., Professional Judgment: a reader in clinical decision making. Cambridge: Cambridge University Press.
Gersick, C. (1991). “Revolutionary change theories: A multilevel exploration of the punctuated equilibrium paradigm.” Academy of Management Review 16(1): 10–36.
Ghemawat, P. (1991). Commitment: The Dynamics of Strategy. New York, The Free Press.
Gresov, C., H. A. Haveman, et al. (1993). “Organizational design, inertia and the dynamics of competitive response.” Organization Science 4(2): 181–208.
Hall, J. and C. W. Hofer (1993). “Venture capitalists’ decision criteria in new venture evaluation.” Journal of Business Venturing 8(1): 25.
Hannan, M. T. and J. Freeman (1977). “The population ecology of organizations.” American Journal of Sociology 82: 929–964.
Hannan, M. T. and J. Freeman (1984). “Structural inertia and organizational change. 49:149–164.” American Sociological Review 49: 149–164.
Hayek, F. A. v. (1937). Economics and knowledge. Individualism and Economic Order. F. A. v. Hayek. London, Routledge and Kegal Paul: 33–54.
Heider, F. (1958). The psychology of interpersonal relations. New York, Wiley.
Hindle, K. (1997). An Enhanced Paradigm of Entrepreneurial Business Planning: develeopment, case applications and general implications, Unpublished disseration, Swinburne University of Technology Melbourne Australia.
Hitt, M. A. and B. B. Tyler (1991). “Strategic decision models: Integrating different perspectives.” Strategic Management Journal 12: 327–351.
Kelley, H. H. (1967). “Attribution in social psychology.” In D. Levine (Ed.), Nebraska Symposium on Motivation (Vol. 15, pp. 192–238). Lincoln University of Nebraska Press.
Kelley, H. H. (1973). “The process of causal attribution.” American Psychologist 28: 103–128.
MacMillan, I. C., R. Siegel, et al. (1985). “Criteria Used by Venture Capitalists to Evaluate New Venture Proposals.” Journal of Business Venturing 1(1): 119–128.
McMullan, W. E. and W. A. Long (1990). Developing New Ventures: The Entrepreneurial Option. San Diego, CA, Harcourt Brace Jovanovich.
Mitchell, R. K. (1995). New Venture Template. Software based venture analysis tools.
Mitchell, R. K. (1998). Possible standards for the comparison of business ventures. Remarks presented at the 1998 USASBE Conference Symposium: New venture evaluations: Is there a standard method on the horizon?, Clearwater, FLA.
Mitchell, R. K. (2001). “Vision Statement: Global Committee For Venture Analysis Standards: 2004.”
Muzyka, D. (1996). “Trade-offs in the investment decisons of European venture capitalists.” Journal of Business Venturing 11(4): 273–287.
Pareto, V. (1896). “The New Theories of Economics.” Journal of Political Economy 5.
Poindexter, E. A. (1976). The efficiency of financial markets: The venture capital case. New York, Unpublished doctoral dissertation: New York University.
Romanelli, E. and M. E. Tushman (1994). “Organizational transformation as punctuated equilibrium: An empirical test.” Academy of Management Journal 36: 701–732.
Roure, J. B. and R. Keeley (1990). “Predictors of success in new technology based ventures.” Journal of Business Venturing 5: 201–220.
Rumelt, R. P. (1987). “Theory, strategy and entrepreneurship.” In D.J. Teece (Ed.) The Competitive Challenge: Strategies for Industrial Innovation and Renewal: 137–158. Cambridge, MA: Ballinger.
Schumpeter, J. (1934). The Theory of Economic Development. Boston, MA, Harvard University Press.
Shapero, A. N. and J. Giglierano (1982). “Exits and entries: A study in yellow pages journalism.” Frontiers in Entrepreneurship Research. K. H. e. a. Vesper. Wellesley, MA., Babson Center for Entrepreneurial Studies: 113–141.
Shepherd, D. A., R. Ettenson, et al. (2000). “New venture strategy and profitability.” Journal of Business Venturing 15(5–6): 449–467.
Shepherd., D. A. (1999). “Venture Capitalists’ Assessment of New Venture Survival.” Management Science 45(May 1999): 0621–0632.
Slater, S. F. (1993). “Competing in high velocity markets, Indust.” Marketing Management 24: 255–268.
StatSoft, Inc. (2002). Electronic Statistics Textbook. Tulsa, OK: StatSoft. WEB: http://www. statsoft.com/textbook/stathome.html.
Stevenson, H. H., M. J. Roberts, et al. (1994). New Business Ventures and the Entrepreneur. Homewood, IL, Irwin.
Stewart, T. R. (1988). “Judgment analysis procedures.” In B. Brehmer and C. Joyce, eds., Human Judgment: The SJT View. North Holland, Elsevier.
Timmons, J. A. (1994). New venture creation: Entrepreneurship for the 21st century. Homewood, IL, Irwin.
Tyebjee, T. T. and A. V. Bruno (1984). “A model of venture capitalist investment activity.” Management Science 30(9): 1051–1056.
Wells, W. A. (1974). “Venture capital modelling: Evaluation criteria for the appraisal of investments.” The Financier ACMT 1(2 (May)): 54–64.
Whetten (2001). “Modeling-as-Theorizing: A Methodology for Theory Development” to appear in Essential Skills for Management Research, David Partington (ed), Sage, Ltd.
Williamson, O.E. 1985. The economic institutions of capitalism. New: The Free Press.
Zacharakis, A. L. and G. D. Meyer (2000). “The potential of actuarial decision models An earlier version of this paper was presented at the 1996 Academy of Management Meetings in Cincinnati.” Journal of Business Venturing 15(4): 323–346.
Zacharakis, A. L. and D. A. Shepherd (2001). “The nature of information and overconfidence on venture capitalists’ decision making.” Journal of Business Venturing 16(4): 311–332.
Zider, B. (1998). “How Venture Capital Works. The Discipline of Innovation.” Harvard Business Review 76(6): 131–139.
2002
Babson College. All Rights Reserved. Last Updated March 2003.