Sweet Spot

Based on research by Haiyang Li and Jade Yu-Chieh Lo

• Functional diversity on innovative projects has a more complex effect than once thought.
• Diversity, in this context, refers to differences in the primary function or specialization of participating firms that are collaborating on projects.
• Technical reviewers react neutrally to industry diversity. Business reviewers react to it less well.

When fund reviewers consider a pitch for an innovative project, does the specialization of the collaborators matter? Does it matter who the reviewer is? To find out, Haiyang Li, a professor of management in the business school, and Jade Yu-Chieh Lo from Drexel University studied a range of collaborative innovative projects that were funded by the Advanced Technology Program (ATP) at the U.S. National Institute of Standards and Technology (NIST).

In order to be considered for funding, the proposed projects had to be both collaborative and innovative. They also needed to show commercial potential that could lead to broad-based economic gains.

For each proposal, a group of business reviewers evaluated the possible economic benefits; then technical reviewers decided if the project showed technical innovation and feasibility.

Some background on the reviewers: when ATP gets a proposal, independent peer reviewers are invited to assess the technical and commercial merits of the pitch. The technical reviewers come from NIST’s technical staff, federal laboratories and the scientific community; the business reviewers are hired consultants with expertise in business operations, venture capital, economics and business development.

While past researchers have studied how participant diversity colors the performance of collaborative innovative projects, Li and Lo’s study examined the links between a project’s participant diversity and its merits from the viewpoint of both technical and business reviewers.

To reach their conclusions, the researchers looked at 138 collaborative projects in 29 technology categories that were funded between 1993 and 2001. For each project, Li and Lo then assigned values for three characteristics:

1. Participant diversity: How diverse was the expertise of the firms that proposed a project? The authors determined this with a formula based on the primary Standard Industrial Classification code for each participating firm.
2. Novelty: Was the project really novel? The scholars rated each project they studied on a scale of 1 to 4, based on the participating firms’ assessment of whether their proposed project reflected a new R&D direction for them.
3. Fuzziness of a technology category: Each year, the NIST assigned its proposals to a technology group (such as electronics) and then to a category within that group (such as semiconductors). The researchers ranked the “fuzziness” of a technology category based on a formula similar to the one they used to rate diversity among the collaborators.

Armed with these descriptors, Li and Lo then analyzed ratings from the project reviewers, comparing those from the business reviewers to those from the technical reviewers. While the business reviewers looked at the projects through a market-value filter, the technical reviewers looked at them through a filter of technological advancement.

As most entrepreneurs know, advanced innovations often are not the most economically successful. So it was to be expected that the two sets of reviewers would view projects differently. For reviewers judging from a research and development perspective, cross-fertilization of ideas signals heightened creativity. Thus, technical reviewers should not necessarily have a negative reaction to participant diversity.

From a market value perspective, however, a project that overlaps into more than one industry category could have trouble connecting with a target market, or may find its resources spread too thin, or may add unnecessary collaboration costs. So for a business reviewer, participant diversity may lower confidence in a proposal’s market potential.

The researchers found that business reviewers and technical reviewers did indeed react differently to proposals by a diverse cast of firms. Though diversity didn’t significantly affect the ratings from the technical reviewers, it prompted a markedly negative reaction from the business reviewers.

While the business reviewers gave lower ratings to diverse projects, however, the effects of that diversity were mitigated when a project was novel or when its category was distinctive.

In fact, projects with diverse participating firms but high novelty scores were spared lower ratings from business reviewers. Meanwhile, diverse projects that were in technology without clear categories saw the negative reaction intensified. When a project fell into a clearly identifiable category, the effect was the opposite: There was actually a positive, albeit marginal, relationship between participant diversity and higher business reviewer scores.

A project’s diversity, in other words, affects more than just its team members. In projects that are based on innovation, diversity or lack of it also influences the reviewers who may hold a proposal’s fate in their hands. The project’s definability–whether its participants fall into one easily identified category or blur into several–also affects reviewers’ reactions. It’s a dilemma that demands deeper attention, both from project innovators, who need the rich intellectual capital of a varied team as well as a profile that attracts financial capital, and from researchers, who need to learn more about how the two types of resources best can be balanced.

Haiyang Li is a Professor of Strategic Management at Jones Graduate School of Management at Rice University.

To learn more, please see: Lo, J.Y. & Li, H. 2018. In the eyes of the beholder: The effect of participant diversity on perceived merits of collaborative innovations. Research Policy, 47(4), 1229-1242.