|
|
          |
 |
Agricultural Research Assessment: A Symposium Summary August 1995
Acknowledgements This symposium was cosponsored by the Council on Food, Agricultural, and Resource Economics (C-FARE); the ESCOP Social Sciences Subcommittee of the National Association of State Universities and Land Grant Colleges; the USDA Cooperative State Research, Education, and Extension Service; the Farm Foundation; and the Regional Research Project NC-208, "Impact Analysis and Decision Strategies of Agricultural Research." We are grateful to the following agricultural scientists who came to Washington, DC in August 1995 to discuss issues in implementing, conducting and using agricultural research assessments: John Antle, Montana State University; Peter Barry, University of Illinois; Bruce Bullock, University of Missouri; Joe Coffey, Southern States Cooperative, Inc.; Kelly Day, Economic Research Service, USDA; Thayne Dutson, Oregon State University; Mike Martin, University of Minnesota; George Norton, Virginia Polytechnic Institute and State University; Susan Offutt, National Research Council; Phil Pardey, International Food Policy Research Institute; Mike Phillips, Office of Technology Assessment; Kitty Smith, Henry A. Wallace Institute for Alternative Agriculture; Karl Stauber, USDA; Bob Thompson, Winrock International; Sally Thompson, University of Illinois; Mason Wiggins, U.S. Congress; Cathy Wotecki, Office of Science and Technology Policy. In addition, we would like to thank the individuals representing agribusiness, the executive and legislative branches of the federal government, state and local government, academia, professional societies, and public interest groups who actively discussed issues in agricultural research assessment. Many of their ideas are captured in this summary.
C-FARE Board of Directors OFFICERS Walter J. Armbruster, Farm Foundation, ChairPeter J. Barry, University of Illinois, Vice-Chair Rueben C. Buse, University of Wisconsin, Secretary Treasurer BOARD MEMBERS Henry M. Bahn, USDA/CSREESJohn Baritelle, Baritelle Vineyards Sam Cordes, University of Nebraska Mark R. Drabenstott, Kansas City Federal Reserve Harold M. Harris, Clemson University J. Charles Headley, University of Arkansas James R. Nelson, University of Idaho Susan E. Offutt, National Research Council Alfred L. Parks, Prairie View A&M University J.B. Penn, Sparks Companies, Inc. Michael J. Phillips, Office of Technology Assessment Bob H. Robinson, USDA/ERS Katherine R. Smith, Henry A. Wallace Institute for Alternative Agriculture STAFF Tracy Irwin Hewitt, Washington, DC Representative
Executive Summary This report summarizes the findings of "A Symposium on Agricultural Research Assessment" held in Washington, DC, on August 29, 1995, co-sponsored by the Council on Food, Agricultural, and Resource Economics; the ESCOP Social Sciences Subcommittee of the National Association of State Universities and Land Grant Colleges; the USDA Cooperative State Research, Education, and Extension Service; the Farm Foundation; and the Regional Research Project NC-208, "Impact Analysis and Decision Strategies of Agricultural Research." The symposium sought to broaden the understanding of research assessment across the full spectrum of agricultural scientists, to illustrate the benefits of applying the assessment process to research agendas at the state and federal levels, and to demonstrate the importance of effective communication of research assessment information to public and private decision makers. Because of increased competition for public dollars, decisions about public funding of agricultural research will increasingly rely on the anticipated value and usefulness of research findings. The symposium presentations and discussion indicated that while many important research assessments have been conducted, much remains to be done. The scientific community needs to better understand the role and benefits of research assessment analysis. Standard assessment tools, such as rate of return analysis, may have to be re-examined so that politically important distributional effects can be captured. Interdisciplinary teams need to be created and encouraged so that data can be integrated across disciplines and used in comprehensive research assessment analyses. Finally, agricultural scientists need to clearly and effectively communicate the benefits of agricultural research to taxpayers, policy makers, and other agricultural stakeholders.
Introduction Decisions about public funding of agricultural research increasingly rely on the anticipated value and usefulness of the research findings. Who will benefit, when, by how much, and what will the research cost, are key questions for scientists, policy makers, and research leaders throughout the agricultural sciences. So are questions about environmental impacts and broader social benefits. To answer these questions, both ex ante and ex post assessments of research payoffs are receiving greater attention. Despite the growing interest, the experience base and expertise for conducting research payoffs are not widespread. Much of this base rests with social scientists, including a group of agricultural economists who have focused on measuring rates of return and multiplier effects of research. The need is urgent, however, to broaden the understanding of research assessment analysis across the full spectrum of agricultural scientists, to apply the assessment process to research agendas at the state and federal levels, and to effectively communicate research assessment information to public and private decision makers. In response to these and other concerns, the Council on Food, Agricultural, and Resource Economics (C-FARE) co-sponsored an event in Washington, DC on August 29, 1995 entitled, "A Symposium on Agricultural Research Assessment." C-FARE is a nonprofit, organization of agricultural economists which has as one goal the identification of key economic issues in agriculture. The focus of the symposium was on understanding the key issues in conducting and using research assessments to enhance the overall contributions of the agricultural research system. An invited audience of over 80 individuals representing agribusiness, the executive and legislative branches of the federal government, state and local government, academia, professional societies, and public interest groups actively discussed issues and concerns about agricultural research assessment. Participants identified factors that facilitate assessments, the types of assessments needed, and ways to better communicate the assessment results to decision makers. This summary report highlights the major points raised in each presentation and discussion. Included are: summaries of what both public and private sector decision makers need from agricultural research assessments; the principles, processes, and products of agricultural research assessments; interdisciplinary issues for conducting assessments; and examples of assessments conducted by a variety of experts. The final two sections of this summary describe some of the issues raised in the discussion including: What kinds of assessments are needed? What do we need to do to get these assessments done? and What characteristics of assessments will increase their influence on decision making?
I. Keynote Presentation Robert L. Thompson, President, Winrock International Competition for public funds is intense. Agricultural research now competes with other programs like K-12 education, welfare, and Medicaid for limited public dollars. This competition makes it essential that agriculturalists are able to demonstrate the benefits of agricultural research to taxpayers, relative to other social investments. Thompson suggests that the recent decline in funding for agricultural research, relative to other research, is due to inadequate communication of the benefits of agricultural research to policy and decision makers. The best way to address this communication problem is for researchers and research administrators to answer two fundamental questions about research in plain, simple English. These are: Who cares? and So what? By answering these questions, both before and after the research is conducted, it is more likely that both applied and basic research will address important real world problems. Also, it is more likely that the public will better understand the value of the research. To effectively answer these questions, researchers need credible measurements and appropriate incentives. Credible measurement requires reliable data on impacts across different groups of agricultural stakeholders including consumers, farmers, environmentalists, and conservationists. It also requires honesty about successes, failures, and the time it takes to do research. Not all research is successful. Similarly, not all research takes years to finish. Rewards should be given for socially beneficial, high-priority research that is clearly communicated, not for the number of articles published in referred journals, or the amount of competitive grants money a researcher secures. Multidisciplinary research is essential in both the reward and incentive structure because it directly addresses broad based problems and it helps tell the story of agriculture's contribution to the public good. Thompson also raised a number of side issues that detract from the reputation of research. There is, he stated, a perception that the land grant colleges and universities are resistant to change, and that there is duplication of research across state lines. Others believe that research and extension priorities are not well linked. He also noted that many observers find estimated rates of return for agricultural research implausibly large. In conclusion, he stressed his main point -- that agricultural research is under-funded largely because of our failure to clearly communicate the benefits of agricultural research to taxpayers, policy makers, and other agricultural stakeholders.
II. Users of Research Assessment: A Federal Perspective Karl Stauber, Under Secretary for Science, Education, and Economics, USDA Public skepticism about the government is pervasive. In the past, we operated under the assumption that all knowledge is good. And, if it is good, it is also valuable. These arguments are losing their weight with the public. Stauber asks, how do we argue that publicly funded agricultural knowledge is valuable and in the public interest? Stauber believes that the social sciences are in a great position to lead this debate. He has been working in the research, education, and extension agencies to create a new culture where biological and social scientists work together. He is concerned, however, with the use of rate-of-return studies by economists for evaluating public research. Are rate-of-return analyses appropriate for the public sector, or are they better used by the private sector? Extremely high rates of return on publicly funded agricultural research may lead some to believe that it should be conducted by the private sector. Stauber believes that our biggest competitor for funds is the budget deficit, not other research institutions. As budgets decline, it becomes increasingly important to set priorities. Stauber asked whether we can do research on how to set research priorities. He is interested in how research administrators can establish apolitical, objective priorities.
Mason Wiggins, Counsel, House Science Committee, U.S. Congress. There is no single model for how research is used by Congress. Research is neither ignored, nor is it heavily relied upon to set priorities. Wiggins indicated that sometimes research is used to inform and make decisions, at other times, it is used to support a position a representative has already taken on an issue. While hill staff are aware that research is available, they do not always know where to look for it. They are most interested in research that can be used to build a defense of their representative's position, help reconceptualize problems, and warn of pitfalls in taking a certain position. They consider the objectivity of the authors and rely heavily on summaries. It is absolutely critical that policy relevant research be forwarded to hill staff in a timely, concise manner. A research paper that arrives either too late or too early is less useful. Finally, simplicity is a must. The more straightforward a piece can be, the broader its potential audience is on Capital Hill.
Cathy Wotecki, Acting Associate Director for Science, Office of Science and Technology Policy The Government Performance and Results Act (GPRA), passed by Congress in 1993, was intended to improve management of federal programs and strengthen the public's confidence in its government. GPRA places new management expectations and requirements on federal agencies by creating a framework for more effective planning, budgeting, program evaluation and fiscal accountability for federal programs. GPRA's focus is on outcomes and outputs of agricultural research. By 1997, all programs utilizing federal funds must have designated relevant performance measures and put an evaluation system in place to monitor and report on their performance. The National Science and Technology Council has recently developed a list of guidelines to be used when developing and using GPRA performance measures. These include:
In closing, Wotecki stated that the National Science and Technology Council is working to ensure that GPRA results in a truly effective basis for research program accountability, without infringing on researchers' creativity and flexibility.
III. Users of Research Assessment: Land Grant and Industry Perspectives Michael V. Martin, Dean of the College of Agricultural, Food, & Environmental Sciences, University of Minnesota It is likely that public support for agricultural research will continue to decline in real, if not nominal, terms. However, demands for new knowledge, new products and new technologies will continue to increase. This disconnect between public expectations and public support reflects the public's lack of understanding about the role and benefits of agricultural research. The public used to accept the importance of research on faith, but not any longer. It is critical for researchers to sell the benefits and products of research.While excellent work has been done by agricultural economists and other researchers, Martin suggests the need to do the following:
These activities will improve the use of and accountability for public funds for agricultural research, and more effectively involve the social science perspective in administrative decisions.
Joseph D. Coffey, Vice President of Economics and Planning, Southern States Cooperative, Inc. Private industry has several interests in agricultural research -- as a taxpayer, a supporter of the land grant universities, and a cosponsor of agricultural research projects. In each of these capacities, there is interest in converting basic research into practical applications, verifying their own findings, and establishing objective evaluation and accountability mileposts (e.g., net present value of costs and benefits). Since 1970, private agricultural research spending has increased at a much faster rate than federal or state agricultural research. Coffey suggests that industry is most interested in research results that have a direct pay-off. This usually includes research that (1) is future oriented, (2) can be commercialized and used by many farmers, (3) consumers will accept, (4) limits liability exposure, and (5) requires minimal up-front investments and quick pay-back to the user. The benefits of agricultural research accrue to consumers, farmers, the nation and the world. And, these benefits can be expected to continue as new, promising technologies are developed. Examples of these technologies include: precision farming to improve input efficiency, satellite scouting and positioning for site specific farming, herbicide-resistant crops to reduce pesticide use, weed and fertility sensors, insect vacuums and air-jet weeders, agricultural based biodegradable products and renewable energy sources, more nutritious crops, and leaner meat. Coffey is optimistic about the future of agriculture. Agriculture is a multi-billion dollar contributor to the U.S. and the global economy, and is an important steward of land resources. However, continued public support of research will be crucial to the well being of the sector.
IV. Agricultural Research Evaluation: Principles, Processes, and Products George Norton, Professor of Agricultural and Applied Economics, Virginia Polytechnic Institute and State University The benefits of agricultural research include: increased productivity, greater per capita incomes in agriculture, enhanced agricultural competitiveness, and reduced environmental problems. Despite past success, resources for agricultural and related science are scare and research systems are being asked to do more for less. As government budgets tighten generally, agricultural research policy makers and administrators face ever sharper pressures to justify budgets and to economize. Accountability becomes increasingly important. Research planning and evaluation may be relatively simple or complex. Over the past several years, agricultural economists have developed and implemented methods for evaluating the economic impacts of agricultural research in a credible, and relatively sophisticated manner. Some of these methods are used primarily for evaluating completed research (ex post), others are used for projecting research benefits (ex ante). Decisions served by research evaluations. Planning and evaluation can serve decisions at several levels. Economic evaluations can be used to help justify a budget allocation to research as a whole, to specific agencies or universities, to particular projects, or to individual research activities. At every level, evaluations can help answer questions such as: Was the previous research investment worth the cost? What are the projected net benefits of a future research investment? How much should be spent on research? How should research funds be allocated to obtain the greatest net benefit? Which groups have or will benefit the most? What is the role of the public versus the private sector? Economic evaluations can provide accountability for resources previously allocated and also help guide future research investments across problems, programs, people, and places. Norton also delineated several caveats. He indicated that evaluations are expensive, outputs are sometimes difficult to measure, and over-evaluating research can be counter-productive. Consequently, quantitative economic assessments are most useful at the aggregate rather than project or activity levels. Finally, formal evaluation and priority setting procedures should not be used as a basis for replacing ingenuity, serendipity, and scientific entrepreneurship with costly bureaucratic procedures. Economic evaluation provides a framework to synthesize scientific and economic data to assess tradeoffs among objectives. Evaluation principles. Several principles influence the cost-effectiveness, accuracy, and usefulness of the evaluations.
The evaluation process. Research evaluation consists of four basic components: (1) defining the problem, (2) compiling the data, (3) analyzing the data and information, and (4) interpreting and using the results. The first step in defining the problem is to define the client, who in turn identifies what decisions the evaluation will serve, what questions it will try to answer, and the objectives for the research system. The analyst works with the client to define the research programs to be evaluated and the level of detail in the measures to be used in the analysis, before collecting the data. The second component of the evaluation process is to compile the data which may include both market-related data and research-related data. Analyzing the data and information, the third component, involves a variety of techniques including economic surplus and econometric analysis. The benefits estimated with either approach can then be used as the basis for a benefit-cost analysis or to estimate impacts on output, income and employment in the economy (multiplier effects). The final component of the evaluation process is interpreting and using the results. Agricultural research evaluations can be presented in several forms. There are, however, two general types of products: (1) information to help justify budgets using ex post analysis, and (2) information to help allocate resources from ex ante analysis. An example of each is described below. In Virginia an econometric approach was used to calculate the rates of return to agricultural research, extension, and teaching over the past 40 years. Annual returns of 58% for research, 37% for extension, and 53% for teaching were estimated. These are very high returns compared to most alternative investments. The analysis also provided estimates of yearly sequences of changes in Virginia agricultural production (marginal products) attributable to agricultural research, extension, and teaching expenditures. For example, a one-dollar increase in research expenditures increases agricultural production approximately nine dollars over twelve years. While ex post studies are more numerous than ex ante studies, several recent refinements of ex ante procedures suggest that future evaluations will focus more in this direction than they have in the past. In an ex ante analysis, information is used to help rank alternative programs or to provide guidance on projected benefits associated with increasing or decreasing budgets to particular programs. These techniques have been used in many developing countries as well as in the United States. In summary, quantitative economic evaluations have a specific role to play in justifying budgets and in providing information for research resource allocation decisions. The time, resources, and data required preclude their use in estimating the benefits of every research program or project on an annual basis. However, every agricultural and related research agency or station can (a) identify their objectives and major programs, (b) periodically assess the economic impacts of their major programs and their total research effort, and (c) apply basic economic analysis to major programs when making strategic decisions about the allocation of research resources.
V. Interdisciplinary Issues in Research Priority Setting, Design, and Assessment John M. Antle, Professor of Agricultural Economics, Montana State University This talk was based on the recently published paper entitled, "Why Scientists Should Talk to Economists," by John Antle and Jeff Wagenet. This essay is based on the belief that publicly funded science has a responsibility to be accountable to the public and to contribute to public policy goals -- and the belief that this can be done while meeting the standards of good science. To be successful, research assessments must be an integral part of research priority setting and research design. The key to success is to avoid the following typical mistakes:
Setting research priorities. Economics provides the scientific community with a systematic framework for priority setting and research assessment that tries to account for the benefits and costs to all segments of society affected by publicly funded research. This framework requires that economic data, and data from the other scientific disciplines, be collected for integration using common units of measurement. Many scientists and administrators incorrectly believe it is too costly and time consuming to implement this kind of framework. The solution to "paralysis by analysis" is not to abandon informed decision making. Rather, the solution is to encourage scientists to collaborate with economists and other social scientists to develop the capacity to provide timely assessments of their research. Impact assessments can be constructed in a timely manner, and at a reasonable cost, if research institutions make the commitment to invest in the requisite data, methods, and expertise. Research design. Designing research to meet the standards of good science while also satisfying the standards of accountability and policy relevance involves careful planning and coordination. By involving scientists in the priority setting process, they become aware of how their research contributes to the mission of publicly funded research. Researchers must then coordinate their research designs so that data can be integrated across disciplines and used for research planning and policy analysis. Researchers need to agree upon a unit of analysis to use in quantifying the impacts of production technologies. The only way to accomplish this is for researchers to coordinate the design of the research before they actually do the research! Valuing research. Several problems arise in valuing the economic, environmental, and human health impacts of agricultural production systems. The key problem is that different types of research impacts are measured in different units. The economic solution is to obtain a common unit of measurement by converting all physical impacts to monetary values. Collaboration between scientists and economists is necessary to determine what science and data are needed to translate physical changes, such as a change in water chemistry, into terms that people associate with their uses of water. With the critical link made between science and valuation, economists can then use suitable methods to translate physical changes into monetary values. In conclusion, research assessment is an integral part of doing research. Every publicly funded research project should contain an assessment component that identifies likely economic, environmental and public health impacts, and that provides data needed to quantify impacts and to support priority setting and assessment activities.
VI. Successes and Experiences in Research Assessment Kelly Day, Economist, Economic Research Service, USDA Day discussed her experience in analyzing a Cooperative Research and Development Agreement (CRADA) between the U.S. Department of Agriculture (USDA), the National Cancer Institute (NCI), and Bristol-Myers Squibb (BMS), a private pharmaceutical company. The objective of her analysis was to identify characteristics that led to the project's success, as well as research and policy issues related to the federal government's role in encouraging technological innovations. The goal of the CRADA was to commercialize a promising anticancer drug, taxol, which is derived from the bark of the pacific yew tree. Taxol was discovered by the NCI in 1962. Over the next two decades, the NCI determined that taxol had powerful cancer fighting properties. To promote wide-spread diffusion, the NCI entered into a CRADA with BMS. Essentially, NCI agreed to share data with BMS, who in turn, agreed to fund research in exchange for exclusive production rights. The taxol research agreement was deemed successful for a number of reasons. First, areas of expertise were clearly and rapidly identified among cooperators in firms, universities, and the government. Second, the division of labor was carefully coordinated between cooperators and disciplines. Finally, the project rapidly cleared hurdles to commercialization. Much of the success of the CRADA can be attributed to dedicated public support for the preliminary research done by NCI and USDA, and strong academic linkages between relevant disciplines. The CRADA also gave BMS the flexibility to consider a broad range of alternatives in research and production of taxol. Finally, BMS had strong financial incentives to meet market demand. When designing policies to encourage technological innovations, Day believes the following research issues should be considered. First, we should consider how to revise the stylistic distinction between public and private sector research. Theoretical comparisons usually represent the private sector as atomistic firms, to be compared with a coordinated monolithic government. Government is actually more diffuse than frequently assumed and the private sector can be very coordinated. Second, we need better information as to whether public-private agreements deter non-cooperating firms from entering the market, even if they have a superior product. Finally, there is a need for comprehensive and balanced economic analysis of potential impacts of policies designed to encourage technological innovation.
Sally Thompson, Associate Professor of Agricultural and Consumer Economics and Interim Assistant Director of the Agricultural Experiment Station, University of Illinois The Council on Food and Agricultural Research (C-FAR) has begun a process to establish and use research assessment analysis to improve coordination and communication between agricultural researchers and the public. C-FARE is a non-profit organization working to secure additional resources to adequately fund relevant and high-quality research and related outreach programs. The results of this research initiative will lead to profitable, consumer-sensitive, and environmentally sound food and agricultural systems in Illinois and the nation. C-FARE will facilitate public participation in planning and evaluating the process and impact of research activities. C-FARE's process was formulated by a group of agricultural stakeholders including representatives from the public and private sectors. Stakeholders identified five objectives for food and agricultural research in Illinois, formulated a strategy for assessing and refining research priorities within the five objectives, and developed criteria for evaluating research programs in terms of those objectives. Five working groups, each based on one of the five C-FARE objectives, were created to:
C-FARE held a retreat last spring. Participants created a "Blueprint for Development of Processes and Organizational Plan" which identified education needs, communications strategies, and target audiences for assessment results. The ultimate goal is to improve coordination and communication between the public and food and agricultural researchers through effective research evaluation and assessment.
Philip Pardey, Research Fellow, International Food Policy Research Institute Pardey spoke about two experiences with research assessments. The first assessment project, undertaken by the University of California (UC), was designed to identify, document, and value the benefits from agricultural research and extension conducted at the University. The primary impetus for the study came from severe cuts in the state's agricultural research budget. In addition, skepticism was increasing as to whether public-sector agricultural research through the UC system was a worthwhile investment. Three techniques were used to document the value of research and extension within the UC system: growth in productivity, commodity case studies, and rate of return analysis. It was found that productivity growth since 1949 added $5.6 billion to the value of California's agricultural output in 1985 -- about two-fifths of that year's total output. Studies of strawberry and dairy production, for example, demonstrated that research conducted in the UC system generated substantial benefits to both producers and consumers. Rate of return studies estimate a 17.1 - 21.5% return on funds spent on research and extension within the UC system. Pardey cautioned practitioners of research assessments to consider measurement problems associated with unaccounted changes in quality of inputs and outputs, and over-looked or omitted inputs (e.g., consumption of unpriced stocks of natural resources; infrastructure) or outputs (e.g., environmental externalities). The second assessment Pardey discussed was undertaken by the Consultative Group on International Agricultural Research (CGIAR). The CGIAR is an informal association of more than 40 countries, international and regional organizations, private foundations, and representatives from national research systems in the less-developed world. It was formed to guide and support a system of international research centers. There are 16 centers that share an operating budget of about $335 million. Within the CGIAR system there are both system-wide and sub-system studies. For example, in 1984-86, an "impact study" was carried out by an external manager and a team of consultants who were charged with assessing the value of the activities carried out by all 16 centers. Within individual centers, a substantial amount of research evaluation work has occurred. Some of this work involves ex post research evaluation studies of specific technologies or commodity specific research programs. The CGIAR system currently is working to strengthen its capacity for research evaluation at both the subsystem level (i.e., within each center), and for the system as a whole. It is expected that research assessment will be used to inform strategic priority setting within the CGIAR system.
Thayne Dutson, Dean of the College of Agricultural Sciences and Director of the Oregon Agricultural Experiment Station, Oregon State University Financially austere times challenge public agencies to prove their worth, but in the public sector, a commitment to accountability pays dividends whether the taxpayers are skeptical or enthusiastic in their support. Dutson reported how the Oregon Agricultural Experiment Station developed a systematic, flexible, and accessible database tailored to communicate the benefits of research to key decision makers and general audiences. This system, Oregon Invests! The Accountability Database, is mission oriented, project-based, and relies on principal investigators as the original source of information. Each project is assessed based on its potential economic, environmental, and social impacts. The key role of Oregon Invests! is to address the University's responsibility to be accountable by reporting the extent to which research fulfills its mission. Oregon Invests! also plays a critical role in helping faculty learn about the work of others, as well as providing an opportunity for faculty to see their own work summarized and presented to the world. The computer program is easy to use. Projects may be called up, for example, by geographic area of the state; commodity; topic, such as water quality or cancer research; and by department or branch station. Then, the database can be sorted by selected criteria -- for example, by descending order of annual economic contribution, by date of project origination, or alphabetically by project title or by a principal investigator's last name. The database may be used on most personal computers. Dutson has used Oregon Invests! to make presentations on research to the Oregon Legislative Assembly and staff, commodity organizations, the National Association of State Universities and Land Grant Colleges, the U.S. Senate subcommittee on research accountability, the U.S. General Accounting Office, and the U.S. Department of Agriculture. Inquiries from more than 40 states have been received. To be fully effective, an accountability system like Oregon Invests! requires the personal involvement of the chief executive in its design, use, and continuing support. Without such ownership, people in the organization will sense that the system is not very important and it will languish. With it, however, faculty and staff will invest their efforts, and the system can continue to be responsive and vital.
VII. Conducting Research Assessments Discussion Leader: Katherine R. Smith, Director of Policy Studies, Henry A. Wallace Institute for Alternative Agriculture. Smith facilitated audience discussion on two important questions related to agricultural research assessments. The questions and audience responses are summarized below. What kinds of assessments are needed? Participants indicated a need for portfolio assessments to analyze the benefits and effectiveness of general research programs. Assessments should incorporate the full range and distribution of impacts across different stakeholder groups. Assessments should also allow for comparisons with alternative public investments. One person stressed the need for assessments to analyze research investments not necessarily aimed at increasing production and efficiency, such as information technology, institutional design, and policy issues. Another suggested that research assessments should deal with ethical issues in the evaluation framework and anticipate policy needs. Finally, assessments should account for ex ante expectations about how the research will be implemented. How can assessments be implemented? Participants agreed that interdisciplinary teams of researchers are needed both within and outside the social sciences. The teams should be established at the beginning of the research process to ensure that adequate data will be available for the assessment. To create these teams, institutional bridges are needed among disciplines and their professional societies. Appropriate incentives must be established. These should come from within disciplines and professional organizations. Some participants related this level of change to a paradigm shift. One person suggested a need for researchers and research administrators to become more comfortable with qualitative work. Another stressed the need for more research on the definition and measurement of social impacts. Finally, several people suggested a need for a filtering system to maintain and/or regulate the quality of research on research. Other comments raised in the discussion. A participant suggested that different analytical approaches will be more or less appropriate depending on the objective of the assessment. Another cautioned that being explicit about the distribution of benefits may stimulate shifts in research funding sources. Finally, someone suggested that many researchers are not comfortable being advocates. Some perceive a conflict with being both an analyst and advocate.
VIII. Research Assessments in Decision Making Discussion Leader: Susan E. Offutt, Executive Director, National Research Council So far, agriculture has done rather well in the appropriations process relative to other areas of government. However, there are several ways we can improve our communications to decision makers. First, we need to recognize that politics is about the distribution of benefits. Percentages that describe rates of return on research investments are not politically compelling reasons to fund agricultural research. Politicians want to know who benefits, and by how much. It is essential that agricultural research assessments identify the beneficiaries of agricultural research. Second, there is a fundamental communication barrier between scientists and politicians. Politicians like to talk and scientists like to read. Direct personal communications with decision makers are needed if we are to be effective. Offutt then led the audience in a discussion of the following question. What characteristics of assessments will increase their influence on decision making? Much of this discussion focused on how the message is formulated, who relays the message, and how. Suggestions were that assessments should:
IX. Summary and Conclusions Agricultural research assessments are likely to become increasingly important as scientists, policy makers, and research leaders seek to justify public and private expenditures on agricultural research. Decision makers are interested in who will benefit from the research outcomes, when, by how much, and what will the research cost. Questions about environmental impacts and broader social benefits are also becoming increasingly important to both decision makers and the public. The participants in this symposium agreed that while the social sciences -- and agricultural economics, in particular -- have made important contributions to research assessment methodology, much remains to be done. The scientific community needs to better understand the role and benefits of research assessment analysis. Standard assessment tools, such as rate of return analysis, may have to be re-examined so that important distributional effects can be captured. Interdisciplinary teams need to be created and encouraged so that data can be integrated across disciplines and used in comprehensive research assessment analyses. Finally, and perhaps most importantly, agricultural scientists need to clearly and effectively communicate the benefits of agricultural research to taxpayers, policy makers, and other agricultural stakeholders. C-FARE urges the agricultural community to work together to make research assessment analysis even more useful and effective in this time of increasing public and private accountability.
The Council on Food, Agricultural, and Resource Economics An Organization of Agricultural EconomistsC-FARE was created in 1993 to strengthen the national presence of the agricultural economics profession and to enhance its effectiveness. C-FARE is actively working to:
In pursuit of these goals, C-FARE will continue to conduct priority setting exercises for input into funding processes, develop nonpartisan issues paper to enhance the policy dialogue, and hold policy briefings on key economic issues. If you would like more information about C-FARE, please feel free to contact any member of the C-FARE Board, in addition to:
All material contained in this report may be used without permission provided credit is given. C-FARE Publications Symposium Summaries
Other C-FARE-Related Publications
|
