 |
|
|
|
|
Summary of full report
Evaluation of the Future
Agricultural Systems Program -
Evaluation of selected
projects – stage 2
by Dr Jenny Gordon, James Beard, Sarina Fisher, Jane Barnett
December 2001
RIRDC Publication No 01/169
Executive summary
Results of the evaluations
This report presents four benefit–cost evaluations on projects in the Future Agricultural Systems Program. The evaluations are:Agriculture subprogram
- nine projects on trade policy in the Global Competitiveness subprogram;
- two projects on Environmental Management Systems (EMS) in the Resilient
Chart 1 shows the net present value (NPV) benefits and costs for each of the project sets. The estimates are in 2000-01 dollars and assumed a discount rate of 5 per cent. The highest expenditure was on the RAINMAN projects at $2.4 million, which was one reason for undertaking an evaluation to see if this expenditure has paid off. For all project sets the value of the benefits greatly exceeds the cost.
- five projects contributing to the development of RAINMAN, a climate forecasting system, in the Resilient Agriculture subprogram; and
- three projects targeting tractor and utility seats and back pain in the Human Capital and Communication Information Systems subprogram.
This is particularly true for the trade policy projects, with a NPV of $136.3 million.
The high benefit–cost ratio for the trade policy project reflects the high returns, although because the benefits do not start until 2010 the internal rate of return (IRR) is comparatively lower. Using a higher discount rate would substantially reduce the estimated NPV and have little impact on the net present cost. Chart 2 shows the benefit–cost ratios and the IRR. Only the latter can easily be compared.
Compared to the other programs evaluated the range of returns is fairly small.
This may be an artifact of the conservative approach adopted given the considerable uncertainty around the parameters in a number of the evaluations.
Chart 3 compares the internal rates of return for the four RIRDC programs. While the estimates are indicative only due to the largely ex-ante nature of the benefit estimates, comparison would suggest that the payoff to R&D in Future Agricultural Systems compares favourably at least with the new and emerging industry R&D, if not with the established industries R&D.
Nineteen projects are covered by the four evaluations. All project sets satisfied, and in some cases exceeded, the 25 per cent RIRDC target rate of return, with estimates of the IRR ranging from 29 to 49 per cent.
Lessons from the evaluations
Estimating the impact of R&D activities that work through developing information, promoting understanding and skills, improving extension or informing policy debates is inherently difficult.
However, the exercise of undertaking quantitative assessment is an important one. To undertake evaluations it is important to think through how the R&D will come to have an effect on producer and consumer welfare. The lessons from these evaluations stem from this thinking through rather than anything revealed by the numbers.
What the numbers reveal is that the gains from policy advice, in OHS, and improved information for management decisions were considerable and potentially could have been much larger than the estimates given in these evaluations.
- Information must add to understanding to have any influence. Preaching to the converted is of little value. For example, the dissemination of the trade policy material to a wider audience is important. While early projects estimating the value of free trade added to the evidence on the benefits, more projects of this nature are unlikely to add much more. What is required is new ideas on how to progress to this objective.
- Information must be believed to lead to a change in understanding. To do this the quality of the research must be high — if the source is not considered credible the information is of little value even if it is correct. Sometimes charging for advice develops credibility that is not available in advice that comes free. In the evaluation of the EMS projects, the fact that the consultants now charge for the advice may not be a negative (due to slower adoption), if it means producers are more likely to adopt successfully. The target audience for information depends on the political situation. While policy makers may hold the direct power, they may not act unless there is pressure from the broader public. Similarly, for farmers who are more comfortable following standard practices as part of a group, informing the group will be more effective than informing the farmer.
- Bringing benefits forward has high returns. For example, in the Occupational Health and Safety (OHS) projects on tractor seats the more change was brought forward the greater the benefit.
- The greater the number of producers and the higher the value of production affected by the R&D output, the larger the benefits. For example, the trade policy projects have a huge potential benefit because the policy change — reduction in protection — will affect all agricultural industries, with flow on effects to the rest of the economy.
- The size of the impact of management information is likely to vary considerably between producers. In general, the closer producers are to optimal production systems to less the value of the information to them. At an industry level the structure of the industry will impact on the magnitude of the benefits. However, this general rule is violated when new inputs become available. With systems such as RAINMAN it is the most efficient producers who will probably make the best use of the system — that is getting the greatest productivity gains out of the system.
|