2003 RIRDC Stakeholder and Client Survey

Executive Summary Recommendations Stakeholder Ratings of Relative Usefulness The Website Appendices: Appendix 1 – RIRDC Stakeholder Survey (Four-Page) Appendix 3 – Website Survey Appendix 5 – Media Interviews Appendix 7 – Producer Interviews Appendix 9 – List Of Interviewee

Issues And Implications 2003 RIRDC Stakeholder and Client Survey Findings Communication Activities – Level of Usage 29 Media Communications Delivery And Adoption of RIRDC Research   Appendix 2 – RIRDC Client Survey (Two-Page) Appendix 4 – Accountability Stakeholder Interviews Appendix 6 – Advisory Committee and Industry Interviews Appendix 8 – Researcher Interviews

Executive Summary
The Rural Industries Research and Development Corporation (RIRDC) manages research and development investments on behalf of industry and government for the benefit of rural industry participants and the wider community. A measure of the Corporation’s effectiveness is the adoption of research outcomes, so in 1995 the Corporation set in motion a bi-annual process of client and stakeholder monitoring to track its performance over time.

This process monitors the organisation’s performance in the areas of communication and information transfer and how these are impacting on adoption of research outcomes. From the initial survey completed in 1995 the 2003 RIRDC Stakeholder and Client Survey represents the fifth study.

This report presents the findings of the 2003 RIRDC Stakeholder and Client Survey with the results drawn from:

1. A printed four-page survey, which was developed using the 1999 and 2001 RIRDC Stakeholder and Client Survey as a baseline. It was distributed to 530 contacts from the RIRDC researcher, administrator and Advisory Committee member databases.
2. A shortened printed two-page survey was developed specifically for producers and distributed to a random sample of 2848 contacts from the RIRDC publications and general contacts databases.
3. A review of the previous survey reports was conducted to identify emerging trends.
4. A website survey was developed using the 2001 Client Feedback online survey as a baseline. This was attached to the front page of the RIRDC website and flagged in the monthly RIRDC Research Online newsletter. Information collected from this survey was compared to the 2001 survey.
5. A total of 115 telephone interviews were conducted with a selection of internal and external stakeholders to capture their views and opinions and allow the survey data to be validated.

From this detailed process the report has been prepared for the RIRDC Board. The report combines quantitative and qualitative information, presents key findings and identifies a range of issues and implications.

KEY FINDINGS
In previous studies RIRDC had substantially improved its consultation with industry regarding decisions the organisation was making. The improvements from 1995 to 2001 have again been maintained in practically all areas. In most areas the definitive ‘excellent’ rating has either remained the same or improved and there have been no increases in the ‘poor’ rating.

Achievements highlighted in the 2003 results include:

• The delivery of a high level of service is well recognised.
• RIRDC holds a strong position with R&D media access.
• Communication with researchers is highly rated.
• The RIRDC website is a leading site amongst the Rural Development Corporations.
• The Corporation is performing well for accountability stakeholders (Government and NFF).

Opportunities identified for attention include:

• Stakeholders are seeking quantitative information on adoption of research findings.
• Advisory Committees need to take greater responsibility for monitoring producer needs and adoption of research.
• More effective distribution mechanisms need to be identified in light of the changing research and extension environment.
• As identified in previous research, the internal RIRDC databases remain a high risk factor.

Issues and implications
 

ISSUE	IMPLICATION
Program Management	The level of input from the research community has been maintained at a high level with effective interaction between Program Managers and research providers, and an improvement with the Advisory Committees. The personal contact and workshop participation methods are working well. Program management is widely recognised to have improved considerably in recent years. Financial management and reporting is tighter and more professional. RIRDC administration and program managers should be recognised and applauded for these efforts. RIRDC staff are highly valued by the research community and industry. RIRDC needs to ensure program mangers are not over-burdened so they can continue delivering the high level of professional service that stakeholders and clients have come to expect. The current system of personal interaction with workshops should be maintained.  Workshops are a good opportunity to involve the relevant media to ensure wider distribution of information, and to provide cross marketing to other programs.
Corporate Accountability	Accountability is driven strongly through personal representation to the major clients, the National Farmers’ Federation and the Federal Government, supported by a series of publications including the annual report, program plans and publication catalogues.  Consider instigating regular meetings between key RIRDC, NFF and AFFA staff to enhance mutual understanding and encourage the exchange of information. Consider investing in more projects to measure adoption of RIRDC funded research. This information would assist reporting to industry and strategic planning of future research and development investments. Continue the personal interaction and support with additional presentation information, which may be extended to other ‘potential’ clients in this category, including government and RDCs. Proactively seek input from these clients and ensure this input is included in decision-making procedures.
Advisory Committees	The Advisory Groups are an important link to ‘industry’ and are a prime provider of determining needs.  The Board could introduce an initiative to ensure the committees monitor relevant customer satisfaction levels and producer needs.  This would need some standard linkage between the committees, however the majority could be left to individual requirements.  This process could be implemented in alternate years to the RIRDC Customer Satisfaction Survey.  Program strategies could also be targeted by this research. RIRDC could benefit from investing in some detailed analysis of current and potential client groups and then identify some future needs for the industry. This will improve the committees’ understanding of producer issues and ensure producers have continual input and ‘ownership’ of relevant programs.

 

The RIRDC website	The RIRDC website has become a functional tool for researchers and allows a good knowledge of program priorities and reports. The website has developed quickly but the principle users are from the science community and the site does not yet display any interactive capabilities between researchers and Program Managers. The site is not a priority for the producer. By not continuously improving the website opportunities will be lost as producers bookmark favorite sites and form a habit of visiting alternative sites. The website is sufficiently active and developed to now consider the use of improved ‘e-metrics’ to build a greater integration between site content and customer usage. It is anticipated that RIRDC could lead the way in information customisation. Further linkage between the website and e-mail and chat rooms or bulletin boards would provide specific customers with a refined awareness and access medium. In developing the website, do not forget that there is a significant majority of clients who will prefer fax letters and print-based information. Delivery strategies must continue to recognise this issue.
RIRDC Brand	In view of the low awareness of RIRDC with producers and industry, it is timely to review the RIRDC branding policy.  RIRDC needs to consider its desired level of awareness in line with future strategies to distribute research information to end-users.
Service level improvement	Overall improvement in services during the past eight years needs to be consolidated. Having made a continued improvement in selected areas, such as general office communication, there is an expectation the organisation will be able to achieve similar improvements in other areas identified as requiring attention, such as facilitating the adoption of research outcomes.
Customer segmentation	The definition of RIRDC’s customer groups seems too ‘convenient’ as the surveys only ask the people the organisation knows really well. The monitoring still does not tackle the real question of Industry users, for example primary producers.  Communication into industry journals has reduced in perception of being a ‘very useful’’ tool. This is principally driven by competition with the science fraternity’s need to publish ‘refereed reports’ and ‘first’ compared to a completely differing need for the RDC’s outcome requirements. Further investigation of the need to deliver results further down the ‘information chain’ will require consideration and direction from the Board. Varying solutions will be required for each audience. Firstly the variance of industry size serviced by RIRDC is an important consideration. For example Internet research outcomes have a target audience of some 140,000 farmers without including the agribusiness sector, while the other extreme is the Cashew Industry with a handful of players that can be gathered in one small room. Secondly, the partnership arrangement with the relevant research establishment will determine the audience. Some projects will be better serviced if the research is disseminated through ‘second-tier’ delivery links such as consultants, accountants and select farmer organisations.
Publications	The publications have markedly improved in volume and identity. The division of roles between publications and communications has significantly contributed to this reduction in the previously identified bottleneck. It is now appropriate to revisit the publications and modify the processes for determining each project output.  Work towards identifying client needs and allow these to determine report formats. Capture marketing information from publication purchasers to assist planning future promotions.

 

Customer databases

The database issues identified in the 1999 and 2001 Stakeholder and Client Survey reports remain. The accumulation of databases and the lack of internal integrity are an on-going dilemma. RIRDC will reach a close client and stakeholder relationship only when the investment in an integrated database is made. Adopt a policy of making the best possible use of the current database system so that errors and duplications are not unnecessarily detracting from the organisation’s professionalism. Adopt a policy at the Board level that recognises the importance and value of the database information as an asset of the organisation. Clean-up entries on the current database by implementing an ongoing system that notifies the database manager of invalid contacts so they may be deleted.  The poor alignment of databases will in turn develop into another bottleneck in the operation, however an integrated system will allow for better targeting of customers for publications, workshops and other services. It will also reduce wastage and over-production of reports and provide tighter cost analysis etc. Ensure a data entry process that adheres to standards preventing the inclusion of incomplete entries. Disseminate the message of the database ‘value’ to all parties involved in RIRDC work and contributing to the database. Request an internal project to determine the extent, format and use of all databases within RIRDC.

Communications

The Communications Strategy needs to be constantly reviewed and updated to reflect the changing extension and media environments. The challenge is to provide more direct linkages to producers and raise awareness of both the RIRDC brand and research outcomes. Continual evaluation of the relevance and distribution of RIRDC newsletters would be valuable to ensure RIRDC is meeting the needs of recipients. An evaluation form could be distributed with each of the newsletters and with each of the Research Reports. RIRDC does not have high visibility with some agricultural media. Individual media satisfaction needs to be monitored. There is a need to determine response mechanisms such as to measure sales, consultant take-up rates and media coverage etc. The e-mail newsletter remains tight and relevant to the individual customer needs.

Recommendations

Research Requirements

1. Conduct further work to quantify the current rates of adoption of research outcomes. This information will be essential to help set Board policy on relative expenditure on research, development and extension.
2. Conduct further research to measure current awareness and satisfaction levels of RIRDC research and to determine the information needs of individual agricultural industry participants. This needs to be co-ordinated in conjunction with the program advisory groups in close consultation with the program managers.

1. Set a Board policy on RIRDC participation in delivering the results of research outcomes to end-users. Define desired levels of producer and industry awareness of RIRDC’s role in managing research and development for agricultural industries. This directive will help program managers, advisory committees and the communication manager effectively plan further strategies for communicating the results of RIRDC funded research and achieving desired levels of RIRDC recognition for this work.

 

1. Revisit the RIRDC Branding Strategy in view of current low awareness and visibility. It is recommended this is done once RIRDC has defined their desired level of recognition and participation in delivery research outcomes to end-users.
• Review the RIRDC name and livery.
• Review the visibility of the RIRDC logo on published material.
• Review the visibility of RIRDC staff.
• Review the follow-up procedures for participants to RIRDC sponsored workshops and seminars.
2. Update the RIRDC Media and Communications Strategy to build stronger relationships with a wider range of rural media and to achieve wider news coverage.
• Media should be monitored as part of the relationship building exercise.
• Consider providing more access by media to ongoing research and to researchers.
• Consider distributing e-mail press releases in addition to fax releases.
3. Revisit the Publications Strategy to modify the process for determining each project output.
 

1. Continue building strong relationships with major accountability clients: NFF, AFFA and Senator Troeth’s office.
• Organise bi-monthly (or quarterly) meetings between key personnel from these Organisations to allow the development of an improved understanding between all parties.
• Ensure these Organisations receive briefing information about new and important research and that they have the opportunity to provide input towards RIRDC policy and research and development planning.

1.  Conduct an audit of RIRDC databases and develop a strategy to combine existing information into a single system to protect data integrity, increase marketing potential and decrease the possibility of errors while adhering to recent privacy regulations.
8.  Adopt a policy of making the best possible use of current database systems so that errors and duplications are not unnecessarily detracting from the organisation’s professionalism.
9. Adopt a policy at the Board level that recognises the importance and value of the database information as an asset to the organisation. Disseminate this message to all parties involved in RIRDC work.
10. Clean-up entries on the current database by implementing an ongoing system that notifies the database manager of invalid contacts so they may be deleted.
2. It is recommended that the RIRDC Website be further developed to encourage producer and researcher participation and that its existence be promoted throughout the agricultural community.

•  The RIRDC Research Online newsletter could be further developed to encourage ongoing use of the RIRDC Website by producers.
• Online newsletters with links to the relevant home page and Research Reports for individual programs may also be useful.
• Investigate the development of Bulletin Boards and Chat Rooms for specific programs and industries.
• Investigate developing a decision tool to help producers explore diversification options suitable for their circumstances.
 

• De-duplicate the RIRDC contacts, researchers and publications lists – or integrate these into a single list.
• Consider obtaining producer lists from individual programs. Use these contacts to collect more meaningful information on producer satisfaction, awareness and information needs. This research would need to be done in conjunction with the advisory committees and program managers. This could be done in alternate years to the stakeholder surveys.
• Repeat the four-page survey – distribute this to entire Board and all advisory committee members plus samples from the researchers, publications and contacts list.
• Include explanatory letters with surveys sent to Board members and Advisory Committee members. The letter should explain the context of the survey and encourage them to respond.
• Circulate the RIRDC online survey as a separate email to a sample of RIRDC Research Online recipients. This would boost the response rate.
• Consider collecting some satisfaction information from a separate sample of RIRDC Research Online recipients. Email surveys save on printing, postage and data entry. Response rates to this format would need to be tested before attempting this.

Data Collection and Methodology
In preparing this report Agknowledge has drawn information from:

• A printed four-page survey
• A printed two-page survey
• A series of telephone interviews
• A RIRDC website survey
• Review of the 1995 to 2001 RIRDC Stakeholder and Client Survey reports.

The 2003 survey followed the format established in the 1999 and 2001 RIRDC Stakeholder and Client Surveys, allowing the repeat questions to establish trends in key areas. A copy of the 2003 RIRDC Stakeholder Survey is contained in Appendix 1 and this also includes the results.

While broadly covering the same issues, the 1997 survey only repeated four questions from the 1995 survey in a manner that would enable direct comparison of data and identification of meaningful trends. The 1999 and 2001 surveys have repeated these questions, resulting in four clear areas that can be compared across the eight years since the surveys were initiated. Where appropriate, Agknowledge has compared trends over a range of periods between 1995 and 2003.

Two-page printed Survey
Agknowledge was specifically requested to obtain more input from producers and end-users in the 2003 Stakeholder and Client Survey. A second, abbreviated survey was designed specifically to elicit a greater producer response. Questions were worded with producers in mind whilst maintaining the maximum ability to compare responses with previous surveys. The layout was simplified to allow ease of completion. A copy of the two-page 2003 RIRDC Client Survey is contained in Appendix 2 and this also includes the results.

This two-page survey was distributed to a random sample of 1722 contacts from the RIRDC contacts database and 1126 contacts from the publications database. In previous surveys in 1999 and 2001 a random sample from these databases had received the four-page survey.

Analysis
Both surveys were analysed using the SSPS Version 10 statistical software package and results are tabulated to reflect the overall responses. The grouped data was analysed initially using a one way Analysis of Variance to determine if there were any significant differences due to variation between groups.

The analysis also considered T-Tests to compare responses and there were no significant distributions in the two-tailed (bell curve) distribution, which would indicate a strong comparison of paired results in the area of both ‘researcher’ response and ‘producer’ response, supporting the confidence levels in the results presented. All statistical analysis was carried out at the 95% confidence level.

The study used the ‘triangulation’ survey method by supporting the quantitative survey with qualitative input from more than 115 individual interviews of internal and external stakeholders. Consequently, the findings presented below have been supported with sufficient rigour so that where there are areas of clear significance there is a high degree of validity in the results.

Analysis comparing the four-page survey results in 2003 to 2001 results was conducted with producers responses removed from the 2001 results to increase validation of the survey samples.

To avoid confusion, all results discussed will be clearly marked as originating from either the four-page, two-page or online survey.

Demographics of survey respondents, four-page survey
A total of 108 responses to the four-page survey were received, representing a 20.4% response rate. This compares to a 12.2% response rate in 2001. However, it needs to be remembered that the 2001 return rate would be expected to be lower as it includes responses from producers as well as researchers. In fact, return rates for all individual groups were lower in 2003 than in 2001.

In 2001 30% of Advisory Committee members returned surveys compared to less than 20% in 2003. In 2001 three members from each Advisory Committee were contacted by telephone, interviewed and reminded to return their written questionnaires. This year only one member was contacted from most of the Advisory Committees. This may have affected the response rate.

Nearly 60% of the research fraternity returned this survey in 2001, compared to just over 30% in 2003. The lower response rates are most likely due to duplication in RIRDC databases. Some researchers and Advisory Committee members would have received both a four-page and a two-page survey. Although a couple of researchers chose to complete both, it is likely that the majority of people will have taken the simplest and quickest option and completed the two-page survey.

This crossover in mailings occurred because the RIRDC database lists are maintained separately. Maintaining separate lists causes a number of problems:

• It leads to an increased number of database errors, as individual details may only be added to, updated on or deleted from one list.
• And makes it difficult when the organisation wants to target communications to particular groups without the risk of duplication. Sending duplicate mailings is expensive and appears unprofessional. De-duplicating lists by hand is very time consuming and therefore expensive.

Table 1. Breakdown of the 2003 printed four-page survey distribution

Client Group	Surveys dispatched	Responses received	% response  by group
Members of RIRDC Advisory Committees	107	21	19.6%
Recipients of RIRDC funding (administrators)	142	46	32.4%
Providers of research - extension	273	37	13.5%
Total surveys sent	530	x
Total surveys returned		108	x
Responses as a percentage of total surveys sent			20.4%

Figure 1:

The majority of respondents to the four-page survey were Researchers (46%), followed by Industry organisations (23%), Advisory Committee members (21%), Extension personnel (14%) and Producers (3%) (Figure 1). It is unclear how those respondents identifying themselves as producers received the survey. It is possible they were also Advisory Committee members or received research funding from RIRDC.

Figure 2:

Respondents to the four-page survey were distributed from across Australia (Figure 2). The majority of respondents were from New South Wales (28%) and Victoria (28%) followed by Queensland (12%) and Western Australia (10%).

Demographics of survey respondents, two-page survey

A total of 389 responses were received to the two-page survey, representing a 13.6% response rate. This compares to the 6% response rate from the same mailing lists in 2001. As predicted, the two-page survey was easier and quicker, and more likely to be completed than the four-pages.

Table 2. Breakdown of the 2003 printed two-page survey distribution

Client Group	Surveys dispatched	Responses received	% response  by group
Contacts and publications database	2850	389	13.6
Total surveys returned		389	x
Responses as a percentage of total surveys sent			13.6%

Figure 3.

Respondents to the two-page survey were distributed across Australia. The most responses were received from NSW (29%) and Victoria (25%) (Figure 3.)

Table 3. Respondents’ linkage to RIRDC – two-page survey
 

Client Group	Responses received	% response  by group
Producer	142	36.5
Researcher	89	22.9
Consultant	27	6.9
Extension personnel	27	6.9
Vet	7	1.8
Teaching	8	2.1
Hobby farm / Horse owners	16	4.1
Not stated and other	58	14.9

The distribution of respondents to the two-page survey reflects the make-up of the RIRDC publication and contact lists. The greatest number of responses were received from Producers (36%); followed by Researchers (23%); Consultants (7%); Extension personnel (7%) and Horse owners and hobby farmers (4%). Fifteen per cent of respondents chose either not to identify themselves or used an unclassifiable descriptor.

It is worth noting that the group of producers responding to this survey is a select group with a higher degree of contact with RIRDC than the wider community. As such, we would expect higher usage rates than those from the wider community.

In analysis of results we have tabulated all questions by these groups to identify any variance and have discussed any significant issues. However, producer results for the two-page survey will be discussed in most detail, as results for other groups are collected in the four-page survey.

Table 4. Agricultural enterprises of major involvement for producer respondents
 

	Agricultural enterprise of major involvement for producer respondents	Number of responses	Number of producers
Apiculture	Apiculture	30	20
Forestry	Agroforestry Forestry Landcare / NRM	23 13 15	8 4 1
Livestock	Livestock general Cattle Sheep Deer Goats (incl. Dairy) Pigs Dairy Wild game Pastoral and grazing Pasture production	5 33 19 9 5 3 10 4 11 3	1 19 13 9 3 1 2 2 4 1
Poultry	Poultry general Egg production Meat production	9 8 2	3 6 1
Equine	Equine	66	21
Cropping	Fodder Broadacre cropping Rice Sugar Cotton	3 30 4 4 1	1 14 1 0 0
Horticulture	General Vegetables Tree crops/orchards Olives Herbs Floriculture Wildflowers Tea Tree Essential Oils Truffles	23 5 16 5 11 8 7 3 2 1	11 4 9 9 3 3 1 2 0 0
Aquaculture	Aquaculture	1	1
Viticulture	Viticulture	5	3
Organics	Organics	7	1
Other		26	2
No response		17	2
	Total	446*	180*

* note: total exceeds the total survey responses as some respondents gave multiple answers.

Telephone interviews
A telephone survey was undertaken to expand and further understand the results recorded from the posted surveys.

A total of 115 individuals were contacted by telephone and invited to discuss issues relating to RIRDC’s delivery of services to its customers, including comparisons with other research and development organisations.

A list of interviewees can be found in Appendix 9. However, the names of interviewed Advisory Committee members and producers remain confidential.

Phone interviewees spanned the five client groups and included interviews with:

• 16 Advisory Committee members - including at least one member from each

Advisory Committee (Appendix 6).
• 10 Researchers from across Australia (Appendix 8).
• 3 RIRDC staff – Murray Hansen and both internal program managers.
• 15 #9; Media contacts (Appendix 5).
• 53 #9; Primary producers (Appendix 7).
• 13 Government and industry associations (Appendices 4 and 6).

Comments from the telephone interviews have been included in the issues analysis and text findings where relevant.

Website survey
In 2001, Agknowledge initiated a website survey on the front page of the RIRDC site. This same survey was posted on the RIRDC website for a period of just over one month during the data collection phase of the 2003 Stakeholder and Client Surveys. A link to the website was also included in two issues of the RIRDC Research Online monthly newsletter.

The 2003 website survey captured 58 responses, compared to 94 responses in 2001. The lower response rate this year is likely to be a response to the proliferation of surveys currently posted on the Internet. Internet users are becoming tired of repeatedly being asked for feedback. Also, many of the recipients of the RIRDC Research Online newsletter would have also received a written survey.

The complete results to this website survey are detailed in Appendix 3.

Review of 1995 to 2001 RIRDC Stakeholder and Client Survey reports
The 2003 report follows the established reporting format from the 1999 and 2001 reports. Although the earlier trends are more difficult to compare, the update between the more recent surveys is closely connected.

The written reports prepared from the 1995 and 1997 RIRDC Stakeholder and Client Surveys were reviewed to identify trends since the surveys were introduced. There are substantial differences in the detail of these reports. This has made some areas of comparison difficult, however it has still been possible to identify trends.

In previous years a single survey was sent to researchers, producers and other industry contacts. This year the four-page survey was only sent to the researcher, Advisory Committee and administrators databases and a two-page survey was sent to the publications and contacts database, which includes producers, researchers, advisors and industry representatives. For this reason producer responses have been removed from the 2001 result to allow more accurate comparison with the 2003 results to the four-page survey. The 2001 producer results are compared with the 2003 producer results from the two-page survey where possible.

Table 5. RIRDC Stakeholder and Client Survey responses
 

YEAR	1995	1997	1999	2001	2003 (4-page)	2003 (2-page)
Number of Responses	156	142	174	228	108	389

2003 RIRDC Stakeholder and Client Survey Findings
Over recent years RIRDC has substantially improved consultation with industry regarding decisions the organisation is making. There has also been a concerted effort to improve the communication of research findings back to industry to facilitate adoption.

The findings from these surveys are presented and discussed to provide an overview of the following areas:

Overall performance – results from the four-page survey of stakeholders.

Input

Input from industry – feedback from qualitative interviews with the Advisory Committees and industry organisations.
Input from researchers – feedback from qualitative interviews with researchers.
Input from government –feedback from qualitative interviews with NFF, AFFA and Government.
Stakeholder ratings of RIRDC performance in industry collaboration – results from the four-page and two-page surveys of stakeholders.
 

Stakeholder ratings of relative usefulness - results from the four-page and two-page
surveys.
evel of usage – results from the four-page and two-page surveys.
RIRDC website – results from the four-page and two-page surveys and the Online Survey of website users.
Media communications – feedback from qualitative interviews with key journalists.

Adoption of research findings – results from the four-page and two-page surveys.

Comparison with other R&D Corporations and suggestions for improvement.

Overall Performance

Table 6. Question 2 (four-page survey): How would you rate RIRDC’s overall performance in carrying out its role?
Summary of results from question 2 in the 2003 four-page survey and comparative data from the 1999 and 2001 surveys. Figures are collective responses from all client groups. 2001 results have been adjusted to remove producers to allow more accurate comparison with 2003.
 

	Excellent			Good			Satisfactory			Poor
Yr	99	01	03	99	01	03	99	01	03	99	01	03
%	24	33	33	57	54	58	37	11	9	3	1	-

Figure 4:

Eighty four per cent of respondents to the four-page survey rated RIRDC as either ‘excellent’ or ‘good’ in carrying out its role. It is most pleasing that no respondents rated it as ‘poor’.

Ten-Year Trend
Stakeholder opinions of RIRDC’s overall performance in carrying out its role have steadily increased since the first survey in 1995.

In the 2003, 2001 and 1999 surveys ‘good’ was the most frequently recorded rating, followed by ‘excellent’, while in both the 1997 and 1995 surveys ‘good’ was the most frequently cited rating followed by ‘satisfactory, with ‘excellent’ coming in as the third most frequent rating.

The 2003 ‘excellent’ rating was supported evenly by all client groups, with little variance evident between groups.

Input
Primary stakeholder input from government, researchers and industry is essential if RIRDC is to continue to understand and meet client industry needs.

The 2003 survey investigates stakeholder satisfaction with their level of input through a number of questions in the four-page survey. Qualitative interviews with Advisory Committee members, researchers, Government and industry body representatives are used to support quantitative data.

Stakeholder ratings of RIRDC performance in Industry Collaboration
Results from the four-page survey indicate input to RIRDC from government, researchers and industry continues to improve.

Table 7: Question 1 (four-page survey): Please rate RIRDC in terms of(see range of options in table):

		Excellent			Good			Satisfactory			Poor
	Note: Results as a %	99	01	03	99	01	03	99	01	03	99	01	03
1.1	Understanding of industry needs	28	29	35	53	58	50	17	13	14	2	1	2
1.2	Consultation with industry	28	29	34	53	47	51	15	20	14	4	3	1
1.3	Involving industry people in  decisions regarding priorities	37	36	38	46	43	46	12	20	14	5	1	1
1.4	General consultation with researchers	23	28	26	53	46	47	18	24	24	5	3	3
1.5	Provision of clear statements  of R&D priorities	35	31	40	44	56	47	18	10	12	3	3	1
1.6	General availability and openness  of staff	44	47	52	36	38	33	15	12	14	5	2	2
1.7	Level of R&D projects resulting in useful products and information	17	25	23	55	54	56	24	21	19	4	1	2
1.8	Dissemination of information about  its program	25	36	27	45	37	58	24	22	14	6	3	1
1.9	Dissemination of R&D findings  to clients	17	24	22	41	42	52	33	29	20	9	9	6

Summary of results from question 1 in the 2003 survey and comparative data from the 1999 and 2001surveys. 2001 results have been adjusted to remove producers to allow more accurate comparison with 2003. Figures are collective responses from all client groups. Responses 103.

This question was designed to provide a snapshot of customer satisfaction of consultation and collaboration with industry and researchers. The earlier improvements from 1995 and 1997 have again been maintained in practically all areas. In most areas the definitive ‘excellent’ rating has either remained the same or improved and there have been no increases in the ‘poor’ rating.

It is encouraging to see the three questions relating to industry input have increased their ‘excellent’ ratings.

Figures 5, 6 and 7*:

* Results are from the four-page survey. 2001 results have been adjusted to remove producers to allow more accurate comparison with 2003.

Understanding of industry needs (1.1 in Table 7) has maintained an 80% plus ‘good’ and ‘excellent’ rating, with the ‘excellent’ rating increasing by 6%; Consultation with industry (1.2) has increased its ‘excellent’ rating by 5%; and Involving industry people in decisions regarding priorities (1.3), has also increased its ‘excellent’ rating by 2% since 2001.

As the above graphs (Figures 5, 6 and 7) show, all three areas have shown constant improvement since 1995.

However, the industry organisation rating for the above sections was slightly lower than those given by other groups (although this was not significant) and it is worth noting that producers were not asked this question.

These results are backed by interviews with Advisory Committee members (including representatives from peak industry organisations) and industry organisation representatives. No industry organisation expressed an opinion that it was not adequately consulted in RIRDC decision-making.

Figures 8, 9 and 10*:

* Results are from the four-page survey. 2001 results have been adjusted to remove producers to allow more accurate comparison with 2003.

There appears to be little change in RIRDC consultation with researchers (1.4). Qualitative telephone interviews with researchers (see Appendix 8) did not highlight any major issues although a couple of researchers mentioned they would welcome more one-on-one interaction with program managers and Advisory Committees.

Provision of clear statements of R&D priorities (1.5) has maintained the high excellent and good rating of 87% and most relevant interviews supported the case. A number of Advisory Committees reported they were still receiving research applications that did not adhere to the stated priorities, but it is probable this will always occur as researchers put in applications just to try their luck.

The path from industry consultation to setting R&D priorities is well managed and should provide the strong platform for delivery of research to meet industry needs. Interviews with researchers support the view that RIRDC programs are managed professionally with industry priorities clearly defined.

The lynch-pin to making the process work is the Program Managers and their support team, and clearly this area is well serviced as the ‘excellent’ rating from General availability and openness of staff (1.6) was indicated at 51%. In 1999 and 2001 Agknowledge was impressed by the 43% and 44% ‘excellent’ ratings. Clearly RIRDC staff efforts to service industry and researchers is recognised and appreciated. Well done.

Figures 11, 12 and 13*:

* Results are from the four-page survey. 2001 results have been adjusted to remove producers to allow more accurate comparison with 2003.

The level of R&D projects resulting in useful products and information (1.7) has maintained a 23% ‘excellent’ rating.

The final two responses for Question 1 are the ‘excellent’ rating associated with dissemination of information and R&D findings to clients (1.8 and 1.9). The 12% and 7% improvements respectively since 2001 can be attributed to the significant investment in publication dissemination and the public relations efforts over recent years. Advisory Committee efforts to improve communication may be starting to make an impact. However, once more it is worth remembering that no producer input was collected for this question.

Input from industry – Advisory Committees
The Advisory Committees are vital to the effective functioning of individual RIRDC programs. Their membership ensures industry is adequately represented in decision-making throughout the management of the program. Their mix of expertise, experience and industry representation enable research objectives and projects to remain relevant and ensure results are effectively communicated back to end users where appropriate.

It is disappointing that only 21 (20%) of Advisory Committee members responded to the four-page survey. There was little difference between results for the four-page survey from Advisory Committee members compared to total responses. However, as expected, Advisory Committee members had higher general rates of communication, attended more workshops and purchased more publications than other respondents. Interestingly, Advisory Committee members were less likely to receive the e-mail newsletter or access the website than other respondents.

One representative from most of the Advisory Committees was contacted by phone. Two representatives were contacted from the JVAP, Tea Tree and Deer Advisory Committees. The issue of stakeholder satisfaction was discussed from the perspective of support and input from RIRDC, as well as perceived success in reaching intended target audiences and stakeholders.

The identity of interviewed Advisory Committee members has been kept confidential to ensure committee members could speak freely.

Advisory Committee responses are presented in Appendix 6.

Where possible, further comment was sought from some of the industry organisations that did not have representatives on their relevant Advisory Committee. Comments from these organisations are included under their relevant industry section in Appendix 6.

The Advisory Committees recognised they were vital to the successful and appropriate allocation of research funds. In general the committees demonstrated a high degree of ownership towards their program and were satisfied with the process by which they made funding decisions.

Interviewees were generally happy with the functioning of their Advisory Committee and the operation of RIRDC. However, a number of concerns were raised for individual programs. These are listed below.

Setting research priorities

• The Chicken Meat Industry interviewee suggested the committee needed more flexibility in adjusting industry priorities to meet changing situations.
• Both Deer Industry interviewees suggested there were difficulties getting some sectors of the industry to contribute to the five-year plan – and that this had led to criticism. One Deer Industry committee member also suggested that priorities needed updating more regularly than every five years.

Communication with researchers

• Interviewees from five separate programs (Chicken Meat, Deer, Wildflower, Rice and Fodder) indicated there was still a problem with researchers submitting proposals that were not aligned with stated priorities. Researchers in these areas need more guidance in understanding industry priorities.
• On a positive note, interviewees from the Chicken Meat and Wildflower Advisory Committees mentioned that the recent inspections of research projects by Advisory Committee members were a good idea.

Communication to end-users
In general, the Advisory Committee members seemed happier with the effectiveness of communication back to end-users compared to responses in the 2001 survey.

Three programs indicated this was still a problem.

• Both interviewees from JVAP said more work was needed either communicating directly with end-users or co-ordinating the matrix of communication back to producers and end-users. They also stated they recognised the program was now reaching a stage where more ‘farmer-orientated’ publications being produced and looked forward to the new knowledge brokering communications strategy.
• In the Chicken Meat program there was a problem as quite a bit of the work could not be published due to confidentiality issues.
• Both interviewees from the Deer Industry committee expressed some difficulties communicating with producers as many were part-time producers and were difficult to reach.
• There was a suggestion from the Australian Fodder Industry Association that regional developments may be useful to provide regional feedback and engender ownership of RIRDC programs. However, it was recognised that these would be expensive and many of the smaller programs would find the expense difficult to justify.

Project management

• Interviewees from the Deer, Tea Tree and Bee programs indicated their program managers were over-worked and this was affecting their performance and ability to meet deadlines.
• The interviewee from the Wildflower Advisory Committee said project management appeared to have improved recently.

Board

• The Board is generally recognised as being receptive and innovative.
• Two interviewees commented that the lunches between Advisory Committee chairs and the Board were a worthwhile initiative.

All representatives from the industry associations interviewed were generally happy with how RIRDC was managing research and development in their industries. RIRDC is viewed as being very efficient at administering scarce research and development resources.

Even these organisations without direct representation on Advisory Committees still felt they had adequate input and were included within the decision making process.

Input from Researchers
Seven out of ten respondents to the four-page survey (see Table 7) indicated general consultation with researchers was either good or excellent. Although this is a good result and an improvement on previous years (see Figure 8) – three out of ten respondents still thought there was room for improvement.

A total of ten phone interviews were completed with researchers who had applied for funding with RIRDC. The full list of comments is included in Appendix 8.

The majority of researchers had praise for RIRDC’s project management and communication.

The main points to come from the phone interviews with researchers were:

• RIRDC’s program management had become much tighter and more professional in the past couple of years.
• RIRDC program managers were recognised as being efficient, well-qualified with a dedication to their relevant industries and to quality research.
• The RIRDC Online newsletter was highly valued by the research community. The RIRDC Website is a valuable resource for the research community.
• The research reporting systems could be streamlined. Researchers find the volume and detail required in progress reports confusing and time consuming. Researchers need to spend a lot of time on RIRDC paperwork for the money allocated to them. The optimal timing of required reports should also be reconsidered. Spreadsheets should be included in proposal and reporting templates.
• Increased opportunities for interaction between program managers (or the Advisory Committee) and researchers would be valued by the research community.
• RIRDC may need to consider researcher aims to publish research in refereed journals or apply for patenting. Some research is invisible to the international research community. Reports need to be abstracted on Medline and CAB.
• Some researchers were concerned that research results weren’t getting out to end users. Many State Departments are cutting back on extension – they can’t be relied upon.

Input from accountability clients (Government, AFFA and NFF)
Representatives from RIRDC’s major ‘accountability’ customers; NFF, AFFA and Senator Troeth’s office were interviewed. Their general observations were of a well-managed and targeted organisation that was meeting its customers’ needs. RIRDC’s publications were thought to be of a high standard. Details from these interviews are included in Appendix 4.

A couple of points were raised:
 

RIRDC needs to be measuring the rates of adoption of their research in a way that can stand up to intellectual scrutiny. "Case studies are useful, but they illustrate rather than substantiate adoption."

Just because producers hear about research doesn’t mean they will necessarily use the information. Research and Development Corporations need to intelligently manage the release of information to assist adoption.
 

If more funds were allocated to adoption, RDCs would develop a better understanding of which research projects are actually adopted by industry. A better understanding of adoption would help the RDCs focus funds on Research and Development that is needed and actually used by industry.

NFF also commented that State Departments of Agriculture are cutting expenditure on extension and that RIRDC could no longer rely on them to fulfill this role. NFF also suggested that RIRDC needed to reassess its relative expenditure on research and extension.
 

NFF also commented that publication summaries were very useful.
 

NFF would also like to have more opportunities to have input into RIRDC decision making. They would like to provide input into individual program plans. They have provided input to RIRDC in the past (which has been well received) but they have always had to initiate this input. They would like RIRDC to be more proactive in their dealings with NFF.

Communication activities

Stakeholder ratings of relative usefulness
Respondents to both the two-page and four-page surveys were asked to indicate the number of times they had used different communication mechanisms and to rate their usefulness. Any relevant information gathered from the qualitative surveys is also included in this section.

Table 8. Question 7 (four-page survey) – How would you rate RIRDC’s communication mechanisms?

	Very useful			Useful			Not useful			Unable to comment 2003
Note: Results as a % of respondents to the question	99	01	03	99	01	03	99	01	03
Telephone contact with RIRDC staff	47	45	57