Peer Review

 

What is peer review?

Peer review is the evaluation of NSW Government advertising campaigns by experienced marketing practitioners (referred to as the peer review panel). Peer review assesses the need, strategy and management of the campaign, formally reporting the review findings and recommendations to the campaign manager for their response.

Peer review is a requirement under the Government Advertising Act 2011 (Section 7) for all advertising campaigns over $50,000 (excluding GST). This total cost is inclusive of any research, media, creative, and production associated with the campaign. Peer reviews apply to all categories of advertising - recruitment, public notices and public awareness campaigns.

Peer review must be completed prior to media bookings, finalising creative and campaigns going into market so that the campaign team can consider and incorporate the peer review recommendations for improvement into the final plans.

The peer review  can be described as a six-step process: Peer Review Process3

 

Peer review Process

1. Initiate a peer review

To start the peer review process, submit the online Notification for Peer Review form to the Department of Premier and Cabinet (DPC). This should ideally be done at least four weeks in advance of a proposed peer review meeting date. Once received, DPC will be in contact to discuss the campaign; submission documentation required and date and time for peer review. DPC will also send the templates for peer review submission at this time or otherwise upon request. This ensures the most recent templates are used and queries about the peer review requirements can be addressed as early as possible.

2. Peer review preparation

A peer review submission must be prepared for the peer review as well as a draft media plan, budget table, evaluation framework and creative concepts.

The advertising submission must provide a compelling case for the campaign, define the target audience/s, clearly explain how the planned strategy will achieve specific and measureable stated objectives and how the campaign will be managed. The role of advertising should be clearly outlined to demonstrate how the campaign will achieve the objectives and the planned advertising is an efficient and effective means to reach the target audience. A core focus of the advertising submission is also to show the evidence that the proposed campaign will effectively address the issue with the target audience/s  and that there are plans to reasonably measure the effectiveness of the campaign while in market. 

A completed evaluation framework is required to provide a summary of the evaluation approach.

For new submissions in some instances supplementary materials such as focus testing research reports should be made available to the peer review panel. For repeat campaigns, evidence of effectiveness of previous campaigns is required.

You should also refer to the peer review workbook as you prepare your submission. The workbook guides peer reviewers when assessing campaigns during peer review and should assist you in ensuring that you have covered what will be assessed.

Please contact DPC  for copies of peer review submission templates and the peer review workbook.

Peer review documents must be emailed to DPC at least five working days prior to the peer review meeting to ensure they can be circulated to the peer review panel and that adequate time is allowed for reviewers to consider materials prior to the meeting. Peer review meetings may be cancelled if documentation is not received on time.

3. Peer review meeting

The peer review meeting is a two hour face-to-face meeting held with the peer review panel and agency proposing the campaign, typically represented by the campaign manager and a small number of the relevant agency officers who can provide specific knowledge and clarification on aspects of the advertising campaign.

Peer review panel

The panel consists of representatives from DPC and two peer reviewers from different NSW Government agencies.  Peer reviewers are marketing and communications practitioners across NSW Government who have nominated to become a peer reviewer with DPC. Nominations must have the endorsement of their Manager or Director, and demonstrate their knowledge and experience in managing advertising campaigns.  DPC allocates peer reviewers for a specific peer review based on their position, professional experience, the campaign scope and budget. 

 

In the first half hour of the meeting, the campaign team presents the campaign to the peer review panel and responds to questions as they are raised throughout.

 

In the next hour, the peer review panel discuss and formally review the campaign against the campaign checklist set out in the peer review workbook. The peer review panel notes queries, concerns and suggestions during this discussion.  The campaign team is not present during this stage.

 

Once the discussion is complete, the campaign team is invited back for the final half an hour of the meeting. Any further questions are asked of the campaign team and the review Chair provides a summary of the discussion and key findings that will be included in the peer review report.

4. Peer review report

Within a few working days of the peer review meeting the facilitator prepares the peer review report based on the peer review panel’s findings as discussed during the meeting, including their recommendations to enhance the campaign.  The report is then emailed to the panel for their additional comments and endorsement. The final report is provided by DPC to the campaign manager within five working days of the meeting for their response.

 5. Agency response to peer review

The campaign manager should formally respond within the peer review report to all items noted. Updates should also be made to the advertising submission and attachments where appropriate.

The peer review facilitator will work closely with the campaign manager to ensure satisfactory response to the peer review report findings discuss any outstanding items.  If required, the facilitator may consult the peer review panel about the response. 

Once DPC confirms that the agency has addressed all items the facilitator will request a final clean submission (without tracked changes) to complete the peer review. Peer reviewers will by provided feedback by DPC on the the agency’s final response to peer review.

6. Letter of completion

DPC issues a letter to the head of the agency proposing the campaign once a satisfactory response to the peer review has been provided. The letter advises of the completion of the peer review process in accordance with the Government Advertising Act 2011.

Link to top

How are campaigns assessed at peer review? 

The peer review workbook is used by the peer review panel to assess:

  • Need – The compelling reasons for the advertising, its alignment to government priorities and objectives, and timing of the campaign
  • Strategy – The specific audience to be targeted by the advertising, details and rationale for the creative, media, and supporting strategies
  • Management – The approach to budget management, evaluation, risk management, and stakeholder consultation.

 

The peer review report is prepared based on the peer review panel’s assessment of the campaign using the workbook. The report is comprised of two parts:

  1. Campaign checklist - made up of ‘Yes’ or ‘No’ questions. Peer review submissions are assessed against each of the items in the Campaign Checklist.  Reviewers will note a “no” in this section if they have queries or more information is required.
  2. Peer review findings - include the peer review panel’s rationale for a ‘No’ against the campaign checklist and advice and feedback on areas that require greater rationale or clarity. This part includes advice and suggestions by peer review panel members, based on their experience and discussion in the peer review.

 Link to top

Peer review roles and responsibilities

See also the Government Advertising Handbook which sets out the roles and responsibilities for government advertising more generally.

Department of Premier and Cabinet (DPC)

The Strategic Communications and Events Branch of DPC coordinates peer reviews on behalf of NSW Government and is responsible for:

  • Chairing the peer review meeting for the peer review panel;
  • Facilitating the review panel’s discussions and delivery of the review reports;
  • Maintaining a register of peer reviewers and managing peer reviewer nomination and briefing;
  • Guidance and templates used for peer review; and
  • Communicating the requirements of the peer review process to agencies.
DPC is NOT responsible for providing assurance on other agencies’ compliance with the Government Advertising Act 2011.

For each peer review, DPC will

  • Liaise with the campaign manager to facilitate the peer review;
  • Brief the campaign manager about the requirements of a peer review;
  • Ensure the peer review process runs consistently;
  • Summarise the peer review panel’s discussion to close the peer review meeting; 
  • Coordinate the development of the peer review report;
  • Review the campaign team’s response to the peer review report and discuss with the campaign team whether peer review findings are addressed; and
  • Provide feedback to the peer review panel regarding the response to the review report.

Government agency proposing campaign

The government agency is responsible for:

  • Allowing reasonable and realistic timeframes to undertake the peer review process;
  • Following DPC’s guidance and requests around peer review procedures;
  • Providing peer review information in the required format and at the required time; 
  • Nominating the key contact - a senior officer responsible for managing the campaign – who will be the point of contact for the peer review process, outcomes and follow-up activity; and
  • Giving serious consideration to suggestions and recommendations in the peer review report and presenting evidence supporting their position.

The campaign manager will:

  • Prepare and send the campaign documents for peer review to DPC;
  • Coordinate a brief presentation at the peer review meeting to provide an overview and to highlight the key aspects of the campaign;
  • Determine actions required to address peer review report findings; and
  • Responds to peer review report findings and provide a final updated peer review submission.

 

Peer review panel

The peer panel consists of representatives from DPC and two peer reviewers from different NSW Government agencies.  DPC allocates peer reviewers for a specific peer review based on their position, professional experience, the campaign scope and budget.  DPC will select reviewers from agencies not directly connected to the campaign. It is the responsibility of the peer reviewers to disclose any conflict of interest in relation to the campaign to DPC.

Reviewers are responsible for:

  • Following DPC’s guidance and requests around peer review procedures;
  • Reading the peer review submission provided in advance of the peer review meeting; and
  • Assessing the campaign using the peer review workbook and using an evidence-based approach to supporting their suggestions or recommendations.

In cases where DPC develops a campaign that requires a peer review, the following applies:

  • The peer review will be facilitated  by a person independent of DPC; and 
  • Peer reviewers will be independent of DPC, including at least one reviewer independent of the public sector.

Link to top