Guide Filter: Create and Test

Introduction

__________________________________________________________

Click here to access this Guide in Arabic – مراجعة هذا الدليل باللغة العربية، انقر هنا

Click here to access this Guide in Portuguese – Guias em Português

__________________________________________________________

A creative brief is a short, written document used by project managers and creative professionals to guide the development of creative materials (e.g. drama, film, visual design, narrative copy, advertising, websites, slogans) to be used in communication campaigns. Usually, it is no more than two pages in length, sets the direction, defines the audience(s), focuses on the key messages and shows the desired results for an SBCC campaign or materials.

The creative brief is part of the design phase in the communication process. The creative brief should be based on a communication strategy to ensure creative deliverables align with the overarching strategic approach.

Why Develop a Creative Brief?

A creative brief is the guidepost for creative deliverables: it guides in-house experts, an advertising agency or a creative consultant in the development of messages and materials that fit within the campaign’s overall strategic approach.

A creative brief outlines the most important elements of the SBCC campaign (see Creative Brief Template):

• The key health or social issue to be addressed.
• The priority and influencing audiences (who the campaign will reach).
• The importance of reaching those audiences.
• The key behaviors to promote.
• The reason the audience(s) should adopt a specific behavior.
• The benefits of taking that action.

Who Should Develop a Creative Brief?

A small, focused team should develop the creative brief. Members should include communication staff, health/social service staff and, if available, research staff.

When Should a Creative Brief be Developed?

A creative brief should be developed by the team after conducting a situation analysis and audience analysis. Data collected during these analyses inform the creative development process. The creative brief will guide the process of message and materials development.

Estimated Time Needed

Completing the creative brief can take up to two to three days to develop depending on how complete the communication strategy is and whether the team needs to gather additional information.

Learning Objectives

After completing the activities in the creative brief guide, the team will:

• Understand the importance of a creative brief in designing communication interventions.
• Develop a creative brief that clearly identifies the purpose of the campaign.

Prerequisites

• Situation Analysis
• Audience Analysis

Steps

Step 1: Define the Purpose

Prior to developing a creative brief, it is important to have a clear understanding of why messages and communication materials are being created for the health or social issue and audience. Define the purpose of the creative brief by completing the following sentence:

We want [this audience – who?] to [do what?] in order to [benefit how?].

The more specifically tailored the creative materials are to the purpose, the more likely it is that the SBCC efforts will succeed.

Step 2: Determine the Objectives

Creative brief objectives should be clear and specific. One way to write a good objective is to specify what the audience should think, feel or do as a result of exposure to the creative materials:

• What should the audience believe, think or know?
• What should it feel about the proposed behaviors/solutions?
• What should it do to immediately improve its (or its family’s) situation and well-being relative to the issue?

Step 3: Describe the Audience

To develop successful creative materials, it is critical to understand who those materials will address. This section of the creative brief should answer the following questions:

• Who is the audience for the creative materials?
• What does the audience care about?
• What does the audience currently think, feel and do in relation to the objectives set in Step 2?

Refer to the audience profiles developed during the audience analysis to produce a description that will help the creative team understand who the audience is. Include both demographic and psychographic characteristics.

Step 4: List Competing Current Behaviors or Conditions

Make a list of current behaviors and conditions that prevent the audience from adopting the behavior(s) the communication materials will promote. These competing behaviors or conditions will be similar to those that were defined as social, economic and physical challenges or obstacles found in the situation analysis.

By creating this list, the creative team will be able to develop its messages more carefully and accurately by specifically addressing the barriers to adopting the new behavior(s).

Step 5: Highlight the Key Issue

Identify the most important issue that needs to be addressed. Think about one improvement (e.g. improving knowledge, increasing self-esteem or addressing myths about an illness or disease) and its potential effect on members of the priority audience. Even if multiple issues exist, each creative brief should focus on one audience, one message and one issue.

Step 6: Determine the Key Promise

A promise expresses how the audience will benefit from using a product or taking an action. A promise to the priority audience must be true, accurate and of real benefit. The promise is not a product (e.g. surgical gloves). It is not an action (e.g. getting circumcised). It answers the question, “why should I do this?” Or “how will this help me?”

This promise hints at an action but highlights the benefit of that action.

Discuss among the creative team to develop a single promise for the campaign. It is best to write it as an “if…then…” statement. It may be useful to develop a few alternative ‘then’ options and pretest them among the priority audience to see which benefit resonates the most with them. Keep the consequences positive, since negative consequences can increase fear and disempower the priority audience.

Step 7: Identify Support Points

The audience needs believable, persuasive and truthful information to support the key promise. These can be in the form of facts, testimonials, celebrity or opinion leader endorsements, comparisons or guarantees. The kind of support points used will depend on what will appeal and be credible to the priority audience.

Step 8: Define the Call to Action

The call to action suggests a specific action the audience should take to receive the benefit of the promise. This action needs to be realistic and do-able. This helps the audience make a quick decision instead of delaying action or forgetting to take action.

Step 9: Determine Creative Considerations

There are several factors that will impact the creative process and overall approach. Review the situation analysis and audience analysis to determine appropriate media and materials, as well as overall tone. Provide the creative team with information on the following creative considerations:

These creative considerations will guide the creative team in their development of messages and materials. Other considerations may include: geographic placement, language, any program requirements, literacy and branding and marking guidelines.

Step 10: Map the Timeline

The creative brief is the guiding plan for the internal team, agency or consultant hired to do the creative work. It should include a realistic timeline. The timeline should include each task (e.g. review, testing, revisions) with a realistic number of days for completion (see Timeline Template under templates). Keep the timeline in a place that is visible and will allow the team to stay on track. Be sure to update the team if tasks or the timeline changes.

Step 11: Develop the Budget

Anything that takes time and labor has a related cost. For example, someone from the team will likely have to travel to test materials. Be sure to identify all tasks and the cost (including pretests and revisions) of each task (see Budget Template under templates).

Templates

Creative Brief Template One

Creative Brief Template Two

Creative Brief Budget Template

Creative Brief Timeline Template

Samples

NURHI Get it Together Creative Brief

Club Risky Business Creative Brief

Older Men and Voluntary Medical Male Circumcision (VMMC) Creative Brief

Tips & Recommendations

• Try to think of different and creative ways to get the message to the priority audience. For example, how would gossip spread in the community? Is it possible to use the same approach to spread a positive message?

Lessons Learned

• The creative brief helps to clarify the project’s goals and objectives.
• The creative brief provides a very good and concise historical record of the thinking behind the products produced for a campaign.
• Keep all creative briefs in one folder (electronic or print) for quick reference.

Glossary & Concepts

• A communication campaign includes a combination of approaches (usually including mass media in addition to community-based approaches) and provides multiple opportunities for exposure through a consistent theme that links program activities together.
• Demographic information is statistical data (e.g. age, sex, education level, income level, geographic location) relating to a population and specific sub-groups of that population.
• Priority audience is the group of people the communication campaign is aiming to reach.They may be the people who are directly affected by the health or social issue or who are most at risk of the issue, such as a new mother wanting to begin using a family planning method. Or they may be people who are best able to address the issue or who can make decisions on behalf of those affected, such as mothers and fathers of young children.
• Psychographics are the attributes that describe personality, attitudes, beliefs, values, emotions and opinions. Psychographic characteristics or factors relate to the psychology or behavior of the audience.
• Social norms are the rules or standards of behaviors shared by members of a social group.

Resources and References

Resources

Developing a Creative Brief

The DELTA Companion: Marketing Made Easy

References

• The BLU Group – Advertising & Marketing. Seven Key Questions to Answers When Writing an Effective Creative Brief.
• Jeff Berry Rules. How to Write a Creative Brief.
• Raven. How to Write a Killer Creative Brief.
• Smart Insights. How to Write a Creative Brief that gets Results.

---

Banner Photo: © 2013 Valerie Caldas/ Johns Hopkins University Center for Communication Programs, Courtesy of Photoshare

Introduction

What is Concept Testing?

Concept testing is the process of sharing creative concepts with the intended audience to get their feedback and identify the best idea before designing materials (See the How to Develop a Creative Concept guide). Results from creative concept testing help the creative team to revise concepts, drop ones that do not resonate well with the audience and identify the ones the audience likes best. Sometimes none of the concepts is appropriate for the audience. In these cases, the creative team returns to brainstorming, keeping in mind lessons learned from the concept tests.

Keep in mind that for social and behavior change communication (SBCC) campaigns and materials to be most effective, they should be tested at several stages of development. In the SBCC process, four types of testing are typically conducted: concept testing, stakeholder reviews, pretesting and field testing. The graphic below demonstrates the relationship between the four types of testing. This guide covers concept testing.

The purpose of creative concept testing differs from that of materials pretesting. With creative concept testing, the intention is to seek audience input into an overall creative concept or idea. Materials pretesting, on the other hand, seeks to assess audiences’ reactions to creative materials before they are finalized. Since the purpose of concept testing is not to pretest the actual copy and visuals of a creative material, concept test materials need not include final copy or visuals. Concept tests only need to communicate the creative idea.

Why Test Creative Concepts?

Testing creative concepts helps identify the best way to meet campaign objectives. Understanding what works—or does not work—saves time and money. It helps ensure that communication materials are developed based on an effective creative concept that connects with the intended audience. Concept testing allows teams to evaluate big ideas side-by-side and select from among alternative versions. Testing also reveals:

• Confusing words, phrases or ideas that should be clarified
• Preferred language used by the intended audience
• Preferred visual styles
• Weak concepts that should be cut
• New ideas

Who Should Conduct Concept Testing?

Ideally, researchers trained in concept testing methodologies – either an external research firm or internal research staff – should conduct concept testing. Sometimes, the hired creative agency conducts the concept tests. It is important that whoever conducts the testing be objective and able to ask questions without biasing or leading participants to a certain decision. The campaign implementation team, including stakeholders, can observe the testing or read through transcripts of interviews and focus group discussions to understand audience reactions and suggestions.

When Should Concept Testing be Conducted?

Concept testing should be conducted after a few viable creative concepts have been developed and before materials development begins.

Learning Objectives

After completing the activities in the concept testing guide, the reader will:

• Select the most appropriate study participants and method for concept testing, based on available resources, timing and objectives
• Prepare tools for the concept test
• Refine a creative concept based on what has been learned during the testing process

Estimated Time Needed

Concept testing typically takes between one to two months depending on the testing method used and the number of revisions necessary. If creative concepts require a complete rework, it could take longer.

Prerequisites

• Creative Brief
• Creative Concept
• Audience Analysis

Steps

Step 1: Determine Which Elements to Test

After developing a few creative concepts that support the creative brief and the audience analysis, determine which of the following elements are most critical to test:

• Attention. Does the idea attract audience attention? This is often measured as a person’s ability to remember an idea, message or image.
• Comprehension. Is the idea clearly understood?
• Motivation. Does the idea inspire the audience to take a desired action?
• Personal relevance. Can the audience connect with the idea? Does it take their point of view into consideration?
• Cultural appropriateness. Is the idea consistent with the values, attitudes, beliefs, traditions and history shared by the intended audience?

Step 2: Identify Testing Audience

The best way to get a sense of how the audience will react to a concept is to test it with them. Review the audience profile created for the primary and influencing audiences. These descriptions can help determine where to find audience representatives, as well as the methods that will be used for concept testing. Some examples include recruiting pregnant women from antenatal care clinics, female sex workers from known hot spots, like night clubs, or mobile men from truck stops.

Based on the audience profiles, create a list of characteristics that concept test participants must have in order to participate. These will become part of the screening questionnaire in Step 5 that establishes criteria for participation. Consider demographic (age, sex, relationship status, number of children), psychographic (needs approval from friends, values intimacy), behavioral (never used family planning, do not want more children) and geographic (rural, urban, density) characteristics.

Also consider getting feedback from people who influence the intended audience, such as health care providers, teachers, religious leaders or spouses. Testing concepts with these influencers is a good way to learn about cultural appropriateness and to gauge their support for the campaign.

Step 3: Select the Best Testing Method

Decide which method(s) will be used to test the concepts: focus group discussions, in-depth interviews, intercept interviews or self-administered questionnaires. The method selected will depend on the depth of feedback required, characteristics of the audience representatives, time and budget.

Test Method	Sample Size	Appropriate For
Focus Group Discussions (FGDs)A discussion on the concept guided by an experienced moderator. Can be conducted online or face-to-face.	Six to ten people; Usually several FGDs/ concept.	Good for generating new ideas and gauging emotional impact. Work best when it is fairly easy to bring together a group of people.
In-Depth InterviewsOne-on-one discussion between an interviewer and a participant. Can be conducted in person or over the phone.	At least ten	Work best when testing concepts among people from several different geographic locations, or when the creative content is sensitive, such as reproductive and sexual health, making the audience feel uneasy discussing it in a group.
Intercept InterviewsTrained interviewer shows concept materials to the intended audiece in a place they frequently visit (markets, clinics) and conducts a quick survey with them.	60-300	Work best when many audience representatives frequent the intercept location, when testing for one element, such as attention or cultural relevance, across alternative concepts. They help reduce recruiting costs.
Self-Administered QuestionnairesQuestionnaires of mostly close-ended questions are completed by respondents. Can be done in-person, via email or mobile, or online.	20-200	Good for large numbers of respondents with a short timeframe. Work well when testing creative concepts for websites, mobile applications or communication campaigns intended for audiences who come from diverse geographic locations, such as from across countries or regions of one country.

Step 4: Develop Testing Methods

Once the testing method(s) have been chosen, the team will need to develop mockups or visual presentations of each alternative creative concept that can be shared during concept testing. How these materials are presented will depend on the method of testing, the nature of each concept and the characteristics of the people who will participate in the testing exercise. Often, the best way to present creative concepts is through a visual presentation that includes samples of various materials that could be used. This helps to minimize the effort and costs of preparing full sets of campaign materials for each creative concept. Presentations can be projected electronically, shared online and as poster boards, or printed as handouts. It is important that mockups of each creative concept are comparable in quality to minimize bias.

When using intercept interviews, it might be best to present each concept via a poster to minimize on time and eliminate the need for technology. When using in-depth interviews or focus group discussions, an electronic presentation, such as Powerpoint® or Prezi, might work well since they are more interactive and there is sufficient time to explore. When pretesting concepts for a video or film, storyboards work well, since they visually show a sequence of events. When testing concepts for a multi-channel campaign, an electronic presentation allows audience representatives to see mock-ups of a variety of media. On the other hand, when testing concepts for radio or TV spots among rural residents, where electric power is not readily available, dramatic readings work better than electronic presentations.

Several creative concepts were developed and tested for the Safe Male Circumcision Campaign Uganda. A Powerpoint® presentation was printed as poster boards and shared during focus group discussions with uncircumcised men to test the creative concept for a national campaign promoting voluntary medical male circumcision in Uganda. See the Samples section for the full presentation.

Step 5: Prepare Testing Tools

Develop the tools that will be used during concept testing. The tools needed depend on the testing methods to be used. Four types of tools may be required: recruitment screener, consent form, moderator’s guides and questionnaires.

• Recruitment Screener. A short questionnaire that is administered to potential participants—either in-person, online or by telephone—to ensure that they meet the criteria for participating in the concept testing, as described in Step Two. It should be used to guide the selection of testing participants, regardless of testing method used (see the Samples section for sample Screeners).

• Consent Form. A brief form that describes the research and what a participant should expect, including the risks involved, and allows a participants to voluntarily consent to participate. The form can be read to participants if they are not literate (see the Samples section for sample form).

• Moderator’s Guide. An important tool for focus groups and in-depth interviews, the moderator’s guide serves as an outline to help keep the moderator on track during discussions. The guide contains topics, questions and activities to spark discussion. Normally, when testing more than one creative concept, testing involves sharing each concept individually, followed by a few questions. After all alternative creative concepts have been shared and discussed, the moderator asks participants which of the concepts they prefer and why. In developing the guide, determine the key questions to ask and probes for obtaining answers in a conversational manner. A typical guide contains the following (see the Samples section for examples of moderator’s guides):
  • Objectives and elements to be tested
  • Brief respondent profile, including the number of interviews or discussions to complete
  • Where and when the interviews are to take place
  • An introduction script to be read to participants
  • Questions and activities
  • Closing script to be read to participants

• Questionnaire. A standard set of questions and response options that may be completed by the respondents themselves or by interviewers (see the Samples section for sample questionnaires). In general, questionnaire questions should:
  • Contain only one idea or question
  • Define the scope to consider, such as the time period or activities that are relevant to the question
  • Be written with neutral language to avoid leading the respondent to a specific answer
  • Use language that enables respondents to easily understand the question
  • Contain response options that are simple, clear, consistent and include the full range of responses that might occur
  • Provide mutually exclusive response options so that a respondent can pick only one option

Here are some sample questions that can be asked after showing each individual concept. These questions are designed to test concepts for each of the five elements listed in Step One.

Testing Element	Recommendation	Sample Question
Attention	Provide pre-coded questions	Have you seen similar concepts/advertisements? Compared to other advertising you have seen, would say this concept is very memorable, memorable or not so memorable?
Comprehension	These are open-ended questions that will be coded during analysis	What is the first thing that comes to your mind when you see this concept/poster/presentation? What is this concept/poster/presentation telling you? (Record the time required to understand the main message.)
Motivation	Provide pre-coded questions	After seeing this concept/poster/presentation, how likely are you to adopt the practice/use the service or product?
Personal Relevance	Provide pre-coded questions	Would you say that you completely believe/partially believe/do not believe what this presentation/poster is telling you? Is this concept made for people like you or someone else? If someone else, what sort of person is this concept for? (Provide alternative characteristics, such as older/younger, more/less wealthy, more/less educated.
Cultural Appropriateness	Use pre-coded questions to gauge degree of appropriateness; use open ended questions to probe how a concept is inappropriate	How well does this concept reflect the values and traditions of most people in your culture, such as very well, to some extent, not at all? Would you or others in your community think there is anything offensive about this concept? If so, what?
Comparing/rating concepts	Ask after all concepts have been shared and tested	Please rank these concepts in order of preference, from most preferred to least preferred.

Step 6: Conduct the Testing

Proceed with testing as detailed in the moderator’s guide or survey instrument. At the start of each focus group, interview or survey, be sure to welcome the participants and provide a brief overview on the purpose of the testing. Reinforce that there are no “right” or “wrong” answers, and encourage everyone to provide their honest views. Make sure to obtain permission from participants if planning to video or audio record the sessions. Take notes and/or record the feedback on an audio or video recorder, as appropriate.

Step 7: Analyze Testing Outcomes and Summarize Results

Review all notes and recordings from the testing exercise and write a report outlining the process and the findings. The report should have the following sections:

• Background. What was tested? What were the objectives of testing? Who participated in the testing? How was the testing done?
• Highlights. Summarize the main points that came up during the testing.
• Findings. Present a complete report on the findings. Where appropriate, describe the participants’ reactions, incorporate key quotes, describe which creative ideas and concepts worked the best versus those that were not appealing or effective, and share suggestions given by participants.
• Conclusions. Describe the key learnings that came up and/or the major differences that were observed across individuals and/or groups.
• Recommendations. Suggest and prioritize revisions for the tested creative concepts based on the findings and conclusions.

Step 8: Apply Testing Results

Share the testing report with the campaign implementation team and apply the recommendations to sharpen and hone in on one creative concept. Use the research to inform decisions and help develop improved concepts and creative materials. Sometimes the testing will identify one creative concept that is clearly preferable and the results can be used to improve that concept. In other cases, there will be no clearly preferred creative concept, and the results can be used to inform new concepts.

Be aware of the limitations of testing. Keep these things in mind to avoid using the results incorrectly.

A test is only as objective or fair as the person designing and conducting the research and interpreting the results. Testing cannot take the place of experienced judgment. Rather, it can provide additional information to help make good decisions. Testing results should not be overgeneralized. For example, the opinions of two or three people in a focus group of women ages 25-35 cannot be applied to the entire population of women in this age group. These are just a few perspectives to help see how concepts or materials might be received by some. Although testing is very important, it does not guarantee that a creative idea or product will succeed in reaching or appealing to an intended audience. Once developed, continually assess whether the campaign is meeting its objectives. If not, changes might be needed.

Templates

Informed Consent Form Template for Qualitative Studies

Samples

Sample Focus Group Screener Form

Sample Focus Group Moderator’s Guide

The CDC’s VERB[TM] Campaign

Concept Testing Discussion Guide

CP Phase 2 Concept Testing

Safe Male Circumcision Communication Campaign Logo and Concept Test Report

Concept-­Testing Research Findings for CP Tulizana Campaign Phase 2

Focus Group Participant Recruitment Screener

Intercept Questionnaire

External Resources

https://www.surveymonkey.com/mp/concept-testing/

Tips & Recommendations

• Be sure to test the concept test tools/resources developed for each method before using them with the audience
• Emphasize at the beginning of the concept test that these are only rough drafts/prototypes, and that they should evaluate based on the big idea, not the specifics. Colors, fonts, models and details will be further refined at a later point.
• Do not try to merge concepts following the concept test, picking the most well liked pieces from different concepts and combining them into one. This dilutes and confuses the entire campaign.

Lessons Learned

• When asked to compare creative concepts, respondents sometimes get distracted by the way the concepts are presented, and rate a concept based on the quality of artwork, color schemes or typeface used. A way around this is to test each creative concept on its own merits and not ask respondents to rate concepts against each other. Being consistent with the quality of visual materials also helps. Moderators also need to be well trained in how to steer the conversation away from this level of feedback and focus it on the bigger picture.
• Sometimes, the order in which creative concepts are introduced creates bias. This sort of bias can be reduced by varying the order in which the concepts are presented in each interview or focus group discussion.
• Concept testing is useful not only to test creative concepts for a campaign, but also to get audience feedback on design features of communication products. For example, when designing websites, it is useful to allow prospective users to interact with the site during the design stage to test the layout, features and look. Testing design features can save time and money over the long run by preventing costly redesign.
• Often, the differences between creative concepts is so slight that audience representatives cannot distinguish them. When testing more than one creative concept at a time, it is important that each concept is unique from one another and there is no ambiguity.
• The location where focus group discussions or in-depth interviews take place can influence results. Focus groups and interviews that take place in an office often yield different results from those that take place in communities where the respondents are from.
• Too many concepts produce concept fatigue in the respondent. The maximum number of concepts to test in a focus group or interview is three.

Glossary & Concepts

• Audience analysis: A description of the defining characteristics of the people who are targeted by a health communication campaign or product.
• Concept testing: seeks feedback about general ideas, concepts and creative concepts; typically done before materials are developed.
• Creative brief: A short document outlining the audience, objective, promise, key message content and the communication channels that will be used; a succinct account of a campaign strategy.
• Creative concept: The overarching artistic theme that ties together all elements of an SBCC campaign. It is an imaginative plan for capturing the audience’s heart and mind.
• Field testing: allows practitioners to observe whether the SBCC materials are used effectively in their intended settings and contexts, usually through observation and focus group discussions. It determines whether the material meets the intended purpose.
• Pretesting: is the process of bringing together members of the priority audience to react to the components of a communication campaign before they are produced in final form.
• Stakeholder reviews: are input from technical experts, partners and decision-makers prior to finalizing materials. These reviews do not replace pretesting with the priority audience and can be done before or after pretesting.
• Storyboard:
• Dramatic reading: is a public reading or recitation of a work of literature with an interpretative or dramatic use of the voice and often of gestures.
• Pre-coded questions: are a type of question that is asked by interviewers as though they are open-ended questions, but they have pre-coded responses that interviewers use to match (code) respondents’ answers, rather than copy down the verbatim response.

Resources and References

Resources

Making Health Communication Programs Work

A Step by Step Guide for Focus Group Research

Conducting Focus Group Interviews

Developing and Testing Creative Concepts

Six Concept Testing Strategies

References

• Making Health Communication Programs Work: A Planner’s Guide. December 2001. Washington, DC: U.S. Department of Health and Human Services, Public Health Service, National Institutes of Health. Office of Cancer Communications, National Cancer Institute. NIH Publication No. 02-5145. Pp 53 – 89. www.cancer.gov/pinkbook
• Concept Testing in Qualitative Market Research, http://www.focusgrouptips.com/concept-testing.html

---

Banner photo: © 2005 Jennifer Orkis, Courtesy of Photoshare

Introduction

What is a Creative Concept?

A creative concept is an overarching “Big Idea” that captures audience interest, influences their emotional response and inspires them to take action. It is a unifying theme that can be used across all campaign messages, calls to action, communication channels and audiences. Typically, the creative concept is embodied in a headline, tagline and a key visual. Successful creative concepts are distinctive, memorable, unifying and relevant. Some examples include: Nike’s “Just Do It” campaign, the “Got Milk?” campaign and the Red Ribbon Campaign.

The “Big Idea” behind the UNICEF Tap Project was to brand tap water in New York City. The campaign asked restaurant diners to donate $1 for the tap water restaurants usually serve free of charge. Diners were told that their $1 donation would provide clean, safe water to a child for 40 days.

Creative concepts are based on the communication strategy and creative brief. This ensures that concepts are informed by a strong understanding of the situation, the audience, the channels that will be used, the objectives the campaign seeks to reach and the benefits the audience will respond to. The creative team develops multiple creative concepts based on this information and then concept tests them to determine which one resonates best with the audience.

The Green Star campaign in Tanzania developed several creative concepts for their family planning campaign. The winning concept was to recognize anybody who was positively involved with family planning as a star. The concept could be used across audiences. For example, married couples who decided together which method to use were stars, media houses that reported accurately on family planning were stars and dedicated family planning service providers were stars. The concept worked across many channels, including print, community and mobile.

What is NOT a Creative Concept?

Creative concepts are NOT final products. They are rough drafts that give an idea of how the campaign could take shape. Keeping this in mind, creative concepts are not:

Different colors or fonts	These are graphic design elements that will be developed in the executions (materials) after concept testing. Colors and fonts should be tested during the pretesting stage.
Different models	Concepts typically use stock photography or rough illustrations to convey the general idea. Talent selection for the actual models to be used in the final executions can be older/younger/shorter/ taller/happier/more serious. Models should be tested during pretesting, once the concept is solidified.
Different informational messages or calls to action	The concept should be strong and overarching enough to work for all messages/calls to action.

Why Develop a Creative Concept?

In today’s busy and constantly changing communication environment, social and behavior change communication (SBCC) must be creative and strategic to compete with commercial advertising and capture the audience’s attention. The creative concept shapes the core of the SBCC campaign, which helps create consistency across messages and materials. Developing creative concepts is also important because it allows the team to:

• Test which benefit is most appealing to the audience
• See how the campaign will work (or not) across multiple media channels
• Identify an idea that ties the whole campaign together
• Reach the audience more effectively by finding ‘big ideas’ that resonate with them

Who Should Generate the Creative Concept?

There are different approaches to generating creative concepts. One approach is to outsource creative development to an advertising or creative agency. The creative agencies develop several concepts and test them with the audience. Often, a committee made up of SBCC and government professionals reviews the concepts and makes a final decision based on concept test results and campaign objectives. Another option is to hold a design workshop with multiple stakeholders, including SBCC professionals, technical/topical experts, service delivery partners, government officials, multi-sectoral bodies, audience members and a creative team, including creative directors, graphic designers, illustrators, script writers, copy writers, producers, actors and musicians. During the design workshop, the participants review the creative brief and hold creative concept brainstorming sessions. The team needs a leader and it is sometimes helpful to have a creative expert facilitate the exercise.

When Should the Creative Concept Be Developed?

The creative brief marks the beginning of the creative process and acts as a “creative contract” agreed upon by all partners. The creative brief provides a road map for developing the creative concepts. Materials and activities are developed based on the winning creative concept.

Learning Objectives

After completing the activities in the creative concept guide, the team will:

• Produce a set of creative concepts based on research findings and strategic insights
• Select and refine the ideas that best support the SBCC strategy and creative brief

Estimated Time Needed

Developing a creative concept can take anywhere from a few days to two months depending on who is on the team, how experienced team members are at creating campaign concepts, campaign size and number of channels selected, time needed for individual brains

Prerequisites

• Creative Brief

Steps

Step 1: Develop a Problem Statement

Before holding a brainstorming session, develop a problem statement based on information in the creative brief and the communication strategy. It may help to review and revise (as necessary) the problem statement developed during the situation analysis. A problem statement is a concise description of the issue that needs to be addressed by the SBCC campaign. It answers three questions:

• What is the problem or communication challenge to be addressed? This explains why the communication campaign is needed. [Example: Low uptake of family planning services]
• Who has the problem? This describes the audience to be reached. [Example: Young married women ages 15 – 24 in Kirongo region]
• What will the audience do as a result of the campaign? This will explain how the project will measure success. [Example: Speak to a health worker about contraceptives]

Sample problem statementEven though services are available, very few young married women in Kirongo region use modern contraception. This campaign will inspire young married women ages 15-24 in Kirongo to talk with a health worker about contraceptives.

Step 2: Select the Brainstorm Team

Assemble a team to conduct a creative concepts brainstorm. If outsourcing, creative agency staff may do the brainstorming. If conducting a design workshop, be sure to invite members from a variety of backgrounds, including government staff, creative professionals, SBCC experts and technical experts. Clarify the team member roles before brainstorming sessions begin:

• Team leader: responsible for developing the problem statement and inviting people to the brainstorm, presents the problem statement with background information.
• Facilitator: leads the brainstorm, encourages full participation and the flow of ideas, and keeps the group focused on the task. The facilitator must be familiar with the creative brief, situation analysis, communication strategy and other background information.
• Note taker: writes down ideas generated during the brainstorm, recalls ideas during the session as requested and compiles the notes into a [brief] report.
• Team members: share ideas even if they seem unlikely or crazy, respect the contributions of others and actively participate in the brainstorm session.

Step 3: Organize the Brainstorming Session

Arrange a time and place for the session and invite participants. Ensure that the room is comfortable and has markers and a flip chart, white board or blank paper to capture ideas. To encourage creativity, it may be helpful to decorate the room, provide toys to play with or play fun music.

Participants are more likely to be creative if their minds are open and prepared for brainstorming. When inviting participants, include any tasks they should do before arriving at the brainstorming session. For example, it may help them to read or view something inspirational, try something new or interrupt their usual routines.

Step 4: Provide Background for the Brainstorm

At the beginning of the brainstorm session, the facilitator states the objectives and expected outcomes of the meeting to create a common purpose. The facilitator then outlines what will happen in the meeting and the methods that will be used, such as individual idea generation, group brainstorming, small group work or critique sessions.

Next, the facilitator helps the participants set ground rules for the brainstorm. For example:

• Quantity over quality
• No idea is a bad idea
• Only one person can talk at a time
• Present ideas as quickly as possible
• No comments on any of the ideas during the brainstorm

Remind participants of what they have agreed to by posting these rules around the room. The facilitator will likely need to guide participants to follow the rules.

The team leader (or facilitator) then presents the problem statement and key points from the creative brief. Team members, including partners, share summaries of research findings from the analyses conducted. This will help ground the idea generation, making it more strategic.

Step 5: Break the Ice

To produce creative ideas, participants need to open their minds and feel comfortable with each other. A silly ice breaker or fun activity can help participants get in a creative mind-set and not be as concerned about what others think of them. See the Resources section for links to ice breakers and other activities.

There are many ways to conduct a brainstorm session for creative concepts. The following steps present one common method. Choose a method that works best for your group of participants.

Step 6: Individual Idea Generation

Start by giving participants a set amount of time to generate ideas individually (30 minutes is usually good). Provide individuals with sticky notes and ask them to think of themes or ideas that would encourage the desired change and communicate a benefit for making that change. Usually, this stage involves coming up with an image or overarching idea, but individuals may think of a headline or tagline, as well. Individuals can draw, write or express their ideas in other fun ways on their sticky notes.

The purpose is to generate as many ideas as possible. Encourage all ideas, not just “good” ones; ideas tend to get better as the session progresses. Remind participants that now is not the time to evaluate their ideas. Taking time to sort good from bad will break their creative momentum.

Once the time limit is reached, ask everyone to put their sticky notes up on a board. If time allows, participants can read their ideas out loud as they put them on the board. Then let participants walk around so they can see all the ideas up on the boards. If the group is smaller, the facilitator can ask everyone to work together to group the ideas. If the group is large, assign a few people to group them.

Step 7: Expand Ideas

Next, expand on the groups of creative ideas. This is often done in small groups, but it also can be done all together, especially if the group is small. Divide the participants into small groups of three or four, ensuring a good mix of backgrounds (creative, technical, communication, government). Assign each a grouping and ask them to further develop that creative idea. The small groups should come up with a visual and a headline, slogan or tagline that goes along with the visual. Set a time limit and remind participants of the ground rules.

The facilitator can provide participants with methods for expanding on ideas, such as:

• Dissection: Take the idea apart and discuss how to improve or change each part.
• SCAMPER: Think of ways to substitute, combine, adapt, modify, put to another use, eliminate or reverse the idea.
• Challenge assumptions: Ask what assumptions the idea makes and challenge them. Dig deeper to challenge the way the group views the problem and the audience.
• Opposites: Ask what is the opposite of the idea and whether it could work.
• “Yes, and…”: Read the idea aloud. Then, say, “yes, and…”, adding a new piece to the idea.

A campaign to reduce concurrent sexual partnerships developed two top concepts. 1. Tagline: “Some things are not meant to be shared.” The concept involved people sharing unexpected items like sugarcane, gum, and toothbrushes. 2. Tagline: “How many are we?” The concept involved presenting scenarios where a couple thinks they are alone (car, restaurant, bedroom). Additional characters would continue to enter the scene, creating obvious romantic linkages to the main characters and each other. The couple ends up thinking, “Is it really just the two of us, or are we many more?”

When the time is up, each group presents their ideas to the larger group. During presentations, participants can help refine ideas using positive phrases like, “Yes, and…” The note taker should record every idea presented. Ideas can be written on a white board or flipchart, recorded using visual notes or graphic recording, or captured using various applications and software.

If participants get stuck during a brainstorm, the facilitator can encourage creativity using various techniques:

Roadblock removal. Get rid of all barriers. Ask what participants would do if there were no limit on time, money or other resources. Inspirational objects. Select a physical object from the room and ask participants to come up with ideas that include or are related to it. Existing ideas. Ask participants to share ideas from existing communication programs they believe are creative and inspiring. The ideas do not have to be related to health. Ask the group to imagine how these ideas could work for the problem they are trying to address. Roadblock introduction. Create a ridiculous barrier to get participants’ brains working in a new way. For example, tell them all the audience members are blind, or that they only have a very small amount of money. Outrageous thoughts. Ask participants to suggest ideas that are fun, but risky – and those that could get them into trouble. Sometimes the best ideas start off crazy! “What Would He or She Do?” Ask participants to imagine how someone else might solve the problem. This could be someone famous, a farmer or a small child. “What is the opposite of what we want?” Ask participants to describe the opposite of what they want to achieve. Once they have some good ideas flowing, ask them to switch directions to generate ideas of what they do want. Rotate. Ask participants to change their physical position. For example, they can switch to a different group or move to a new location.

Step 8: Select and Refine the Two to Three Best Ideas

Once all of the ideas have been presented, the facilitator leads participants in an exercise to identify the strongest ones. The facilitator may have participants fill out a rating matrix (see Templates for an example), or vote on the best ideas using a dot vote or other method. Participants should keep the following criteria in mind when selecting ideas:

• Does it follow the creative brief and communication strategy?
  • Is it appropriate for the audience?
  • Does it address the communication objectives, key benefits and barriers?
  • Can it be rolled out in multiple ways across several media channels?
• Is it ‘fresh’, new, outside the box? Does it capture attention?
• Does it have the potential for expansion?

The facilitator reviews the results and identifies the top three ideas. Using methods like Round Robin and 6 Thinking Hats, the facilitator then helps the group to refine the top ideas into final creative concepts. Some questions to consider include:

• What are the good and challenging/potentially problematic aspects of this idea?
• How might we fix the challenging aspects?
• How can this idea be presented in a way that makes it real for the audience?
• How can this idea work throughout the campaign/program/strategy?

Step 9: Check Feasibility

After the brainstorming session, check the three final concepts again to ensure they are feasible and consistent with the creative brief. For each, consider each of these questions:

• Will this work?
• Will it interest the audience? Would it offend any audience members?
• Is budget sufficient to support the idea?
• Can this idea be promoted through the communication channels selected for this program/campaign?

For each viable concept, search the Internet and query personal contacts to ensure the idea is not already in use by another organization or for another issue.

Step 10: Prepare Visual Presentations of Concepts

The creative team now develops the final three concepts visually and verbally so that they can be tested with the audience. The most common approach is to develop a concept board containing a headline, tagline and a key visual. The visual is an image that complements the idea and serves as a catalyst for discussion. Concept boards can be illustrated or stock photography can be used. Sometimes the concepts can be presented through a storyboard for a TV spot, an outline of a radio spot or a written description of how the concept would roll out across multiple media. Below are a couple examples of concept boards. See the Samples section for more.

For Folic Acid use:

For controlling alochol consumption:

Creative concepts are rough drafts – just enough to convey an idea. The team can use electronic presentations to develop them, with stock photos or basic images, or they can draw basic figures on paper. Once developed, it is helpful to present the concepts to the original group of participants to vet them. Then, the concepts are tested with the audience to get feedback before developing materials.

Templates

Creative Concept Rating Matrix Template

Samples

Alive & Thrive Infant and Young Child Feeding Community-based Support Group Model in Viet Nam

Stand Proud, Get Circumcised

Creative Concept Mockup Example

External Resources

http://www.ted.com/talks/tim_brown_on_creativity_and_play)

http://www.ted.com/talks/amy_tan_on_creativity

http://www.ted.com/talks/david_kelley_how_to_build_your_creative_confidence

http://scottberkun.com/essays/34-how-to-run-a-brainstorming-meeting/

Tips & Recommendations

• Before brainstorming, participants should open their minds to inspiration from the world around them. Some ways to do this include:
  • Read a novel, look at magazines and newspapers, or visit websites.
  • Change normal routines—work from a new location, listen to a different radio station, interact with new people or do things in a different order.
  • Talk to people in different fields to learn about the creative ideas that have worked for them, and think about how these ideas might apply.
  • Look at examples from other countries to spark ideas.
• Hold the brainstorming session in a creative environment. Decorate the room with unique objects, such as sculptures, architectural models and toys, photographs or drawings that may spark creative thinking. Play unusual music as the session begins and during breaks.
• Mix up small groups, making sure there are creative, technical, communication and government people in each brainstorming group.
• Do not get discouraged if the first brainstorming session fails to produce good ideas. Hold another session with different participants.
• Do not evaluate ideas when they are presented. It kills the new ideas.

Lessons Learned

• Many people think they are not creative, and divide the world into creative and non-creative people. This is not the case and creative concept team members need to believe in their ability to be creative. Facilitators can nurture this self-confidence by offering encouragement and a non-judgmental attitude.
• Most people fear the judgment of their peers, even at work. This fear causes people to think and speak conservatively. They might have a wild idea, but are afraid to share it. Helping people feel secure by providing a trusted environment makes it more likely they will feel free to express their ideas.
• Let people brainstorm in whatever language they feel most comfortable.
• During brainstorming, focus on the quantity of ideas, not the quality. When people come up with as many ideas as possible in the shortest time allowed, they are more likely to innovate.
• Brainstorming generates much more creative outcomes when everybody follows agreed-upon rules. It helps to post brainstorming rules on the walls—things like: “defer judgment,” “go for quantity” or “think big.” Providing written rules helps people break old rules and norms that they might bring to the creative process.
• When leaders and facilitators start out sharing bad or silly ideas, it frees other participants to speak their ideas. It is critical to have an environment where everybody feels comfortable sharing their ideas.
• A good facilitator is essential to a productive brainstorming session. Choose a facilitator with good listening skills, sharp group awareness and the ability to help people express their ideas, especially those who are usually quieter.

Glossary & Concepts

• Communication challenge: The gap between what a group of people is doing and what they should be doing to correct a health problem. This gap is what a communication campaign is designed to address.
• Concept board: A visual representation of an idea that usually contains a headline, visual and tagline.
• Creative brief: A short document outlining the audience, objective, promise, key message content and the communication channels that will be used—a succinct account of a campaign strategy.
• Creative concept: The overarching artistic theme that ties together all elements of an SBCC campaign. It is an imaginative plan for capturing the audience’s heart and mind.
• Headline: The lead line that is usually at the top of a print execution with the most prominence.
• Tagline: A phrase or slogan that identifies a product or behavior. Taglines are short and memorable. They give the audience a reason to do what they are being asked to do.
• Problem statement: A concise description of the issue that the communication campaign will address. It should indicate why a communication campaign is needed, the people who are affected and the behavior(s) those people should adopt.
• Stock photography: Photographs of common people, places, landmarks or things that can be bought and used for design purposes. Typically, they are used to save money on a photographer.
• Storyboard: A set of illustrated panels, displayed in sequence, which visually tells a story. A storyboard shows what happens in a story by drawing out the contents of the story.
• Stakeholders: Those who are affected by, have a direct interest in or are somehow involved with the health or social problem.

Resources and References

Resources

Developing and Testing Creative Concepts

References

• Mark W. Stuhlfaut & Chan Yun Yoo, A tool for evaluating advertising concepts: Desirable characteristics as viewed by creative practitioners, Journal of Marketing Communications, Volume 19, Issue 2, 2013, pages 81-97, http://www.tandfonline.com/doi/abs/10.1080/13527266.2010.550009 (accessed May 2015)
• Swee Hoon Anga, Siew Meng Leonga*, Yih Hwai Leea & Seng Lee Loua Necessary but not sufficient: Beyond novelty in advertising creativity, Journal of Marketing Communications, Volume 20, Issue 3, 2014, pages 214-230. http://www.tandfonline.com/doi/abs/10.1080/13527266.2012.677464?src=recsys#.VVXMndGJjcs (accessed May 2015)
• Erin Schreiner, The difference between advertising campaigns, strategies and concepts. http://smallbusiness.chron.com/difference-between-advertising-campaigns-strategies-concepts-17682.html (accessed May 2015)
• Riku Vassinen, The seven characteristics of the great concept, Stand Up Strategy, http://standupstrategy.org/2012/07/11/the-seven-characteristics-of-the-g… (accessed May 2015)

---

Banner Photo: © 2011 Katherine Lin/Johns Hopkins Bloomberg School of Public Health, Courtesy of Photoshare

Introduction

__________________________________________________________

Click here to access this Guide in Portuguese – Guias em Português

__________________________________________________________

Pretesting is the process of bringing together members of the priority audience to react to the components of a communication campaign before they are produced in final form. Pretesting measures the reaction of the selected group of individuals and helps determine whether the priority audience will find the components – usually draft materials understandable, believable and appealing.

Components of a communication campaign that benefit from pretesting include:

• Key benefit and support points
• Messages
• Story boards
• Draft materials
• Name of campaign and logo
• Signature tune/music
• Translated text
• Interpersonal communication activities such as those used by peer educators or field workers

Keep in mind that for social and behavior change communication (SBCC) campaigns and materials to be most effective, they should be tested at several stages of development. In the SBCC process, four types of testing are typically conducted: concept testing, stakeholder reviews, pretesting and field testing. The graphic below demonstrates the relationship between the four types of testing. This guide covers only pretesting.

Why Conduct Pretesting?

Pretesting increases the impact of SBCC materials by determining if what has been designed is suitable for the audience. It can save money, time and energy overall as the resulting material will be effective.

Pretesting should be conducted to gather information from the audience on the basic aspects or elements of the material, including:

Do not skip the pretesting phase. If there are not resources or time to conduct a large-scale pretest, even a small-scale pretest can offer useful insights if it is thoughtfully designed.

Who Should Conduct Pretesting?

A small focused team of key program staff (3-4 people) should develop the plans for pretesting. For the pretest to be most effective, however, it is best to find people most like the priority audience – who are trained in pretesting – to lead the actual pretesting exercises. Having someone who is like the audience will encourage honesty and openness during the pretesting process. Some organizations may consider hiring a research firm to conduct the pretesting.

When Should Pretesting be Conducted?

Pretesting should be completed after concept testing, message design, and materials development, and before components of the communication campaign are finalized, produced and disseminated.

Estimated Time Needed

Completing pretesting typically takes between two weeks and two months depending on the testing method, the objectives of the pretest, the number of campaign elements to be tested, and the number of revisions necessary. If materials or messages require a complete rework, it could take longer.

Learning Objectives

After completing the activities in the pretesting guide, the team will:

• Understand the steps and the stages of testing SBCC materials.
• Define and list the elements of pretesting.
• Know how to choose a pretesting method.
• Know how to conduct a pretest.

Prerequisites

• Audience Analysis
• Situation Analysis
• Creative Brief
• Message Design
• Materials Development

Steps

Step 1: Outline Pretest Objectives

To guide the pretest process, the team should develop a plan with a clear set of objectives for each component or material being tested. The objectives describe the aims of the pretest and the information to be gathered. Start by reviewing the creative brief(s) for the SBCC campaign. The creative brief’s description of the priority audience, the promoted behavior and the key promise can be used to inform the pretest objectives.

Step 2: Choose the Pretest Method

After the pretest objectives are established, select the pretest method. Choosing the right method(s), described in the table below, depends largely on the following:

Review the table below for a list of pretesting methods. Keep in mind that using one method might limit the assessment. The use of mixed methods (e.g. survey questionnaire and in-depth interviews) is one way to capture additional information and fill gaps. Project teams should be able to articulate why they have chosen a certain method or methods for their pretest.

Step 3: Plan the Pretest

Plan the details of the pretest. This includes identifying the location and meeting site, recruiting participants, identifying facilitators and interviewers, determining incentives, and designing survey questionnaires or focus group discussion guides as needed. Below are some key points to keep in mind during this process:

Location:The priority audience should feel comfortable with the pretesting location. For example, it might be best to conduct the pretest in areas or places (e.g. clinics, churches) where the priority audience is most likely to encounter the materials.

Facilitators/Interviewers/Note-takers: For focus group discussions and in-depth interviews, make sure to identify trained or experienced moderators or facilitators. Trained facilitators can be found at universities, research firms, or partner organizations. If possible, use a facilitator who has similar characteristics (e.g. age, background) to the priority audience. This helps to develop trust and comfort among the participants. It is also important to have a trained note-taker who is familiar with the topic and speaks the local language.

Participants: Use the creative brief to identify key characteristics of the priority audience. Select a sample of participants that match those characteristics to participate in the pretest. Participants should not have had any involvement in the development of your materials or concept testing. The sample size and collection method will depend on the selected pretest method (see Pretesting Methods Table in Step 2). It is often helpful to over-recruit participants in case some do not show up or complete the pretest. The image below provides some ideas on where to recruit participants. Some organizations have membership lists that can be used for recruitment.

Cost: Create a budget to reflect costs for the meeting site, travel/accommodation, equipment rental, facilitator/moderator’s time, copies of draft materials, stakeholder meetings and incentives. Thoughtful budgeting can help ensure all pretesting costs are accounted for.

Step 4: Develop Pretesting Guide

Develop a pretesting guide that will serve as a reference for keeping the activity on track (see How to Conduct an Effective Pretest for sample pretest questions). The guide should include the following:

• Background information from the creative brief (e.g. description of SBCC campaign and priority audience)
• Pretest objectives
• Pretest plan (description of pretest method, location, participants, facilitators/moderators/note-takers)
• Pretest questions
• Plan for use of information gathered

Step 5: Develop Questions

The goal of pretest questions is to understand the value of the materials. For example, how effective are the posters in influencing young parents to practice birth spacing? A series of open-ended questions will gather specific details about the audience’s preferences. Avoid close-ended (yes or no) questions or those that lead participants to respond in a certain way.

When developing questions, it is helpful to review the pretesting elements listed in the introduction. This will ensure questions are effective and meaningful (see example in table below). It is also important to include questions that will capture demographic information (e.g. age, education level, marital status, number of children) and details on how participants spend their day (e.g. media use, social gatherings). The program and creative teams should work together to contribute questions about behavior and design.

Salazar, 2008

Step 6: Conduct Pretest

Consent Forms: It is important to obtain participant consent (verbal or written) prior to the pretest. Consent forms are written agreements that show the individual has volunteered to participate in the activity. It also informs the participant of the risks involved (or clearly states there is no risk).

Recording and Note-taking: Some pretests use a self-administered questionnaire. When this is not the case, use a pretest answer sheet to note verbal and nonverbal responses to the material. This promotes consistency among interviewers and pretest sessions. Include on the data sheet the date, time, place, name and type of material, audience, respondent number, element (e.g. image, text, font, audio/video segment, character), pretest questions, and other relevant information as appropriate. Pretests can be recorded to help remind or clarify, but recording should not take the place of note-taking (see Resources Section).

The specifics on how to conduct a pretest will differ based on the method. The pretesting guide in the samples section outlines how to conduct a Focus Group Discussion (FGD) pretest. For details on how to conduct other types of pretests (see Resources Section).

With any type of pretest methods, it is important to use open-ended and probing questions to obtain rich information and avoid unduly influencing respondents.

Step 7: Analyze Data and Interpret Results

Analyze the data and interpret the results of the pretest. To analyze:

• Look for trends in responses. If a certain problem or change is mentioned multiple times, it is something that likely needs to be addressed.
• Determine whether results highlight fundamental flaws with the design, messages, or format. If so, the material may need to be completely redesigned. Otherwise, basic revisions should address the problems.
• Consult materials development experts about the suggested changes or problems highlighted. Do not feel compelled to make every change participants suggest.

Step 8: Summarize the Results

Communicate the results of the pretest. Write a report outlining the process and the findings. The report should have the following sections:

• Background: What was tested? What were the pretest objectives? Which audience was involved in the process? Why? How? How many participants were involved in the pretest?
• Highlights: Summarize the main points that came up during the testing.
• Findings: Present a complete report on the findings. Where appropriate, describe the participants’ reactions, incorporate key quotes and describe which creative ideas and concepts worked the best versus those that were not appealing or effective.
• Conclusions: Describe the patterns that came up and/or the major differences that were observed across the individuals and/or groups.
• Recommendations: Suggest and prioritize revisions for the tested creative ideas, concepts, and/or materials based on the findings and conclusions.

The results should be discussed among those involved with designing the messages, creating the materials and conducting the pretest. This includes program staff, designers, writers, editors, interviewers and note-takers (see Pretest Report Sample under samples).

Step 9: Revise Materials and Retest

If the results of the pretest indicate that major revisions are needed, a complete redesign may be required. Once the materials have been revised, pretest the new version if budget and time allow. The same questionnaire or FGD guide can be used as before with questions added or changed as needed on the particular areas of concern. This is to make sure the problem from the first design is addressed in the newer version.

Samples

Making Health Communication Programs Work

Methodology for Pretesting Rock Point (RP) 256 Comic Book

Pretest Discussion Guide for VMMC [Kenya]

Sample Focus Group Discussion and In-Depth Interview Guides

CHCT Materials Pretest Report

Pretest Report of Triple S, ‘Sexy, Smart and Safe’ Health Promotion Campaign

Tips & Recommendations

• Pretest some options and alternate concepts if possible, not only one version of a material.
• Even if you are pretesting a draft and not a final draft, the draft should be as close to the final version as possible so that those reviewing it can judge it appropriately.
• Be open-minded about the outcome of the pretest. If you have already decided what you will find, you will only hear that and miss important insights.
• Don’t use convenience sampling. The group of people most convenient for you to gather may not best represent your priority audience for the material.
• Check with the donor and local government to see whether IRB approval is needed prior to pretest.

Lessons Learned

• Pretesting is the key to understanding how a priority audience will react and respond to SBCC messages and materials.
• Pretesting can save money, time and energy overall as the resultant material will be most suitable for the priority audience and will not run the risk of being inappropriate, misunderstood or rejected.

Glossary & Concepts

• Concept testing seeks feedback about general ideas, concepts and creative concepts; typically done before materials are developed.
• Field testing allows practitioners to observe whether the SBCC materials are used effectively in their intended settings and contexts, usually through observation and focus group discussions. It determines whether the material meets the intended purpose.
• FOG Readability Test measures the readability of writing by estimating how many years of formal education are needed to understand a text. The fog index is calculated by selecting a passage of text, determining the average sentence length, counting words with three or more syllables, adding the average sentence length and percentage of complex words, and multiplying by “4”.
• Incentives are small gifts of appreciation to participants. They might include cash, a snack (e.g. drink, tea, biscuits), phone credit, transport money or hygiene products (e.g. soap, toothbrush).
• Probing questions are follow-up questions that reiterate a participant’s comment and clarifies the comment while asking for further information. Some probes can be developed in advance.
• SMOG Readability Test is a measure of readability that estimates the years of education needed to understand a piece of writing. It is calculated by reading 30 sentences, counting words with more than 3 syllables, and using a formula.
• Stakeholders can include all those involved in or affected by the health or social issue, including public, private and NGO sector agencies, relevant Government Ministries, service delivery groups, audience members, advertising agencies, media and technical experts.
• Stakeholder reviews are input from technical experts, partners and decision-makers prior to finalizing materials. These reviews do not replace pretesting with the priority audience and can be done before or after pretesting.

Resources and References

Resources

Beyond the Brochure: Alternative Approaches to Effective Health Communication

Making Health Communication Programs Work

Testing SBCC Materials

7 Tips for Conducting Intercept Surveys

Readability Formulas

Conducting A Focus Group

Steps for Conducting Focus Groups or Individual In-depth Interviews

References

• Betrand, Jane T. 1978 Communications Pretesting, University of Chicago, Community and Family Study Center
• Brown, K., Linderger, J. & Bryant, C. (2008). Using pretesting to ensure your messages and materials are on strategy [Electronic version]. Health Promotion Practice, 9 (2), 116-122.
• C-Change (Communication for Change) (2012). C-Bulletins: Developing and Adapting Materials for Audiences with Lower Literacy Skills. Washington, DC: FHI 360/C-Change.
• Escalada, M. (2007). Pretesting and evaluation of communication materials.
• Escalada, Monina M. 1985 Pretesting Radio Dramas on Integrated Pest Management, Manila, Philippines
• Flanagan, Mahler and Cohen. How to Conduct Effective Pretests: Ensuring Meaninful BCC Messages and Materials, AIDSCAP Handbook
• Johns Hopkins Bloomberg School of Public Health. Fundamentals of program evaluation: Course 380.611. Communication pretesting, needs assessment (U.S.).
• Lozare, Benjamin V. et al. 2011. Leadership in Strategic Health Communication: Making a Difference in Infectious Disease and Reproductive Health. Johns Hopkins University Bloomberg School of Public Health, Center for Communication Programs
• Population Services International. Pretesting Toolkit.
• Population Services International. The effects of interview method on self-reported sexual behavior and perceptions of community norms in Botswana.

---

Banner Photo: © 2006 Basil Safi, Courtesy of Photoshare