Developing and Pretesting Concepts, Messages, and Materials

Transcription

1 Developing and Pretesting Concepts, Messages, and Materials Ms. Williams: Today we are going to talk about step two, which is developing and pretesting concepts, messages, and materials. We are also going to talk about implementing and managing health communication programs. So, you will have step one and step two under your belt, and then you can work on your plan. We will use the communication plan template that comes out of the pink book to do that. But, first, before we get into that, we wanted to come back to what we talked about just briefly yesterday towards the end of yesterday. That was, once you develop this profile and you have your indicators in place, and you kind of have a sense of probably where your population is, how do you make decisions about what approach you are going to take? The example we showed yesterday was that you have southern small state, central small Page 1 of 33

2 state, and northern small state. They all have sort of a different demographic profile. How are you going to make a decision about we are going to use health communication approaches in this community and we are going to use policy approaches in this community, and we are going to do something else in that community? It is a process of looking at a number of different factors. So, you might say southern small state has high unemployment, low income, high uninsured, a few barriers, a few providers. There are a lot of barriers to people making behavior change. If you were doing a social marketing campaign and there is sort of this idea that you go for the low-hanging fruit, southern small state is not low-hanging fruit. However, central small state or northern small state might be considered more low-hanging fruit. The challenge for us who work in the government and use public money is that is not always how we are expected to spend the tax Page 2 of 33

3 dollars, right, on people who are ready to change? We are expected to work on the tough problems. That is why they have us. Maybe some of our partners can work on some of the more low-hanging fruit. What do you do in these cases of the barriers are so high and there s multiple problems going on? How do you make a decision about the appropriate approaches? One place that you might start is back to your literature review and your environmental scan. What is that telling you about different approaches you might take in communities that face similar challenges? There are a number of organizations out there that have put out best practices and guidelines. One place you can look is the Community Guide to Preventive Services has a set of recommendations around health communication and marketing. You can look at what they have recommended. Robert Wood Johnson has put out some work on policy. I think Kellogg has some Page 3 of 33

4 information about strategies. The Public Health Institute has strategies. You can look at Healthy People 2012 now. What are we on? Whatever the next one, 2020, you can look at Healthy People, those recommendations, and see what they are suggesting in terms of how you approach some of the issues that you are looking at. The Community Guide also has recommendations for diabetes, heart disease, physical activity. You can look and see across what all of these sort of people who have been looking at the literature are saying are what may be best practices right now. You can also work with your stakeholders. If you say this is the picture of where things are, these are the different strategies we might take. We might take a policy approach. We might take a worksite approach. We might take a communication approach. Discuss amongst your stakeholders what seems like it is going to be something that is going to be meaningful Page 4 of 33

5 and impactful. Frankly, your own priorities within your organization are going to dictate a lot of what you do. You might have categorical funding that says you must do health communication, and you are looking at southern small state and saying, Well, what can we tell them when they can t see a doctor or they don t have access to healthy food or access to resources, but we have to do communication? You might look at, then, what are your options, given the conditions. You might consider can you apply communication in another way. Instead of reaching the people who don t have access to resources, can you use your communication funding to reach decisionmakers, to reach communities that might influence access, to address some of the policy and environmental barriers? Can you use your communication funding to do some community collaboration or some kind of community mobilization kind of work? There are a lot of Page 5 of 33

6 different approaches that you could take, even in these seemingly difficult areas for doing communication. But it is multiple approaches to making a decision about, given all of these conditions, what would be our best approach to take? You can always, also, ask us at NDEP. I mean, you can run by, This is what we are thinking. These are the strategies that we are thinking. We are happy to provide some technical assistance and some advice in terms of how you might approach any given situation. Mr. Petty: Another thought that I had is, looking at these three different profiles, you are going to have three very different types of campaigns that you would want to roll out in each area. So, if I am looking at southern small state, you are probably looking at more environmental things, dealing with access, maybe building parks and walking trails so that people have the opportunity to do more physical activity, maybe working a little more with Page 6 of 33

7 trying to get different policies put in place that allow for some of these things to improve. It looks like they have got some of the highest smoking rates, but their obesity isn t too bad. Maybe there is more of an emphasis on tobacco in that area. Whereas, if you are looking over at the northern small state, that is probably more where you are trying to get down to individuals and help work on behavior change for those that aren t quite there. I mean, just looking at the different characteristics says to me that you would have very different types of approaches in each different area that you would just want to think through. Ms. Williams: Another thing you might find is maybe central small state has tons of campaigns already going on. They are sort of in the middle, and they are sort of what people like that do health communication or do interventions. There may be lots of other programs already happening. Maybe the place Page 7 of 33

8 for you, then, is southern small state because there is less happening. So, there are more opportunities to get things done. There are a lot of environmental factors, evidence factors. There is a lot that goes into that decisionmaking process of deciding we are going to take these approaches in this area with these people. It is a process that you have to go through, and you bring a lot of different factors to bear in that decisionmaking. Let s talk about step two, which is developing and pretesting concepts, messages, and materials. I was at a presentation once, and this young woman was developing a breast cancer awareness campaign for African-American women. She had done all the things you are supposed to do in step one. She had done a series of focus groups with African-American women. She asked about what their preferred communication channels were. One of the channels they mentioned was the links. She thought that was kind of weird, Page 8 of 33

9 links being another word for golf course. But she had developed this whole golf campaign where they were going to ride around in golf carts. This is a true story. They were going to ride around in golf carts and distribute breast cancer information awareness. She hadn t launched the campaign yet, but she was in the process of putting this all together. I was like, I m sorry, but are you sure they meant a golf course? Because you know there is an organization of African-American women called The Links? And she just kind of looked at me. She was like, Oh, no, I didn t know that. She missed that, too, which is pretesting your messages, materials, and concepts. Once she had put together her golf, her links, and her focus groups, and once she had decided on the concept of a golf course intervention, she should have pretested that. She could have just run it by three African-American women, and they would have said, Well, they probably didn t Page 9 of 33

10 mean golf course. They probably meant The Links organization. If she had done the pretesting, she would have saved herself a lot of work. Especially when you are not as familiar with the community or you are an outsider to a community, it is important to do pretesting and to keep in mind that focus groups, they are good for a certain amount of information, but you have to check what you hear in a focus group. Sometimes the facilitator will latch onto a novel idea. That doesn t mean that your audience is going to like that. But if there is something that you heard that you didn t hear anywhere else, and you thought it was a good idea and it rises to the top for some reason, you have to go back and check that. When I was doing a secondary analysis of the focus group data for the Surgeon General s Report on Physical Activity, they had made a recommendation that one of the partners we should work with to disseminate the Surgeon General s report for African-Americans was the Page 10 of 33

11 Nation of Islam. That was in the report from the focus group, the contractor. I went back and reviewed the tapes. It was one person out of they did probably nine focus groups with African-Americans one person in one focus group and nobody else in the room. There was no non-verbal agreement with that one person. But it was something that the facilitator had not heard before, and so she put it in as a novel idea. Can you imagine CDC or the Surgeon General partnering with the Nation of Islam because they heard one thing from one person in a focus group that no one else -- I mean, there was not even people nodding their heads, Oh, yes. There was none of that. You have to check what you hear from your audience. That is why pretesting is so important. Pretesting allows you to assess the cultural appropriateness of what you are developing. It helps you assess whether people will pay attention to your message or your Page 11 of 33

12 strategy, whether they understand it. You can assess how motivating your message is or your concept, whether people can remember, whether people are remembering accurately. You put out a message and people interpreted it in an entirely different way. You need to know that before you move forward with your campaign. Is it meaningful? It can also help you identify controversial areas. In one case, there was a tobacco campaign that had done these focus groups with teenagers that were smokers. One of the stories that was told was that, before you start smoking when you are 14, you think that being addicted to cigarettes is like really wanting a piece of chocolate. Once you become addicted to cigarettes, you realize that it is more like you are drowning and you need a next breath of air. That is what it feels like to be addicted. And so, they came up with this whole campaign about drowning, right? Smoking was Page 12 of 33

13 like drowning. When they launched the campaign, they got blowback from the people who had children who drowned, right? They had to pull the campaign. They didn t know that that was going to happen. That wasn t their intent. They didn t mean to upset people. But you need to check. Especially when you do something that is potentially controversial, that is potentially edgy, that is potentially out there, you just need to run it by some people to make sure you are going to be okay, and be prepared if there is going to be any kind of blowback or outcry over your message. Pretesting allows you to assess the strengths and weaknesses before the campaign launches. Now keep in mind that there are formal ways of doing pretesting that are expensive and complex. You can also just run things by people. You can do a check with a few people that represent your target audience, that represent people in your organization. Page 13 of 33

14 You can do it with your stakeholders. You can go to opinion leaders, gatekeepers, and run your message by them. So, there are also less-formal ways to do pretesting. It is not objective. It is not predictive or statistically-precise. It is not a substitute for a sound model or theory. You still have to have your messages based on something. They still need to come out of some kind of framework. Pretesting is not going to make up for poor planning or flawed execution and implementation. You still have to do all of the steps, even if you pretest. If you go to the pink book, what it is going to explain to you about pretesting is the steps that you take are you review the existing material. So, you go back to your literature review and review other campaigns. Or you go back to your environmental scan. You review what has already been done. You look at national organizations. You look at what Page 14 of 33

15 messages are already out there. You develop and test your message concepts, which are ways of presenting information. Are we going to take a funny approach or a serious approach, a scary approach? Are we going to use this kind of person or that kind of person? Are we going to use photos or illustrations or comics? There are all different kinds of concepts that come out, and you test these concepts to find out which resonate with your audience. You decide what materials to adopt or tailor. You develop those messages and materials, and then you pretest them. It is a complex process. It takes people who are really skilled at developing concepts to do this kind of work. It is oftentimes people that we don t have. A lot of times, if you don t have a contractor that can do this work for you, it can be very difficult to do it on your own because it does take a certain amount of skill to develop the concepts and then test the Page 15 of 33

16 concepts in a way that is going to give you meaningful information. So, that is kind of a red flag. It can be an expensive and complex process. However, you can work with materials that are already out there where some of this work has already been done, and it will save you some steps. You can work with NDEP. NDEP has already developed and tested the message concepts around the diabetes prevention and control materials that we put out. So, review NDEP materials. Decide what materials to adopt and tailor. Based on what you have learned about your audience and the approaches you are going to take, and the strategy you want to take, you can modify and adapt those messages and materials. Then, you want to pretest. Pretest the adaptations you made to make sure they are still on point, to make sure people still are getting the message. It saves you the step of doing that concept testing, which can be very complicated Page 16 of 33

17 and expensive. So, leverage the materials that are already out there. Start with NDEP and go on from there. Okay? You review our existing materials. You can search by risk status, ethnicity, language, what group you are trying to reach. You can also call us or send us an because we do that. People are thinking, We are trying to do this and we were thinking these materials you have. We will say, Well, these materials, this much material, or that material might work better for what you are trying to accomplish. There is a lot of stuff on the website, and sometimes it can be difficult to navigate. We are working on that, but we are happy to help you work through what is there, to identify what might work. What you want to look for is, are the messages relevant? We learned all of this in our formative research. We think this is the message that is going to resonate. This is what Page 17 of 33

18 we are going to try to accomplish in our community. Which of these materials? Are the messages relevant in these materials to what you are trying to accomplish? Some are prevention messages. Some are control messages. If you are trying to focus on control, you may not want to use the prevention materials. You may want to focus on the control material. Okay? So, make sure the messages are relevant. Are the materials appropriate, the format, the style? Are there cultural considerations? The language, the literacy level, you want to take a look at that. Those are things that can be adapted, too. Just because you see something like our Paso a Paso, which is step by step, or you might see one of our American Indian materials, it doesn t mean it can t be adapted for your population if the message seems to be on target. Are the materials likely to meet your communication objectives? Whittle down. Ask Page 18 of 33

19 yourselves these questions. Ask your stakeholders these questions, and whittle down your choices to the ones that are most likely to be yes. Decide which materials to adopt. You may want to test existing materials. You may get three or four NDEP resources and you may say, I m not sure which of these is going to be the best fit. So, you might want to run that by your target audience just to be sure. What do you think about this? Oh, this is fine, but it s wordy. Or This is fine, but I don t want to read it on paper. I want to read it on my phone, or something like that. So, you might at this stage want to run something by your target audience. For program activities, you might want to test the concept. If you are deciding to do like Road to Health, you might want to run that by. How will we do Road to Health in this community? What is going to be the best way to make this happen? Or power to prevent? Or, if we are Page 19 of 33

20 going to do some kind of provider outreach with one of our provider tools, how is that going to work? You might want to test, actually, that concept for making that activity work. Does that make sense? Not testing necessarily the material itself, but how you are thinking you are going to implement or disseminate the activity. Okay? Then, you can decide if any tailoring is needed or if you are going to need to add anything to make this really work. It doesn t necessarily have to be a whole big thing, right? You don t necessarily have to convene another round of 18 focus groups to do this. It can be a matter of sitting down with your stakeholders and going through the material, sitting down with members of your target audience, running it by people who are knowledgeable or who have done this work before, and working through this. So, it can be done very effectively. It can be done without using a lot of resources. Page 20 of 33

21 It can be done in a very timely fashion. Step three will be modify or adapt messages and materials. The first place to start is the principles established in the NDEP Guidelines for Partner Collaborations. That is on our website. There is a link to it from the phconnect site. You can also find it on our website. But, basically, the materials are in the public domain. There are no copyright restrictions on them. You can brand them with your logo or your partner logos. You can add your contact information. You can tweak the examples that are used. You can change the pictures. I think in Utah they were using Managing Diabetes: It Is Not Easy, But It Is Worth It. They used local people because they were trying to tell local stories about diabetes. They just took the pictures we put on there off and put the pictures of their local people on. You can alter the format. We have a Page 21 of 33

22 brochure. You convert that to HTML for your website. Or we have a PSA canned. You can have someone else read the script. That s fine. You can adjust for programs the delivery and the timeline. We have the Power to Prevent curriculum, which I think is 12 weeks. You can make that shorter; you can make that longer. You could add Road to Health to that. You can do Road to Health in one sitting. You can do Road to Health in three sittings. You can do Road to Health in six sittings. If you have a question, ask. Okay? You can complement and supplement existing materials. You might want to tailor for cultural or local relevance. We have, say, the Fat and Activity Counter. You might have some restaurant or something in your community that everybody goes to, and you want to add that menu or you want to provide that menu as well, those nutrition facts for those folks. You can provide that as a supplement to our existing materials. Page 22 of 33

23 We can work with you on literacy if literacy level of the materials is an issue. We can work with you on language if the language of the materials is an issue. Things that should not be modified: the health topic. If we have something that is prevention, it is prevention. Those are scientifically-approved. We go through a process of clearance, clearing our messages, that people look at and say, okay, it is okay for you to say this. So, if it is a prevention message, those messages have been crafted for prevention. If it is a management message, those messages have been tailored for management. You have to be very careful about switching them, about saying, Okay, well, we are going to use these prevention tools in our self-management class. You have to be careful about that because there are differences between the messages for the two audiences. Ask us if you have a question or a concern. Page 23 of 33

24 You can t really make substantial alterations to the text that alter the meaning. Like I said, that has been scientifically-approved. You shouldn t be making substantial changes that really alter the meaning of what is being said. Really, attaching and inserting new text is not allowed, although there is an exception for adding culturally-relevant examples, for the most part. And you can t imply that NDEP endorses commercial products. There is language in the Partner Guidelines about, if you are using a commercial, you have Walgreens or Pfizer or something like that who is local you are adding, what the language is that needs to go on there to make it clear that NDEP doesn t endorse that particular company. If you adapt the material to the extent that the NDEP logo has to be removed, generally, our test is, if you have made changes to the extent that we would have to get it cleared Page 24 of 33

25 again. If you run it by us and we say, We would have to run this through clearance, then you would have to take the logo off. It doesn t mean you can t do it and you can t use it. It just means that the logo has to come off because it is not what has been cleared by CDC or NIH, and you put this message on it. This is also in the Partner Guidelines. I think Utah also changed some of the text on some of the posters to be preventing and managing diabetes. They added prevention and management messages together. So, they took the NDEP logo off because it was sort of pulling messages from different places and putting them into one place. You can always check with us, always ask. Let s do an example. This campaign s messages are Make a plan and manage your ABCs. Make an appointment to visit your healthcare provider and keep it. And make a referral to the small state quit line. So, they are using our Managing Diabetes Page 25 of 33

26 posters. They added the state s website, and they used a local couple. Can they keep the NDEP logo? How many people say yes? How many people say no? They can keep the logo in that case. So, it is okay to add your contact information, add your local agency information. You can switch out the pictures and still keep the logo. Okay? What about they pick this heart disease flyer. It was developed for American Indians. They changed the photographs and converted it to HTML. Can they keep the logo? Yes. Good. They took Four Steps for Life, and they reprinted it with information about smoking cessation in the back. Can they keep the NDEP logo? No. For one thing, it is not four steps. No. (Laughter.) Participant: Yes, and they changed it to five steps. Ms. Williams: And they changed it to five steps. Page 26 of 33

27 You can t attach new information that hasn t been cleared. You could add the smoking cessation information as a supplement, something that is offered. They get the Four Steps brochure and they get another thing about smoking cessation. That s fine. But you can t physically attach new content to one of our materials without us getting clearance for it. Or you could go ahead and do it and just take our logo off. You could change it to five steps, add the smoking, take the NDEP logo off, add that it was adapted on the back cover, and go on. Okay? And then, this is our poster about working with your pharmacist and your optometrist and your dentist and your podiatrist. They added the phone number for the quit line to this poster. Can they keep the NDEP logo? Yes. Yes, you can because all you did was add local contact information, and that is fine. Does that make sense? At the end of the day, just call us, send Page 27 of 33

28 us an . Tell us what you are thinking. We are happy to let you know. Step four is pretesting the message and materials. This happens after you have substantially adapted a material. We took this, we changed the format, we tweaked the intervention, we did whatever, and it is different from what it originally was. You want to pretest that. Or you have come up with sort of how we think this campaign is going to run or this intervention is going to run, or whatever the strategy is. You might want to do a small pilot, just to make sure that it is working the way that you intended, that those messages are resonating the way that you intended. Again, it doesn t have to be a whole big thing. It is just running it by people who know to make sure that you are still on target, you are still on a message, your messages are making sense. It helps you to avoid missteps in Page 28 of 33

29 execution. So, you are not driving around on a golf course looking for Black women and not finding them, and wondering why the people in your focus group lied to you. It helps you confirm that messages and materials are consistent with the audience research and the testing. It also helps you gain buy-in. So, we have all these people we want to carry our message for us. They need to have some kind of input into the development of those messages. This pretesting gives you a chance for them to say, Hey, this looks really good. We can t wait to start using these materials. Or It would be good if you would tweak this a little bit for us. Or This is on point. Or This could change. Pretesting is a way to gain buy-in from your stakeholders. Again, it is always helpful to run your messages, your materials, by someone who knows. Mr. Petty: Just another quick example on Page 29 of 33

30 this. I am not sure exactly where this came from, but I was at a social marketing conference a year or two ago and heard about an example of some folks that got some money to do an HIV campaign. They decided that, through their environmental scan, they really wanted to target prostitutes, street workers. They appropriately discovered that the government should not be the right message deliverers for this program because it is an illegal activity. But, nonetheless, they recognized that it was a problem. And so, very creatively, they went out and got several of them in a room. Basically, they created this program for themselves. Talk about getting buy-in and avoiding all these potential problems, because they recognized we know nothing about their life and what they do. But they really went out and got the people who are in that situation to craft messages to help them make more healthy life choices and had a lot of really good success with it. Page 30 of 33

31 It is kind of out of the box, but a great example of really pulling in the right people at the right time to make sure you don t make some of these mistakes. Ms. McDivitt: And that is recommended, also, for working with kids and youth, is getting them involved in the development. Mr. Petty: Early, yes. Ms. Williams: When you are looking at developing and pretesting messages and materials, this step is key to ensuring the effectiveness of your program materials and activities. Sometimes you go through all your formative research and you come up with a great concept, and you just want to get started. You just want to run out and implement it. You think you are okay because you did a pretty good job with your formative research. But don t shortcut step two. Even if it is just a matter of running your ideas by some people, it is worth the time and effort to make sure that you avoid some missteps in the Page 31 of 33

32 execution of your campaign or your activities. NDEP materials can serve as the basis for most campaigns. We are willing to work with you to make those work. You can look at other materials, and we are willing to work with you on that as well. Test the materials and activities for relevance to your intended audience. Test them to make sure they are going to be okay in your agency, that your stakeholders are okay with them. If you adapt materials significantly or you adapt an approach significantly, pretest or pilot before launching the program, just to make sure that everything is working the way you intended, and not just the messages, but your 800 number, your website, whatever systems you put in place to make this work. Make sure those things work. I worked on a campaign that crashed NCI s Call Center. That is not something you want to do. That didn t make them happy. But it was Page 32 of 33

33 because we didn t have all the right pieces in place when we launched our campaign. We had no idea how many people were going to call. We had no idea that the Call Center was going to be understaffed the day that we launched our campaign because half the people were in training. And so, we crashed the Call Center. Running through your plan, not just your message, but how you are going to execute your plan, is important, too. It is not helpful to make people in your agency upset with you about crashing the Call Center. So, this is a really important step. Page 33 of 33