John Tanza Mabusu Blog Week 5: Public Diplomacy Measurement and Evaluation

Evaluation is an important part of feedback that is needed in every program to ensure its success. Public Diplomacy just like other program at US Missions overseas should be evaluated to measure its effectiveness against the goals set by Public Affairs department. Since Public Affairs cuts cross multi disciplinary boundaries and uses different tools for different situations. There is no one standard procedure that should be can be used across the board to measure the success Public Diplomacy requires evaluation in the following areas.

Impact:
It is difficult to measure the impact of Public Diplomacy program within a short time because changes happens gradually and is affected by several factors. It may be several years before the real impact of the student exchange program becomes known in a particular country. It is also difficult conclusion between a Public Diplomacy program and its desired result. For instance teaching English in the Arab World or learning Arabic that not changes the hostilities towards the United States in the Arab World. Time, external events, and other actors can intervene to complicate the cause-effect equation. Most of the time, Public Diplomats look for quick results and they end up measuring contribution, rather than attribution.

Since Public Diplomacy targets the elite mass, getting them to agree to an interview, fill out a survey, or participate in a focus group is difficult. Getting them to do it over the course of several years or more for the purposes of a longitudinal study is truly problematic hence a lot of attention is needed is assessing the impact of the program because new leadership at the White House comes with it new programs and Policies, as well as new approaches to performance measurement. What constituted a priority for one administration may be discarded in the next. As problems become more globalized, however, joint solutions will be required to resolve them. Measuring success in such an environment will likely demand new evaluative approaches and hence more money to investment in measuring success of the program.

Communication
Media changes today with lightning speed. Social media may pose a particular challenge in this regard. It is important to examine the usefulness of several tools that are used by Public Diplomats. It is not about how many newspapers publish your press release, opinion piece or commentary. But rather it is about whether you are reaching the right audience with your Public Diplomacy messages. Measuring the impact of your media strategy can be also very difficult because Public Diplomats are competing against other loud voices from critics of the United States that are out there with their massages. Constant evaluation of the media strategy is important to measure the impact of messages and their desired outcome on the target audience.

 Programs 

No organization likes to admit that one or more of its programs are ineffective. Prestige, funding, even jobs, are on the line. Fear of such consequences can lead to inaction, avoidance, or even burying the results of evaluation. Benefits of evaluation include funding and opportunities for new programs and justification for more funding. It also reveals best practice and it can serve to boost morale and the performance of both permanent staff and contractors. Evaluation, however, can help identify those audiences, areas, and circumstances in which well-researched, well-funded, and innovative Public Diplomacy programs can reasonably be expected to produce positive results.
For instance the State Department’s Fulbright programs can produce substantive results; it can stake a claim to not only more consistent funding but also a more significant role in foreign policy deliberations- it should be used for the success of soft power. Hence publicizing the results can attract the public interest from both citizens and legislature. The aim is to get the legislature to support it. It also allows policymakers to consider it in their plans because of its impact and track record in effecting change.

Evaluation of Public Diplomacy program planning, design, and implementation
helps practioners to think ahead of time. The end game in Public Diplomacy is finding and reaching the desired audience, regardless of whether one is trying to inform, advocate, listen, or connect. This has become more important as the spread of democracy has given more people a voice in government, as new non-state actors gain influence, and as developments in communications technologies offer increased access to information and more avenues for direct, non-mediated communication.

The goal of evaluation program is to look for scorecard and story board as a method of doing the evaluation because it is easy to control our actions, but not the results of our actions. Long term benefits outweigh the short term benefits because short terms usually end up as quick fixes.

Measuring impact calls for engagement as opposed to broadcasting, shouting louder. The world out there is where you need to engagement, not in the office. You can have a beautiful policy and branding, but if you fail to reach out to the elite mass using bottom up approach, then your Public Diplomacy remain on paper as a policy. Public Diplomats need to understand that different situations will require different approaches. Sometimes the most effective Pubic Diplomacy program will be conducted in the media spotlight. But sometimes Public Diplomacy is more effective when it isn’t carried out in public. What matters in Public Diplomacy is not what you say, but what you do. It should not be for domestic consumption, but rather to engage foreign public.

It is proven that the most powerful voice out there should not be your voice. It should be the voice of someone who understands your program, who benefited from your program and is ready to spread the word out, and hence change takes place as a result of soft power. Joseph Nye defines, soft power as ‘’the ability to getting others to want the outcomes you want’’. It requires understanding how they are hearing your message, and fine-tuning it accordingly’.

Advertisements

Kimberly- Week 6

1) What do you think are the limitations or problems with the turn to new and social media for US public diplomacy? Do these critiques outweigh the advantages?

 

Social Media can be a powerful tool in advancing the goals of US public diplomacy. However, it is important to note that every tool has its limitations. According to the American Security Project (ASP) article by Matthew Wallin the actual and perceived power of social media are not the same. There is still much need for research to find out how much influence social media can actually have on users.

Part of the challenge when working through special media is that users can respond immediately and the comments made by those users are not necessarily linked to how those people will act. Another challenge is the issue of access. Not everyone has easy or regular access to the internet and if public diplomacy relies too heavily on this medium, they risk excluding a large audience.

Not only is the internet inaccessible to many people, but for those who can access it, ensuring that certain content is being read can be another challenge. According to the ASP article, the “half-life” of internet links, meaning the amount of time the link is shared usually while it is still appearing near the top of users newsfeeds, is very short (less than 3 hours). This means that in order to ensure that content is being read, that content must be constantly re-posted and refreshed.

With all the challenges of reaching the intended audience through social media it seems that it is not to be the best choice for public diplomacy’s primary medium however with the proper time and effort it can be made a strong supplemental medium.

Public diplomacy officers should get more training than traditional Foreign Service Officer ( John Tanza Mabusu)

 

Public diplomacy officer requires more training than traditional Foreign Service Officer because public diplomacy is a different assignment that requires different skills America’s foreign policy interest. The ideal Foreign Service training provided by State Department is a foundational basis for any US Mission employees going abroad and it equips US government employees working abroad with the necessary tools for executing their duties. US Mission activities abroad come with a set of challenges that requires a specialized training in program and personnel management, interpersonal communication and reporting skills.
Management skills are a pivotal to Public Diplomacy officer because the job involves handling people with different cultures which are foreign to America’s way of doing business, hence skills for handling people comes handy. Since Public Affairs, is a department in most America’ Missions abroad, then Public Diplomacy Officer requires training in development and implementation of programs, which includes writing reports and drawing budgets among others.
Communicator and Persuader
Public diplomacy officer is the communicator of US foreign policy, society and culture to the outside world. This particular sensitive duty comes with the responsibility on the part of the public diplomacy officer where she/he has to write well because the job requires reporting to local officials at the host country as well as to Washington on the local public opinion. Understanding the local public opinion requires knowledge of the local language. Mastery of the host local language is an added advantage for Public Diplomat because it opens doors in hostile environment such as Iraq and Afghanistan.
Interpersonal communication skills is another important requirement for the success of Public Diplomacy because officers engaged at Public Affairs department at various US Missions abroad are tasked with the responsibilities of persuading and mobilizing local public opinion in the interest of the US foreign policy. Persuading foreigners to buy in to specific cause of action is not an easy task. It entails developing contacts in print and broadcast media, reaching out to University students and faculty members, and other actors of track two diplomacy.

Week 6 Blog- Russian PD in Ukraine (Alexis)

I think the use of social media and public diplomacy is particularly relevant and interesting in the case of Russian propaganda in Ukraine right now. As someone who has been working on the Ukrainian elections since the Euromaidan, I can see first hand that the US government is struggling to combat the Russian PD machine. The Russians have virtually mastered the art of public diplomacy, arousing the emotions and minds of ethnic-Russians, especially in the east of the country where violence continues to worsen. 

Why has the US been so unsuccessful in combating this damaging Russian public diplomacy strategy? For one, as pointed out in this week’s readings, the Department of State and USAID, the two largest American organizations in Ukraine right now, have no clear public diplomacy strategy. Both agencies share an annual Joint Strategic Plan that insufficiently lays out a communications strategy to deal with new age communications such as social media. Russia on the other hand has been able to integrate its social media strategy using VKontakte (Russia’s version of Facebook) with its dominant control over Russian-language television in the region to flood the country with pro-Russian ideas in an attempt to discredit the new Ukrainian government.  I believe this crisis in Ukraine will definitely serve as a lesson learned to USG and perhaps encourage key decision makers in State and Congress to take the role of public diplomacy seriously once again and revamp our efforts like in the Cold War. 

Just thought it was interesting how applicable this was to our discussions this week. 

Emily Kaiser – Week 6 Blog

Q: What do you think are the limitations or problems with the turn to new and social media for US public diplomacy? Do these critiques outweigh the advantages?

A: It seems to me social media, in the realm of Public Diplomacy, is a double edged sword. It’s a challenge to take a thoughtful and specific approach to managing social media in the context of engaging and influencing foreign publics; I think the strategy is something that is yet to be determined. Social media is relatively young, and while it has certainly generated a “shift in geopolitical power from hierarchies, to citizens and networks of citizens.” as Wallin cited Alec Ross, I think as with much technology there is still a lot to unfold, strategy to be determined, and long-term results realized.

Major limitations are found in this week’s readings, but some present benefits with merit that are worth exploring and debating. Social media is interactive, favors personal relationships, is cheap but laborious, is a behemoth in technologically advanced countries while mostly irrelevant in countries with limited internet access. It can seem to be a jumble of contradictions.

I don’t believe that it can singularly take on the role of tried and true Public Diplomacy as it’s not universal.Consider cultural diplomacy, the idea is that it is a means for bridging gaps and using the human experience as a vehicle for relationship building and experience exchanging. While social media is a means of creating a conversation which is the foundation of relationships, the same type of emotional experience cannot be achieved through social media since it is not a universal experience or tool. Similarly, cultural diplomacy is not a means that is traditionally used to discuss politically-charged topics—rather like, social media, it favors relationship building over persuasion. Social Media can build trust and credibility, but the slow-moving process of creation, clearance, and translation in addition to maintaining a constant flow of information is difficult to overcome. And when overcome, social media posts are not guaranteed to be shown to every single Facebook or Twitter follower—who may or may not be part of your target audience. It also does not always guarantee that the primary goal of PD—action—is met, and if it does happen to influence action, it is even harder to track and link back to social media. Therefore, exposure and action are possible, but not sure-fire nor easily trackable. Without clear strategy (including crisis management) and vigilant creation, monitoring, and response the social media platform can devolve into an unsavory, unproductive, and harmful arena for internet trolls.

Some particular concerns include the accidental sharing of very confidential or crucial security information. A few months ago the White House accidentally blew the cover of a CIA Station Chief and the power of social media compounded this problem. Similar accidental leaks are capable of happening via embassy social media accounts which endangers human lives and sheds a poor light on America.

Additionally, access to information is a hurdle in countries that censor the internet. Public Diplomacy goals could be tweeted all day, but if it links to a site on the Russian Internet Blacklist, then that information and subsequent intended action are invalid. Or in terms of crisis or political unrest governments in Egypt, Syria, and China have shut the internet down or greatly reduced its speed and operability. Again, hindering the delivery of the intended message and influence. Another cause for concern, is the Chinese blogger program Gregory mentions. With the internet freedom debate widely controversial, and the blogosphere being targeted by various governments worldwide including China, I hesitate to employ a blogger to potentially place them in harms way.

Ultimately, I think social media is an asset in the toolbox of Public Diplomacy, but careful consideration must be given to strategy. While it should not be used singularly I do think instances like the Virtual Embassies Gregory mentions are attempts at engaging and building a relationship with populations to which we have limited access. Just as multiple voices, perspectives, and outlets contribute to a thriving society I think multiple tools and approaches contribute to a thriving Public Diplomacy strategy.

Erica – Blog Week 5

Do you think there are consequences to how the call for more measurement impacts the practice of PD?

There are absolutely consequences of impacting the practice of PD through emphasizing the increase of measurement. It is important first to clarify the nature of measurement and evaluation (M&E). Evaluation assumes a measurable link between action and objective; cause and effect. But when dealing with PD, we see that programs/projects/initiatives are not easily measured through scientific, tangible ways of thinking. If only those channels of PD that are measurable are praised /funded, this severely limits the kinds of PD that will be implemented. PD offices already have the tendency to quickly remove or not communicate about the programs that did not ‘work’ or at least bore no tangible outcomes, so increasing the stakes surrounding M&E (outputs) would only increase this trend. In a world where sources of PD continue to proliferate through a larger and larger variety of stakeholders, to limit rather than adapt to these increased channels, is a detriment to PD efforts.

Additionally, we know that those that carry out PD on a daily basis are trained to be generalists. PD officers/diplomats are not (and arguably, should not be) experts in M&E. While the call for increased measurement tools has led the government to strengthen their capacity of staff to include M&E experts, we know that doing so will likely take even more time, more resources and more cost. While it is important to measure impact, to do so in the limited ways we have so far with the only aspect changing being the amount of staff that are experts in M&E, does not seem to fit strategy nor the nature of aspects of PD like social media (which moves very quickly). This increased capacity might also severely undermine the PDs leadership and decision-making.

Speaking of increased capacity, as PD becomes more complex in its network of stakeholders, an increase in M&E may complicate the standards of evaluation. This is what Professor Hayden spoke of when saying there could be ‘too many cooks in the kitchen’ in the Week 5 lecture. With more people involved in M&E, where should the standards and methodology be coming from and how could we create an effective way to make sure each stakeholder follows these tools of assessment?

Lastly when it comes to potential negatives impacts, we must consider how reliable M&E is to the overall goals of PD. Many PD program/projects/initiatives cannot be analyzed in the short term. Some require historical, long-standing M&E to demonstrate change. We also have the issue of M&E of projects geared towards measuring ‘negatives’ such as the Center for Counterrorism’s initiatives to counter extremists’ narratives or Stop Fakes initiatives to counter false information about events in Ukraine. How can we get reliable information on these programs initiatives through traditional M&E?

There are positive consequences of course, as good evaluation can reveal best practices as well as determine better allocation of resources and provide an alternative to the use of hard power as Professor Hayden discusses. If PD ‘plays the game’ of M&E, PD could also potentially increase attention and influence policies, other areas of the government and society.

Also should increased M&E mean it can adapt to the changing landscape of PD, it can be used to analyze information not previously able to be analyzed with a smaller capability of resources. Information found in social media through evaluation such as text analysis for example may be one area where increased M&E would be very valuable. Social media allows us to know more about engagement. Using new measurement tools– such as software similar to what private sector businesses use – could greatly assist in the effectiveness of PD. Broadcast Board of Governors has started using these types of metrics.

Just to add as a side note – it is quite amazing how much information about your audience you can learn through social media. The Yepsen Study was a great example of this use new metrics to determine the nature of the different social media networks, how audience become part of the ‘story’, the key players (identifying people) and figuring out best strategy for proliferating messages. In my work recently we have started thinking critically about how to reach and influence our audience through social media. A colleague of mine shared an excellent video that taught us more about our audience (adolescence around the world) called Generation Like which was featured by Frontline on PBS. I would encourage you to watch it if you have the chance!

http://www.pbs.org/wgbh/pages/frontline/generation-like/

Alexis-Week 4

How much freedom should public diplomacy officers have to speak their minds? Why? 

Public diplomacy officers should have as much freedom to speak their minds to the extent that their opinions do not interfere with the objective of the PD program. In other words, as representatives of an entire government, a public diplomacy officer should be allowed to speak their minds to the extent that the balance between their home-country’s national interests and their host countries public opinion is not altered to favor one side or another. Chapter 3 of this week’s readings on the public affairs officer’s duties states that the task of a PAO is “to take into account the U.S. government’s global and regional priorities, as well as its bi-national policies, and seek to match them as much as possible against local concerns. A unilateral monologue that ignores local views would be ineffective, and a focus exclusively on local priorities that ignores American interests would also fail.” So the extent to which a PDO can speak their mind has to be fairly limited and strategic in order for their programs to be effective. 

However this is not to say that public diplomacy officers should necessarily be confined to the strict limitations set by Department of State whose priorities in a particular country may not be as conducive to the methods and priorities of an embassy’s public diplomacy office. The closing of the USIA and the fact that only 8.4% of DOS staff are public diplomacy officials illustrates the rather significant difference between DOS’s tendency toward traditional diplomatic measures and public diplomacy. This difference gives reason to give public diplomacy officers more leeway with speaking their minds because the nature of their job includes a thorough understanding of the host-country culture, economy, ideals, and politics as well as US national interests there whereas traditional FSO’s with the Department of State have more objective, scripted duties in their more traditional version of diplomacy.