Home
Videos
Photos
Welcome
About
Legal
Search
Archive

Share This Post

Recent Posts

Spirit of Chicago Evokes American Experience in Brussels

How Can the U.S. Effectively Support Democratic Efforts in the World Today?

U.S. Embassy Kabul Salutes Friendship, Solidarity

We the People…

History Resonates at July Fourth Celebration in Vietnam

View all Feeds

More information on Social Networks

Subscribe to Dipnote by Email

Dipnote for the BlackBerry

Dipnote for the iPhone

Follow Dipnote on Twitter

Dipnote on YouTube

Dipnote on Facebook

AFSA Survey Falls Short
Posted by Sean McCormack on Jan 10, 2008 - 09:35 AM

Screenshot of AFSA survey results and Washington Post story published on Jan. 8, 2007. [State Dept.]

Sean McCormack is the Spokesman for the U.S. Department of State

Tom Shannon and Richard Boucher both posted on stories yesterday referencing a "survey" conducted by the American Foreign Service Association about issues related to the Foreign Service. If you take a look at my briefing Tuesday, you'll see what I thought about the survey, so my post is not about re-hashing those thoughts, but I wanted to fill you in on how Richard, Tom, and others (perhaps) came to post entries on DipNote.

In the Department morning staff meeting Tuesday, I talked about a story appearing in the Washington Post that morning about AFSA's "survey", and I asked for budget information concerning congressional outlays for the Department during the previous three years as a way to talk about what Secretary Rice has done on the Hill on behalf of the Department (and the Foreign Service) during her tenure. My bringing up the story prompted comment from a number of senior staff around the table about how the impression left by the story couldn't have been further from the reality of what they had witnessed of the Secretary's efforts fighting for the building. Many volunteered that they wanted to speak to journalists writing stories on the topic. I saw this willingness to engage the public on this topic as an opening to encourage my colleagues at the Assistant Secretary level and above to make posts to DipNote. The content and actual writing of the posts would be up to them. I thought this brief explanation was worth a short post, as it has not been the norm that we have had multiple posts on the same topic from senior management folks at the Department.

One additional note about the "survey". In talking to one media organization, I found that AFSA denied the journalist's request for a copy of the questionnaire, which the media organization's polling experts said was critical in determining whether to ascribe any weight to the results. The experts also noted, as I did, that the lack of random sampling technique raised serious questions about the value of the "survey" results in drawing conclusions about attitudes in the Foreign Service. I read one news report in which AFSA officials said the "survey" aimed to get a general sense of members' views. Seems questions about methodology undercut any utility of this "survey" in getting even a general sense of members' views, and the refusal to release the actual questions asked raises even more issues leading to the core question of what one might usefully conclude from the AFSA survey.

Commenting is not available in this weblog entry.

Follow Entry's Comments Via RSS

Do you want to know when a comment is added to this entry? Stay up-to-date:
Comments

David writes:

@ Joseph in Virginia -- While your simplistic approach to just subtract the number that is dissatisfied from 100% to come up with the number of satisfied people is compelling, it is wrong. There is no way to derive a number of any significance from the percentage of people that did not do the survey; how many would vote one way versus the other (i.e. approval/disapproval of Secretary Rice).

Posted on Mon Jan 21, 2008

Peter writes:

@ Sean McCormack -- Let's set aside the question regarding the validity of the survey and deal with the essence of this matter which you touched upon in your posting: "What has Secretary Rice done on the Hill on behalf of the Department (and Foreign Service) during her tenure."

By any measure, the Secretary has failed to achieve satisfactory results. The budget adopted this fiscal year is yet another strike at the heart of our Service. In comparison to Secretary Powell or to the Department of Defense during the last three years, we have been stuck in neutral or going backwards from a resource perspective. The results speak for themselves.

Perhaps the Secretary is fighting the good fight, but whatever it is she is doing on the Hill, it is not working. You may know different, but we who serve on the frontlines of the Foreign Services live with the realities. Cable after cable informs us of budget cuts, elimination of programs and benefits, removal of positions from FS Bid because of lack of personnel, and decreases in post allowances to fund the Iraq effort. These are not indicators of success.

Collin Powell's Number One Leadership Rule was "Being responsible sometimes means pissing people off." Well Sean, perhaps it's time Secretary Rice took a lesson from Secretary Powell and started to piss off the Congress and stood up for the best darn Foreign Service in the world.

Posted on Sat Jan 19, 2008

Department Spokesman Sean McCormack writes:

@ John in Washington, DC -- I appreciate your and AFSA's readiness to engage the public in this forum. The issues at hand are important to the future of the State Department, as well as the Foreign Service, the public discourse here is enhanced by your willingness to share AFSA's response.

Posted on Thu Jan 17, 2008

Joseph in Virginia writes:

The "survey" respondents only constituted under 40% percent of the foreign service as a whole. Any administration statistician will tell you that that must mean the other 60-something percent probably thinks the exact opposite. So keep up the good work! And don't forget to continue putting the word "survey" in quotes -- I think you missed one in your last entry.

Posted on Wed Jan 16, 2008

Michael in North Carolina writes:

Anyone interested in this string really needs to see AFSA's response (sent to its listserve, but presumably available to anyone interested at http://www.afsa.org. The survey more than met the requirements for a legitimate survey, and in making it public AFSA has done no more than what organizations representing military officers and enlisted men have done. Instead of carping about it, senior Department management ought to take it seriously and redouble efforts to improve the staffing levels and issues facing the Service.

(from a retired career officer who served as ambassador)

Posted on Tue Jan 15, 2008

J in U.S. writes:

As others have noted, the simple fact that senior officials within the Dept more or less dismiss the opinions of over 1/3 of their employees speaks volumes to the issue. I must have missed that management skill in business school. You want people to fall in line? Unquestioningly volunteer for service in Iraq? Accept the pay cuts involved in serving abroad? We signed up to do this and we're all (most of us, at least) more than happy to salute leaders that we respect. In return, though, show some respect for our opinions and act on them as appropriate. Debating the scientific validity of the survey itself simply shows the all too frequent State "leadership" attitude of "suck it up quietly" as opposed to inspiring us.

Posted on Tue Jan 15, 2008

John in Washington, DC writes:

The American Foreign Service Association has issued a statement responding to the criticisms of our employee survey. That statement was e-mailed to the nearly 10,000 people on our AFSANet list serve this afternoon. We will post it on our website at http://www.afsa.org (http://www.afsa.org/011508critics.cfm) as soon as our IT folks can manage it.

Regards, AFSA President

---
Department Spokesman Sean McCormack: writes:

@ John in Washington, DC -- I appreciate your and AFSA's readiness to engage the public in this forum. The issues at hand are important to the future of the State Department, as well as the Foreign Service, the public discourse here is enhanced by your willingness to share AFSA's response.

Posted on Tue Jan 15, 2008

Katherine in New York writes:

This is not a clinical drug trial--this is a simple satisfaction survey. I don't think that I've ever heard of a satisfaction survey that adhered to scientific method. What government agency has the financial resources and time to conduct a true scientific study of something as subjective as employee happiness? The AFSA survey is what it is, so instead of completely discounting its findings why don't we use it as the ice breaker for addressing concerns that it was intended to be?

I'm very disappointed by the official response to try and rob this survey of all meaning. Sorry to haul out an overused saying, but if you're not part of the solution, you're part of the problem.

Posted on Tue Jan 15, 2008

Robert in Virginia writes:

Folks, I can't address the substance of this dispute, but I would like you to know that putting the word survey in quotation marks does not strike me as very convincing. Instead, it strikes me as reflecting elitist arrogance and is inappropriate and potentially ineffective, especially when writing about an organization which represents employees.

I am prepared to believe that there has been way too much whining and second guessing on the part of Foreign Service worker bees, but it may well be that the public will draw the necessary conclusions from AFSA's own words. The Department does not serve it own interests by acting in an officious and condescending manner.

Posted on Tue Jan 15, 2008

Department Spokesman Sean McCormack writes:

N.S. wrote that our reaction to the survey had something to do with how Secretary Rice came out in the "survey". I admit to frustration arising from a story line that foreign service officers don't believe Secretary Rice is working hard on their behalf, and I'm sure that frustration comes through in my public remarks. Don't read that frustration as turning a blind eye to issues on the minds of my colleagues. I'm sure there are those concerned about issues like "locality pay" and other benefits, as well those who struggle with how to manifest their views on policy issues. No doubt, we have within the foreign service a range of views any given topic, as one might expect in any large organization whether private or public sector.

The problem arises when the general public, important opinion makers, and members of congress are left with impressions about the foreign service that are inaccurate. You have heard similar concerns from several of my colleagues posting to this blog. That is where the "survey" comes in. While some in the comment section say anecdotal evidence supports the idea of disagreement within the foreign service over Iraq policy or frustration over management "not working hard enough" to win resources for the Department, we do not know on the basis of the "survey" to what extent those sentiments are shared by our colleagues. It is irresponsible to speak authoritatively in public about attitudes within the foreign service based on the AFSA survey . And this is where my frustration reaches a boil. I know (after seeing an e-mail from a member of AFSA leadership) that AFSA actively sought to promote high-profile public dissemination of the "survey's" results by leaking it to members of the media before public release of the "survey". I note that in today's Washington Times in talking about the "policy-disagreement factor", AFSA vice president Steve Kashkett is quoted as saying, "Perhaps we set up the question in the wrong way from a statistical point of view, because respondents could check more than one option." I have no problem with an elegant media roll out strategy, but to now claim the equivalent of oops regarding the integrity of the "survey" does nothing to repair the damage done to perceptions of the State Department in wake of news stories actively promoted by AFSA leadership about the "survey's" results. While we have all made our share of mistakes, my advice would be to "look before you leap" next time. Stories like the ones I have seen over the past week based on the results of the "survey" do material damage to efforts to win the resources all clamor for in the halls of the department. And that is too bad, because there are a lot of our colleagues in dangerous and even not-so-dangerous places who could use the additional support.

Posted on Mon Jan 14, 2008

U.S. writes:

I would think that, since many Foreign Service union members filled out the survey at work, that some were reluctant to criticize their leader. I would be. A complete survey would likely be below 14%.

Posted on Fri Jan 11, 2008

N.S. writes:

Interesting how the survey you criticize has been conducted by AFSA for years. Were there any complaints about it before support for the Secretary of State fell to record lows? Somehow I doubt it.

I bet I could criticize the science and methodology behind the "Best Places to Work" survey that the State Department uses on its recruiting website.

You just didn't like validation that Condi Rice is viewed by a record number of employees as a bad and unsupportive Secretary of State. I read that State Department employees overseas now get a 20% cut in salary when they leave DC! A supportive boss wouldn't allow that.

Posted on Fri Jan 11, 2008

Edward in Iraq writes:

While the methodology of the survey is unscientific, it is a useful tool to understand the feelings and perceptions of many of the Foreign Service (FS) "rank-and-file" members. (I would not go so far to say it reflects what a substantial portion of the FS believes - that's a bit too strong.) It begs several questions: If the Secretary is making serious efforts on behalf of the FS, why do so many members of the FS not feel that she is doing so? Why is the perception of those efforts so out of sync with the reality of those efforts? What can be done to communicate to the members of the rank-and-file that management does care about them?

Posted on Fri Jan 11, 2008

J in Iraq writes:

First a comment on methodology. The issue of a "random sample" is a red herring. This survey was announced in October via both Department Notice and ALDAC; so the sample includes everyone with access to one or the other - a group which approximates the universe of the foreign service. (The sample would exclude people in long term training, diplomats in residence, some PRT staff; etc.) This is the same sampling approach that the Department uses in surveys that it conducts.

"Selection bias" in responding on the other hand is a legitimate concern. The survey was sent to almost everyone, but not all of us responded. For a voluntary survey, a response rate of over a third is very impressive, but people choosing to respond may not be representative.

I am confused by the concern for a "copy of the questionnaire." The survey questions are posted by AFSA.

Finally, I am puzzled that the first reaction of senior leadership would be to "engage the public" about the achievements of the Secretary. It is not the public which has expressed concerns, but employees of the Department. I would hope that senior leaders are also engaging their staff because apparently the message has not been filtering down. To be fair, since October the Director General has (imho) been addressing some of the concerns raised by the survey and it is possible that opinions now might differ from those in October.

Posted on Fri Jan 11, 2008

Rachel in New York writes:

The Department denying that the AFSA survey has ANY merit at all does not make much sense, and comes across as overly defensive. The minute increases in State budget over the past few years do not come close to meeting the increased demands placed on State's resources.

Posted on Fri Jan 11, 2008

Aline in New York writes:

As parent of a long-time member of the State Department, I am always glad to read anything that comes out about it, and I find the blog, especially the one entered here about the AFSA survey, very helpful to my understanding what my family members live with.

Posted on Thu Jan 10, 2008

Robin in Venezuela writes:

Perhaps the Department, working with AFSA, and perhaps an independent facility less prone to charges of favoritism, could do a survey on similar topics, ensuring that it is scientifically done and thus have that not be a reason to dismiss the comments/results. I have heard many of my fellow officers express sentiments that reflect the survey may be more accurate than the Department would like; however, it would be good to see results in a form that all sides believe is fair and accurate. If the scientific survey reveals the same results, the Department could put more efforts into addressing those issues. If it reveals a different perspective, that would also help AFSA know where to concentrate its resources.

Posted on Thu Jan 10, 2008

Peter in Kansas City writes:

Sean: You are correct that the survey may not give a true picture of the views of the entire Foreign Service (FS). Without using a random sample approach, that is not possible.

On the other hand, the survey is not without merit. This survey does give you a very clear picture of the views of about 4300 FS members -- over one third of the entire FS. Whether you like what they collectively said or not, it reflects what a substantial portion of the FS believes.

The more the senior leadership at State tries to dismiss this survey and the views of the people who took part in it, the more it will reinforce the impression that the seventh floor lives in a bubble.

Posted on Thu Jan 10, 2008

Andrew in Washington, DC writes:

The WaPo story you're linking to makes no reference to the AFSA survey (which makes sense, as it's from Jan 5). This is also true of Tom's and Richard's posts.

---
Dipnote Bloggers write: @ Andrew in Washington, DC -- Nice catch Andrew! Our mea culpa. The link is now updated.

Posted on Thu Jan 10, 2008

Charlie writes:

Sean: Can you speak more extensively to the issue of Overseas Comparability Pay? This seems like it should be a major priority for the Department. How long has this problem existed, and why hasn't the Department been able to get the funding from Congress to fix it? Given the size of the Foreign Service, what would be the cost of implementing such a fix? This seems like a bedrock human capital problem; the incentive structure is totally upside down. How can we talk about transformational diplomacy or expeditionary diplomacy without putting in place the right incentives?

Posted on Thu Jan 10, 2008

Page 1 of 1 pages