SIGINT is hard freakin work
Published on December 14, 2005 By greywar In Politics

            Folks, this one won’t be for the easily bored or for people who only stop by my blog to see creative use of the F-word. If you don’t care about how intelligence is collected, processed, and disseminated by the U.S. military just stop here. Might I suggest Fark.com instead?

            Ok, now that those guys are gone we can get right to the geeky heart of the matter at hand:

 

Intelligence Theory:

The Right and Wrong Way to Process Voice Traffic

(radios, phones, wiretaps…whatever (don’t bail out now… I warned you this would be esoteric for most of you)).

 

            Let’s start with a discussion of why accountability and quality checking are essential in this field. The basic question asked of every single voice collector in the field is, “What did you hear?” and the answer to that question will vary wildly depending on one single factor: “Will anyone be checking the veracity of what I say I heard?”

            Let me give a sanitized real-world example. During the last war in Iraq most collection stations (especially the Air Force) were not required to prove what they reported from voice exploitation. They were not required to keep recordings, not required to provide a syllable by syllable target language transcription, or even provide a full-text translation to support their field reports. This resulted in an ever increasing volume of reports from these unaccountable sources with fantastic intelligence content (and by fantastic I don’t mean good I mean that these were works of purest fantasy by the collectors).

            It is frankly amazing how often an operator (usually a U.S. soldier, sailor, airman, or marine) will hear things like “We will place the anthrax booby trap 2 kilometers from the police station in Basra and the sarin gas trap under the bridge at 2 pm tomorrow.” when they know that they will never have to prove it to anyone. Strangely in stations that are required to retain recordings and produce target language transcripts these cuts are extremely rare to non-existent. Those same station will report something like, “We need to go to the police station to see if Lt. Haidr will help us get more patrols on our street.” for the exact same time and source as the quote I gave above.

            There were still dozens of ridiculous reports like the first one coming through every day in Iraq when I left there earlier this year. They of course almost never (less than 1%) turned out to be even remotely true but that hasn’t stopped their production since intelligence Commanders care about two things : numbers of reports issued and number of high profile reports issued. Rarely do they care whether the reports corresponded in even a nodding fashion with “ground truth”.

            This brings me to the various methods currently employed by the military in voice collection.

 

            Version 1.0 Unchecked Operator Gists

            This is the version used in the first example. The operator (Army MOS 98G) gives his opinion on what he heard to a report writer (98C) who usually has zero training in the language and even less on analysis of this specific target (analyst training is very general and generally very inadequate) resulting in reports that run from very skeletal gists (by operators who don’t want to make assumptions about what they might have heard) to the flights of fantasy fiction cited earlier in this article. For the hard of thinking this is a bad thing. I am sad to say that this is the most common of all collection methods.

 

            Version 1.5 Traffic is Quality Checked Only When a Report is Questioned

            This version is a bit less common and is actually even worse that version 1.0 since it polarizes the results of collection into only two categories: Hyper-conservative skeletal gists (so they will never be wrong since they never report anything pf importance to be questioned) or Erratic reporting. The latter is when the station still puts out loads of seemingly “juicy” reports only to issue cancellations for 99% of them when they are required to back them up.

 

            Version 2.0 Every Cut of Traffic is Transcribed by One Level Transcription

            This is supposed to be the US SIGINT Service standard. In reality it is followed almost exclusively by large permanent stations only while the “field” does Version 1.0. This version starts out fairly slowly as operators tend to be very conservative when they first start (since they know their stuff is being checked (usually by a native)), slowly progresses to a fair/good level as their confidence increases, and then tends to wobble a bit after becoming well established due to overconfidence by the transcribers themselves (who of course are not checked themselves since they are the checker). Lastly this method is prone to both intentional and unintentional sabotage as I will explain later.

           

            Version 3.0 Every Cut is Scribed Using a 2 Scribe+1 Quality Control System

            (I have never seen this in use anywhere largely because I invented it only recently after thinking over some lessons learned)

            This version (which I will detail later on) has a slow starting pace much like version 2.0 but over time its implementation would result in vastly increased operator speed, confidence, and transcript accuracy.

 

 

(To be continued in the next installment)

 

Sorry about breaking it up but I hate overly long posts….

 

           

           

 

           

Site Meter
Comments (Page 1)
2 Pages1 2 
on Dec 14, 2005
"Who watches the watchmen," eh?

I've had similar observations, but I haven't really approached it from the angle you're attacking from. As I am naive enough to believe that most people are "good," I usually approach this from a competence standpoint, not a virtue one. Also, because I'm projecting, I tend to think that everyone will be cautious/conservative about reporting. This of course isn't the case

I haven't seen a situation where Version 1.0 is in place, although I know you've mentioned it to me before. I suppose the reason I haven't seen this is because of the standard you set against it when you came to our unit...

Version 1.5 is what I've heard of run recently with some non-live-collect work. I concur that it encourages the conservative side, and those operators that do try and make something out of the audio will just have it shot down at the QC/Xscribe level, anyway.

Honestly, I've seen many more honest errors than outright lies (although it's hard to believe that GUS was just that wrong sometimes). However, it still leads to the same conclusion, that some form of Quality Control and accountability for the original gist errors be in place. We've had the prior much more than the latter... ("You can't yell at me! I'm a contractor!")
on Dec 14, 2005
First, was 'US Intelligence' spelled incorrectly on porpoise?

Second, you mentioned that it was '(especially the Air Force)' that has had the fantasy problems... to me this is not surprising given the difficulty that members of said service have distinguishing between WOW and their real lives. ... At least the ones that *I've* known!

It seems like a two year immersion program might be fun and worthwhile. At least for the long termers.
on Dec 14, 2005
I am amazed. Only because I think that I actually understood the gist of this blog. The fact that some persons might want to enlarge, distort, and give misleading intelligence does not not surprise me. Integrity in many people, no matter what their job, is hard to come by in this day and age.

I hope your next installment is written as clearly and as interestingly as this one was. I'll be waiting. Keep it up. I am also happy that you didn't have to use your creative genius in the use of the F-word.
on Dec 14, 2005
Hey. I may not know the language, but I know car bomb when I hear it.
on Dec 14, 2005

First, was 'US Intelligence' spelled incorrectly on porpoise?

oh bite me:)

on Dec 14, 2005
Grey:
It is frankly amazing how often an operator (usually a U.S. soldier, sailor, airman, or marine) will hear things like 'We will place the anthrax booby trap 2 kilometers from the police station in Basra and the sarin gas trap under the bridge at 2 pm tomorrow.' when they know that they will never have to prove it to anyone.

Heh heh heh... where's Gus when you need him? At least he kept us Charlies on our toes. We could always count on him for a laugh, especially when he kept mistaking Farsi for a previously-unheard Arabic dialect.
Version 3.0 Every Cut is Scribed Using a 2 Scribe+1 Quality Control System

I like your reform ideas, but whatever happened to "time-sensitive reporting"? Or "finite resources"? Could you imagine all the fights that would ensue if certain contractors had to simultaneously work on a cut? Yikes, stand back, Mr. I Am (Kurd)...

X-SPC:
but I know car bomb when I hear it


Really? In Arabic?
on Dec 14, 2005
(really nbs)

Really? In Arabic?


mutafa(noise like coughing up a dead animal)hah. IED

Mutafahahah sayyarat. VBIED

Koombala. Bomb (easy to remember because it rhymes with big kaboombala)

Abu nasif. Explosives.

Really. In arabic. As in, "Boy do you remember that car bomb that went off two months ago? Man it was loud. The one that broke the neighbors windows? That's exactly how loud you just farted." That kind of useful tidbit.
on Dec 14, 2005
You have schooled me. Truly schooled. Thank you for bringing the bus around, let's go home from school.
on Dec 15, 2005
Whatever, Linguists are best imployed in the field, pulling guard duty or administrative errata. Their mediocracy is a higher standard than other MOSs, resulting in impressing your senior officers when they visit. As long as the weeds are pulled and the cheese and crackers are on the table, the Army is proveably running smoothly.
on Dec 15, 2005
Really? In Arabic?


For a Korean linguist, you're at least better than GUS.

Link

I don't know how often it happened (because I wasn't QCing you much, especially after my outburst), but I know there was the occasion where you really wanted to hear those words... part of the problem being that those are a good percentage of the words you do know, so you'll hear them more often.

I'm not saying you're part of the "incompetence" problem; you weren't officially trained! Your diligence in self-training in the language, along with throwing yourself headlong into the work really paid dividends for our unit and the mission. You. Were. An. Asset. Don't doubt that. But we certainly would have been better off had you been a 2/2/1+ AD.
on Dec 15, 2005

I like your reform ideas, but whatever happened to "time-sensitive reporting"? Or "finite resources"? Could you imagine all the fights that would ensue if certain contractors had to simultaneously work on a cut? Yikes, stand back, Mr. I Am (Kurd)...

 

     Well actually I have solved the "working" together thing too (as I will detail in the next installment) and as for time sensitiviity issues...? Well frankly the "timely" reporting being done right now in and out of theatre is doing *far* more harm than good and it is the strategic term reporting that was producing the real results in country.

     Fact of the mater is that you don't catch weapon smugglers, insurgents, agents etc... through time sensitive reporting... they simply don't drive around in trucks full of guns and then get on the radio to tell Billy-bob Mustafa about it so that we can hear them, figure out what he said, report them in five minutes while somehow magicking up a good enough DF to get the report to some fictional MP or Infantry unit through channels that don't exist so that they can pick them up. That is the Myth of current collection and it needs to die today.

     The major breakthroughs from the Intel side of the house have come through careful correlation of minutiae over time. This can only work when the material you build your investigations is as close to 100% accurate as possibel. Polluting this pool with faster and less resource intensive materials that are badly flawed makes things worse. Do you remember the Kill-Chain operation that we attempted to do with the Marines? Jesus, that was 5 seconds from an international disaster!

     Fast and bad intel kills friendlies, innocents, and harms the war effort.  

on Dec 15, 2005
That is the Myth of current collection and it needs to die today.


This was a fantastic post. I hadn't heard it put quite that way before, but you're 100% correct here.
on Dec 15, 2005
Great summary. Always wondered how collecting worked. Kinda hoped all collection was at the level 2 that you described. A bit frightening the prevelance of '1'. Keep up the great posts.
on Dec 15, 2005
Hello
This is a real good blog. I really enjoyed it. if i were you i would go to www.autosurfmonster.com and submit them this blog for free. let thousands see your blog. Well, I look forward to all the updates.
jennifer
on Dec 15, 2005
figure out what he said, report them in five minutes while somehow magicking up a good enough DF to get the report to some fictional MP or Infantry unit through channels that don't exist so that they can pick them up. That is the Myth of current collection and it needs to die today.


I always knew that all our reporting went into some black hole here at Meade never to see the light of day. Now that I know it's true, since I have now seen it for myself, it is somewhat disheartening. All our electronic efforts...
For all those times when we heard "great work" and "wow, you really pulled some good intel tonight", it was just blowing smoke up our collective asses...

Do you remember the Kill-Chain operation that we attempted to do with the Marines? Jesus, that was 5 seconds from an international disaster!


Yes I do. Yes it was. Glad that worked out, oh yeah.
2 Pages1 2