ThinThread is the name of a project that the United States National Security Agency engaged in during the 1990s, according to a May 17, 2006 article in the Baltimore Sun.
The program involved wiretapping and sophisticated analysis of the resulting data.
ThinThread, was developed to handle greater volumes of information, partly in expectation of threats surrounding the millennium celebrations.
Sources say it bundled together four cutting-edge surveillance tools. ThinThread would have:
* Used more sophisticated methods of sorting through massive phone and e-mail data to identify suspect communications.
* Identified U.S. phone numbers and other communications data and encrypted them to ensure caller privacy.
* Employed an automated auditing system to monitor how analysts handled the information, in order to prevent misuse and improve efficiency.
* Analyzed the data to identify relationships between callers and chronicle their contacts. Only when evidence of a potential threat had been developed would analysts be able to request decryption of the records.
In what intelligence experts describe as rigorous testing of ThinThread in 1998, the project succeeded at each task with high marks.
For example, its ability to sort through massive amounts of data to find threat-related communications far surpassed the existing system, sources said.
It also was able to rapidly separate and encrypt U.S.-related communications to ensure privacy.
ThinThread was designed to address two key challenges: The NSA had more information than it could digest, and, increasingly, its targets were in contact with people in the United States whose calls the agency was prohibited from monitoring.
With the explosion of digital communications, especially phone calls over the Internet and the use of devices such as BlackBerries, the NSA was struggling to sort key nuggets of information from the huge volume of data it took in.
By 1999, as some NSA officials grew increasingly concerned about millennium-related security, ThinThread seemed in position to become an important tool with which the NSA could prevent terrorist attacks. But it was never launched. Neither was it put into effect after the attacks in 2001.
Despite its success in tests, ThinThread’s information-sorting system was viewed by some in the agency as a competitor to Trailblazer, a $1.2 billion program that was being developed with similar goals. The NSA was committed to Trailblazer, which later ran into trouble and has been essentially abandoned.
A number of independent studies, including a classified 2004 report from the Pentagon’s inspector-general, in addition to the successful pilot tests, found that the ThinThread program provided “superior processing, filtering and protection of U.S. citizens, and discovery of important and previously unknown targets,” said an intelligence official familiar with the program who described the reports to The Sun.
What made ThinThread ingenious is that, “Instead of vacuuming up information around the world and then sending it all back to headquarters for analysis, ThinThread processed information as it was collected—discarding useless information on the spot and avoiding the overload problem that plagued centralized systems.”
ThinThread was so thorough and good at collecting information that it picked up intelligence on American citizens without the NSA intending it to.
The program was discontinued after the September 11, 2001 attacks due to the changes because of bureaucratic infighting and a sudden White House expansion of the agency’s surveillance powers, according to several intelligence officials.
But the NSA, then headed by Air Force Gen. Michael V. Hayden, opted against both of those tools, as well as the feature that monitored potential abuse of the records. Only the data analysis facet of the program survived and became the basis for the warrantless surveillance program.
The Pentagon report concluded that ThinThread’s ability to sort through data in 2001 was far superior to that of another NSA system in place in 2004, and that the program should be launched and enhanced.
Both programs aimed to better sort through the sea of data to find key tips to the next terrorist attack, but Trailblazer had more political support internally because it was initiated by Hayden when he first arrived at the NSA, sources said.
NSA managers did not want to adopt the data-sifting component of ThinThread out of fear that the Trailblazer program would be outperformed and “humiliated,” an intelligence official said.
Without ThinThread’s data-sifting assets, the warrantless surveillance program was left with a sub-par tool for sniffing out information, and that has diminished the quality of its analysis, according to intelligence officials.
Sources say the NSA’s existing system for data-sorting has produced a database clogged with corrupted and useless information.
The privacy protections offered by ThinThread were also abandoned in the post-Sept. 11 push by the president for a faster response to terrorism.
Once President Bush gave the go-ahead for the NSA to secretly gather and analyze domestic phone records — an authorization that carried no stipulations about identity protection — agency officials regarded the encryption as an unnecessary step and rejected it, according to two intelligence officials knowledgeable about ThinThread and the warrantless surveillance programs.
“They basically just disabled the [privacy] safeguards,” said one intelligence official.
Another, a former top intelligence official, said that without a privacy requirement, “there was no reason to go back to something that was perhaps more difficult to implement.”
However two officials familiar with the program said the encryption feature would have been simple to implement. One said the time required would have involved minutes, not hours.
Encryption would have required analysts to be more disciplined in their investigations, however, by forcing them to gather what a court would consider sufficient information to indicate possible terrorist activity before decryption could be authorized.
While it is unclear why the agency dropped the component that monitored for abuse of records, one intelligence official noted that the feature was not popular with analysts. It not only tracked the use of the database, but hunted for the most effective analysis techniques, and some analysts thought it would be used to judge their performance.
The mass collection of relatively unsorted data, combined with system flaws that sources say erroneously flag people as suspect, has produced numerous false leads, draining analyst resources, according to two intelligence officials. FBI agents have complained in published reports in The New York Times that NSA leads have resulted in numerous dead ends.
The strength of ThinThread’s approach is that by encrypting information on Americans, it is legal regardless of whether the country is at war, according to one intelligence official.
Officials familiar with Thin Thread say some within NSA were stunned by the legal flip-flop. ThinThread “was designed very carefully from a legal point of view, so that even in non-wartime, you could have done it legitimately,” the official said.