SUBSCRIBE NOW
IN THIS ISSUE
PIPELINE RESOURCES

The Federal Big Data Factory


Just how big is the United States government’s Big Data factory?
But surely a warrant and a high level of clearance are needed to peer into the communications of everyday Americans, right? Wrong. Even analysts near the bottom of the NSA hierarchy can use XKeyscore to access “whatever emails they want, whatever telephone calls, browsing histories, Microsoft Word documents,” said Glenn Greenwald, the journalist who broke the story, in an interview with ABC News. “And it’s all done with no need to go to a court, with no need to even get supervisor approval on the part of the analyst.”

Is it possible?

The apparatus to gather and store vast amounts of data is certainly in place. As Pipeline reported last July, the feds have enough virtual space to store and process every digital transmission in the US—and maybe more. Increasingly, the NSA is a Big Data analysis factory.

“The agency is collecting so much information every day that without a regimented, factory-like system, analysts would never have the chance to look at it all,” wrote investigative reporter Shane Harris for Washingtonian magazine in February.

However, given the fact that data is only stored for five years in the Marina database, to say nothing of the sheer volume of information that moves throughout global networks 24 hours a day, there are technological and physical limits on how much data is actually scooped up and retained. Verizon’s landline business alone handles more than a billion phone calls each day.

Even when taking into consideration the NSA’s massive data center in Bluffdale, Utah, there is simply not enough storage space to save every daily call and communications transmission from every service provider for five years, let alone five days. While the capability exists to capture everything, all the way down to digital pocket lint, much more data is falling through the net than is being captured and stored.

The official response

After flatly denying the existence of domestic surveillance, the US government responded to allegations over the summer. Pressure from concerned legislators, privacy rights advocates, a worried public, and, last but not least, CSPs and other companies that desire greater accountability certainly hastened that response.

On August 9 President Obama gave a press conference in which he promised to overhaul the NSA and improve transparency. He outlined four steps to achieve this initiative:

“First, I will work with Congress to pursue appropriate reforms to Section 215 of the Patriot Act, the program that collects telephone records ... Second, I’ll work with Congress to improve the public’s confidence in the oversight conducted by the Foreign Intelligence Surveillance Court, known as the FISC ... Number three, we can, and must, be more transparent ... The intelligence community is creating a website that will serve as a hub for further transparency, and this will give Americans and the world the ability to learn more about what our intelligence community does and what it doesn’t do, how it carries out its mission and why it does so. Fourth, we’re forming a high-level group of outside experts to review our entire intelligence and communications technologies.”



FEATURED SPONSOR:

Latest Updates





Subscribe to our YouTube Channel