Meta data and their collection; two approaches.
March 31, 2014 |
The collection of metadata is a growing area of concern for privacy practitioners, advocates and regulators. Big data, with every more powerful computing power and more sophisticated algorithims, have made sorting and sifting phone reocrds, emails and internet browsing history to obtain and use, and misuse, information against individuals.
In the United States it has attracted the interest of the courts and the Congress. The impetus has been the Snowden disclosures/leaks etc.. The politics of the action is one issue. What it exposed, the very broad powers and their liberal use by United States Government agencies has caused regulators, the executive and the body politic to pause and consider the need for controls. In the Guardian’s Obama formally proposes end to NSA’s bulk collection of telephone data the Government the US Government is looking to end the bulk collection of phone data, a key form of metadata.
It provides:
The Obama administration on Thursday formally proposed ending the National Security Agency’s bulk collection of all US phone data.
Nearly 10 months after the Guardian exposed the controversial program, based on leaks from Edward Snowden, President Obama announced that he would seek legislation that would require the NSA to seek an individual order from the secret Fisa court before phone companies turn over data on their customers.
“I have decided that the best path forward is that the government should not collect or hold this data in bulk,” Obama said in a statement. “Instead, the data should remain at the telephone companies for the length of time it currently does today.”
The move goes further than Obama’s position on bulk surveillance in January, when the president left the door open to the possibility of the data being held by a private-sector third party. That position was vigorously opposed by the phone companies and criticised by proponents and critics of the NSA alike.
Bulk phone data would no longer be collected by NSA under the latest proposals. Instead phone companies would, in response to a court order, turn over a suspicious phone number as well as all the numbers it called and received, and all numbers those numbers called and received, on an “ongoing and prospective basis”, according to an administration official.
The administration has yet to decide on a specific time limitation for querying the data, but “there would be some limited time period,” the official told reporters on Thursday. “That’s something we’re going to have to talk with Congress about.”
The Obama administration is seeking legislation to enact the changes, but it has not settled between competing proposals currently before Congress.
“I am confident that this approach can provide our intelligence and law enforcement professionals the information they need to keep us safe while addressing the legitimate privacy concerns that have been raised,” Obama said.
But the NSA is not yet out of the business of harvesting phone data in bulk, which it has done in secret in various forms since late 2001. The administration said it would seek approval from the Fisa court to continue the programs for another 90-day period under restrictions in place since January, until Congress passes a bill along the administration’s guidelines.
A senior administration official indicated that the legal standard by which the court could order phone companies to turn over customer data would be a “reasonable articulable suspicion” of a phone number’s connection to terrorism or espionage. That is a lower threshold than relevance to an ongoing terror investigation, the language of Section 215 of the Patriot Act, the current authorisation the administration claims for bulk domestic phone data collection.
Verizon’s top lawyer, Randall Milch, sounded a tone of wariness over the specifics of the proposal while praising it overall. “If Verizon receives a valid request for business records, we will respond in a timely way, but companies should not be required to create, analyse or retain records for reasons other than business purposes,” Milch wrote Thursday on Verizon’s blog.
Since January, the NSA has been permitted to query its phone data troves only after the Fisa court first certifies it possesses reasonable, articulable suspicion of a record’s connection to terrorism. “So that provides, I think, a good baseline, and a good point from which we can work with Congress to develop the proposal,” said the senior official, who would not agree to be identified.
It also aligns the proposal with a bill put forward on Tuesday by the leaders of the House of Representatives intelligence committee, Republican Mike Rogers of Michigan and Democrat Dutch Ruppersberger of Maryland, which uses that standard.
But a key difference between the committee’s proposal and the one outlined by administration officials on Thursday was a judge’s prior approval for individual phone numbers. The House panel wants the surveillance judges to review the specific collection after the fact, something the administration rejects in all but emergency cases.
A rival bill proposed by members of the Senate and House judiciary committees would require prior judicial approval for specific phone data, but would set the legal threshold for acquisition of the data higher than what the administration desires.
Earlier this week, the chairwoman of the Senate intelligence committee, Democrat Dianne Feinstein of California, said she intended to schedule a hearing to examine Obama’s surveillance proposals alongside those of her House counterpart.
The Obama administration left several aspects of its desired surveillance policy unaddressed on Thursday.
Although officials explaining the policy on a conference call with reporters said they wanted the government to no longer “hold” the data, they did not unveil any changes to the NSA’s so-called “corporate store” of analysed phone records. That store, according to the government’s official privacy and civil liberties watchdog, contains tens of millions of phone numbers, and analysts do not face any restrictions on searching through it.
Caitlin Hayden, a spokeswoman for the National Security Council, clarified that the Fisa Court will approve a new set of minimisation procedures to provide privacy protections around the use, retention and dissemination of phone data.
“The details of where the data would be stored and accessed once it is received would be governed by those minimization procedures, just as minimisation procedures currently govern how we handle the data,” Hayden said.
Nor did the administration outline any changes to its consideration of privacy rights for non-Americans abroad, something Obama said in his January speech the NSA needed to consider.
NSA’s ability to search for Americans’ identifying information in its troves of phone and internet communications content appears to be unimpeded, a function the USA Freedom Act would prevent. Nor would NSA be prevented from surreptitiously undermining online encryption standards.
Critics of bulk surveillance, in and outside Congress, praised the administration for its shift. “I’m glad the president has embraced my bipartisan legislation to end the dragnet collection of Americans’ private phone records,” said Senator Mark Udall, a Colorado Democrat, who said he had long felt like a “voice in the wilderness” on the NSA.
But Udall, like privacy advocates, urged Obama to “immediately end” the phone records dragnet rather than re-submit it for another 90-day Fisa Court approval.
Privacy groups also expressed wariness that Obama’s proposals on Thursday only covered phone data. “This raises the possibility that the government could collect other types of information in bulk, including internet metadata, location information and financial transactions,” said Harley Geiger of the Center for Democracy and Technology. “Unless legislation addresses all types of data, not just phone records, then businesses remain at risk of receiving an order to turn over records on all of their customers and to keep quiet about it.”
Obama’s position on the proper scope of the NSA has changed substantially, by degree, in the 10 months since the Guardian and other news outlets began publishing stories based on documents Snowden provided.
In June, Obama, a former constitutional law professor, greeted the revelations of bulk domestic call records collection by saying he thought he had “struck the right balance.” Over the course of 2013 and early 2014, two high-level review panels, one of which Obama personally empowered, disagreed and proposed changes, while a federal judge in December found the collection to be on the precipice of constitutional violation.
Opposition in Congress was substantial, if short of a majority to end the practice. But in recent weeks, members of the House of Representatives publicly threatened to allow the provisions of the Patriot Act the administration relied upon for bulk collection to expire next year if Obama did not act first.
The fear of losing the basis for a program that officials consider critical – although they have backed away from earlier claims it has prevented terrorist attacks – alongside resistance from the telecos appears to have contributed to Obama’s most recent shifts in position.
Left unspoken on Thursday was the fate of Snowden, the former NSA contractor whose disclosures prompted the administration to restrict its surveillance dragnets.
Snowden is in Russia on temporary asylum after the State Department cancelled his passport. The Justice Department is considering an indictment on espionage-related charges.
Even as US government officials in both the executive and legislative branches have moved closer to ending bulk domestic surveillance, the practice Snowden says prompted his disclosures, they have accused him without public evidence of endangering the lives of US troops in the future and even laid responsibility for the Russian invasion of Crimea at his feet.
At the White House, the reversal in positions by Obama on bulk data collection has not translated into leniency for Snowden. “There’s no change in our position that he needs to return and face the felony charges against him,” Hayden said.
In Australia there is a debate but the quality of the input is variable. In that debate the role of an opposition is important. It tests and, if necessary, challenges government’s proposals. The Government’s position on collection of meta data without warrant is not finalised. What is concerning is the opposition’s preparation to loosen, rather than tighten, any limited constraints on intelligence agencies accessing metadata and requiring telecommunications companies to collect and store data for longer periods. This story is covered in the Guardian’s Tanya Plibersek: we would give spies more tools to fight domestic terror.
It provides:
Labor’s deputy leader, Tanya Plibersek, has signalled she is open to giving intelligence agencies more tools to deal with a possible increase in domestic security threats from Australians being radicalised in the Syrian conflict.
Plibersek on Sunday gave a strong signal she was comfortable with telecommunications companies collecting and storing intercepted data for longer periods in order to assist intelligence agencies in their domestic anti-terror investigations.
The shadow foreign minister said the collection of metadata – the information we all generate whenever we use technology, from the date and time of a phone call to the location from which an email is sent – had aided the disruption of terror plots in Australia.
Plibersek played down the invasions of privacy posed by metadata sweeps, reasoning the intercepted material was the “envelope”, not the contents. “People describe it as keeping the haystack so you can go back and look for the needle afterwards,” she said.
She said metadata collections had been an important tool in safeguarding Australia’s national security. “We have disrupted some very serious terrorist plots in Australia,” Plibersek told Sky News on Sunday. “We’ve done it because we’ve got a strong intelligence community here. They do a good job.”
She was asked whether strengthening of the interceptions regime was justified in the wake of new threats posed by radicalised fighters returning from the Syrian conflict – an issue the Abbott government and intelligence agencies have expressed concern about.
“There continue to be threats. Those threats may increase,” Plibersek said. “I want to give (intelligence) agencies the maximum ability to do their job well, within the bounds that people would expect.”
Plibersek suggested she was comfortable in-principle with telecommunications companies collecting metadata and storing it for a mandatory retention period.
She said the community had a right to privacy, and to expectations of living in an open and democratic society – but her view was government needed to make it as “easy as we can” for intelligence agencies to protect against established and emerging threats.
The comments Sunday suggest Plibersek would be happy to revive a controversial plan shelved by the former Gillard government. That proposal would force Australian telecommunications companies, internet service providers and social media sites to collect metadata from Australian users and store it for two years.
Australia’s domestic intelligence agency, Asio, wants telecommunications data to be collected by companies and stored for two years – longer in some cases – for law enforcement agencies to access during their investigations.
A series of disclosures by the former National Security Agency contractor-turned-whistleblower Edward Snowden revealing the extent of coordinated online surveillance by the US, the UK and its allies, including Australia, have generated significant debate about whether governments are achieving the right balance between national security and privacy.
The Snowden disclosures prompted a significant rift in the Abbott government’s relationship with Indonesia (when it was revealed that Australia in 2009 attempted to tap the phone of the president and his inner circle); and also prompted the Greens and Labor to combine in the senate to establish a new inquiry into Australia’s telecommunications interception regime.
In submissions to that inquiry, the privacy commissioner and Australia’s intelligence watchdog, the Inspector General of Intelligence and Security, have flagged potential adjustments to the interception regime to ensure agencies strike the correct balance between protecting consumer privacy and conducting necessary investigations to prevent crimes.
Those agencies suggest more can be done to safeguard community privacy.
But while accepting the need for some adjustments, Asio has used the same inquiry to defend the central importance of a broad interception regime and the mandatory storage of private communications data for lengthy periods.
“Telecommunications interception of content is a key tool in preventing harm because people engaged in activities of security concern must communicate to progress their intentions,” Asio says in its submission to the senate.
“Access to telecommunications content provides essential details of activities of security concern and enables Asio to provide advice and take action to protect Australia,” it says.
“In terms of data retention periods required for Asio to effectively discharge its functions, at least two years is required in some cases, whether by carriers, carriage service providers, or ancillary service providers. Due to the nature of activity by clandestine foreign actors, retention for longer than two years would be ideal.”
The government has been signalling for months it is exploring steps to increase protections for Australians given the Syrian situation.
The reported comments by Ms Plibersek are silly and ill thought out. That was a them picked up by Zdnet in Plibersek loses our privacy in a haystack of envelopes.
It provides:
Dear oh dear, Tanya Plibersek, dear oh dear oh dear oh me oh my! Where do I begin?
I know that the Australian Labor Party has a lot to think about, having been soundly thrashed in the last federal election. And I guess there must be a lot on your plate personally, what with being the party’s deputy leader at a time when it still seems to be missing some sort of leader for you to be deputy to. Tough gig. Sympathies.
But re-bleating the discredited analogies of the digital spooks? Really? After everything we’ve learned about the vast power of metadata collection in the last year, thanks to Edward Snowden?
I was astounded to read The Guardian’s report of your TV interview on Sunday.
Plibersek played down the invasions of privacy posed by metadata sweeps, reasoning the intercepted material was the “envelope”, not the contents. “People describe it as keeping the haystack so you can go back and look for the needle afterwards,” she said.
I’ve already written how metadata is just more personal data, and how attempting to portray it as less revealing of our private lives than the “content” is disingenuous. Anyone still pushing that angle is either a fool or a liar. (I’m sure you’re neither, Ms Plibersek; as I say, you’ve been busy.) There’s also a slide from Electronic Frontier Foundation kicking around on Twitter today that makes the same point.
But I’m wondering just how big a haystack Ms Plibersek reckons our spooks need, when they’ve already got more hay than ever before — and can cut and bale it more effectively than ever before, what with their data centres full of computers, and nifty analytical software, and ever more warm bodies and keen minds at their disposal?
After all, the spooks already have the power to issue “ongoing domestic preservation notices” — that is, to get internet service providers to start logging a customer’s activity — as soon as they have a reasonable suspicion they might be up to something that falls within a rather wide range of potential crimes.
The only possibility left, logically, is the surveillance of people who are *not* under suspicion. I thought that we who lived in western democracies frowned upon that sort of thing. Either way, it’s certainly something to consider when reading another paragraph from The Guardian’s report.
[Plibersek] said the community had a right to privacy, and to expectations of living in an open and democratic society — but her view was government needed to make it as “easy as we can” for intelligence agencies to protect against established and emerging threats.
Have you ever noticed that the phrase “The community has a right to privacy, but” has the same structure as “I’m not a racist, but”? How there’s lip service to the idea of privacy, then in the very next breath a proposal that would comprehensively trash it? In western democracies we don’t make things “as easy as we can” for spooks, we seek a balance between their power and our freedoms — all the while keeping the supposed “threats” in perspective.
Here’s some perspective.
How many people have been killed in a terrorist attack in Australia in the past decade?
Zero.
In the past two decades?
Zero.
In the past three decades?
One.
On 23 November 1986, the unfortunate Hagob Levonian blew himself up with his own poorly conceived device in what’s known as the Turkish consulate car-bombing.
The last times Australians have been killed in terrorist attacks anywhere in the world were when a British-Australian man was killed in Nairobi’s Westgate shopping mall attack last year, then — a long gap — when four died in the Bali bombings of October 2005, and a Melbourne man died in the London bombing earlier the same year. The more serious and better-known Bali bombings, and the so-called 9/11 incident in New York, tragic as they were, were more than a decade ago.
Despite all the talk of terrorists — and this has been pointed out so many times before — we’d be better off launching a war against bathtubs. Or getting all the spooks into sporting kit and out on the paddock, leading the kiddies in some exercise in the battle against obesity — which is something far more likely to kill us in the long run. But instead, they’re thinking up new terrorist threats.
[Plibersek] was asked whether strengthening of the interceptions regime was justified in the wake of new threats posed by radicalised fighters returning from the Syrian conflict — an issue the Abbott government and intelligence agencies have expressed concern about.
“There continue to be threats. Those threats may increase,” Plibersek said …
Well, in response to that I’ll point to two things.
One, an eminently readable analysis by Gary Brecher aka The War Nerd, which points out that jihadis from countries like Australia are small in number and rather low in effectiveness. I reckon we could track them individually as they left the country and returned, and assign each one their own personal ASIO agent.
Two, RAND Corporation research which suggests that self-radicalisation through the internet — without having to go to Syria or whatever the fashionable conflict might be — isn’t even a thing.
Dear Ms Plibersek, we already have one side of politics wallowing in their own fantasy of the last century, or the century before that. We don’t need another. There’s plenty of real research out there, about real threats and the real risks.
Leadership is about navigating those waters, not being a mouthpiece for outdated clichés.
And Scott Ludlum, Greens Senator from Western Australia, chimes in with his own excoration of the Ms Plibersek with Labor’s capitulation to ever more invasive surveillance is shameful which provides:
It won’t make the front page but on 30 March, Labor’s deputy leader Tanya Plibersek quietly caved in on one of the most invasive of recent proposals for mass surveillance in Australia.
The mandatory data retention regime would compel phone companies and internet service providers (ISPs) to retain telecommunications data – “metadata” – for at least two years for the whole Australian population.
As Plibersek noted in her announcement, it is not the recordings of phone calls or content of emails that is proposed to be retained; it is data around these things. Who you called, and for how long; who you email, and the size and type of the attachments you send. If you carry a mobile phone, metadata can record your location every minute of the day.
The unprecedented invasions of privacy made possible by the proliferation of ever-more detailed clouds of metadata are a key reason why the revelations of US whistleblower Edward Snowden caused such deep revulsion around the world. This is one essential thread of Orwell’s waking nightmare 1984, brought into being through a mix of dogged persistence, bowties and bland euphemism.
The idea hasn’t come from nowhere: it was floated and rejected twice during the Rudd/Gillard years. It emerged into the public domain in 2009 with a leak to the Fairfax press that the attorney general’s department had coerced ISPs into a round of secret meetings to establish how expensive it would be to implement. I was able to initiate a Senate inquiry into the plan which flushed more details into the public domain, and in the ensuing uproar the plan was quietly withdrawn.
In July 2012, the data retention proposal merited a few sketchy lines in a discussion paper published alongside an inquiry by the Joint Intelligence and Security Committee. Data retention then inevitably became a focal point of public alarm, such that the Committee was unable to come to a consensus recommendation as to its merits. In the ensuing uproar the plan was quietly withdrawn, again.
But the attorney general’s department is nothing if not patient, and it’s hard to imagine a more pliable attorney general than senator George Brandis, whose first confidence-building initiative was to employ ex-ASIO director general Paul O’Sullivan as his chief of staff.
All of this makes Labor’s spontaneous capitulation to ever more invasive surveillance that much harder to understand. Data retention is not about targeted, evidence-based intelligence gathering. It is the indiscriminate collection of detailed, real-time metadata on everyone. It is not just for people suspected of serious crimes or political violence, and no judicial oversight is required for a proliferation of hundreds of agencies and local government authorities to get their hands on it. Because the access threshold is so low, there were more than 320,000 of these requests made of telecommunications companies in the last financial year, rubber stamped without a single warrant being issued. We don’t know how many times ASIO or the Australian Signals Directorate vacuum up this material, because they are exempt from the reporting requirements that apply to local governments or the federal police.
One such euphemism is that metadata snooping is about the “envelope” and not the content, and that these envelopes are so innocuous they should be open season for warrantless access by an unknown number of agencies. It’s a deliberately deceptive metaphor.
In 2009, German Green party politician Malte Spitz subpoenaed six months worth of these envelopes from his service provider, and mashed them into a google map. The result is breathtaking: you can watch Spitz’ entire life unfold in real time; everywhere he goes, who he calls, when he sends text messages, where he sleeps, when he jumps on a train.
Now imagine all of us mapped into this fine-grained simulacrum of social life on an industrial planet, all our social interactions through many degrees of separation, swarming around like microchipped pets under a gargantuan microscope.
Anyone who downplays the importance of assembling billions of these metadata “envelopes” either doesn’t understand the technology, or hopes to impede your understanding. We expect it from Brandis. We could have done without it from what remains of the Labor party.
The PM program iiNet attacks the ‘fallacy’ of metadata and mandatory data retention highlights both the fallacy of Ms Plibersek’s analysis and the cost of storing vast amounts of metadata.
It provides:
MARK COLVIN: The country’s second-biggest internet service provider iiNet has criticised the growing number of law enforcement agencies pushing for the introduction of a mandatory data retention regime.
The Federal, NT (Northern Territory), Victorian and WA (Western Australia) police forces have all recently supported the introduction of a scheme which would store the internet use of citizens for two years.
iiNet says assurances that it’s only metadata, not content, that would be stored are a fallacy, and the system would cost its internet users $60 million.
Will Ockenden reports.
WILL OCKENDEN: The country’s second-largest ISP iiNet has hit out at a growing number of Australian police forces pushing for the introduction of mandatory data retention policies.
STEVE DALBY: They want to keep everything on everybody, and we just think that’s a massive overkill.
WILL OCKENDEN: Steve Dalby is the chief regulatory officer at iiNet.
STEVE DALBY: At the moment they can get a warrant or a court order or some other appropriate instrument and request us to record information on people of interest.
WILL OCKENDEN: That lasts for 90 days, and the data is recorded and stored.
STEVE DALBY: It’s a long way from recording information about a person of interest to recording everything about everybody in Australia.
WILL OCKENDEN: In the wake of the leaks from whistleblower Edward Snowden, a Senate Committee is reviewing surveillance legislation.
The submissions from law enforcement and intelligence agencies have supported the idea of making ISPs store internet data for two years.
STEVE DALBY: Our concern is the commentary that we’ve seen from Northern Territory Police and other police forces – including the Attorney-General’s Department in the previous government – was that we should keep everything on everybody.
WILL OCKENDEN: Interception of Australian’s telecommunications data is already widespread.
Last financial year around 40 government departments and agencies made nearly 300,000 requests for telecommunications metadata for a “law enforcement purpose”.
While police agencies like the Australian Federal Police feature heavily, the same access laws have been used by the RSPCA (Royal Society for the Prevention of Cruelty to Animals) and – increasingly – local councils.
STEVE DALBY: I don’t disagree with the argument that law enforcement needs the tools to do their job properly. There’s got to be a bit more evidence about people misusing telecommunications infrastructure to commit crimes or do something illegal, and then you focus in on those people. It’s just an overreach and an overkill to store the calls and the website activity of my 12-year-old niece.
WILL OCKENDEN: In the submission, iiNet questions the practicality of storing internet use data as more and more devices end up online.
STEVE DALBY: If we have to store all that information for two years, we’d have to build a new data centre, which we’ve estimated to cost about $60 million.
WILL OCKENDEN: So that would be the once-off fee which would then presumably be passed back on to consumers. What would the yearly cost be?
STEVE DALBY: Internet usage is growing daily. Sixty million might be enough for today’s traffic, but in the future it won’t be sufficient.
WILL OCKENDEN: The Federal Government is not currently considering any proposal for mandatory data retention.
Privacy activists hoping for Labor support are unlikely to find happiness. While it isn’t her portfolio, Labor’s shadow foreign minister Tanya Plibersek on Sunday told Sky News that mandatory data retention was essential.
TANYA PLIBERSEK: We always need to balance the expectations people have of living in a democratic and open society, but I certainly want to make it as easy for security agencies to do their job of protecting Australians from threat as we can.
I think there is a misconception in the Australian public about the sort of data that’s retained in these circumstances. I think some people imagine that security agencies can go back and listen to the phone call you made to your mum 18 months ago about what time you’re going to be home for dinner.
The information that is kept in these circumstances is basically, you could describe it as the envelope that the message comes in; who called whom and when.
People describe it as keeping the haystack so you can go back and look for the needle afterwards.
WILL OCKENDEN: But as PM broadcast last month, telephone metadata shows far more about an individual life than is often admitted.
Stanford University graduate student Jonathan Mayer runs a project called Metaphone, where the phone metadata of volunteers is tracked.
JONATHAN MAYER: We were able to learn about medical conditions, we were able to learn about gun ownership, we were able to learn about religious denominations. I think our latest results substantially undercut the view that ‘it’s just metadata’.
WILL OCKENDEN: It’s a sentiment Steve Dalby from iiNet agrees with.
STEVE DALBY: You can’t put the word ‘just’ in front of it and pretend that it means something less than what it actually is. Metadata is the address details of the content. So if you know the address details of the content, you can see what the content is.
MARK COLVIN: Steve Dalby from iiNet ending Will Ockenden’s report.
Itnews also covers iinet’s critisism about untargeted collection of meta data here.
