Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
FP7-SEC-2007-217862 DETECTER Detection Technologies, Terrorism, Ethics and Human Rights Collaborative Project Meeting on Data Mining, Human Rights and Ethics. Report on meeting. D03.1 Due date of deliverable: 31.6.2010 Actual submission date: 22.6.2010 Start date of project: 1.12.2008 Duration: 36 months Work Package number and lead: WP06 Dr. Daniel Moeckli Author(s): James Thurman, University of Zurich Project co-funded by the European Commission within the Seventh Framework Programme (2002-2006) Dissemination Level PU PP RE CO Public Restricted to other programme participants (including the Commission Services) Restricted to a group specified by the consortium (including the Commission Services) Confidential, only for members of the consortium (including the Commission Services) X DETECTER Project Meeting 3 Report Thematic Programme: 10th – 11th June 2010 The thematic programme began with a series of panels that were held on 10 June 2010. The first panel provided an introduction to data mining and its applications in law enforcement and counter-terrorism in particular. The aim of the second panel was to provide law enforcement and intelligence perspectives on counter-terrorism, common problems in intelligence work, and the role that data mining may play in addressing or contributing to those problems. The third panel provided an introduction to data protection law and its implications for data mining. The fourth and final panel addressed human rights issues posed by data mining in counterterrorism and proposed technical solutions for the preservation of privacy. Panel 1 – “What is Data Mining?” Stephen E. Fienberg, Department of Statistics, Machine Learning Department, Cylab, and i-Lab, Carnegie Mellon University Prof. Feinberg began by offering a definition of data mining and providing examples of different types of data mining functions. He then explained the notion of machine learning and its relation to data mining. He explained that machine learning can be used in fraud detection as well as in the detection of terrorists. He focused on systems designed to detect suspicious individuals in specific environments, such as airports. Fienberg compared this application of information technology to the use of the polygraph, which also represented a form of technology that detects deception. He argued that the efficacy of the polygraph had never been adequately established, noting that an attempt by the US National Academy of Sciences to do so had limited its attention to a selected number of studies, all of which had methodological flaws. Fienberg questioned the efficacy of the US programmes “FAST” and “SPOT”, and emphasized the near-impossibility of developing adequate models for rare events. Above all, Fienberg stressed the need for systematic evaluation and careful experiments for testing any kind of technology or system before relying on it in practice. 2 Colleen McCue, SPADAC, Inc. Dr. McCue began by stressing the importance of understanding analysis as a process. She presented four different process models that may be applied to data mining activities. McCue’s presentation also highlighted the problems of dealing with rare events. In her work she used data mining as a tool for determining how best to position police assets in anticipation of crime. Allocating assets so as to increase police presence where a particular incident is expected, for example, might help to prevent crime. She provided two examples of what she considered to be effective data analysis. One involved the application of supervised learning to the problem of random gun fire on New Year’s Eve. Data analysis was used to identify the times and places where the most incidents occurred. This information permitted local police to deploy officers strategically, resulting in a 47% reduction in the number of reported incidents and a reduction in personnel costs. The second example concerned the use of unsupervised learning in a “hostile surveillance” situation (i.e. a situation in which people watched and monitored a place or facility at which they were planning to commit a future crime). Data was derived from reports of suspicious activity around a particular location. The activities were assessed in terms of their riskiness. For instance, approaching a guard or trying to seek admission to the place were rated as high risk, since these were highly conspicuous activities which drew the attention of security personnel and increased the possibility that the person would be apprehended. Analysis of the data indicated that people’s activities became more risky over time. Mapping the activities spatially in relation to the facility also revealed that incidents began to gravitate towards a particular part of the facility. This helped analysts to predict where a security incident might occur and what the parties might be planning. McCue pointed out that this kind of analysis focuses on the behaviour rather than the characteristics of targets. Overall she sees her approach as one that seeks to “leverage predictive analytics in support of meaningful operationally relevant information-based tactics, strategy, and policy”. The moderated discussion and subsequent floor discussion raised the following points: There are difficulties providing adequate evaluation of programmes. Any system should be subject to extensive testing to demonstrate its effectiveness before being deployed in the field. Interest in using data mining in counter-terrorism seems to stem from its success in commercial settings, but it is a mistake to think that just because it succeeds in one setting, it will automatically be successful in another. Also, the risk of harm in commercial settings is simply not as significant as it is in counter-terrorism. Confirmation of why certain behaviour is occurring in a particular context may be more important than the mere discovery that such behaviour is occurring. Before trying to combine data, one should ask whether adding more data will provide any real additional benefit. Application of data mining in counter-terrorism will always encounter the problem of the infrequency of terrorist events. 3 Prevention of terrorist attacks may be an unrealistic goal, and it may therefore be better to aim for something else. Focus on behaviour—and detectable behaviour specifically—may provide a better operational approach than looking at personal characteristics. When data mining programmes combine databases there will inevitably be substantial errors of identification. Such errors may also arise when different countries identify individuals in different ways and then share information. This has implications for the effectiveness of data mining projects but also for privacy, as people may be linked with information that doesn’t pertain to them. The notion that we can discover new patterns that we’ve never seen before simply by linking more databases is flawed. Much of the discussion about data mining in counter-terrorism is essentially concerned with ensuring a rational approach to risk management. We have to ask ourselves: What are acceptable levels of risk? What are the economic and social costs of increasing security only marginally? Panel 2 – “Fighting Terrorism – Law Enforcement & Intelligence Perspectives” Sam Lincoln, ex-UK military intelligence officer and currently the UK Chief Surveillance Inspector, Office of Surveillance Commissioners (presenting in a private capacity) Sam Lincoln discussed the challenges facing intelligence officers, the role of technology and the importance of adhering to the law when engaged in intelligence activities. He argued that solid human analysis and critical thinking should be the basis for intelligence work. Technology could provide useful tools for intelligence but ultimately the “human in the loop” was key. He expressed concern that technology had a seductive appeal—presenting “pretty pictures” or easy solutions— to which decision-makers may too often surrender. He was concerned that we may increasingly rely too much on technology. For example, we allow CCTV cameras to replace community policing. Additionally, human bias or preconceptions as well as context-related analytic failures could undermine good intelligence work. The use of any technology is unlikely to overcome such failures. He repeated a challenge to data mining and profiling that had been expressed earlier: In many cases, the individuals flagged up are those who have made mistakes. It is therefore questionable whether a profile or model based on their characteristics or behaviour would be useful. He also noted that we have to be careful not only in the way that we obtain a profile but in the way we apply the profile once we have it. He was uncertain as to whether these kinds of details would always be understood by the rank and file of law enforcement who often don’t have 4 the time to deliberate. If data mining or technology-generated results were too appealing, the background issues might be forgotten. With respect to the privacy debate, he stressed that the fact that an action or measure becomes technologically possible does not by itself mean it should be permitted. The fact that current technology often operates automatically means that it sometimes collects information without being asked to do so, or without being asked to do so in a specific way. As a result, the user often ends up getting more information than he or she actually needs. This superfluous information is increasingly retained on databases “just in case”. Retaining information forever is unlikely to be acceptable in the future, given the growing and justified concern with the implications for privacy. There is a tendency amongst law-enforcement officers to continue surveillance for long periods in order to obtain enough evidence for a criminal conviction. Lincoln argued that it was important to consider the privacy of individuals whose information or communications get sucked up with those of the individual targeted. However, he also argued that a problem with the privacy debate was that it was dominated by extreme positions with few occupying the middle ground and that it was often being driven by the media. Christopher Westphal, Visual Analytics, Inc. Chris Westphal of Visual Analytics, Inc. spoke about law enforcement tasks from the perspective of an independent data analysis contractor. Some of the major themes of his presentation included the importance of having a sound business process in place and providing analysts with appropriate training. Changes in business process could take the form of altering collection practices or the methods that analysts use when examining and analyzing collected data. Westphal argued that too often no one thought about why data was being collected in the first place or how it would be used, consumed, and analyzed. Such things should be thought about in advance in order to ensure that the data would best serve its intended purpose and to address data quality concerns. He pointed out, for instance, that sometimes simply the design of the input forms used for collection could play a very significant role in these endeavours. He also discussed efforts to achieve standard-setting in the US, such as the National Information Exchange Model (NIEM) standard. These kinds of efforts could provide standard layer-on interfaces that would permit users to query across multiple government databases in a consistent and reliable manner since the use of a common schema would ensure that data would all be structured in the same way. Standard-setting initiatives, however, had not addressed the content of the data, how it was being managed, shared, integrated, and so on. He contended that the quality control mechanisms that are in place after collection could be improved substantially: Often information entered into the database was not verified in any way, there were inconsistencies such as typos and misspellings, incorrect data, incomplete data, etc. These quality issues would inevitably affect the reliability of results. Business processes, however, could also include the methods that analysts apply. He therefore thought the methodology should be included in analyst training. There 5 were too many people sitting in front of multiple databases who were told to simply find something of interest but had no clue what to do. The sort of analysis that Westphal is usually involved in concerns looking for connections between data points across multiple databases. Often one analytic task associated with this approach concerns process resolution – taking raw data and extracting the basic process structure to determine essentially how the data points are related. Part of process resolution will often include resolving entities where the same entity may appear multiple times in the raw data. Entity resolution did raise privacy issues, he pointed out, but once the methods were in place, the entities could then be anonymized so that the analyst would not see who they are dealing with initially. Westphal suggested that this kind of safeguard might be in use with the exchange of flight passenger data between the US & EU. Querying across multiple databases also allowed network and process data to be combined with various referential data. This provided additional context and could help the analyst to prioritize in their efforts to extract further details. Analysts, however, generally did not have the time to go into 30 different databases and run checks on various leads; therefore, the goal was to automate many of these processes. One instance in which he saw good potential for automation was in the discovery of known patterns. But this potential was often not realised, as discovered patterns often were not communicated within the community or from one organization to another. Westphal also stressed the necessity of having a human in the loop – there was always an exception to the pattern and exceptions to the exceptions, which only human intelligence could spot. The points of his conclusions were as follows: • It is critical to have a human in the loop for decision-making. There are always exceptions to patterns, and often exceptions to the exceptions. A human can confirm when there is an anomaly and determine if it is “actionable”. Often data is wrought with inconsistencies and therefore, until a “pattern” can be deemed reliable, a human decision-maker should be involved in the process. • Processes should be automated where possible, for instance where data mining results are confirmed by existing operational knowledge. There are many meta-data and value added calculations that can be performed on data. These can often be automated (e.g., checking the OFAC/SDN, the SSDMI, Most Wanted, etc). The results from these scans can then be incorporated into the analytical process and their value weighted to determine what types of patterns (or inconsistencies) are viable. • Systems that are capable of learning from the past and can communicate with one another (known as adaptive systems) should be used. These communicate patterns in a manner which allows them to be used in other investigations, domains, or systems. Thus, if we find a temporal pattern in, say, financial transactions, it should be investigated whether similar patterns can be found in, say, travel movements. At present the knowledge often stays with the individual analyst, rather than being incorporated into the overall systems – or communities – they serve. • The overall quality of data needs to be improved. Often, better collection instruments can be defined to help minimize inconsistencies. It was discussed that a business-process must change to address/deal with the 6 patterns/anomalies identified through analyses. If inconsistent data is a pattern, it needs to be addressed. • The exploitation of meta-data should be expanded. Often people/agencies do not even understand what is achievable from the data. Something as simple as a DATE has at least 15-20 different dimensions that can be exploited in the context of an analysis. There should be a master-list or reference source to describe what types of meta-data can be “extolled” from the raw source. • Non-standard sources should be incorporated in data analysis. We should not attempt to re-create the wheel for each system. A common system with upto-date data, fast response times, and consistent interfaces should be made available to all investigative systems. For example, the list of all “most wanted” people - which would contain data from FBI, DEA, Interpol, Europol, etc. • Different technologies and capabilities should be combined in order to generate better results and more efficient processes. In a nutshell, this would be a mash-up of technologies – ranging from databases, entity extractors, language translators, analytics, reporting, etc. Often, systems are stove-piped into specific functionality and their interoperability is (then) somewhat limited. • Improved training and instruction of analysts is needed. The moderated discussion and subsequent floor discussion raised the following points: Police and intelligence agencies can be transparent about the methods they employ in surveillance and analysis—often the types of technology are publicly known anyway. What is important is that who they are being used on and when they are being used remain secret. Governments should ensure that data collectors understand why the collection is being done and how the methods of collection affect the process, including where data collection has been outsourced to the private sector, e.g. Suspicious Activity Reports (SARs) in the financial sector. On the subject of data retention, the US has certain regulations in place for the intelligence community: data is subject to review every five years and must be purged if no longer needed. The current trend in the EU seems to be around 3 - 5 years, a timeframe which seems to be defined by the length of time it takes the legal system to conclude a trial. Most investigations rely on data that is at most 2 - 3 years old, so historical data is often not of real interest. However, in some counter-terrorism cases 10-year-old data is still relevant. The length of data retention has to be decided in each context. There has been a review of the data retention directive which examined how data is being used, for what kind of crimes, how old those crimes are, etc. Generally, data is used in relation to crimes that were committed within the last 6 months. So, therefore it does not make much sense to retain that data for more than 6 months. Perhaps the ideal data retention scheme for law enforcement and intelligence is one in which the relevant agencies have the option to keep the data longer 7 than the standard time, but have to provide an explanation as to why it needs to be retained beyond the usual limits. The issue of “office politics” in intelligence work is also significant, and it is important to keep in mind who is making the decisions. That is an issue that is outside the box in this discussion: we are assuming that the people who are conducting data mining are the ones making law-enforcement decisions on the basis of the results, but that is not the case. It is easy to see how data mining can be used in information processing, but it is unclear whether it has any place in providing contextual analysis or insight. The issue of the contextualization of data is significant. Often data ends up being passed around, perhaps even transferred abroad, and takes on a life of its own, appearing in different databases. EUROPOL and some Member States developed systems which work in law enforcement: Data is given a rating reflecting its reliability and that rating follows it wherever it goes. The question is whether this kind of system could be useful in intelligence. The culture of an organization is very important. It is usually determined by the most senior person and can affect the type of collection that is done and what goes into a database. Biases and preconceptions may also affect decisions about data collection and use. The issue of trust is equally important. In some cases, you will never be able to get one big database within a jurisdiction because the intelligence agencies will not want to let anyone else have access to their prime information source. On the other hand, excessively wide access can also be a problem, because often everyone thinks someone else will take responsibility for action points or issues that crop up; so, in the end, no one takes responsibility. Different agencies also have different preferences. Some agencies may have poor information systems (legacy systems), but be unwilling to change. In short, culture, design, and business process are all vitally important. Since there is a very strong cultural element that gets lost between technology and law, perhaps someone should be applying the methodology of ethnographical research to national security & law enforcement. Human intelligence (HUMINT) is likely to be far more effective than data mining in the search for terrorist suspects. Data mining does have a role in intelligence work, but there is no cast iron solution or holy grail. The question is how we can provide people with the best tools and training. We must not reach a point at which people begin to believe that data mining is going to do the analysis for them. Too often police and intelligence these days want to rely on covert intelligence gathering rather than a direct, up-front approach, which may be both more effective and more respectful of people’s rights. However, if they are to adopt an open approach to intelligence gathering, management must first be willing to accept the risks of getting it wrong in public. There are cases in which local law enforcement has to follow protocol with regard to individuals included on watch-lists, etc., despite the fact that they know the person is not of interest. This can end up radicalizing people. The problem of entity resolution becomes endemic as we accumulate more automated data merger applications. In many instances, no one is cleaning up databases, and there is no way to make corrections. It is a big problem that is destined to get worse and have significant negative impact on people’s lives. 8 Panel 3 – “Traditional Limits on Processing Personal Data: Data Protection” Hanspeter Thür, Swiss Federal Data Protection and Information Commissioner Hanspeter Thür provided an overview of the Swiss legal framework regulating data processing of data by police and intelligence agencies. He also outlined the possibilities for Swiss federal authorities to perform data mining and related analytic methods. He noted that the Swiss Federal Data Protection Act did not cover the processing of data in the context of criminal investigations; in that area the law of criminal procedure applied. Thus, decisions about the use of data mining tools had to be made under procedural law and in some cases would require the involvement of a judge. Processing personal data by federal bodies in Switzerland requires explicit statutory authority. In addition, any processing had to be in conformity with the principles of finality and proportionality. Thür described two statutes which provide the Federal Intelligence Service with some authority to process personal data: the Bundesgesetz über die Zuständigkeiten im Bereich des zivilen Nachrichtendienstes of 3 October 2008 and the Bundesgesetz über Massnahmen zur Wahrung der inneren Sicherheit of 21 March 1997. According to Thür, the first law allowed the Federal Intelligence Service to search and evaluate information from abroad—most of which is acquired by satellite—on behalf of the administration and Federal Council. The Service may process personal data, including sensitive data and personality profiles without informing the data subject. This sort of processing might take forms that could be considered “data mining”. The second law, according to Thür, permitted the Federal Intelligence Service to search information necessary for the accomplishment of aims specified in the statute. Processing may take place without the knowledge of the data subject. Activities under this statute could include: 1) use of publicly accessible sources; 2) information requests; 3) consultation of documents located abroad; 4) the reception and exploitation of communications; 5) inquiries about the identity or place of residence of persons; 6) observations or facts including photographic or sound recordings conducted in freely accessible locations; 7) collection of information pertaining to the location and contacts of persons. Searches conducted by the intelligence service are mostly done on the basis of the name of a person or organization. Such activity was not considered data mining. 9 These statutes did not provide a basis for conducting a profile-based search (“Rasterfahndung”). Such an action would have to be contemplated by the law of criminal procedure and would be regulated by that body of law. Bénédicte Havelange, Office of the European Data Protection Supervisor (EDPS) Bénédicte Havelange began by speaking about the missions of the EDPS and noted that the EDPS saw data protection law not only as a limitation but also a roadmap to successful data processing: Often principles of data protection were also principles of sound data management. She contended that data mining put the principles of data protection to the ultimate test because it often went much further than other forms of data processing. Two of the main issues which she felt illustrated the profound impact that data mining had were the following: 1) Data mining could be based on existing databases; but very often those databases were constructed for another purpose originally (e.g. PNR data). The use of data mining in these instances thus flew in the face of the purpose limitation under data protection law. Additionally, it often made it difficult to know what law applied since there would be a conflation of processing for commercial purposes and law enforcement purposes. 2) Data mining contributed massively to the establishment of a surveillance society. It could overturn the presumption of innocence by placing everyone under surveillance. She emphasized three points about the surveillance society in her presentation: 1) In a democracy, citizens should be able to scrutinize the intentions and actions of the government, but not the other way around. Constant and generalized monitoring of individuals should not be allowed. 2) Point 1 applied even when we were fighting terrorism and serious crime. 3) There were limitations to most fundamental rights, but they must be established by law, have legitimate aims, and be proportionate. Havelange also addressed the implications of data protection as a fundamental right. Fundamental rights were universal, which meant that data protection rights applied to all persons and not only EU citizens. Additionally, data protection applied independently of any actual harm or the risk of harm. As a fundamental right, violations of data protection also needed to be provided with some form of redress. Lastly, she contended that data protection required the supervision of an independent authority. She noted that there were often tricky problems with the requirements for purpose definition and limitation under data protection law that regulate data mining. The EDPS was often told that those using data mining programmes often would not know beforehand what exactly they were looking for. For this reason, the EDPS was frequently confronted with very vague statements of the purposes of a suggested operation. These kinds of statements were not really consistent with the purpose 10 limitation, Havelange argued. She noted that this kind of issue was highlighted in an EDPS opinion from 2 years ago with respect to the use of risk assessment profiles (EU-PNR Opinion). The proportionality of data mining was also frequently questionable. Proportionality meant both that the processing that takes place be proportionate to its stated aim and that the data relied upon should be proportionate and adequate to that aim. The first principle entailed that massive data mining should in principle not be conducted simply to fight petty crime. She also noted with respect to reliance on profiles that the European Network of Independent Experts for Human Rights found that the development of profiles could only be proportionate where there was a significant demonstration that there was a connection between the profile characteristics and the risk it wanted to address. Such a demonstration so far had not been made convincingly with regard to terrorism. She did not address the issue of data retention since that subject had been discussed extensively in the previous panels, but she argued that the retention of data should be limited, and that the algorithm or programme should be constantly revised. She also argued that some level of transparency should always be ensured – whether visà-vis parliaments, data protection authorities or other relevant regulators. And lastly, she noted that data mining also presents particular challenges for the provision of redress and the contestation of the results of such programmes. There might be considerable reaction on the part of the public if individuals were singled out incorrectly by data mining and detained or interrogated. Herbert Burkert, Research Center for Information Law, University of St. Gallen Herbert Burkert began by spelling out the “leading escape routes” that would permit the processing of personal data, including data mining, under data protection law. These were: 1) when the processing was authorized by law BUT the law had to a. contain measures to safeguard the interests of the persons concerned; b. observe the principle of proportionality; c. contain redress measures, e.g. notification, etc.; d. be necessary in a democratic society; 2) where there was consent BUT a. there were areas where consent is not possible – e.g. public sector b. the consent had to be informed c. the consent had to come prior to the processing 3) on the basis of “collateral processing” – e.g. while administering a contract, etc. BUT 11 a. there would always be a question of whether the data had been lawfully collected; b. it would have to meet the purpose limitation requirements; c. there were problems with the quality of data; 4) where we were not dealing with “personal data” – i.e. the data were anonymous and it could not be re-personalized. Burkert then went on to enlarge on considerations important to a project such as DETECTER . He pointed out that law was a dynamic system and he argued that it was necessary for a project like DETECTER to see law as a whole process, a cycle, examining 1) how law reacts to technological changes; 2) how technology in turn reacts to changes in law; and 3) how law and politics react to these changes. He suggested, for example, that one could take legal decisions on data mining and perform some data mining on the decisions to see if a pattern was emerging. He then turned to the specific example of the German case on data retention, which was decided by the German Constitutional Court. This decision held that storage as such for a limited time was not a problem; the problem lay with the particular use of the stored data. There had to be a clear description of the use, security measures, and the installation of a redress procedure—in part to address the issue of false positives. The decision also established that it was possible to demand certain contributions from the private sector for the public sector. However, Burkert thought it was unclear how this one case could be used to predict the future direction of the law on data processing. For example, he argued that the Court’s holding on “storage as such” reflected the fact that the Court did not want to pre-empt the ECJ on that issue. Trends that Burkert saw emerging included the privatization of security and the emergence of hybrid institutions, referred to as public-private partnerships. These developments, he contended, were of questionable transparency and accountability. Technology was heading toward the development of real-time machine-based “triage” systems – systems that no longer just select but also prioritize intervention. He foresaw the following reactions to these kinds of developments: 1) the establishment of NGOs that were specialized in security issues, not just human rights organizations that look at impacts but also NGOs that wanted to be involved in the design process of security systems; 2) the development of devices that would secure programme authorization oversight of these software and organizational mechanisms; 3) the development of real-time auditing systems that were designed in such a way that they could not be manipulated; 4) the establishment of strict liability rules for false positives and 5) far stronger involvement of civil society in the design of security. He called for a more holistic view of the limits in the designs of systems like data mining. He noted that transparency played an essential role in data protection and needed to be extended not only to freedom of information or access to documents but also had to reveal the structure of the providers of data mining. Lastly, he noted that our current conceptualization of security was problematic for a number of reasons and suggested that a re-conceptualization of safety and security in terms of a global public good was in order. 12 The moderated discussion and subsequent floor discussion raised the following points: The EDPS issued an Opinion in 2008 on the EU PNR proposals that is listed on its website. The EDPS identified many issues with the proposal. The proposal was listed as having a dual purpose: to identify persons involved in terrorism or organized crime and to locate persons who might fall within the criteria of the proposal. This was too vague. In addition, there was simply too little solid evidence that such systems were effective and therefore necessary. The risk assessment indicators that would be used to generate the risk profiles was not revealed, which cast doubt on the proportionality of the measure, as the consequences for an individual positively identified were significant, e.g. being prevented from boarding a flight. Furthermore, the data were initially collected for a commercial purpose but then used for a law enforcement purpose, which made it unclear what legislation would apply. Under the plan, PIUs would collate data and send it to authorities. It was not clear whether these PIUs would be public or private entities. It is difficult to assess proportionality of data mining when it is used in counter-terrorism. Assessments should be based on a sliding scale which takes a range of factors into consideration. One cannot predict what a court will determine to be inconsistent with proportionality. But the fact that it is open to interpretation keeps the law flexible. There can be no proportionality where core values of democratic society are undermined, as with general surveillance. One has to consider values. The human rights framework is not a magic wand, it is not like a computer programme: It involves a debate of competing interests. Proportionality also requires that a measure achieves its aim. Too often we discover that the old measures already in place do not work, but we do not get rid of them, we just heap new ones on top. Since police agencies do not have much experience with terrorism, it makes it difficult to know what works and what does not. Do we need to change data protection laws in order to promote transparency? Is it time to get rid of the exception for national security? We should question the extent to which secrecy is contributing to security in any single case. Some of the same issues arise with encryption: if you have to keep the algorithm secret, there is something wrong with the encryption system. The secrecy of the algorithm can be broken far more easily than the algorithm itself. And police work is like any other line of work. Secrecy is to some extent a kind of rationalization, an economizing device; it gives you a chance to do more with less since you have a time advantage. Some national security services want to have a broader discussion. Data protection authorities provide a default switch for society. In the past, you had to justify transparency; nowadays it is secrecy that must be justified. Similarly, people who defend data mining now have to give an account of its effectiveness. It is no longer taken for granted that it works. The interaction of institutions and their function in this sector is also an area which merits further attention. Lawyers & legislators once thought it was enough to have access to certain documents but that is not everything. 13 Generally, transparency concerns are partly satisfied by being provided with general information about what intelligence agencies are doing. Having legislation that spells out what agencies can do also provides transparency. Although it is often mentioned in discussions on surveillance, the model of the Panopticon may not be apt any longer. Firstly, the model of the Panopticon assumes that there is one observer with one point of view but in reality information is not unified in this way. Secondly, the claim that surveillance overturns the presumption of innocence should be nuanced. For instance, if there is a facility under constant surveillance, anyone passing by is caught on camera. In that scenario, it would be wrong to claim that everyone is under suspicion or has lost the presumption of innocence. Such a scenario differs greatly from one in which a person is singled out and subjected to a great deal of surveillance on very slim grounds. In one case maybe there is a threat to the presumption of innocence but in many cases there is not. UK discourse often refers to the profusion of CCTV cameras that people can see every day. But these cameras are very often trained on public places where anyone can conduct surveillance if they want to. So it does not seem useful to assume that all uses of surveillance serve the Panopticon model. And it is important to distinguish between surveillance of public places and private places. The pertinence of the Panopticon model is the feeling that one is being watched all the time. It does not matter that it is being done by different people. There is a principle of informational power and control that is often lost in the transatlantic discussion. Every individual has certain expectations and would want to have some remedy if those expectations are not respected. It is the same situation with Facebook: I may like to be able to advertise myself, but at some point I would like to have the ability to remove the information. Privacy, just like security, is not a binary thing. It is rather a scaled thing. Technology is constantly expanding our capabilities. The question of whether we should develop a particular capability or not stimulates the kinds of debates that we are having now. Those debates in turn lead to the construction of frameworks that become constraints in some way on the misuse of technology. So much of what the technology can do is positive that we would be luddites to say “let’s uninvent the technology”. We have data today that are made available in ways that we cannot envision and it is important that we consider the implications of losing control in the future. The problem is not, for example, that someone is an exhibitionist today, and cannot get a job in 10 years because of what they did; it is the fact that they never gave Facebook the permission to share their data and had no idea how it was going to be used because there was no real contract that was understandable between the parties. We need to consider both the right to be recompensed if data are misused and the right to have data repaired or corrected. We know that we are now propagating errors. How will we empower citizens to demand that the records be fixed? This is a problem not so much for government but for the private sector. EU-level databases are trying to address the issue of data being spread around to other databases by imposing obligations to send not only the data itself but subsequent corrections of the data. This helps to ensure that sound 14 and traceable audit trails are established. However, it remains to be seen how this will work in practice. It is important to keep in mind the purpose of a new technological measure. Policy-makers are often dazzled by the technical things. Technology is part of the solution but should be put in context. We have to ask what operational decisions the technology is meant to support. Will new information provide any solid basis for operational decisions? Will we have the resources to deal with the new information and implement operational action? EU data protection law applies to both public and private sector. There are more exceptions for the public sector, but that may be changing since more public-private partnerships are emerging. Section 55 of the UK Data Protection Act makes it a criminal offense to sell data, but the fines are not high enough to stop this practice and so data handlers treat it as a business expense. The fact that everything that happens in a public place is automatically becoming public domain nowadays raises further questions. Questions that need to be addressed are: What use is going to be made of the data afterwards? Who is the owner of the data? Who do we prosecute? If there is a loophole, it will be exploited. There is a section of the UN that is working on a recommendation addressing the cooperation of the public and private sectors in law enforcement work. So, maybe next year there will be something published in this field. There is a long tradition of dealing with political violence in Europe, but we forget the lessons of the past. Panel 4 – “Data Mining and Human Rights: Conflicts and Solutions” Gus Hosein, London School of Economics/ Privacy International Gus Hosein spoke about the role of NGOs in the human rights debate surrounding government data mining measures. He argued that politics could not be removed from the counter-terrorism sphere. For this reason, NGOs had to play the political game and be “unreasonable”. NGOs were essentially locked in a competition against the unreasonable arguments of policy makers for the hearts and minds of individuals. Additionally, NGOs had to get people’s attention because otherwise they would have no impact. Only once they had people’s attention could they then sit down and have a reasoned discussion about legal issues, etc. Hosein also suggested that reasoned arguments concerning technological issues were ineffective since politicians understand neither the technology nor the law. He also complained that too often NGOs fell into arguments concerning the efficacy of technological proposals when they should really be talking about whether the proposal was really something we want or should have. In concluding, he focused on the following points: 15 1) All options should remain on the table, including being unreasonable; until we were unreasonable, people would forget about the extremes and the extremes could come to sound reasonable. People forgot that other options were available. 2) We have to abandon the quest for perfection. Politicians are still looking for the perfect solution, but there isn’t one. 3) We have to get on top of the collection issue. Too often the focus is placed on the use of the data without addressing the question of why the data needs to be collected in the first place. 4) We have to deal with the accountability and transparency issues. A lot of the abuses we have seen began as data integrity problems. 5) This debate is not about technology assessment or technology policy, it is about what world we are capable of building. Ramon Barquin, Barquin International Ramon Barquin indicated that he was a believer in the ability to use technology to improve the human condition. The question was how we use it and best harness its power. He argued that there were movements in the right direction towards finding useful applications and ways to protect privacy in data mining. Data mining was something that held promise as a way of dealing with the explosion of bits and bytes that we faced and might provide beneficial new applications in health care, education, and logistics, among others. Yet, it faced the classic dual-use dilemma of being capable of both positive and negative applications. The question then became, how we could perform data mining and protect ourselves from the potential for harm? Barquin suggested that a starting point for privacy might involve an examination of what people consider to be sensitive information. In addition to the question of how touchy people were with regard to various types of information was the question of who “owns” the information where that information was generated or held by an institution. Barquin suggested that the Fair Information Practices provided an initial framework for the privacy discussion, but mentioned 4 basic, initial techniques that had been developed to attempt to preserve privacy within data mining applications. These were: 1) generalization; 2) de-identification and re-identification; 3) “anonymization” and 4) cryptography. He claimed that generally two approaches were relied upon: a randomization approach or a cryptographic approach. The first injected random “noise” to hide or disguise the data. The second often involved inter-operation of two different databases to prevent disclosure of personally identifiable information. 16 Barquin also examined the role of trust in privacy concerns related to data mining. In many instances, there was a crisis of trust vis-à-vis the government. He turned to the founders of the United States and the notion of a system of checks and balances for a suggestion that internal controls were needed. He also pointed to a set of principles developed by the European Group on Ethics in Science and New Technologies and to the Ten Commandments of Computer Ethics developed by the Computer Ethics Institute as sources of inspiration for notions of human dignity, the right to the integrity of the person, and the protection of personal data. In sum, Barquin claimed that the solution lay in identifying second order consequences of technology (in this case data mining), identifying those that are unintended, unanticipated, and undesirable, and then finding ways to reduce them. The moderated discussion and subsequent floor discussion raised the following points: We are currently undertaking the research to get us to where we have data mining tools that we can use, but we are not there yet. There is a desire to research and develop technology but what damage are we doing in the meantime? Everyone agrees we would like to have the end result, but it is not clear we can do it quickly enough without doing a lot of damage. Some would doubt that the end result is controllable or therefore desirable. The machine decision system may not be a real alternative. The alternative is probably to have a bundle of human decisions that is transformed into a hardware/ software system. You then have a set of repetitive human interventions that involve human decision-making. So, it is really just two kinds of decision-making. The second system is simply more adaptable. That kind of repeated re-examination is important. Government decisions about what technology to develop or contract out for & employ do not happen in a vacuum. The human rights framework has to be taken into account. Many governments have a formal acquisition process. It is essential to make acquisition authorities aware of these issues so that they are included in the contracts. Even beyond the human rights discussion, we also need to do self-criticism/ social criticism. Perhaps there are practices we seem to be engaging in voluntarily that may be weakening our sense of privacy. In other words, start a conversation with the average citizen who does not care about what NGOs or academics say and ask whether we are doing things in such a way that certain values are being squandered by us, not the governments. We should live in a world in which people can live as openly as possible but still have rights. There is the question whether there can be such a thing as corporate ethics. Some may argue that ethics is a matter for individuals. But unless we have 17 these debates, we are not going to converge on the kinds of rules that we need as a society. The description of boundaries is incredibly important: We are never going to be able to get the perfect solution. The metaphor of drug development may be illustrative: there is no perfect drug as they all have side effects. There are those side effects we know about and those we do not know about. And it is the ones that we do not know about that end up biting us in the long run. As long as we keep an open mind and have an open debate and understand the boundaries and work to find the mid-point, we have reason to be optimistic. The optimal spot will be fluid depending on the context. Even if we find the optimal solution, all the safeguards will be thrown out the window when the next terrorist attack comes, and the technology will remain. But it would be antidemocratic to develop a system that cannot be changed or is tamper-proof. It is definitely important for open societies and for citizens to become involved, but in a closed society, there is not that possibility. Yet, there are closed countries that are primary users of these technologies and are rapidly moving toward becoming suppliers of these technologies. It may be unclear whether the technology in these closed societies is coming from or going to countries that may or may not have had these debates. As long as societies exist where the debate cannot take place, the technology risks will always be present. It is the dual-use problem: Do you avoid selling anything to countries run by dictators? Dictatorial regimes will use any technology they can get their hands on. There is nothing to say that democratic societies or so-called democratic societies are immune to the same issues. Wrapping Up The thematic programme concluded with an open floor discussion held on the morning of 11 June 2010. The moderators decided to focus on four main themes: Definition of Data Mining, Approaches, Effectiveness, and Proportionality. The discussion yielded the following points: Definition One approach would be to simply take a definition from some other source and annotate it. The annotations could include information explaining why it is important you link it to a police agency or intelligence agency. In that way, you can enlarge the definition, and, at same time, you have a discussion of issues involved so that in the end you have mapped out the space. The narrow choice for the Federal Data Mining Reporting Act has been broadly recognized as not only overly narrow but exclusionary. The report from the Committee on Technical and Privacy Dimensions of Information for 18 Terrorism Prevention and Other National Goals featured a definition that was intentionally broad, but it also features a detailed appendix. It will be impossible to dig up everything. Some people will tell you certain programmes do not exist, despite the fact that others will be able to describe those very programmes to you. It is really the breadth of activities and not the narrowness that should be revealed, and from there it will be possible to identify issues that are comparable across the spectrum of activities. An all-encompassing definition is not needed for our purposes. One of the things that will come up is the connection with profiling and classificatory schemes. It is the way that stereotypes are applied to the data that is interesting. Some connection with automatically applied classificatory schemes would start to capture some of the things we go on to discuss. It is not the nature of a database per se but the way it is used in this context that is important. And maybe being explicit about the context will help to make choices about what to address and what to leave out. Perhaps we can find elements for a definition by focusing on the impacts on individuals. In other words, by identifying what elements of the programmes have implications for individual rights. For example, is the programme used to make predictions? If so, we would be more cautious as opposed to programmes that trace a known suspect. Is it profile-based? Does it involve abstract patterns? Is it based on one individual or a group of individuals? Is it applied case-by-case or generally applied? Is it only used within the scope of a single investigation? Many definitions focus on a predictive element. But with prediction, you lose the association and relationship elements which also would be a concern for privacy and civil liberties. The existing definition is broad enough but the reference to “usually large data sets” may not be apt. SARs (Suspicious Activity Reports), for example, can be very small data sets. To specify the scope of the data set may be too restrictive. Approaches The initial idea for the direction of the work package rests on the notion that we need to look at data mining in a holistic fashion. With data mining, we are dealing with a process that involves data handling, and it is perhaps necessary to address the different stages of data handling. The approach essentially sounds like a restatement of the fair information practices. The only things missing are subject access, and the lawfulness or purpose requirement—i.e., is there a clear and stated purpose and how is that lawful & necessary? A framework is very nice, but in an emergency situation, the rules go out the window. Things will come about that are unexpected, and there is always a political requirement to provide immediate answers. There is always the concern for avoiding the restriction of future uses. But we now have historic examples of what were clearly abuses, and, in some instances, laws were passed to circumvent previous legal safeguards. There is a difference where we have data for which no protection was initially offered 19 and data for which there was. We should distinguish between these two situations whether it is a legal guarantee or merely an implicit promise for protection. It is one thing to have regulations concerning data mining, and another to have the implementation of controls and enforcement—there you have the oversight mechanisms. We should also ask ourselves if we need some sort of licensing requirement for data mining and if so what would the procedure for such licensing should look like. Licensing would permit regulators to establish certain requirements as to how those kinds of operations may take place. Many of the things discussed here are already in the data protection directive. Perhaps the project could examine how data mining activities fit within the review of the current framework. Ensuring security may differ in data mining might and another field such as a credit scoring database. In other words, you could spell out where and how data mining fits in to the current framework rather than reinventing new principles. It may also be worth considering the creation of composite variables of other data. Sometimes people get into trouble by creating groups within their data. A Markle Foundation report has a section on being able to document audit trails, the need to know vs. the right to know, and also discusses emergency situations and develops protocols. The notion of allowing subject access requests sounds very nice in theory but in the counter-terrorism world, no one is going to want to allow that within 5 years. Dutch intelligence has a requirement to permit access within 5 years. Yet so far they have not told anyone, but rather keep extending it. It is possible to have indirect access as an alternative. Data protection authorities can do that on the data subject’s behalf. The data protection authorities can then reply that they have made the inquiry and made the necessary recommendations without specifying whether the subject’s data was actually being processed by police and intelligence authorities or not. And in some instances, data protection authorities discover that there are mistakes in the data on police or intelligence service databases. It is in the interest of everybody that that data be corrected. At least there is some authority looking into it. Effectiveness There seems to be some scepticism even from the side of law enforcement and intelligence as to whether data mining can be effective. It will probably be impossible to answer the question of the effectiveness of existing programmes, but maybe we should formulate a requirement that any new measure has to demonstrate its effectiveness on an evidentiary basis before being implemented (or developed). It is also significant to note that in some instances it seems that, regardless of whether the systems are effective or not, most of the alerts are not followed up because police simply do not have the resources to do so. It seems that in establishing criteria for effectiveness, the lowest point has to be that the measure is better than chance alone or randomness. The systems might be demonstrated in such a way that they seem effective, but the question is would you have the same results if it were done by chance alone. 20 Everything after that is a matter of balancing harms and benefits and proportionality. The predictive element of data mining should be questioned strongly. It is a myth that data mining can be used effectively for prediction. But the idea of searching for unsuspected or nontrivial results from a database should be retained. That element has some effective value in counter-terrorism. It is possible to use data mining to confirm gut feelings or standard procedures. It can also be used to anticipate- but not to predict. We can talk about the likelihood that something will happen but not who will be involved and other details. Any compelling applications in that space have not been forthcoming. The predictive functions are useful in terms of resource allocation but not in identifying suspects. If you have known patterns, those can be incorporated into an automated process. In those cases, the analysis was done long ago, and it is now a matter of monitoring. The developers of these systems contend that they are successful, so their claims should be included in the project analysis and examined. Even where techniques are developed that become part of the system, that does not mean that the activity stops there. You do not simply use the credit score coefficients that were developed 7 yrs ago to determine my credit worthiness now. They are updated. The question then becomes the validity of the ongoing uses of the data and the updating of that data. That is one thing that most people leave out. It is easy to implement an algorithm and say this is a good use of this database, but 20 agencies may then begin to use it and these new uses may never be examined. It is not just the successes that need to be examined, it is also all the failures and other negative and positive features. The counter-terrorism community is reluctant to admit failures because one never knows when the failures of the past will be the successes of the future. So much of the information is not in the public domain. The counter-terrorism community also tends not to admit successes for fear of compromising sources and methods. This information concerning effectiveness is the most sensitive information. Even if an operation is shuttered, it does not mean that we should not assess what was done for future work. That does not have to involve giving an account to the public but to some authority. Proportionality How can we be specific about the aims of counter-terrorism measures? What is then proportionate and what not? It seems that the scariness of terrorism and the potential scale of damage, rather than actual damage, drive the scope of the measures that are taken. Perhaps the exact actions in question should be specified when talking about proportionality—is it the set of information collected, the number of people who have access, how the information may be processed? Those are just 21 three domains, there may be others. There are different subsets of the problem that could be discussed. So one approach might be to look at the different stages of data handling in terms of proportionality. Should there not also be something about equality under the law when talking about proportionality? Proportionality when we are dealing with foreigners seems to be something different than when citizens are involved. The proportionality assessment might not hold to people’s risk perception. It will be necessary to inform the public that you are not going to take account of people’s perceptions. There is no such thing as a little bit of privacy. It is all or nothing with databases. It is an issue of access and who has access to information that is essentially public. That point tends to get lost in these discussions, and it needs to be put back in. So, you can talk about the risk of disclosure of information belonging to individuals who are not targets of an investigation. When you are searching through databases, there is a risk of individual disclosure, and once you have opened the door, you cannot close it. That is the problem that needs to be identified, and one also needs to distinguish between the use of databases to amass information about individual suspects as opposed to searching through databases to uncover new groups and then looking at the consequences of revealing that information. In theory we have some good examples. We have case law. In practice, there is perhaps less reason to believe that it will work that well. Scanning large groups of the population is a violation of the presumption of innocence. How far do we want to go and how far can we go without endangering liberal society? Under the European understanding, it is not only the distribution but the processing of data that can affect human rights. That is an added risk that can be taken into account. Maybe the solution to the proportionality problem would involve following the practice of the courts. If you go to case law of constitutional or the European courts, you will see that they ask themselves a number of questions. If we could emulate these questions, we could maybe distil proportionality factors. For example, one question would be how many people are affected? Proportionality is a relative thing. You also consider the purpose and questions of degree, i.e. to what extent personal elements are involved in this kind of processing. It is not an arithmetic exercise, it is a value exercise. The notion that processing in itself is a violation of human dignity raises huge issues for broad-scale data mining: What level of harm would justify that level of infringement? Is there a level of harm that would justify it? For many kinds of data mining operations, it seems to be something that repeatedly goes on in the background. It is not the case that there is a specific investigation with specific target. This makes such operations appear to be no more than fishing expeditions. That raises the question of whether these issues of proportionality and purpose limitation can ever be resolved. As soon as you have a specific investigation it is easy to justify. Maybe it goes back to the marketing application source of data mining. In marketing, you can look all over the place and explore what you could maybe do with the 22 information you discover, but that is just what you cannot do in a proportionality setting. There is this kind of mismatch between background scanning, looking for patterns and whether you can justify it to a court. Is it appropriate to rely on the commercial analogy for counter-terrorism applications of data mining? Fraud monitoring is a more apt model. The purpose is targeted but the actual data used changes over time. You can define the database you want to use but that may change since the data will change over time. There are no auxiliary purposes. Public health surveillance may also be more apt. In counter-terrorism, one instance is actionable. In the commercial setting, one instance is not interesting, you want thousands. A different scale and a different focus are involved. Many technologies used in the commercial setting would skip over these things because they are not geared to look for it. Simply running an algorithm is not going to find a cluster of terrorists. It can uncover something that does not make sense and deserves more attention, more resources. There is a huge psychological component to how we perceive numbers. Despite the relatively high number of people killed in car accidents, we have more of a sense that we have control when driving or being driven than is the case with terrorist incidents. There is also a sense that the numbers could grow if we do not do anything. Numbers must also be understood in context. 99.5 % of vessels pass through the Gulf of Aden safely, but we are throwing tons of resources at prevention of piracy on the high seas because we feel we have to draw a line in the sand. So maybe the allocation of resources is disproportionate to the real risk. 23