Download FP7-SEC-2007-217862 DETECTER Detection Technologies

FP7-SEC-2007-217862
DETECTER
Detection Technologies, Terrorism, Ethics and Human Rights
Collaborative Project
Meeting on Data Mining, Human Rights and Ethics.
Report on meeting.
D03.1
Due date of deliverable: 31.6.2010
Actual submission date: 22.6.2010
Start date of project: 1.12.2008
Duration: 36 months
Work Package number and lead: WP06 Dr. Daniel Moeckli
Author(s): James Thurman, University of Zurich
Project co-funded by the European Commission within the Seventh Framework Programme
(2002-2006)
Dissemination
Level
PU
PP
RE
CO
Public
Restricted to other programme participants (including the Commission Services)
Restricted to a group specified by the consortium (including the Commission
Services)
Confidential,
only for members of the consortium (including the Commission
Services)
X
DETECTER Project Meeting 3
Report
Thematic Programme:
10th – 11th June 2010
The thematic programme began with a series of panels that were held on 10 June
2010. The first panel provided an introduction to data mining and its applications in
law enforcement and counter-terrorism in particular. The aim of the second panel
was to provide law enforcement and intelligence perspectives on counter-terrorism,
common problems in intelligence work, and the role that data mining may play in
addressing or contributing to those problems. The third panel provided an
introduction to data protection law and its implications for data mining. The fourth
and final panel addressed human rights issues posed by data mining in counterterrorism and proposed technical solutions for the preservation of privacy.
Panel 1 – “What is Data Mining?”
Stephen E. Fienberg, Department of Statistics, Machine Learning
Department, Cylab, and i-Lab, Carnegie Mellon University
Prof. Feinberg began by offering a definition of data mining and providing examples
of different types of data mining functions. He then explained the notion of machine
learning and its relation to data mining. He explained that machine learning can be
used in fraud detection as well as in the detection of terrorists. He focused on
systems designed to detect suspicious individuals in specific environments, such as
airports. Fienberg compared this application of information technology to the use of
the polygraph, which also represented a form of technology that detects deception.
He argued that the efficacy of the polygraph had never been adequately established,
noting that an attempt by the US National Academy of Sciences to do so had limited
its attention to a selected number of studies, all of which had methodological flaws.
Fienberg questioned the efficacy of the US programmes “FAST” and “SPOT”, and
emphasized the near-impossibility of developing adequate models for rare events.
Above all, Fienberg stressed the need for systematic evaluation and careful
experiments for testing any kind of technology or system before relying on it in
practice.
2
Colleen McCue, SPADAC, Inc.
Dr. McCue began by stressing the importance of understanding analysis as a process.
She presented four different process models that may be applied to data mining
activities. McCue’s presentation also highlighted the problems of dealing with rare
events. In her work she used data mining as a tool for determining how best to
position police assets in anticipation of crime. Allocating assets so as to increase
police presence where a particular incident is expected, for example, might help to
prevent crime. She provided two examples of what she considered to be effective
data analysis. One involved the application of supervised learning to the problem of
random gun fire on New Year’s Eve. Data analysis was used to identify the times and
places where the most incidents occurred. This information permitted local police to
deploy officers strategically, resulting in a 47% reduction in the number of reported
incidents and a reduction in personnel costs. The second example concerned the use
of unsupervised learning in a “hostile surveillance” situation (i.e. a situation in which
people watched and monitored a place or facility at which they were planning to
commit a future crime). Data was derived from reports of suspicious activity around
a particular location. The activities were assessed in terms of their riskiness. For
instance, approaching a guard or trying to seek admission to the place were rated as
high risk, since these were highly conspicuous activities which drew the attention of
security personnel and increased the possibility that the person would be
apprehended. Analysis of the data indicated that people’s activities became more
risky over time. Mapping the activities spatially in relation to the facility also revealed
that incidents began to gravitate towards a particular part of the facility. This helped
analysts to predict where a security incident might occur and what the parties might
be planning. McCue pointed out that this kind of analysis focuses on the behaviour
rather than the characteristics of targets. Overall she sees her approach as one that
seeks to “leverage predictive analytics in support of meaningful operationally relevant
information-based tactics, strategy, and policy”.
The moderated discussion and subsequent floor discussion raised the following
points:

There are difficulties providing adequate evaluation of programmes. Any
system should be subject to extensive testing to demonstrate its
effectiveness before being deployed in the field.

Interest in using data mining in counter-terrorism seems to stem from its
success in commercial settings, but it is a mistake to think that just
because it succeeds in one setting, it will automatically be successful in
another. Also, the risk of harm in commercial settings is simply not as
significant as it is in counter-terrorism.

Confirmation of why certain behaviour is occurring in a particular context
may be more important than the mere discovery that such behaviour is
occurring.

Before trying to combine data, one should ask whether adding more data
will provide any real additional benefit.

Application of data mining in counter-terrorism will always encounter the
problem of the infrequency of terrorist events.
3

Prevention of terrorist attacks may be an unrealistic goal, and it may
therefore be better to aim for something else.

Focus on behaviour—and detectable behaviour specifically—may provide
a better operational approach than looking at personal characteristics.

When data mining programmes combine databases there will inevitably
be substantial errors of identification. Such errors may also arise when
different countries identify individuals in different ways and then share
information. This has implications for the effectiveness of data mining
projects but also for privacy, as people may be linked with information
that doesn’t pertain to them. The notion that we can discover new
patterns that we’ve never seen before simply by linking more databases is
flawed.

Much of the discussion about data mining in counter-terrorism is
essentially concerned with ensuring a rational approach to risk
management. We have to ask ourselves: What are acceptable levels of
risk? What are the economic and social costs of increasing security only
marginally?
Panel 2 – “Fighting Terrorism – Law Enforcement & Intelligence
Perspectives”
Sam Lincoln, ex-UK military intelligence officer and currently the UK Chief
Surveillance Inspector, Office of Surveillance Commissioners (presenting
in a private capacity)
Sam Lincoln discussed the challenges facing intelligence officers, the role of
technology and the importance of adhering to the law when engaged in intelligence
activities. He argued that solid human analysis and critical thinking should be the
basis for intelligence work. Technology could provide useful tools for intelligence but
ultimately the “human in the loop” was key. He expressed concern that technology
had a seductive appeal—presenting “pretty pictures” or easy solutions— to which
decision-makers may too often surrender. He was concerned that we may
increasingly rely too much on technology. For example, we allow CCTV cameras to
replace community policing. Additionally, human bias or preconceptions as well as
context-related analytic failures could undermine good intelligence work. The use of
any technology is unlikely to overcome such failures.
He repeated a challenge to data mining and profiling that had been expressed
earlier: In many cases, the individuals flagged up are those who have made
mistakes. It is therefore questionable whether a profile or model based on their
characteristics or behaviour would be useful. He also noted that we have to be
careful not only in the way that we obtain a profile but in the way we apply the
profile once we have it. He was uncertain as to whether these kinds of details would
always be understood by the rank and file of law enforcement who often don’t have
4
the time to deliberate. If data mining or technology-generated results were too
appealing, the background issues might be forgotten.
With respect to the privacy debate, he stressed that the fact that an action or
measure becomes technologically possible does not by itself mean it should be
permitted. The fact that current technology often operates automatically means that
it sometimes collects information without being asked to do so, or without being
asked to do so in a specific way. As a result, the user often ends up getting more
information than he or she actually needs. This superfluous information is
increasingly retained on databases “just in case”. Retaining information forever is
unlikely to be acceptable in the future, given the growing and justified concern with
the implications for privacy. There is a tendency amongst law-enforcement officers to
continue surveillance for long periods in order to obtain enough evidence for a
criminal conviction. Lincoln argued that it was important to consider the privacy of
individuals whose information or communications get sucked up with those of the
individual targeted. However, he also argued that a problem with the privacy debate
was that it was dominated by extreme positions with few occupying the middle
ground and that it was often being driven by the media.
Christopher Westphal, Visual Analytics, Inc.
Chris Westphal of Visual Analytics, Inc. spoke about law enforcement tasks from the
perspective of an independent data analysis contractor. Some of the major themes of
his presentation included the importance of having a sound business process in place
and providing analysts with appropriate training. Changes in business process could
take the form of altering collection practices or the methods that analysts use when
examining and analyzing collected data. Westphal argued that too often no one
thought about why data was being collected in the first place or how it would be
used, consumed, and analyzed. Such things should be thought about in advance in
order to ensure that the data would best serve its intended purpose and to address
data quality concerns. He pointed out, for instance, that sometimes simply the
design of the input forms used for collection could play a very significant role in
these endeavours. He also discussed efforts to achieve standard-setting in the US,
such as the National Information Exchange Model (NIEM) standard. These kinds of
efforts could provide standard layer-on interfaces that would permit users to query
across multiple government databases in a consistent and reliable manner since the
use of a common schema would ensure that data would all be structured in the same
way. Standard-setting initiatives, however, had not addressed the content of the
data, how it was being managed, shared, integrated, and so on. He contended that
the quality control mechanisms that are in place after collection could be improved
substantially: Often information entered into the database was not verified in any
way, there were inconsistencies such as typos and misspellings, incorrect data,
incomplete data, etc. These quality issues would inevitably affect the reliability of
results.
Business processes, however, could also include the methods that analysts apply. He
therefore thought the methodology should be included in analyst training. There
5
were too many people sitting in front of multiple databases who were told to simply
find something of interest but had no clue what to do.
The sort of analysis that Westphal is usually involved in concerns looking for
connections between data points across multiple databases. Often one analytic task
associated with this approach concerns process resolution – taking raw data and
extracting the basic process structure to determine essentially how the data points
are related. Part of process resolution will often include resolving entities where the
same entity may appear multiple times in the raw data. Entity resolution did raise
privacy issues, he pointed out, but once the methods were in place, the entities
could then be anonymized so that the analyst would not see who they are dealing
with initially. Westphal suggested that this kind of safeguard might be in use with the
exchange of flight passenger data between the US & EU.
Querying across multiple databases also allowed network and process data to be
combined with various referential data. This provided additional context and could
help the analyst to prioritize in their efforts to extract further details. Analysts,
however, generally did not have the time to go into 30 different databases and run
checks on various leads; therefore, the goal was to automate many of these
processes. One instance in which he saw good potential for automation was in the
discovery of known patterns. But this potential was often not realised, as discovered
patterns often were not communicated within the community or from one
organization to another. Westphal also stressed the necessity of having a human in
the loop – there was always an exception to the pattern and exceptions to the
exceptions, which only human intelligence could spot.
The points of his conclusions were as follows:
•
It is critical to have a human in the loop for decision-making. There are
always exceptions to patterns, and often exceptions to the exceptions. A
human can confirm when there is an anomaly and determine if it is
“actionable”. Often data is wrought with inconsistencies and therefore, until a
“pattern” can be deemed reliable, a human decision-maker should be
involved in the process.
•
Processes should be automated where possible, for instance where data
mining results are confirmed by existing operational knowledge. There are
many meta-data and value added calculations that can be performed on data.
These can often be automated (e.g., checking the OFAC/SDN, the SSDMI,
Most Wanted, etc). The results from these scans can then be incorporated
into the analytical process and their value weighted to determine what types
of patterns (or inconsistencies) are viable.
•
Systems that are capable of learning from the past and can communicate
with one another (known as adaptive systems) should be used. These
communicate patterns in a manner which allows them to be used in other
investigations, domains, or systems. Thus, if we find a temporal pattern in,
say, financial transactions, it should be investigated whether similar patterns
can be found in, say, travel movements. At present the knowledge often
stays with the individual analyst, rather than being incorporated into the
overall systems – or communities – they serve.
•
The overall quality of data needs to be improved. Often, better collection
instruments can be defined to help minimize inconsistencies. It was discussed
that a business-process must change to address/deal with the
6
patterns/anomalies identified through analyses. If inconsistent data is a
pattern, it needs to be addressed.
•
The exploitation of meta-data should be expanded. Often people/agencies do
not even understand what is achievable from the data. Something as simple
as a DATE has at least 15-20 different dimensions that can be exploited in the
context of an analysis. There should be a master-list or reference source to
describe what types of meta-data can be “extolled” from the raw source.
•
Non-standard sources should be incorporated in data analysis. We should not
attempt to re-create the wheel for each system. A common system with upto-date data, fast response times, and consistent interfaces should be made
available to all investigative systems. For example, the list of all “most
wanted” people - which would contain data from FBI, DEA, Interpol, Europol,
etc.
•
Different technologies and capabilities should be combined in order to
generate better results and more efficient processes. In a nutshell, this would
be a mash-up of technologies – ranging from databases, entity extractors,
language translators, analytics, reporting, etc. Often, systems are stove-piped
into specific functionality and their interoperability is (then) somewhat limited.
•
Improved training and instruction of analysts is needed.
The moderated discussion and subsequent floor discussion raised the following
points:





Police and intelligence agencies can be transparent about the methods they
employ in surveillance and analysis—often the types of technology are
publicly known anyway. What is important is that who they are being used
on and when they are being used remain secret.
Governments should ensure that data collectors understand why the
collection is being done and how the methods of collection affect the process,
including where data collection has been outsourced to the private sector,
e.g. Suspicious Activity Reports (SARs) in the financial sector.
On the subject of data retention, the US has certain regulations in place for
the intelligence community: data is subject to review every five years and
must be purged if no longer needed. The current trend in the EU seems to be
around 3 - 5 years, a timeframe which seems to be defined by the length of
time it takes the legal system to conclude a trial. Most investigations rely on
data that is at most 2 - 3 years old, so historical data is often not of real
interest. However, in some counter-terrorism cases 10-year-old data is still
relevant.
The length of data retention has to be decided in each context. There has
been a review of the data retention directive which examined how data is
being used, for what kind of crimes, how old those crimes are, etc.
Generally, data is used in relation to crimes that were committed within the
last 6 months. So, therefore it does not make much sense to retain that data
for more than 6 months.
Perhaps the ideal data retention scheme for law enforcement and intelligence
is one in which the relevant agencies have the option to keep the data longer
7









than the standard time, but have to provide an explanation as to why it
needs to be retained beyond the usual limits.
The issue of “office politics” in intelligence work is also significant, and it is
important to keep in mind who is making the decisions. That is an issue that
is outside the box in this discussion: we are assuming that the people who
are conducting data mining are the ones making law-enforcement decisions
on the basis of the results, but that is not the case.
It is easy to see how data mining can be used in information processing, but
it is unclear whether it has any place in providing contextual analysis or
insight.
The issue of the contextualization of data is significant. Often data ends up
being passed around, perhaps even transferred abroad, and takes on a life of
its own, appearing in different databases. EUROPOL and some Member States
developed systems which work in law enforcement: Data is given a rating
reflecting its reliability and that rating follows it wherever it goes. The
question is whether this kind of system could be useful in intelligence.
The culture of an organization is very important. It is usually determined by
the most senior person and can affect the type of collection that is done and
what goes into a database. Biases and preconceptions may also affect
decisions about data collection and use. The issue of trust is equally
important. In some cases, you will never be able to get one big database
within a jurisdiction because the intelligence agencies will not want to let
anyone else have access to their prime information source. On the other
hand, excessively wide access can also be a problem, because often everyone
thinks someone else will take responsibility for action points or issues that
crop up; so, in the end, no one takes responsibility. Different agencies also
have different preferences. Some agencies may have poor information
systems (legacy systems), but be unwilling to change. In short, culture,
design, and business process are all vitally important.
Since there is a very strong cultural element that gets lost between
technology and law, perhaps someone should be applying the methodology
of ethnographical research to national security & law enforcement.
Human intelligence (HUMINT) is likely to be far more effective than data
mining in the search for terrorist suspects. Data mining does have a role in
intelligence work, but there is no cast iron solution or holy grail. The question
is how we can provide people with the best tools and training. We must not
reach a point at which people begin to believe that data mining is going to do
the analysis for them.
Too often police and intelligence these days want to rely on covert
intelligence gathering rather than a direct, up-front approach, which may be
both more effective and more respectful of people’s rights. However, if they
are to adopt an open approach to intelligence gathering, management must
first be willing to accept the risks of getting it wrong in public.
There are cases in which local law enforcement has to follow protocol with
regard to individuals included on watch-lists, etc., despite the fact that they
know the person is not of interest. This can end up radicalizing people.
The problem of entity resolution becomes endemic as we accumulate more
automated data merger applications. In many instances, no one is cleaning
up databases, and there is no way to make corrections. It is a big problem
that is destined to get worse and have significant negative impact on people’s
lives.
8
Panel 3 – “Traditional Limits on Processing Personal Data: Data
Protection”
Hanspeter Thür, Swiss Federal Data Protection and Information
Commissioner
Hanspeter Thür provided an overview of the Swiss legal framework regulating data
processing of data by police and intelligence agencies. He also outlined the
possibilities for Swiss federal authorities to perform data mining and related analytic
methods. He noted that the Swiss Federal Data Protection Act did not cover the
processing of data in the context of criminal investigations; in that area the law of
criminal procedure applied. Thus, decisions about the use of data mining tools had to
be made under procedural law and in some cases would require the involvement of a
judge.
Processing personal data by federal bodies in Switzerland requires explicit
statutory authority. In addition, any processing had to be in conformity with the
principles of finality and proportionality.
Thür described two statutes which provide the Federal Intelligence Service with some
authority to process personal data: the Bundesgesetz über die Zuständigkeiten im
Bereich des zivilen Nachrichtendienstes of 3 October 2008 and the Bundesgesetz
über Massnahmen zur Wahrung der inneren Sicherheit of 21 March 1997. According
to Thür, the first law allowed the Federal Intelligence Service to search and evaluate
information from abroad—most of which is acquired by satellite—on behalf of the
administration and Federal Council. The Service may process personal data, including
sensitive data and personality profiles without informing the data subject. This sort
of processing might take forms that could be considered “data mining”.
The second law, according to Thür, permitted the Federal Intelligence Service to
search information necessary for the accomplishment of aims specified in the statute.
Processing may take place without the knowledge of the data subject. Activities
under this statute could include:
1) use of publicly accessible sources;
2) information requests;
3) consultation of documents located abroad;
4) the reception and exploitation of communications;
5) inquiries about the identity or place of residence of persons;
6) observations or facts including photographic or sound recordings conducted in
freely accessible locations;
7) collection of information pertaining to the location and contacts of persons.
Searches conducted by the intelligence service are mostly done on the basis of the
name of a person or organization. Such activity was not considered data mining.
9
These statutes did not provide a basis for conducting a profile-based search
(“Rasterfahndung”). Such an action would have to be contemplated by the law of
criminal procedure and would be regulated by that body of law.
Bénédicte Havelange, Office of the European Data Protection Supervisor
(EDPS)
Bénédicte Havelange began by speaking about the missions of the EDPS and noted
that the EDPS saw data protection law not only as a limitation but also a roadmap to
successful data processing: Often principles of data protection were also principles of
sound data management.
She contended that data mining put the principles of data protection to the ultimate
test because it often went much further than other forms of data processing. Two of
the main issues which she felt illustrated the profound impact that data mining had
were the following:
1) Data mining could be based on existing databases; but very often those
databases were constructed for another purpose originally (e.g. PNR data).
The use of data mining in these instances thus flew in the face of the purpose
limitation under data protection law. Additionally, it often made it difficult to
know what law applied since there would be a conflation of processing for
commercial purposes and law enforcement purposes.
2) Data mining contributed massively to the establishment of a surveillance
society. It could overturn the presumption of innocence by placing everyone
under surveillance.
She emphasized three points about the surveillance society in her presentation:
1) In a democracy, citizens should be able to scrutinize the intentions and
actions of the government, but not the other way around. Constant and
generalized monitoring of individuals should not be allowed.
2) Point 1 applied even when we were fighting terrorism and serious crime.
3) There were limitations to most fundamental rights, but they must be
established by law, have legitimate aims, and be proportionate.
Havelange also addressed the implications of data protection as a fundamental right.
Fundamental rights were universal, which meant that data protection rights applied
to all persons and not only EU citizens. Additionally, data protection applied
independently of any actual harm or the risk of harm. As a fundamental right,
violations of data protection also needed to be provided with some form of redress.
Lastly, she contended that data protection required the supervision of an
independent authority.
She noted that there were often tricky problems with the requirements for purpose
definition and limitation under data protection law that regulate data mining. The
EDPS was often told that those using data mining programmes often would not know
beforehand what exactly they were looking for. For this reason, the EDPS was
frequently confronted with very vague statements of the purposes of a suggested
operation. These kinds of statements were not really consistent with the purpose
10
limitation, Havelange argued. She noted that this kind of issue was highlighted in an
EDPS opinion from 2 years ago with respect to the use of risk assessment profiles
(EU-PNR Opinion).
The proportionality of data mining was also frequently questionable. Proportionality
meant both that the processing that takes place be proportionate to its stated aim
and that the data relied upon should be proportionate and adequate to that aim. The
first principle entailed that massive data mining should in principle not be conducted
simply to fight petty crime. She also noted with respect to reliance on profiles that
the European Network of Independent Experts for Human Rights found that the
development of profiles could only be proportionate where there was a significant
demonstration that there was a connection between the profile characteristics and
the risk it wanted to address. Such a demonstration so far had not been made
convincingly with regard to terrorism.
She did not address the issue of data retention since that subject had been discussed
extensively in the previous panels, but she argued that the retention of data should
be limited, and that the algorithm or programme should be constantly revised. She
also argued that some level of transparency should always be ensured – whether visà-vis parliaments, data protection authorities or other relevant regulators. And lastly,
she noted that data mining also presents particular challenges for the provision of
redress and the contestation of the results of such programmes. There might be
considerable reaction on the part of the public if individuals were singled out
incorrectly by data mining and detained or interrogated.
Herbert Burkert, Research Center for Information Law, University of St.
Gallen
Herbert Burkert began by spelling out the “leading escape routes” that would permit
the processing of personal data, including data mining, under data protection law.
These were:
1) when the processing was authorized by law
BUT the law had to
a. contain measures to safeguard the interests of the persons
concerned;
b. observe the principle of proportionality;
c. contain redress measures, e.g. notification, etc.;
d. be necessary in a democratic society;
2) where there was consent
BUT
a. there were areas where consent is not possible – e.g. public sector
b. the consent had to be informed
c. the consent had to come prior to the processing
3) on the basis of “collateral processing” – e.g. while administering a contract,
etc.
BUT
11
a. there would always be a question of whether the data had been
lawfully collected;
b. it would have to meet the purpose limitation requirements;
c. there were problems with the quality of data;
4) where we were not dealing with “personal data” – i.e. the data were
anonymous and it could not be re-personalized.
Burkert then went on to enlarge on considerations important to a project such as
DETECTER . He pointed out that law was a dynamic system and he argued that it
was necessary for a project like DETECTER to see law as a whole process, a cycle,
examining 1) how law reacts to technological changes; 2) how technology in turn
reacts to changes in law; and 3) how law and politics react to these changes. He
suggested, for example, that one could take legal decisions on data mining and
perform some data mining on the decisions to see if a pattern was emerging. He
then turned to the specific example of the German case on data retention, which was
decided by the German Constitutional Court. This decision held that storage as such
for a limited time was not a problem; the problem lay with the particular use of the
stored data. There had to be a clear description of the use, security measures, and
the installation of a redress procedure—in part to address the issue of false positives.
The decision also established that it was possible to demand certain contributions
from the private sector for the public sector. However, Burkert thought it was unclear
how this one case could be used to predict the future direction of the law on data
processing. For example, he argued that the Court’s holding on “storage as such”
reflected the fact that the Court did not want to pre-empt the ECJ on that issue.
Trends that Burkert saw emerging included the privatization of security and the
emergence of hybrid institutions, referred to as public-private partnerships. These
developments, he contended, were of questionable transparency and accountability.
Technology was heading toward the development of real-time machine-based
“triage” systems – systems that no longer just select but also prioritize intervention.
He foresaw the following reactions to these kinds of developments:
1) the establishment of NGOs that were specialized in security issues, not just human
rights organizations that look at impacts but also NGOs that wanted to be involved in
the design process of security systems;
2) the development of devices that would secure programme authorization oversight
of these software and organizational mechanisms;
3) the development of real-time auditing systems that were designed in such a way
that they could not be manipulated;
4) the establishment of strict liability rules for false positives and
5) far stronger involvement of civil society in the design of security.
He called for a more holistic view of the limits in the designs of systems like data
mining. He noted that transparency played an essential role in data protection and
needed to be extended not only to freedom of information or access to documents
but also had to reveal the structure of the providers of data mining. Lastly, he noted
that our current conceptualization of security was problematic for a number of
reasons and suggested that a re-conceptualization of safety and security in terms of
a global public good was in order.
12
The moderated discussion and subsequent floor discussion raised the following
points:

The EDPS issued an Opinion in 2008 on the EU PNR proposals that is listed on
its website. The EDPS identified many issues with the proposal. The proposal
was listed as having a dual purpose: to identify persons involved in terrorism
or organized crime and to locate persons who might fall within the criteria of
the proposal. This was too vague. In addition, there was simply too little solid
evidence that such systems were effective and therefore necessary. The risk
assessment indicators that would be used to generate the risk profiles was
not revealed, which cast doubt on the proportionality of the measure, as the
consequences for an individual positively identified were significant, e.g.
being prevented from boarding a flight. Furthermore, the data were initially
collected for a commercial purpose but then used for a law enforcement
purpose, which made it unclear what legislation would apply. Under the plan,
PIUs would collate data and send it to authorities. It was not clear whether
these PIUs would be public or private entities.

It is difficult to assess proportionality of data mining when it is used in
counter-terrorism. Assessments should be based on a sliding scale which
takes a range of factors into consideration. One cannot predict what a court
will determine to be inconsistent with proportionality. But the fact that it is
open to interpretation keeps the law flexible.

There can be no proportionality where core values of democratic society are
undermined, as with general surveillance. One has to consider values. The
human rights framework is not a magic wand, it is not like a computer
programme: It involves a debate of competing interests.

Proportionality also requires that a measure achieves its aim. Too often we
discover that the old measures already in place do not work, but we do not
get rid of them, we just heap new ones on top.

Since police agencies do not have much experience with terrorism, it makes it
difficult to know what works and what does not.

Do we need to change data protection laws in order to promote
transparency? Is it time to get rid of the exception for national security? We
should question the extent to which secrecy is contributing to security in any
single case. Some of the same issues arise with encryption: if you have to
keep the algorithm secret, there is something wrong with the encryption
system. The secrecy of the algorithm can be broken far more easily than the
algorithm itself. And police work is like any other line of work. Secrecy is to
some extent a kind of rationalization, an economizing device; it gives you a
chance to do more with less since you have a time advantage. Some national
security services want to have a broader discussion. Data protection
authorities provide a default switch for society. In the past, you had to justify
transparency; nowadays it is secrecy that must be justified. Similarly, people
who defend data mining now have to give an account of its effectiveness. It
is no longer taken for granted that it works. The interaction of institutions and
their function in this sector is also an area which merits further attention.
Lawyers & legislators once thought it was enough to have access to certain
documents but that is not everything.
13

Generally, transparency concerns are partly satisfied by being provided with
general information about what intelligence agencies are doing. Having
legislation that spells out what agencies can do also provides transparency.

Although it is often mentioned in discussions on surveillance, the model of the
Panopticon may not be apt any longer. Firstly, the model of the Panopticon
assumes that there is one observer with one point of view but in reality
information is not unified in this way. Secondly, the claim that surveillance
overturns the presumption of innocence should be nuanced. For instance, if
there is a facility under constant surveillance, anyone passing by is caught on
camera. In that scenario, it would be wrong to claim that everyone is under
suspicion or has lost the presumption of innocence. Such a scenario differs
greatly from one in which a person is singled out and subjected to a great
deal of surveillance on very slim grounds. In one case maybe there is a threat
to the presumption of innocence but in many cases there is not. UK
discourse often refers to the profusion of CCTV cameras that people can see
every day. But these cameras are very often trained on public places where
anyone can conduct surveillance if they want to. So it does not seem useful
to assume that all uses of surveillance serve the Panopticon model. And it is
important to distinguish between surveillance of public places and private
places.

The pertinence of the Panopticon model is the feeling that one is being
watched all the time. It does not matter that it is being done by different
people.

There is a principle of informational power and control that is often lost in the
transatlantic discussion. Every individual has certain expectations and would
want to have some remedy if those expectations are not respected. It is the
same situation with Facebook: I may like to be able to advertise myself, but
at some point I would like to have the ability to remove the information.
Privacy, just like security, is not a binary thing. It is rather a scaled thing.

Technology is constantly expanding our capabilities. The question of whether
we should develop a particular capability or not stimulates the kinds of
debates that we are having now. Those debates in turn lead to the
construction of frameworks that become constraints in some way on the
misuse of technology. So much of what the technology can do is positive that
we would be luddites to say “let’s uninvent the technology”.

We have data today that are made available in ways that we cannot envision
and it is important that we consider the implications of losing control in the
future. The problem is not, for example, that someone is an exhibitionist
today, and cannot get a job in 10 years because of what they did; it is the
fact that they never gave Facebook the permission to share their data and
had no idea how it was going to be used because there was no real contract
that was understandable between the parties. We need to consider both the
right to be recompensed if data are misused and the right to have data
repaired or corrected. We know that we are now propagating errors. How will
we empower citizens to demand that the records be fixed? This is a problem
not so much for government but for the private sector.

EU-level databases are trying to address the issue of data being spread
around to other databases by imposing obligations to send not only the data
itself but subsequent corrections of the data. This helps to ensure that sound
14
and traceable audit trails are established. However, it remains to be seen how
this will work in practice.

It is important to keep in mind the purpose of a new technological measure.
Policy-makers are often dazzled by the technical things. Technology is part of
the solution but should be put in context. We have to ask what operational
decisions the technology is meant to support. Will new information provide
any solid basis for operational decisions? Will we have the resources to deal
with the new information and implement operational action?

EU data protection law applies to both public and private sector. There are
more exceptions for the public sector, but that may be changing since more
public-private partnerships are emerging.

Section 55 of the UK Data Protection Act makes it a criminal offense to sell
data, but the fines are not high enough to stop this practice and so data
handlers treat it as a business expense. The fact that everything that happens
in a public place is automatically becoming public domain nowadays raises
further questions. Questions that need to be addressed are: What use is
going to be made of the data afterwards? Who is the owner of the data?
Who do we prosecute? If there is a loophole, it will be exploited.

There is a section of the UN that is working on a recommendation addressing
the cooperation of the public and private sectors in law enforcement work.
So, maybe next year there will be something published in this field.

There is a long tradition of dealing with political violence in Europe, but we
forget the lessons of the past.
Panel 4 – “Data Mining and Human Rights: Conflicts and Solutions”
Gus Hosein, London School of Economics/ Privacy International
Gus Hosein spoke about the role of NGOs in the human rights debate surrounding
government data mining measures. He argued that politics could not be removed
from the counter-terrorism sphere. For this reason, NGOs had to play the political
game and be “unreasonable”. NGOs were essentially locked in a competition against
the unreasonable arguments of policy makers for the hearts and minds of individuals.
Additionally, NGOs had to get people’s attention because otherwise they would have
no impact. Only once they had people’s attention could they then sit down and have
a reasoned discussion about legal issues, etc. Hosein also suggested that reasoned
arguments concerning technological issues were ineffective since politicians
understand neither the technology nor the law. He also complained that too often
NGOs fell into arguments concerning the efficacy of technological proposals when
they should really be talking about whether the proposal was really something we
want or should have.
In concluding, he focused on the following points:
15
1) All options should remain on the table, including being unreasonable; until we
were unreasonable, people would forget about the extremes and the extremes could
come to sound reasonable. People forgot that other options were available.
2) We have to abandon the quest for perfection. Politicians are still looking for the
perfect solution, but there isn’t one.
3) We have to get on top of the collection issue. Too often the focus is placed on the
use of the data without addressing the question of why the data needs to be
collected in the first place.
4) We have to deal with the accountability and transparency issues. A lot of the
abuses we have seen began as data integrity problems.
5) This debate is not about technology assessment or technology policy, it is about
what world we are capable of building.
Ramon Barquin, Barquin International
Ramon Barquin indicated that he was a believer in the ability to use technology to
improve the human condition. The question was how we use it and best harness its
power. He argued that there were movements in the right direction towards finding
useful applications and ways to protect privacy in data mining.
Data mining was something that held promise as a way of dealing with the explosion
of bits and bytes that we faced and might provide beneficial new applications in
health care, education, and logistics, among others. Yet, it faced the classic dual-use
dilemma of being capable of both positive and negative applications. The question
then became, how we could perform data mining and protect ourselves from the
potential for harm?
Barquin suggested that a starting point for privacy might involve an examination of
what people consider to be sensitive information. In addition to the question of how
touchy people were with regard to various types of information was the question of
who “owns” the information where that information was generated or held by an
institution.
Barquin suggested that the Fair Information Practices provided an initial framework
for the privacy discussion, but mentioned 4 basic, initial techniques that had been
developed to attempt to preserve privacy within data mining applications. These
were:
1) generalization;
2) de-identification and re-identification;
3) “anonymization” and
4) cryptography.
He claimed that generally two approaches were relied upon: a randomization
approach or a cryptographic approach. The first injected random “noise” to hide or
disguise the data. The second often involved inter-operation of two different
databases to prevent disclosure of personally identifiable information.
16
Barquin also examined the role of trust in privacy concerns related to data mining. In
many instances, there was a crisis of trust vis-à-vis the government. He turned to
the founders of the United States and the notion of a system of checks and balances
for a suggestion that internal controls were needed.
He also pointed to a set of principles developed by the European Group on Ethics in
Science and New Technologies and to the Ten Commandments of Computer Ethics
developed by the Computer Ethics Institute as sources of inspiration for notions of
human dignity, the right to the integrity of the person, and the protection of personal
data.
In sum, Barquin claimed that the solution lay in identifying second order
consequences of technology (in this case data mining), identifying those that are
unintended, unanticipated, and undesirable, and then finding ways to reduce them.
The moderated discussion and subsequent floor discussion raised the following
points:

We are currently undertaking the research to get us to where we have data
mining tools that we can use, but we are not there yet.

There is a desire to research and develop technology but what damage are
we doing in the meantime? Everyone agrees we would like to have the end
result, but it is not clear we can do it quickly enough without doing a lot of
damage.

Some would doubt that the end result is controllable or therefore desirable.

The machine decision system may not be a real alternative. The alternative is
probably to have a bundle of human decisions that is transformed into a
hardware/ software system. You then have a set of repetitive human
interventions that involve human decision-making. So, it is really just two
kinds of decision-making. The second system is simply more adaptable. That
kind of repeated re-examination is important.

Government decisions about what technology to develop or contract out for &
employ do not happen in a vacuum. The human rights framework has to be
taken into account.

Many governments have a formal acquisition process. It is essential to make
acquisition authorities aware of these issues so that they are included in the
contracts.

Even beyond the human rights discussion, we also need to do self-criticism/
social criticism. Perhaps there are practices we seem to be engaging in
voluntarily that may be weakening our sense of privacy. In other words, start
a conversation with the average citizen who does not care about what NGOs
or academics say and ask whether we are doing things in such a way that
certain values are being squandered by us, not the governments.

We should live in a world in which people can live as openly as possible but
still have rights.

There is the question whether there can be such a thing as corporate ethics.
Some may argue that ethics is a matter for individuals. But unless we have
17
these debates, we are not going to converge on the kinds of rules that we
need as a society.

The description of boundaries is incredibly important: We are never going to
be able to get the perfect solution. The metaphor of drug development may
be illustrative: there is no perfect drug as they all have side effects. There are
those side effects we know about and those we do not know about. And it is
the ones that we do not know about that end up biting us in the long run. As
long as we keep an open mind and have an open debate and understand the
boundaries and work to find the mid-point, we have reason to be optimistic.
The optimal spot will be fluid depending on the context.

Even if we find the optimal solution, all the safeguards will be thrown out the
window when the next terrorist attack comes, and the technology will remain.
But it would be antidemocratic to develop a system that cannot be changed
or is tamper-proof.

It is definitely important for open societies and for citizens to become
involved, but in a closed society, there is not that possibility. Yet, there are
closed countries that are primary users of these technologies and are rapidly
moving toward becoming suppliers of these technologies. It may be unclear
whether the technology in these closed societies is coming from or going to
countries that may or may not have had these debates. As long as societies
exist where the debate cannot take place, the technology risks will always be
present.

It is the dual-use problem: Do you avoid selling anything to countries run by
dictators? Dictatorial regimes will use any technology they can get their hands
on.

There is nothing to say that democratic societies or so-called democratic
societies are immune to the same issues.
Wrapping Up
The thematic programme concluded with an open floor discussion held on the
morning of 11 June 2010. The moderators decided to focus on four main themes:
Definition of Data Mining, Approaches, Effectiveness, and Proportionality.
The discussion yielded the following points:
Definition

One approach would be to simply take a definition from some other source
and annotate it. The annotations could include information explaining why it
is important you link it to a police agency or intelligence agency. In that way,
you can enlarge the definition, and, at same time, you have a discussion of
issues involved so that in the end you have mapped out the space.

The narrow choice for the Federal Data Mining Reporting Act has been
broadly recognized as not only overly narrow but exclusionary. The report
from the Committee on Technical and Privacy Dimensions of Information for
18
Terrorism Prevention and Other National Goals featured a definition that was
intentionally broad, but it also features a detailed appendix. It will be
impossible to dig up everything. Some people will tell you certain
programmes do not exist, despite the fact that others will be able to describe
those very programmes to you.

It is really the breadth of activities and not the narrowness that should be
revealed, and from there it will be possible to identify issues that are
comparable across the spectrum of activities.

An all-encompassing definition is not needed for our purposes. One of the
things that will come up is the connection with profiling and classificatory
schemes. It is the way that stereotypes are applied to the data that is
interesting. Some connection with automatically applied classificatory
schemes would start to capture some of the things we go on to discuss.

It is not the nature of a database per se but the way it is used in this context
that is important. And maybe being explicit about the context will help to
make choices about what to address and what to leave out.

Perhaps we can find elements for a definition by focusing on the impacts on
individuals. In other words, by identifying what elements of the programmes
have implications for individual rights. For example, is the programme used to
make predictions? If so, we would be more cautious as opposed to
programmes that trace a known suspect. Is it profile-based? Does it involve
abstract patterns? Is it based on one individual or a group of individuals? Is it
applied case-by-case or generally applied? Is it only used within the scope of
a single investigation?

Many definitions focus on a predictive element. But with prediction, you lose
the association and relationship elements which also would be a concern for
privacy and civil liberties. The existing definition is broad enough but the
reference to “usually large data sets” may not be apt. SARs (Suspicious
Activity Reports), for example, can be very small data sets. To specify the
scope of the data set may be too restrictive.
Approaches

The initial idea for the direction of the work package rests on the notion that
we need to look at data mining in a holistic fashion. With data mining, we are
dealing with a process that involves data handling, and it is perhaps
necessary to address the different stages of data handling.

The approach essentially sounds like a restatement of the fair information
practices. The only things missing are subject access, and the lawfulness or
purpose requirement—i.e., is there a clear and stated purpose and how is
that lawful & necessary?

A framework is very nice, but in an emergency situation, the rules go out the
window. Things will come about that are unexpected, and there is always a
political requirement to provide immediate answers.

There is always the concern for avoiding the restriction of future uses. But we
now have historic examples of what were clearly abuses, and, in some
instances, laws were passed to circumvent previous legal safeguards. There is
a difference where we have data for which no protection was initially offered
19
and data for which there was. We should distinguish between these two
situations whether it is a legal guarantee or merely an implicit promise for
protection.

It is one thing to have regulations concerning data mining, and another to
have the implementation of controls and enforcement—there you have the
oversight mechanisms. We should also ask ourselves if we need some sort of
licensing requirement for data mining and if so what would the procedure for
such licensing should look like. Licensing would permit regulators to establish
certain requirements as to how those kinds of operations may take place.

Many of the things discussed here are already in the data protection directive.
Perhaps the project could examine how data mining activities fit within the
review of the current framework. Ensuring security may differ in data mining
might and another field such as a credit scoring database. In other words,
you could spell out where and how data mining fits in to the current
framework rather than reinventing new principles.

It may also be worth considering the creation of composite variables of other
data. Sometimes people get into trouble by creating groups within their data.
A Markle Foundation report has a section on being able to document audit
trails, the need to know vs. the right to know, and also discusses emergency
situations and develops protocols.

The notion of allowing subject access requests sounds very nice in theory but
in the counter-terrorism world, no one is going to want to allow that within 5
years. Dutch intelligence has a requirement to permit access within 5 years.
Yet so far they have not told anyone, but rather keep extending it.

It is possible to have indirect access as an alternative. Data protection
authorities can do that on the data subject’s behalf. The data protection
authorities can then reply that they have made the inquiry and made the
necessary recommendations without specifying whether the subject’s data
was actually being processed by police and intelligence authorities or not.
And in some instances, data protection authorities discover that there are
mistakes in the data on police or intelligence service databases. It is in the
interest of everybody that that data be corrected. At least there is some
authority looking into it.
Effectiveness

There seems to be some scepticism even from the side of law enforcement
and intelligence as to whether data mining can be effective.

It will probably be impossible to answer the question of the effectiveness of
existing programmes, but maybe we should formulate a requirement that any
new measure has to demonstrate its effectiveness on an evidentiary basis
before being implemented (or developed). It is also significant to note that in
some instances it seems that, regardless of whether the systems are effective
or not, most of the alerts are not followed up because police simply do not
have the resources to do so.

It seems that in establishing criteria for effectiveness, the lowest point has to
be that the measure is better than chance alone or randomness. The systems
might be demonstrated in such a way that they seem effective, but the
question is would you have the same results if it were done by chance alone.
20
Everything after that is a matter of balancing harms and benefits and
proportionality.

The predictive element of data mining should be questioned strongly. It is a
myth that data mining can be used effectively for prediction. But the idea of
searching for unsuspected or nontrivial results from a database should be
retained. That element has some effective value in counter-terrorism.

It is possible to use data mining to confirm gut feelings or standard
procedures. It can also be used to anticipate- but not to predict. We can talk
about the likelihood that something will happen but not who will be involved
and other details. Any compelling applications in that space have not been
forthcoming.

The predictive functions are useful in terms of resource allocation but not in
identifying suspects.

If you have known patterns, those can be incorporated into an automated
process. In those cases, the analysis was done long ago, and it is now a
matter of monitoring.

The developers of these systems contend that they are successful, so their
claims should be included in the project analysis and examined.

Even where techniques are developed that become part of the system, that
does not mean that the activity stops there. You do not simply use the credit
score coefficients that were developed 7 yrs ago to determine my credit
worthiness now. They are updated. The question then becomes the validity of
the ongoing uses of the data and the updating of that data. That is one thing
that most people leave out. It is easy to implement an algorithm and say this
is a good use of this database, but 20 agencies may then begin to use it and
these new uses may never be examined. It is not just the successes that
need to be examined, it is also all the failures and other negative and positive
features.

The counter-terrorism community is reluctant to admit failures because one
never knows when the failures of the past will be the successes of the future.
So much of the information is not in the public domain.

The counter-terrorism community also tends not to admit successes for fear
of compromising sources and methods. This information concerning
effectiveness is the most sensitive information.

Even if an operation is shuttered, it does not mean that we should not assess
what was done for future work. That does not have to involve giving an
account to the public but to some authority.
Proportionality

How can we be specific about the aims of counter-terrorism measures? What
is then proportionate and what not?

It seems that the scariness of terrorism and the potential scale of damage,
rather than actual damage, drive the scope of the measures that are taken.

Perhaps the exact actions in question should be specified when talking about
proportionality—is it the set of information collected, the number of people
who have access, how the information may be processed? Those are just
21
three domains, there may be others. There are different subsets of the
problem that could be discussed.

So one approach might be to look at the different stages of data handling in
terms of proportionality.

Should there not also be something about equality under the law when
talking about proportionality? Proportionality when we are dealing with
foreigners seems to be something different than when citizens are involved.

The proportionality assessment might not hold to people’s risk perception. It
will be necessary to inform the public that you are not going to take account
of people’s perceptions.

There is no such thing as a little bit of privacy. It is all or nothing with
databases. It is an issue of access and who has access to information that is
essentially public. That point tends to get lost in these discussions, and it
needs to be put back in. So, you can talk about the risk of disclosure of
information belonging to individuals who are not targets of an investigation.
When you are searching through databases, there is a risk of individual
disclosure, and once you have opened the door, you cannot close it. That is
the problem that needs to be identified, and one also needs to distinguish
between the use of databases to amass information about individual suspects
as opposed to searching through databases to uncover new groups and then
looking at the consequences of revealing that information.

In theory we have some good examples. We have case law. In practice, there
is perhaps less reason to believe that it will work that well. Scanning large
groups of the population is a violation of the presumption of innocence. How
far do we want to go and how far can we go without endangering liberal
society?

Under the European understanding, it is not only the distribution but the
processing of data that can affect human rights. That is an added risk that
can be taken into account. Maybe the solution to the proportionality problem
would involve following the practice of the courts. If you go to case law of
constitutional or the European courts, you will see that they ask themselves a
number of questions. If we could emulate these questions, we could maybe
distil proportionality factors. For example, one question would be how many
people are affected? Proportionality is a relative thing. You also consider the
purpose and questions of degree, i.e. to what extent personal elements are
involved in this kind of processing. It is not an arithmetic exercise, it is a
value exercise.

The notion that processing in itself is a violation of human dignity raises huge
issues for broad-scale data mining: What level of harm would justify that level
of infringement? Is there a level of harm that would justify it?

For many kinds of data mining operations, it seems to be something that
repeatedly goes on in the background. It is not the case that there is a
specific investigation with specific target. This makes such operations appear
to be no more than fishing expeditions. That raises the question of whether
these issues of proportionality and purpose limitation can ever be resolved.
As soon as you have a specific investigation it is easy to justify. Maybe it goes
back to the marketing application source of data mining. In marketing, you
can look all over the place and explore what you could maybe do with the
22
information you discover, but that is just what you cannot do in a
proportionality setting. There is this kind of mismatch between background
scanning, looking for patterns and whether you can justify it to a court.

Is it appropriate to rely on the commercial analogy for counter-terrorism
applications of data mining?

Fraud monitoring is a more apt model. The purpose is targeted but the actual
data used changes over time. You can define the database you want to use
but that may change since the data will change over time. There are no
auxiliary purposes.

Public health surveillance may also be more apt.

In counter-terrorism, one instance is actionable. In the commercial setting,
one instance is not interesting, you want thousands. A different scale and a
different focus are involved. Many technologies used in the commercial
setting would skip over these things because they are not geared to look for
it. Simply running an algorithm is not going to find a cluster of terrorists. It
can uncover something that does not make sense and deserves more
attention, more resources.

There is a huge psychological component to how we perceive numbers.
Despite the relatively high number of people killed in car accidents, we have
more of a sense that we have control when driving or being driven than is the
case with terrorist incidents. There is also a sense that the numbers could
grow if we do not do anything. Numbers must also be understood in context.
99.5 % of vessels pass through the Gulf of Aden safely, but we are throwing
tons of resources at prevention of piracy on the high seas because we feel we
have to draw a line in the sand. So maybe the allocation of resources is
disproportionate to the real risk.
23