Download 1 - Information Systems - Technische Universiteit Eindhoven

Merging of incoherent field feedback
data into prioritized design information
(S.H.I.T.T. fusion)
A Project proposal for the IOP Integrated Product Creation and Realization
Dr. Y.Lu
Faculty Industrial Design
Technische Universiteit Eindhoven
Page 1 of 21
Executive summary
Contents
1
PROJECT TITLE................................................................................................ 3
2
THEMA ................................................................................................................ 3
3
EXECUTIVE SUMMARY ................................................................................. 3
4
PARTICIPANTS.................................................................................................. 4
4.1
4.2
5
NAMES AND AFFILIATIONS OF PARTICIPANTS .................................................. 4
EXPERTISE, CONTRIBUTION AND COOPERATION .............................................. 4
PROJECT PROBLEM, GOAL AND RESULTS ............................................. 6
5.1
RESEARCH PROBLEM, RESEARCH GOAL AND RESULTS..................................... 6
5.2
INNOVATION: RELATION TO (INTER)NATIONAL STATE-OF-ART........................ 9
5.2.1
Existing field feedback information ....................................................... 9
5.2.2
Trends in modern product creation process and their impact on
consumer complaints via service centers ............................................................... 9
5.2.3
Additional field feedback consumer databases .................................... 11
6
REFERENCES ................................................................................................... 13
7
CRITERIA.......................................................................................................... 15
7.1
7.2
7.3
7.4
8
MEETING THE SPECIFIC AIMS OF THE IOP CONCERNED ................................. 15
QUALITY AND INNOVATION .......................................................................... 15
ECONOMICAL PERSPECTIVE........................................................................... 16
SUSTAINABILITY ........................................................................................... 17
PROJECT APPROACH ................................................................................... 17
8.1
STRUCTURE OF THE PROJECT......................................................................... 17
8.2
PROJECT PHASES AND DELIVERABLES ........................................................... 19
8.2.1
Phase 1: exploratory case studies........................................................ 19
8.2.2
Phase 2: Field case studies and developing of prototype .................... 19
8.2.3
Phase 3: writing of the thesis ............................................................... 19
8.3
PROJECT ESTIMATE SCHEDULE AND COST ..................................................... 19
8.4
PROJECT RISKS .............................................................................................. 20
8.5
TRANSFER OF KNOWLEDGE ........................................................................... 20
8.6
VALORISATION PLAN .................................................................................... 20
9
EQUIPMENT ..................................................................................................... 20
10
PATENTS ....................................................................................................... 21
11
TRAINING/INTERNSHIPS ABROAD ...................................................... 21
12
FINANCIAL PLANNING AND ORGANIZATION STRUCTURE ........ 21
FINANCIAL PLANNING ............................................................................................... 21
PROJECT MANAGEMENT ORGANIZATION ................................................................... 21
Page 2 of 21
1 Project title
Merging of incoherent field feedback data into prioritized design information (S.H.I.T.
fusion)
2 Thema
This project proposal is for the IOP Integrated Product Creation and Realization.
3 Executive summary
In the past decades the context of new product development, especially for highly innovative
products, has changed. Four major trends in the industry have been identified:
 Increasingly complex products, due to new technology becoming available at lower prices
ever faster
 Strong pressure on time-to-market
 Increasingly global economy
 Decreasing tolerance of end-users for quality and reliability problems
In order to deal with these increasing and often conflicting trends, the development
organization requires that field feedback information be in detail and much earlier available in
the development process so as to deliver the products to meet customer requirements.
However, traditional field feedback information (e.g. service center) is not able to provide
companies the required information (sometimes unavailable until the third generation) and the
information itself is less detailed which can hardly identify the mismatches between technical
specifications and customer requirements. Mainly thanks to advances in ICT (e.g. Internet),
potentially much more information could be available for use in the development process
provided that it arrives in the right format, properly condensed and prioritized, at the desk of
the developer. This project concentrates on five primary sources of information:
 Service data
 Helpdesk data
 Internet data extracted from, for example user forums
 Trade
 Test data obtained during the product creation process
Most of the datasets are not collected to support product development. Some of them are free
text based and very unstructured. This research project focuses on extracting structural and
richer field feedback information from the existing consumer information databases. Fusing
data from the existing information databases can be potentially used to enrich the information
in the new database. We will use the term "development data warehouse" for this database.
Together with the input from product developers, information related to mismatches between
product specifications and customer requirements can be obtained.
Given the scientific and industrial context of this project, this project requires cooperation
between both industrial academic researchers as well as industrial partners. In this context, the
Business Information System research group from Rijksuniversiteit Groningen, and
Information System research group and Business Process Design research group from
Technische Universiteit Eindhoven will act as "mede-indiener" as well as the research expert
in this project. Three PhD students will be recruited at the three research partners and act as
the front end researchers in this project.
1. One PhD student from the field of Business Process Design, concentrating on
generating the information requirements and developing a new information process to
provide development with the required information.
Page 3 of 21
2. One PhD student from the field of (Business) Information Systems, concentrating on
knowledge acquisition methods to extract structural information from different types
of consumer databases into a development data warehouse. The initial goal of
building a “static” data/development data warehouse will be enlarged later in the
project to allow for more dynamic storage structures
3. One PhD student from the field of Information Systems, concentrating on the
identification and deployment of suitable data and process mining techniques that can
be applied to identify various patterns in the development data warehouse
Philips Medical System, BU Cardio Vascular, Océ Technologies B.V. and Bang & Olufsen
A/S will act as "risco-dragende" industrial partners in this project to provide industrial input
and to implement the methods/tools developed in their own organisations.
4 Participants
In this section, we first introduce the project participants, followed by the description of their
contribution, expertise and cooperation in this project.
4.1
Names and affiliations of participants
Proposers:
 Dr. Y. Lu, research group Business Process Design, Faculty of Industrial Design,
Technische Universiteit Eindhoven (TU/e-BPD)
 Prof.dr.ir. J.C. Wortmann, research group Information Management, Faculty of
Management and Organisation, the University of Groningen (TU/e-IS)
 Dr. A.J.M.M. Weijters, research group Information Systems, faculty of Technology
Management, DTI / Technische Universiteit Eindhoven (RUG-BIS)
Industrial partners1:





Philips Medical System, BU Cardio Vascular
Océ Technologies B.V.
Bang & Olufsen (B&O) A/S
Xerox
Philips Consumer Electronics
4.2
Expertise, contribution and cooperation
Getting feedback from actual usage is an extremely important field for both manufacturing
and service industries engaged in supply of physical products. ID-BPD is well-known in the
area of reliability information flow analysis for highly innovative products. Their experiences
related soft reliability problems and working relation with industrial partners can be viewed
already for the previous IOP Soft Reliability Project. For ID-BPD, the scientific contribution
is to identify the information requirements and to develop an information process model to
identify the required information to support product development from the multiple
incoherent databases. RUG-BIS is specialised in enterprise information systems. For them,
the scientific contribution is to extract knowledge from heterogeneous sources (ultimately in
an automated way) into a development data warehouse so that life cycle information can be
fed back to the development organization in a dynamic way. TM-IS has well-known expertise
in data and process mining techniques. Their scientific contribution is to select and
1 The role of- and the relation with- the industrial partners in this project will be explained in section 4
Page 4 of 21
successfully apply the relative new techniques in the area of data and process mining on the
development datawarehouse. The various industrial partners are market leaders in the
consumer electronics industry, medical system and printer business. Their major contribution
in this project will be
 Providing the for this project required datasets, including the support (man-hours, ICT
support, etc.) required to understand, access and use the data in the context of this project
o In Philips PMS, S.H.T.T. databases are available
o In Océ, S.H.(I).T.T. databases are available
o In B&O, S.H.I.T.T. databases are available
o In Xerox, S.H.T.T. databases are available
o In Philips CE, S.H.I.T. databases are available where T stands for testing
data.
 Providing for this project an operational "shadow team" where the results of this project
will be used in their companies business processes and where feedback will be given to the
research team on the operational use of the developed methods/tools in practice. There are
ongoing/to be started improve projects in all industrial partners related to this project.
All partners, including both research partners as well as industrial partners, participating in
this project, form a "New Product Development" process to generate prioritised consumer
complaints information to support product improvement in future products.
TU/e-BPD: Development information process model
RUG-BIS: extract consumer complaints information
from existing databases into a development warehouse
Industrial partners: provide input
Industry partners:
Implement in industry
consumer databases
Discovery
Development
Use
TU/e-IS: Develop data
mining techniques to
identify patterns
in the development
warehouse
Industrial partners:
provide input/feedback
Commercialization
TU/e-BPD, RUG-BIS, TU/e-IS:
identify the required mismatch information
Figure 1: Structure of the partnership
All partners, working together in this project, and have jointly been working on these kind of
inter-disciplinary projects since 2001 and in some cases even before. This is not only the case
for the research groups participating in this project but also for the industrial partners B&O,
Xerox, Philips and Oce; with all companies there have been ongoing research projects already
for many years.
Page 5 of 21
5 Project problem, goals and results
In Section 5.1, the research problem, primary goal and results of the project are discussed.
Section 5.2 elaborate further how the project goal is related to (inter)national state-of-the-art.
5.1
Research problem, research goals and results
Modern innovative industrial product development processes are facing a number of
increasing and conflicting trends that strongly influence the industrial product creation
process. The continuous influx of new technology not only allows new product functionality
but also can (potentially) increase the complexity of the product. Both for customers and
manufacturers this may create unanticipated problems. Due to this continuous influx
manufacturers are also confronted with an increasing pressure on time to market; no
manufacturer wishes to enter the market with his innovative product to learn that all the
market positions are already taken by competitors offering the same product but then earlier.
In addition, globalisation may have a strong impact on Product Creation in this context;
products are currently no longer created by stable, often monolithic companies but by
increasingly volatile and dynamic international business networks.
Traditional field feedback information (e.g. service center) has supported development
organisation to prioritize their development activities in preventing mismatches between
product specifications and customer requirements for the product family or even for the same
product generation. However, as demonstrated by Petkova (2003) and others (Ouden 2006)
the trends discussed above result in that companies will get later (sometimes unavailable until
the third generation) and less detailed information about the mismatches while the
development organization requires such information in detail and much earlier in the
development process. Interestingly, mainly thanks to advances in ICT (e.g. Internet),
potentially much more information could be available for use in the development process
provided that it arrives in the right format, properly condensed and prioritized, at the desk of
the developer. User forums at the internet, Frequently Asked Questions (FAQs) at service,
user helpdesk and many other, related, information sources can potentially provide valuable
decision support information very quickly. The main problem, however, is that these data
sources contain huge amounts of data that are often very dynamic and volatile in structure.
The aim of this project is to develop a system that has the ability to extract relevant
information from these dynamic, incoherent, data sources and to translate it into valuable
decision support information for use in the product creation process. It concentrates on five
primary sources of information:
 Service data
 Helpdesk data
 Internet data extracted from, for example user forums
 Trade
 Test data obtained during the product creation process
These datasets, however, are not all collected to support product development. Some of them
are very unstructured. The fast development of ICT has made it technically feasible
 to extract smaller but far richer databases from those structurally very diverse
databases.
 to identify the (consumer complaints, use, and etc. ) patterns in different databases
In order to induce meaningful information from heterogeneous texts, first of all ontology has
to be developed for usage feedback to the development team. Ontology is a set of concepts
and their relationships used for description of a particular field of knowledge. An initial
version of such ontology can be obtained by classical knowledge acquisition methods (such as
KADS). However, these methods are quite labour-intensive. Moreover, these methods cannot
dynamically adapt to new products or new insight that customers may be able to provide.
Page 6 of 21
Therefore, the ambition should be to allow for automatic knowledge acquisition that can be
used as a basis for storage dynamic and adaptive information structures.
Furthermore, many companies are using data ware houses in combination with data mining
techniques to extract important management information out of the different information
systems of a business. The name Business Intelligence (BI) is often used for the challenge to
store the appropriate data in a data warehouse and the challenge to extract useful management
data out of the immense quantity of data stored in the data warehouse. The challenge in this
project proposal is to extend both, the data warehouse concept, and the mining techniques to
make them more appropriate for the mining of product development data. To distinguish the
more traditional warehouse from the ware house primary designed to support the
development process we will use the obvious term “development data warehouse” for the data
warehouse specially designed and used to support the development process. For the extraction
of useful development information many data mining algorithms such as decision trees, k-NN
classification, Bayesian classification, artificial neural networks, genetic algorithms (Mitchell,
1997) are available. However, within the mining domain there has been a shift from decision
rule mining to process orientated mining (van der Aalst and Weijters, 2004; Kosala and
Blockeel, 2000). The process oriented mining techniques seems specially attractive for the
mining of data in development data warehouse (e.g. (i) how are people using the helpdesk on
the internet that supports a new product, (ii) how are people really using a product, (iii)
mining of the primary product development processes). Again, selecting and successful
applying of the relative new techniques on the development data warehouse is a challenge.
This research project will extract much structural, richer and prioritised field feedback
information from the existing consumer information databases. Fusing data from the existing
information databases can be potentially used to enrich the information in the new database
(development data warehouse). Together with the input from product developers, information
related to mismatches between product specifications and customer requirements can be
obtained.
The goal of this project is to support product development with prioritised
information related to mismatches between product specifications and customer
requirements.
This research project, in long term, aims at preventing consumer complaints through product
development. To achieve this objective, it is necessary to develop a new consumer
information process. This process should be organised in such a way that information related
to mismatches between product specifications and customer requirements can be identified. In
this way prioritised design information can be generated. Our hypothesis is that combining
the consumer information from the existing datasets and related input from product
developers can generate the required information. Assuming the validity of this hypothesis,
this research concerns the design of a new consumer information database based on existing
consumer databases in order to provide the mismatch information to the development
organisation.
In the project, models will be developed, based on actual industrial field test cases, in order to
answer the following questions:
1\
How to develop an information process model to combine the existing consumer information
databases to generate the required information to support development organisation?
2\
Page 7 of 21
How to extract structural information; create ontology as basis for building a development
data warehouse?
3\
How to select and successful apply data and process mining techniques on the development
data warehouse to identify various patterns?
As mentioned in Section 3, the idea is to cover this work by means of three PhD students
from three different disciplinary filed but working closely together in one team:
1. One PhD student from the field of Business Process Design, concentrating on generating
the information requirements and developing a new information process to provide
development with the required information.
2. One PhD student from the field of (Business) Information Systems,, concentrating on
knowledge acquisition methods to extract structural information from different types of
consumer databases into a development data warehouse. The initial goal of building a
“static” development data warehouse will be enlarged later in the project to allow for
more dynamic storage structures
3. One PhD student from the field of Information Systems, concentrating on the identification
and deployment of suitable data and process mining techniques that can be applied to
identify various patterns in the development data warehouse.
Combined these students will gather the following knowledge and results:
1. Information process model to identify mismatches between customer requirements and
product specification
2. Ontology to support extracting structural data from existing databases
3. Knowledge acquisition method and prototype system to extract much richer and structural
data from existing databases into a development data warehouse
4. Data and process mining method and prototype system to identify various patterns in the
development data warehouse
Since these topics are not easy to be solved individually it is a necessity that the students work
closely together; the output of one student is, in many case in this project, the input for other
students. See also the figure below.
TU/e-IS
Topic 2:
Extracting
structural
data into one
data
warehouse
Topic 3:
Mining
Figure 2: interaction between the different parts of the project
Page 8 of 21
Topic 3a: Designing
prototype
RUG-BIS
Topic 1:
Analysing
datasets
Topic 2a: Anslysing needs
TU/e-BPD
Research topics
Topic 1a: Making Ontology
Partners
The main innovation of this project lies not within the applied techniques itself but in the
combination of these techniques and the resulting ability to apply these techniques in a highly
dynamic industrial context. It cannot be expected that this project will result in any patents.
For the same reason it cannot be expected that existing patents will negatively influence the
(progress of the-) project. The paragraph on “Project Approach” will describe the proposed
project structure in further detail.
5.2
5.2.1
Innovation: relation to (inter)national state-of-art
Existing field feedback information
Till mid-nineties product portfolio was mainly defined by incremental innovations, i.e.,
existing technologies to existing markets (Ouden, 2006). At that moment,
- All customer requirements could be captured in formal specifications and field
complaints are the result of defective (out-of-spec) products.
- Service organizations were able to cope with consumer complaints and to disseminate
the resulting information to the corresponding departments.
- Adequate models were available to monitor the consumer complaints trends over
time; these models have sufficient detail to determine when action is necessary and
when and where to initiate an adequate corrective action.
Service organisation provided product creation processes of product families the information
related to mismatches between product specifications and customer requirements. With such
information, the product creation process was able to collect cost of non-quality information
and to take improvement actions accordingly. The following paragraphs will demonstrate
that, however, (the new product development processes of-) modern, highly innovative,
electronic/ electro mechanic systems do not meet with these assumptions.
5.2.2
Trends in modern product creation process and their impact on
consumer complaints via service centers
In the past decades the context of new product development, especially in the consumer
electronics industry, has dramatically changed. Brombacher and de Graef (2001) have
identified four major trends that have put product development under pressure and may affect
product quality:
 Increasing product complexity, due to new technology becoming available at lower
prices ever faster
 Strong time-to-market pressure
 Increasingly global economy
 Decreasing tolerance of end-users for quality and reliability problems
These four trends and their impact on consumer complaints are discussed briefly below.
Increasing Product Complexity
Many products that use different technologies and offer diverse functionalities have been
introduced to market with a faster speed and at a lower price. Often many different functions
are integrated into one product and using such a product is not performing one single function
but experiencing multi-functionalities. The very first mobile telephone can be only used to
call people, but the newest mobile telephone can, for example, call people, send sms, connect
to internet via WiFi, make photos, send emails, watch real life TV, and work as a car
navigation system together with the TOMTOM mobile software. Developing such
complicated products and still meeting customer requirements is an increasing challenge.
Brombacher et. al (2005) showed an increasing amount of complaints reported at the service
center where their root causes could not be determined.
Page 9 of 21
Strong time-to-market pressure
Being first in the market gives companies the opportunity to set the standard and to secure a
larger market share with increased product revenues from the extended sales life; early entry
into the market also gives companies an opportunity to command a price premium and hence
higher profit margin; reducing the product development time allows companies to start later.
Although the development time becomes shorter and shorter, the time to feedback field
information to development via service centers is not reduced accordingly (Brombacher et. al,
2005). As a result, the development projects get much later field feedback information via the
service centers (Brombacher et. al, 2005)..
Increasingly global economy
Meeting the diverse expectations/requirements of consumers from the different regions of the
global market but delivering the required products simultaneously for the different regions in
the global market is an ongoing challenge for new product development projects.
Furthermore, the increasingly global economy not only requires companies to compete in a
global market, but also to utilize capabilities of local sub-contractors all over the world to
achieve an optimal product cost structure. Outsourcing the service and repair activities is done
for efficiency: consumers will get their serviced product back as soon as possible. The side
effect of this is that there is less focus on gathering detailed consumer complaints information
(Petkova, 2003). On the other hand, business processes become increasingly complex due to
globalisation and outsourcing of development and service activities. Companies are working
with national service organisations at different countries where different formats, languages
and structures are used to collect field feedback information. This has resulted in much less
structured field feedback information that are difficult to analyze,
Tolerance of consumers for quality problems
Consumers are getting less tolerated for quality and reliability problems but their
understanding of what can go wrong with these highly innovative systems is decreasing.
People see a mobile phone as the functional equivalent of a plain old wired telephone without
realising that there are complex radio-frequent and digital systems required to allow the use of
these phones. No matter how complex the product is, the consumers just want to use it in their
way that they expect the product will work. In addition, warranty periods and coverage have
also been widened. In the past consumers could return products only if the product was not
according to the technical specifications, and within the warranty period of usually 1 year
(Berden, 1999). Currently for many products there is an extended warranty time (2 or even 3
years becomes more common). In the USA, with ‘no questions asked’ return policies
consumers can bring any products back as long as they are not satisfied. This trend of
extended warranty periods and wider coverage has confronted new product development with
increasing number of consumer complaints with in-spec products and unknown root causes
and urge for more understanding of consumer expectations and requirements.
As discussed above, especially increasing product complexity and increasing customer
demands on product performance in combination with extended warranty package has led to
increasing number of consumer complaints with unknown root causes. In addition, with
current field feedback time, it is impossible for the development organisation to obtain much
faster consumer complaints information from the existing field feedback system to deal with
the increasing time to market pressure (Petkova, 2003). Furthermore, outsourcing of service
activities in different countries has resulted in much unstructured and less detailed field
feedback information. Reducing cost of non-quality based on the information from service
organisation alone is then not possible. Interestingly, the fast development of ICT has
introduced other modes of field feedback. The following section will discuss these feedback
systems in detail.
Page 10 of 21
5.2.3
Additional field feedback consumer databases
Traditionally, industries often rely on service centres (i.e. repairs) to gather information on
problems in the field. This was a very effective system until the mid nineties, and it has not
changed much since. Earlier research has already revealed problems in the field information
available from the service centres: it is incomplete, unstructured and too late for
improvements (Petkova, 2003). Den Ouden (2006) has analysed further other three sources
consumer information, such as helpdesk, internet and trade. She revealed that these sources
do contain information on consumer complaints although they are not set-up to support
product development. In addition, test data generated in the product creation process
(including development tests and consumer tests) do provides more consumer complaints
information (Petkova, 2003, Geudens et. al, 2005). Each of those sources will be described
briefly, with an indication of the information that is available and the way this information is
currently used in the business.
Trade
Trade information is information gathered by suppliers, distributors and/or retailers of a
company’s product. The potential of this information is vast since retailers have direct contact
with the customers. However, since a lot of companies have a “no-questions-asked" policy for
customers returning a product, this sort of information is not collected structurally by
companies in consumer electronics (Den Ouden et al. 2005c). Contacts with trade are often
about new products and explaining sales promotions.
Helpdesk/call center
Many customers do not directly return their products to the service centers when they
experience a problem, but first call a call center or contact a helpdesk. These databases are
therefore a collection of the ‘voice of the customer’. They typically contain records of various
types of customer comments and questions related to the product (Menon 2004).
Companies see call centers as an after sales service to customers and not as a source of
information to support product development. A helpdesk or call center is meant to serve the
customer better (Walker and Johnson 2004), to attain a minimal service level at minimal cost
(Koole 2004).
Main difference with service center databases: call center/helpdesk a database not only
contain records about product failure, but also records with information about other problems
the customer can experience. These include difficulties in installing the product, difficulties in
operating the product or questions about product features. (Den Ouden et al. 2005c).
Internet Forum
Recently through the widespread use, the internet is becoming a very useful source of
information. Forums, websites and surveys through the internet contain information on
problems consumers experienced with the product. On internet forums consumer interact with
each other about products they have bought and problems and solutions they have found
related to those products. Data on forums gives more insight into the use phase of the
consumer process.
Customers view forums as sources of product information, which in comparison with retail or
dealers, is of greater credibility and relevancy. Forums also have the ability to evoke empathy,
creating a shared feeling associated with a product. The greater credibility is perceived since
other customers are not focused on making money by selling you the product. Greater
relevancy refers to the fact that experiences of customers with similar use profiles are
important to customers in assessing whether the product will be of benefit to them as well.
(Bickart and Schindler 2001)
Page 11 of 21
The literature on internet forums is mainly focused on how to develop a successful
community, successful being: perceived as valuable as information source by customers
(Stockdale and Borovicka 2006, Schoberth et al. 2002). However very little is known about
what the potential uses may be for companies producing high-tech innovative products. Fuller
et al. (2004) describe how the innovative potential of an online community can be used in the
product development process in the consumer goods industry. They provide a useful
framework for selecting different customer groups, based on forum activity, which can
develop different aspects of the new product. Lead users can be used to identify future vision
on product features, early adopters could be used to give valuable information on the different
variations of the product and heavy users could give insight into the potential weaknesses of
the products performance. Main benefit of the online community was that it helped in
targeting potential customers (who have shown interest in similar products) instead of
targeting the potential customers perceived by the marketing team.
Information on the internet is usually not very structured but can be very rich in content since
the problem is often described in detail and pertains to a single product (Den Ouden et al.
2005c). Detail means not only the problem is stated but also the environment and use process
are mentioned. The information is usually presented in a free text form. However there are no
statistics available so problems that are mentioned may not be relevant for all instances of the
product. Information in internet forum is created by the product users and/or ‘experts’, who
offer the solution to the customer’s problem. This means no distortion is added by an agent,
like in call center databases. The use of internet information is not structurally done by
companies, but sometimes a business unit is aware of its potential and uses an approach to
gather specific quality and reliability information (Den Ouden et al. 2005c). Furthermore,
internet forums are often created immediately after the product is introduced in the market.
Therefore this information has a high potential to reduce the feedback time.
Tests data obtained during product creation process
The development test database consists of information about how to conduct tests and test
results and it is meant to verify whether the designed product meet the earlier defined the
specifications. Consumer test has demonstrated its potential to provide development with
more detailed information related mismatches between customer requirements and product
specifications (Petkova, 2003; Geudens et. al, 2005). Currently a number of PhD students
from TU/e-BPD are working on improving the test strategies to proactively identify these
mismatches in product development processes.
As discussed above, many databases of (consumer) information available do not provide the
required richer, faster and structured information alone. However, each database does provide
some relevant information to a certain extent (see figure 3).
Page 12 of 21
Richness
Helpdesk
Internet
Test
Service
center/trade
Speed of
feedback/timeliness
Structureness
Figure 3: Characteristics of different field feedback information
Compare with helpdesk and service center information, internet information is much richer in
the content and much easier to be improved further because of the fast evolution of ICT.
However, its validity is still questionable. A study has been recently conducted on the LCD
TV quality based on internet information. It was found that there are large differences about
LCD TV quality from different internet sites. It is largely due to the fact that the backgrounds
of the web sites and the people behind the comments are often unknown. One of the ways to
improve the validity of the internet information is to employ a dynamic adaptive structure via
an interactive website so that various control and take place to ensure the validity. Combined
with the analysis with other sources of field feedback information, a complete overview of the
mismatches present in the field can be achieved.
In this project, ontology is to be identified to extract much structural information from the
various databases and to store the identified data in one development data warehouse. With
the structured information in the data warehouse, data and process mining techniques can be
applied to identify the various patterns in the data warehouse including consumer complaints
patterns and use patterns. Ultimately, the required information, mismatches between
consumer requirements and product specifications, can be potentially generated.
6 References
Armstrong, D., "People, Ideas Move Into the Light Companies", Forbes magazine, August
2006
Berden, T. P. J., Brombacher, A. C., and Sander, P. C., The Building Bricks of Product
Quality: an overview of some basic concepts and principles, International Journal of
Production Economics, vol. 67, Iss. 1, p. 3, 2000.
Bickart, B. and Schindler, R. M., "Internet forums as influential sources of consumer
information", Journal of Interactive Marketing, vol. 15, Iss. 3, p. 31, 2001.
Page 13 of 21
Brombacher A.C., de Graef, M.R., Anticiperen op trends, "Betrouwbaarheid van technische
systemen - anticiperen op trends", Redactie: Dr. M.R.de Graef, Stichting Toekomstbeeld der
Techniek 64, 2001
Brombacher, A.C., Sander, P.C., Sonnemans, P.J.M., Rouvroye, J.L., "Managing product
reliability in business processes 'under pressure' ", Reliability engineering and system safety,
88(2005), No. 2, p. 137-146
Cellular-news, "Misunderstood Phones Costs Industry US$4.5 bllion",http://www.cellularnews.com/story/18310.php, posted on 18th July 2006
Davidson, S. "Towards an understanding of no trouble found devices", 23rd IEEE VLSI Test
Symposium, 1-5 May 2005, Page(s):147 - 152
Den Ouden, Development of a design analysis model for consumer complaints : revealing a
new class of quality failures, PhD thesis, Technische Universiteit Eindhoven, 2006
Den Ouden, E., Lu, Y., Sonnemans, P. J. M., and Brombacher, A. C., Quality and reliability
problems from a consumer's perspective: an increasing problem overlooked by businesses,
accepted for publication at Quality and Reliability Engineering International, 2005.
Fuller, J., Bartl, M., Ernst, H., and Muhlbacher, H., "Community Based Innovation",
Proceedings of the 37th Hawaii International Conference on System Sciences, 2004.
Geudens, W, Sonnemans, P. J. M., Petkova, V. T., and Brombacher, A. C., "Soft Reliability, a
new Class of Problems for Innovative Products: 'How To Approach Them'", IEEE Annual
Reliability and Maintainability Symposium 2005, Alexandria, VA USA, 24 Jan. 2005.
Kosala, R., Blockeel, H., "Web mining research: A survey. In: SIGKDD Explorations",
Newsletter of the ACM Special Interest Group on Knowledge Discovery and Data Mining
2(1) (2000) 1-15
Koole, G., "Performance analysis and optimization in customer contact centers", Proceedings
of the First International Conference on the Quantitative Evaluation of Systems, 2004.
Lucky, R.W., "Is Industrial Research and Oxymoron?", IEEE Spectrum, September2003
Menon, R. (2004), Mining of textual databases within the product development process, PhD
thesis, Technische Universiteit Eindhoven, 2004.
Mitchell, T., Machine Learning, 1997, McGraw Hill
Petkova, V. T., An analysis of field feedback information in consumer electronics industry,
PhD dissertation, Technische Universiteit Eindhoven, Eindhoven, 2003.
Schoberth, T., Preece, J., and Heinzl, A., "Online communities: a longitudinal analysis of
communication activities", Proceedings of the 36th Hawaii International Conference on
System Sciences, 2002.
Stockdale, R. and Borovicka, M., "Developing an Online Business Community: A Travel
Industry Case Study", Proceedings of the 39th Hawaii International Conference on System
Sciences, 2006.
Van der Aalst, W.M.P. and Weijters, A.J.M.M., " Process mining : a research agenda",
Computers in industry, 53, 2004, No. 3, p. 231-244
Page 14 of 21
Walker, R. and Johnson, L., "Managing the customer-service provider relationship with
technology-enabled services", Mt Eliza Business Review, vol. 7, Iss. 2, p. 57, 2004.
7 Criteria
This section will describe how the project meets the following criteria:
 Specific aims of the IOP concerned;
 Quality and innovation;
 Economical perspective;
 Sustainability.
7.1
Meeting the specific aims of the IOP concerned
The project will develop a prototype system to merge diverse, incoherent and heterogeneous
field feedback databases in order to identify the mismatch information between product
specifications and customer expectation. Such information can support product development
to proactively prevent customer dissatisfaction in the product creation process. The project
addresses an important bottleneck that prevents product designer from improving product
design by using large amount of available field feedback data and to bring technically feasible
products from being accepted and appreciated by customers in the market.
The project meets with the IOP criteria in so far that it tackles a highly innovative problem in
the field of product creation and realisation with a multi-disciplinary team in a consortium
with industry.
7.2
Quality and Innovation
This project consists of expert researchers from various disciplines. ID-BPD is very wellknown in the area of reliability information analysis for highly innovative product
development. They are one of the pioneers in recognizing increasing NFF and taking research
measures against these failures. The early approved IOP Soft Reliability Project was also
initiated by Prof. dr.ir Aarnout Brombacher, the head of this department. The recently
published thesis of Elke den Ouden (2006) has attracted world wide attention on the research
results in this group. TM-IS has a world-wide academic reputation because of their research
in data and process mining techniques. By simply searching on www.scholar.google.com, one
can easily find this group holding a prominent position in this research field. The BIS group
headed by Prof. Hans Wortman at RUG has very good research reputation in the area of
enterprise information systems. He is actively involved in national wide projects related to
business information system modelling. For the topic of merging diverse and heterogeneous
field feedback databases to derive prioritise design information, this multidisciplinary
collaboration among these three groups is essential. By having these three groups on board in
this project the quality of the output can be ensured.
This type of applied research has, since the end of the Second World War, been the realm of
(often large) industrial companies. Companies like AT&T, General Electrics and Philips had
(and to a certain extend still have) considerable research facilities that, in some fields, were
leading. The main focus of these research labs was to develop basic technology and to
translate the technology into products that can be sold on the market.
Due to increasing pressure on costs and time to market this position has gradually changed
(Lucky 2003). While in the sixties and seventies it was common that application oriented
conferences were dominated by industry currently the same conferences are dominated by
Page 15 of 21
academia. Universities, however, lack the natural drive towards application in- or related toactual products. This has created a gap, relevant both for academia and industry.
One of the major problems of applied research is that problems, derived from the field,
usually do not follow the traditional lines of scientific mono-disciplines. This means that
application oriented researchers will have to be able to think across boundaries. In the
traditional application oriented research environment this happens naturally; people are
working on (concepts of-) products and have to solve the issues that arise. In a more academic
environment focus on multidisciplinary aspects may not grow naturally and will therefore
have to be taken into account; both in education and in research.
Earlier research (Petkova et. al, 2003; Ouden, 2006) have shown that current field feedback
database alone is incompetent to provide designers with useful design information. Analysing
the diverse, unstructured and heterogeneous field feedback databases requires not only
understanding of mismatches between technical specifications and customer expectation but
also on how to extract this information from these incoherent databases in actual industrial
business processes.
Getting feedback from actual usage is an extremely important field for both manufacturing
and service industries engaged in supply of physical products. For ID-BPD, the scientific
challenge is to develop an information process to identify the required information to support
product development from the multiple incoherent databases. For RUG-IS, the scientific
challenge is to extract knowledge from heterogeneous sources (ultimately in an automated
way) into a development data warehouse so that life cycle information can be fed back to the
development organization in a dynamic way. For TM-IS, the scientific challenge is to select
and successfully apply the relative new techniques in the area of data and process mining on
the development data warehouse.
This project is a direct spin-off project of the IOP Soft Reliability Project (SRP). SRP focuses
on understanding soft failures while this project will identify prioritized information to
support product design for future innovative products.
7.3
Economical perspective
As stated earlier in this proposal, companies active in the field of strongly innovative products
are faced not with one trend but with four simultaneous trends that affect the way the product
is developed in their business processes. Products are getting more and more complicated.
Customer requirements for this type of products cannot be fully understand by the
development. Consumers cannot fully express their expectation, and requirement before
really using these products. A consequence is that specifications are often only partially
known. This has serious impact on the product performance. In those cases, the anticipated
specifications often do not meet with customer requirements during actual product use. This
will result in unanticipated complaints with respect to product performance. These latediscovered consumer complaints can have a considerable impact in an industrial context
because these problems can be the cause of very costly interventions such as product field
changes or even product market recalls.
Traditionally used field feedback information is no longer capable of providing prioritised
consumer complaints information. The following figure shows the development of the
percentage of “No Fault Found” (Failures where the cause of a complaint could not be
determined) at a major manufacturer of high-tech, high volume consumer electronics over the
last two decades2.
2 Name and details of the manufacturer can not be disclosed here due to reasons of confidentiality. Full details,
however, are known to the authors of this proposal.
Page 16 of 21
Figure 4: Percentage No Fault Found in field return reports of modern high-volume consumer
electronics (Brombacher et.al, 2005)
This increasing number of No Fault Found problems can lead to situations where the product
development has lost the control of consumer complaints. For the development organisation
then it is common to see that the same complaints appear in different product generation again
and again. This situation can dramatically influence product image and reputation for future
products in the market, and cause enormous amount of economical loss to the company as
well as to the society. In order to get grip on this new and growing class of problems it is also
for industry highly relevant to gain more prioritised consumer complaints information.
On the other hand, more and more consumer electronics companies are encouraging their
consumers to use more web-based customer service with cost reduction in mind. The
increasing use of internet and mobile telephone has also put the helpdesk/call center and
internet consumer databases into a newer role than what they were. The recent development
in setting up consumer test lab in both industry as well as universities (e.g., Philips Innohub,
Homelab, Georgia Tech Awareness Home, TU/e Consumer Test Lab) has also provided
additional consumer information. How to benefit from combining these existing consumer
databases in providing development with more prioritised and faster information related to
mismatches between customer requirements and product specification before investing in
collecting alternative databases is certainly worthwhile for industries to explore further.
Ultimately this project aims to support designers to improve product design and to reduce
NFF to reach the economical benefits. Hopefully, the billion's loss in various companies and
industries can be prevented or reduced.
7.4
Sustainability
This project focuses on enabling companies and industries to translate prioritised customer
complaints, which are originally hidden in diverse heterogeneous unstructured databases, into
design decision to prevent the same type of complaints (NFF) from re-appearing. As a result,
products/parts will not longer be simply replaced or swapped at service center. Based on the
actual mismatch information, decisions can be made on the sustainability of these
products/parts.
8 Project approach
8.1
Structure of the project
As a project this proposal consists of three main activities:
 Developing the information process model: During this activity the way that the existing
consumer databases should be processed in order to obtain information related to
mismatches between customer requirements and product specifications will be defined.
Page 17 of 21
The industrial partners will actively support these studies by making available facilities, by
sponsoring subject involvement and by their knowledge about product technologies and
markets. We are interested in various consumer information databases and in how
information related to mismatches between customer requirements and product
specifications can be identified. In the project already available databases
(market/marketing information, consumer test information, development information,
SHIT rate) will be used and we are not going to collect information from unknown and
new databases. In the first step, the existing consumer databases at industry partners will
be identified and analysed on whether required mismatch information can be identified
(Topic 1). It is done through case studies in industry partners with actual field feedback
information databases. Secondly ontology will be developed to support the extraction of
much more structural data from the existing datasets (Topic 1a). Thirdly, the requirements
on field feedback information from the developers' point of view will be investigated to
support further mining of the extracted datasets to identify various patterns in the
improved databases (Topic 2a). Finally the resulted data from data and process mining will
be analysed on their ability on identifying the information related to mismatches between
customer requirements and product specifications (Topic 3a). The results from these
activities will be used to support extracting more structural data, support data and process
mining in the development data warehouse and generate the required information. Major
research activities including
 Identifying the available sources of consumer information databases
 Jointly developing ontology with other PhD students to help extracting structural
information from the existing consumer information databases
 Jointly working with other PhD students to identify the requirements of product
developers w.r.t. field feedback information
 Jointly working with other PhD students on fusion of databases to identify the
required information.
 Knowledge acquisition methods to extract structural information from different
types of consumer databases into a development data warehouse. First of all
ontology has to be developed for usage feedback to the development team (Topic 1a);
followed by the development and validation knowledge acquisition methods to extract
structural data from the diverse heterogeneous databases into a development data
warehouse (Topic 2). Thirdly, the requirements on field feedback information from the
developers' point of view will be investigated to support mining of the extracted datasets to
identify various patterns in the improved databases (Topic 2a). Finally the resulted data
from data and process mining will be analysed on their ability on identifying the
information related to mismatches between customer requirements and product
specifications (Topic 3a). Major research activities include:
 Jointly with other PhD students working on developing ontology to support the
data extraction.
 Develop and validate the prototype to perform the data extraction
 Jointly with other PhD students working on identifying the requirements of
product developers with respect to field feedback information
 Jointly working with other PhD students working on analysing the resulted
information from data mining in the data warehouse to identify the required
information.
 Identification and deployment of suitable data and process mining techniques that can be
applied to identify various patterns in the development data warehouse. For the extraction
of useful development information many data mining algorithms such as decision trees, kNN classification, Bayesian classification, artificial neural networks, genetic algorithms
(Mitchell, 1997) are available. However, within the mining domain there has been a shift
from decision rule mining to process orientated mining (van der Aalst and Weijters, 2004;
Kosala and Blockeel, 2000). The process oriented mining techniques seems specially
attractive for the mining of possible data in the development data warehouse (e.g. (i) how
Page 18 of 21
are people using the helpdesk on the internet that supports a new product, (ii) how are
people really using a product, (iii) mining of the primary product development processes).
Again, selecting and successful applying of the relative new techniques on the
development data warehouse is a challenge (Topic 3). Major research activities include:
 Jointly with other PhD students working on developing ontology to support the
data extraction.
 Develop and validate the prototype to perform the data mining
 Jointly with other PhD students working on identifying the requirements of
product developers w.r.t. field feedback information
 Jointly working with other PhD students analysing the resulted information from
data mining in the data warehouse to identify the required information.
8.2
Project phases and deliverables
The project will consist of two, quite similar, main phases and a third phase when the final
results of the project are presented in three different PhD theses. The two main phases are
centred on case studies with the industrial partners and each consists of three sub-phases, each
with a corresponding set of deliverables.
8.2.1
Phase 1: exploratory case studies
During the first phase of the project products will be selected as subject for research at the
industrial partners. The consumer datasets from these products will be used as input for the
different research topics. This phase includes explorative analysis of existing customer
databases w.r.t. the required mismatch information and ontology studies to find the criteria’s
to extract more structural datasets into a data warehouse. Next to that data mining techniques
will be selected. Although product and process information, obtained at the industrial
partners, will not be disclosed to third parties the (development of-) models mentioned will be
published in the academic literature and will be made available to other participants in the
IOP project.
8.2.2
Phase 2: Field case studies and developing of prototype
The ambition is, during this phase, to apply the ontology to extract much more structural
datasets into a data warehouse, to apply the developed data mining methods and techniques to
identify various patterns in the data warehouse and eventually to identify the required
mismatch information in the data warehouse. For these purpose prototypes will be developed
and consecutively applied in actual cases at the industrial partners. This phase will also
include the validation of the developed prototypes and analysing of the results. It is the
intention that also the results of this phase will be published in the academic literature under
the guidelines given above.
8.2.3
Phase 3: writing of the thesis
During the final phase of the project the three PhD students will not only write their thesis but
will also actively participate (see also “Transfer of Knowledge”) in organising workshops and
seminars for the associated industries and for other parties in the IOP-IPCR community.
8.3
Project estimate schedule and cost
Explorative case
study
Field case studies
Estimation Duration
1.5 year
1.5 year
costs
cost of Aio's salary, supervising cost
and support from industry in terms of
data
cost of Aio's salary, supervising cost
Page 19 of 21
and developing of
prototype
writing of the thesis
8.4
1 year
and support from industry in terms of
data
cost of Aio's salary, supervising cost
and support from industry in terms of
data
Project risks
The main risk for the project is a lack of industrial case-study material. Since all partners
(Philips, Oce, B&O and Xerox) have already expressed their interest to provide the project
with cases this risk is not considered as a major risk for the project.
8.5
Transfer of knowledge
The project will work with the industrial data released by the industrial partners. If students
(PhD students and master students) need to conduct field case studies and/or trainingship at
the industrial partners, they can sign agreements with the companies that they will obey the
house-rules at the partners with regards to behaviour on site, safety and confidentiality.
The project results will not be used commercially within the consortium. All the partners have
therefore the full rights to use the results in their own organisations.
To disseminate the knowledge obtained from the project, the plan is not only to publish
papers and thesis via the traditional academic media, but also to organize a number of
workshops/seminars for the (IOP-IPCR) design community in the Netherlands. Another
means to disseminate the design knowledge gathered in this project is via presentations at
(international) conferences. The consortium BC (begeleidingscommissie) will act to prevent
the disclosure of potentially confidential information.
8.6
Valorisation plan
A new report from WDSGlobal has revealed that 63% of mobile devices returned as faulty
are in perfect working order. This 'No Fault Found' (NFF) returns rate exceeds the industry
average for general consumer electronic devices by 13% and is costing the mobile industry
US$4.5 billion globally (Cellernews, 2006). Elke den Ouden (2006) in her PhD research in
consumer electronics industry has found that more than 50% of the returned products were in
reality in full working order, but just too complex to be operated successfully.
This situation is not unique for consumer electronics industry only. No-fault-found findings
are turning up at the extraordinarily high rate of 50 to 60 percent at commercial airlines,
military repair depots, microsystems (Davidson, 2005).
As a targeted result, NFF will be reduced. it is possible to estimate by reducing the 50% NFF
in various industries how much economical values can be created to the society. Here is a
simple example. According to Forbes magazine in August 2006 (Amstrong, 2006), Consumer
electronics is still Philips' largest division, responsible for $13 billion in annual sales. Assume
that 2% of the products are returned and 50% of them are due to NFF. This project can
eventually help save $13 million cost of NFF.
9 Equipment
It is expected that the existing infrastructure, either at the three research partners or at the
industrial research partners, will be sufficient to carry out the planned experiments. There will
be 3 super PC's required for executing the project by the Aio's.
Page 20 of 21
10 Patents
Since this project focuses largely on models and methods it is not expected that the project
will result in patents.
11 Training/internships abroad
All research partners are located with Europe. However, the to-be-analysed data may be
collected outside Europe. When it is necessary, the students may need to visit the sites.
Exposing the students to the industrial partners will not only give the students insight in the
industrial relevance of this project but will also expose them to cultural differences between
companies, industries as well as different continents which are directly linked to the
background of the diverse heterogeneous field feedback databases.
12 Financial planning and organization structure
Financial planning
Three researchers, during 4 years.
1 PhD students TU/e Industrial Design (ID), please refer to the "projectbegrotingplan" for the
budget.
1 PhD students TU/e Technology Management (TM), please refer to the
"projectbegrotingplan" for the budget.
1 PhD student RUG-BIS, please refer to the "projectbegrotingplan" for the budget.
Please refer to the "projectbegrotingplan" for the total budget.
No direct financial contribution of third parties involved.
Project management organization
This project requires both synergy between the project members and also a close
collaboration with the “home base” in the universities/faculties/departments. In this context it
is expected that the PhD students will spend approx. 50% of their time at industrial partners
during the case studies and the other 50% in the contributing faculties.
The project will be coordinated by a Steering Group, made up of the heads of the research
groups of the TU/e and RUG and the department heads (or senior researcher) of the industrial
partners. The steering group will decide on the main direction of the project, in close cooperation with the Industrial Advisory Group (begeleidingscommissie), chaired by a member
of the IOP committee.
The operational management of the project will be executed by the Project Manager, who will
co-ordinate the tasks of the researchers involved in the project. The Steering Group will meet
about once a month, depending on the phase that the project is in. The project team will meet
at least twice per year with the Industrial Advisory Group.
The project management structure is subject to the formal regulations that will be set by the
IOP IPCR program rules.
Page 21 of 21