Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Neo4j for Process Mining Big Data and Hadoop Users Group of West Michigan Gordon Food Service 3/9/2016 Agenda • • • • • • Introduction What is a Process What is a Process Model What is Process Mining Data Prospecting with Neo4j Tool Demo Introduction In the domain of information systems it is very difficult to see a product let alone determine how efficiently it was developed. In most cases we are left with textual information to understand and make assessments of a process. Introduction The purpose of this proof of concept is to explore the application of graph database techniques in conjunction with process mining to improve the assessment of software development processes. *Better ways to capture, visualize, and analyze processes* What is a Process? What is a Process? • A series of actions or steps taken in order to achieve a particular end. • A collection of linked tasks which find their end in the delivery of a service or product to a client. • A set of activities and tasks that, once completed, will accomplish an organizational goal. *There are as many definitions of process as there are people* Definition for Presentation • A process comprises an identified body of work that can be characterized as a set of related activities or a defined sequence of steps and decisions • The process name is in action verb - noun form (usually) • The name is singular • The result must be discrete and countable • The result is what the customer of the process wants • The process is initiated by a triggering event that could be an action, time, or a condition Workflow Modeling: Tools for Process Improvement and Application Development, Sharp & McDermott Process Examples Suggested Process What We Call It Why Not a Process Customer Relationship Management Process Area Doesn’t deliver a single, specific result; a set of related business processes meeting an overall objective Acquire New Customer Business Process Delivers a single, specific result, and meets all other criteria. An “end-to-end business process Assess Prospect Financial Status or Setup Customer Sub process Too small - both deliver specific results, but are intermediate results in an E2E process Calculate Customer Credit Limit or Create Customer Account Activity, Step, Task Much too small - a part of a sub process. Possibly described in a procedure or use case. Workflow Modeling: Tools for Process Improvement and Application Development, Sharp & McDermott What is a Process Model? BPMN Basic Elements Activities Collapsed Sub-Process Task Gateways Events Exclusive Start End Inclusive Timer Parallel Start Intermediate Throwing Receiving Message Throwing Signal Signal Message Data Data Object Data Store White Box Black Box Pools Team B Throw 1 Team C Pool with Swimlanes Process 1 Team A White Box Pools Black Box BPMN Pools Receive 1 Process A External Entity BPMN Pool Communication Data Store What is Process Mining? Process Mining Process Mining Discovery, Conformance, and Enhancement of Business Processes, Wil M. P. Van Der Aalst Three types of Process Models Event Log Event Log Model Discovery Model Conformance Checking Diagnostics Enhancement New Model Event Log Model Event Logs • A process consists of cases • A case consists of events such that each event relates to one case • Events within a case are ordered • Events can have attributes Play-Out Process Mining Discovery, Conformance, and Enhancement of Business Processes, Wil M. P. Van Der Aalst Play-In Process Mining Discovery, Conformance, and Enhancement of Business Processes, Wil M. P. Van Der Aalst Play-In L3 = [<a,c,d>45, <b,c,d>42, <a,c,e>38, <b,c,e>22] a d c b e a b c d e - # > # # b # - > # # c < < - > > d # # < - # e # < # - a # XOR-split pattern: c>d, c>e, and d#e Process Mining Discovery, Conformance, and Enhancement of Business Processes, Wil M. P. Van Der Aalst Replay Process Mining Discovery, Conformance, and Enhancement of Business Processes, Wil M. P. Van Der Aalst Replay Process Mining Discovery, Conformance, and Enhancement of Business Processes, Wil M. P. Van Der Aalst Data Prospecting with Neo4j General Approach • • • • • Given that the artifacts from the SLDC are recorded and linked in JIRA (i.e. issue tracker) they can be viewed as a graph Find a promising project in this case Wholesale Order Capture (WOC) and load the issues with their associate links into a labeled graph (i.e. Neo4J) Use Cypher queries to assess the “Prospect” for developing event logs Remediate good prospects (i.e. if possible) Extract the event log from the graph database and load it into Prom Traceability Network Wiki (Confluence) CA Service Desk Use Cases, Epics High Level Reqs Prototypes Issue Tracker (JIRA) Release Notes Requirement Change Management Release Deploy Task Test Case Spec. Test Run Incident Management Defect Automated Code Analysis (SonarQube) Standards Quality Profile Problem Management Issue Repository Management (Fisheye) Manual Code Analysis (Crucible) Source Code Source Code Repository Code Review Paired Programming Configuration Management Graph Model is in Version 10062 {key} {issue type} {priority} {create date} {resolution name} {resolution date} Version Epic 16 Fix Version is in Fix Version 10062 Is Component of 10062 Comp ENH 8 Is related to 10030 Is Validated By 10040 Is Implemented By 10062 Task 3 Is Implemented By 10062 REQ 6 Is child of 10012 Is Validated By 10040 TCS 12 Is Recorded By 10070 Is Impacted By 10071 Has a sub-requirement of 10012 SubTask 7 Sub-REQ 20 TBD Test 13 Bug 1 Commit TBD TBD Code Review File TBD TBD TBD TBD Scan Defect Comment TBD Issue TBD Rule Metric Challenges Compiling Event Logs • Picking events and activities for the model • Finding good the timestamps • Assessing the relevancy and accuracy of the timestamps • Relating and ordering events and activities across systems Neo4j Demo ProM Demo Conclusions • • • • While the WOC project was very successful from a project context, it was not as successful in a process improvement context Controls for workflow and traceability between issues needs to be improved Neo4j is a very effective tool for prospecting and processing event logs Process mining of the SDLC is feasible but will take significant effort on the data side (no surprise here !)