Download Loading Data with Talend

Loading data with Talend
UW’s approach to loading data into Kuali Student
August 23, 2012
9:30am – 11:30pm
Agenda




Introductions
Product Review
Dave’s Job Design
Demo
Introductions
David McClellan is lead ETL developer for UW’s
MyPlan and KS CM implementation. David is a
Senior Java Developer with 25 years of
programming experience.
Talend Overview
Talend is:
 An open source company
 Provides middleware for data and application integration
 Wide product suite including ETS, MDM, Data integration,
Governance, ESB, BPM. Free/Premium versions
UW applied Talend:
 Extract, transform and integrate data from UW’s Student
Database into Kuali Student CM database
 Data includes ATP, Org, Course
Talend
+
-
Uses Eclipse
Uses Eclipse
Documentation pretty good (for less
complicated connectors)
Job design is not well documented
100’s of Prebuilt connectors
Only two WS connectors, one broken. Won’t
generate attributes that we need
Join across databases and different data
sources
Forum Support (ok)
Quality is erratic:
Eclipse Plugins are erratic, buggy
Open Source
Open Source
Possible to write custom Java code
Require Java Developer
Part of a broader integrated product line
Best features are only available in premium
version
Job design - Process
1. Create properties file
2. Configure connector to source database (user/pass/query to
run/jdbc url etc)
3. Design mapping between source and target destination
(contains transformations)
4. Configure Web Service connector
5. Generate java classes needed by Web Service connector
6. Run job
7. Review log
Job Design - Desktop
Demo
 ATP (simple)
 Course (complex)