Download Exploit the Power of Enterprise Data Management

Anatomy of a Archiving Project
Basic Principles To Consider
Eric Offenberg, Product Marketing Manager
Tim Smith, Technical Product Manager
Princeton Softech
© 2007 Princeton Softech, Inc.
Addressing The Challenges
 Key challenges for sites with
Custom Applications
-
Managing Application
Performance
Controlling Costs
Mitigating Risks Associated with
Data Retention and E-Discovery
Requirements
How can archiving help?
© 2007 Princeton Softech, Inc.
2
Explosive Database Growth
 Mergers & acquisitions
 Organic business growth
- eCommerce
- ERP/CRM
 Records retention:
- Healthcare – HIPAA
- Pharmaceutical – 21 CFR 11
- Financial – IRS and SEC Rule 17a-4
 Data multiplier effect
 According to industry analysts, annual compound growth
rates for databases will exceed 125%
© 2007 Princeton Softech, Inc.
3
The Ongoing Problem
Risk
Compliance $
Downtime
IT Resources
How many copies?
© 2007 Princeton Softech, Inc.
The Data Multiplier Effect
Data Multiplier Effect
Actual Data Burden = Size of production database + all replicated clones
200GB Test
200 GB Production
Total
1200GB
200GB Backup
200GB Disaster
200GB Development
Recovery
© 2007 Princeton Softech, Inc.
200GB Quality
Control
5
Who is Impacted and how they benefit
•CEO
 Data Growth
 Retention and
Compliance
 Portfolio Optimization
•CIO
•Applications
•DBAs
•Quality Assurance
•Business Users
•Capacity Planners
© 2007 Princeton Softech, Inc.
6
Analysts Projections
 A new ESG report, "Digital Archiving: End User Survey and Market
Forecast 2006-2010," regarding their purchasing intentions for
archiving solutions.
- 48% of organizations say they will purchase and deploy a
database archiving application within the next 24 months
- An additional 35% say they expect to purchase a database
archiving application at some point beyond 24 months.
- Database-resident information will be the fastest growing type of
archived information between now and 2010, growing at a
CAGR of 79%. Over 4000 Petabytes of database archives will
exist in 2010.
 The database archiving market will grow at a CAGR of 38.5 percent
through 2009- Gartner
© 2007 Princeton Softech, Inc.
7
Basic Principles for Archiving Data
© 2007 Princeton Softech, Inc.
Components
© 2007 Princeton Softech, Inc.
9
Assess
 Determine application types
- Mission critical
- Business critical
- Targeted for sunset
 Decide where to locate the archive
- Which storage devices
- When to deploy each type
 Determine access requirements
- Who, what, how, when?
© 2007 Princeton Softech, Inc.
10
Classify
 Identify “Business Objects” to archive
- Historical reference snapshot
- Examples: Activities, Service Requests
 Determine retention requirements
- Cross functional consensus
- Time value of business object
- Deletion requirements
 Identify post-archive use cases
- Customer service inquiries, audit, ediscovery, trend analysis
- SLA for access
- Retrieve from archive
- Reload to temporary DBMS
© 2007 Princeton Softech, Inc.
11
Functional Requirements for Archive
Application
Retention
(Years)
Archiving
Recovery / Access
Requirements
Lead
Time
GL
3
Yearly
Audit; Trend analysis
Y
Ledgers, Journals, fully posted
AP
3
Yearly
Audit; Trend analysis
Y
Vouchers, Payments, fully paid and
posted
AR
3
Yearly
Audit; Trend analysis
Y
Invoices, items
Billing
3
Yearly
Audit; Trend analysis
Y
Invoices
Billing
Interface
1
Quarterly
Troubleshooting
Y
Billing input
AM
3
Yearly
Audit; Trend analysis
Y
Retired assets
AM Interface
1
Quarterly
Troubleshooting
Y
Asset input, GL interface
Payroll
2
Yearly
Audit
Y
Paycheck processing data and
balances
© 2007 Princeton Softech, Inc.
Type of Data to Archive
12
Archive
 Determine operational practices
- Frequency of archive
- Automated or manual operations
- Online or offline
 Define file management
- Across storage tiers
- Manual or integrated (Tivoli,
Symantec, etc.)
© 2007 Princeton Softech, Inc.
13
Evolving Business Value
Inactive
Data
Value
Access Frequency
Active
Data
Historical
Data
Time
© 2007 Princeton Softech, Inc.
14
Define Storage Strategies
Active
Data
Inactive
Data
Value
Access
Tier 1
Historical
Historical
Data
Data
Tier 2
Tier 3
Time
© 2007 Princeton Softech, Inc.
15
Set Migration Policies
CLOSE_DATE
> 01-JAN-2005
Access
Tier 1
Tier 2
Value
CLOSE_DATE >
01-JAN-2001&
< 31-DEC-2005
CLOSE_DATE <
31-DEC-2000
Tier 3
Time
© 2007 Princeton Softech, Inc.
16
Archiving a Complete Business Object
Ledgers
© 2007 Princeton Softech, Inc.
17
Store
 Determine format of archives
- Archive file system
 Define hardware targets
- Number of tiers
- Types of devices
 Establish security parameters
- Integration with existing framework
 Database, application, network
© 2007 Princeton Softech, Inc.
18
Cost-Effective Tiered Storage
Current
Production
Production
Database
Database
History/Reporting
Online
Archive
Off-Line
Archive
SAN /
NAS
Archive
Database
Flat Files
Tape
WORM
Files
Time
© 2007 Princeton Softech, Inc.
19
Access
 Analyze use cases vs. cost of access
- Goal: match SLA to value to cost
- Application independent access
- Native application access
 Communicate access terms & conditions
- SLAs
- Resource provisioning
- Training on access paths
© 2007 Princeton Softech, Inc.
20
Dispose
 Build cross-functional team
- Business, legal, audit, IT
- Business owns data, IT manages
supporting infrastructure
 Determine data deletion policies
- Signoff by stakeholders
- Which records to delete, and when
 Ensure orderly disposal
- Automated or manual delete
- Audit trails
© 2007 Princeton Softech, Inc.
21
Getting Started with Archiving Data
© 2007 Princeton Softech, Inc.
Step 1: Business Policies Drive Archiving
 Identify applications that manage regulated data
 Build consensus among stakeholders on retention and
retrieval:
- Business owners, application developers, storage
- Include CFO, legal, compliance, security
 Document your business policies:
- Types of data (Active, Inactive/Historical, Reference)
- Processes for Archiving, Viewing, Retrieving Objects
- Processes for Compliance and Disposal
© 2007 Princeton Softech, Inc.
23
Define Retention Policies at Business Layer
Order
Management
Archive Orders for any Order Type, Order Category,
Customer, Order Numbers, Order Dates, Creation Date
values
Purchase Order
Archive Blanket Agreements and Purchase Orders by a
specified Last Activity Date
Work in Process
Archive Discrete Jobs and Repetitive Schedules for any
Accounting Period
Accounts
Receivable
Archive Transactions (other than transactions applied to
commitments) posted to General Ledger or prior to a Cut Off
Date value
© 2007 Princeton Softech, Inc.
24
Step 2: Define the Storage Architecture
 Technical Safeguards (Security)
 Data integrity safeguards
- Access controls – authentication, authorization
- Recording media (WORM media or subsystems)
- Secure audit trails, duplicate copies, etc.
 Data privacy safeguards
- Access controls – authentication, authorization
- Data encryption
- Access logs, audits and reports
*Exact requirements depend on regulatory environment
© 2007 Princeton Softech, Inc.
25
Storage Goals and Criteria
Goals:
 Cost effective
 Easy to manage and scale
 Ensure accessibility for many years
Selection Criteria:
 Storage capacity
 Availability
 Manageability
 Performance
 Cost
Existing storage technology to be combined with new storage
technology (e.g. ATA disk storage) to help reduce cost.
© 2007 Princeton Softech, Inc.
26
Step 3: Don’t Forget About Process
 Important regulatory requirements specify that the data must
remain unaltered and accessed only by the proper
individuals.
 Accessibility, storage and audit policies each result in a
specific set of processes that govern their maintenance and
education.
 Consistent, repeatable, controlled, documented archive and
access methods and tools
© 2007 Princeton Softech, Inc.
27
Possible Alternatives to Archiving
 Tune or partition the database
 Add capacity
- Processors, storage
 Back up the database
 Purge data
 Alleviate symptoms temporarily,
but…
- Inflate costs
- Do not address underlying
data growth
© 2007 Princeton Softech, Inc.
28
Archiving Solution Technical Requirements
 Basic Requirements
- Support archive, purge and retrieve operations, including selective retrieve
- Ensure referential integrity of archived data
- Increase database performance and minimize batch windows
- Ensure security and maintain access control of archived data
- Archive data stored in database as well as the File System, and maintained
linkage
 Archival Definition
- Allow scope of archive and cascading purge to be controlled
- Maintain schema information in addition to archive data
- Provide pre-defined archival configurations for key objects
- Allow pre-defined archival configurations to be modified to reflect configurations
made to applications
© 2007 Princeton Softech, Inc.
29
Archiving Solution Technical Requirements (2)
 Archive Data Storage and Access
- Provide access to archived data from within the application
- Allow data to be archived to another database and offline storage,
and integrate with hierarchical storage management
 Example – Archive to IBM DR550, long term retention
 Archive Management
- If there is an interruption in the archive, purge and retrieve
processes, be able to recover from the point of the interruption
- Report on what data is archived
- Provide administrative tools to manage the archives
© 2007 Princeton Softech, Inc.
30
Creating and Managing Archived Data
1. Identify the data to archive
2. Define the data to delete
3. Select Archive File
storage
4. Create the archive
5. Research, report, retrieve
© 2007 Princeton Softech, Inc.
31
Choosing the Best Access Method
 Native application access
- Convenient for functional users
- Can slow down online transaction processing
 “Self-Help” access (Canned Reports, Query Tools)
- Convenient for functional users
- No IT services required
 Application independent access
- Preserves a complete view of historical business records
regardless of originating application or version
- Facilitates decommissioning, upgrade and migration paths
© 2007 Princeton Softech, Inc.
32
Summary of Advice
 Recognize that IT owns Infrastructure, but the Business owns
the data
 Improve functional processes by tiering services by functional
need
- Higher service levels on current transactions
- Lower-cost, lower service levels on historical transactions
 Limit liability by ensuring real-time compliance controls are
sustained and documented in your historical retention
processes and tools
- Respond quickly and accurately to audit requests
- Reduce costs of discovery
© 2007 Princeton Softech, Inc.
33
Introducing Princeton Softech Optim™
© 2007 Princeton Softech, Inc.
Princeton Softech Optim™
 Single solution for managing
enterprise application data throughout
the information lifecycle
 Applies business rules to assess, classify, archive, subset,
de-identify, store, retain and access enterprise application
data
 Supports and scales across applications, databases,
operating systems and hardware platforms
 Optimizes the business value of your IT infrastructure
© 2007 Princeton Softech, Inc.
35
Princeton Softech Optim
Production
Historical
Retrieved
Current
Archives
Archive
Retrieve
Historical
Data
Reference Data
Reporting
Data
Open Access to Application Data
Application
© 2007 Princeton Softech, Inc.
ODBC / JDBC
XML
Report Writer
36
Optim™ - The Enterprise Data Management Solution
Custom
& Pkgd
Apps
Oracle PeopleSoft
E-Biz
Suite Enterprise
JD
Edwards
E1
Siebel
Amdocs
CRM
Enterprise Data Management Functions
Relationship Engine
Oracle
SQL
Server
AS400
© 2007 Princeton Softech, Inc.
Sybase
Informix
IMS
VSAM
DB2
UDB
DB2
Adabas
Seq. Flat
Files
Legacy
37
Optim Controls Data Growth
Reverse Multiplier
80 GB Archive
X 6 Environments
Active Data
120 GB
Inactive Data
80 GB
© 2007 Princeton Softech, Inc.
480 GB Reclaimed
Archive
80 GB
38
Success: Data Growth Management
 Finlay Fine Jewelry, $900mm fine jewelry retailer
 55% data growth in key retail management applications
- Slow response time impaired inventory
replenishment; threatened sales during peak periods
- Exhausting DBA resources with intensive tuning;
increasing storage capacity
 Optim Enterprise Data Management yields success
- 60% response time improvements
- Increased “open for business” hours ensured
inventory stocking levels, supported sales during
holiday shopping season
- Reclaimed 100 GB storage capacity at first pass;
$1.8mm 5-year savings
© 2007 Princeton Softech, Inc.
39
Princeton Softech, the Leader in Enterprise Data Management
© 2007 Princeton Softech, Inc.
Princeton Softech
 Proven leader in Enterprise Data Management
- Solving complex data management issues since 1989
- In-depth functional knowledge of mission-critical
applications and the business rules that govern them
- Over 2,400 customers worldwide
 Including nearly half of the Fortune 500
- Only true enterprise solution: across applications,
databases, hardware platforms and operating
systems
© 2007 Princeton Softech, Inc.
41
Support that Scales the Enterprise
 Custom & packaged applications
 ERP & CRM applications
- Oracle® E-Business Suite
- PeopleSoft® Enterprise
- JD Edwards® EnterpriseOne
- Siebel®
- Amdocs® CRM
 Databases: Oracle, DB2, UDB,
Sybase, SQL Server, Informix, Legacy
 Platforms: Windows, Unix, Linux, z/OS
 All storage environments
© 2007 Princeton Softech, Inc.
42
Princeton Softech: Customers
© 2007 Princeton Softech, Inc.
43
© 2007 Princeton Softech, Inc.