Download Remote data replication

February Webinar: The Basics of Remote Data
Replication
Presenter: John Tkaczewski
Duration: 30 minutes
Presenter Introduction
John Tkaczewski
• Co-Founder and President of Unlimi-Tech Software
• Oversees creative and technical vision for
FileCatalyst Webmail and FileCatalyst Workflow
• Blog: http://www.filecatalyst.com/blog
• Sign up for our next user forum in London, UK next
week on Monday (Feb 25th, 2013)
2 | © Copyright FileCatalyst, 2013
Agenda
•
•
•
•
•
•
•
•
•
•
•
Remote data replication overview
Remote data replication scenarios
Data replication vs. sync
Replication tools within FileCatalyst
Platform specific gotchas
Data replication across operating systems
File system events
HotFolder to HotFolder replication
DEMO
30 day free trial and licensing options
Questions (please type them in as we go)
3 | © Copyright FileCatalyst, 2013
Remote data replication overview
Remote data replication
• Data replication, also known as remote folder mirroring or
folder synchronization, occurs when the same folder contents
is duplicated across 2 or more computers, often across wide
area networks (WANs) or geographically dispersed offices
• Must support large data sets:
– Complex directory tree structures
– Thousands or even millions of files
– Combination of small and very large files up-to terabyte
• Wide Area Files System, ideally must work at file system level
to prevent continuous scanning of folders.
4 | © Copyright FileCatalyst, 2013
Remote data replication scenarios
Remote data replication is often used when organizations
require consistent data across multiple geographical locations.
• Possible use cases:
–
–
–
–
Remote Backups for disaster recovery
Mirroring of Data across multiple offices (for faster data access)
Moving the data to the edge (closer to the consumers)
Global Content Distribution Network (CDN)
5 | © Copyright FileCatalyst, 2013
Data replication vs. sync
• Replication (or mirroring) involves a single master node and
one or multiple slave nodes
• Sync involves multiple nodes all acting as master nodes. Every
node can modify any file in the folder.
– Sync requires complex rules to prevent and handle collisions
– Sync across more then 2 master nodes is even more challenging
– Sync is not recommended with FileCatalyst HotFolder
6 | © Copyright FileCatalyst, 2013
Replication tools within FileCatalyst
HotFolder
7 | © Copyright FileCatalyst, 2013
Replication tools within FileCatalyst (Cont’d)
Hotfolder features useful for data replication:
• Scheduler for tasks and bandwidth
• File System Events
• Delta (RSYNC)
• Data-set rules
• Monitoring and notifications
• ZIP chunking
• MD5
• File transfer cache
8 | © Copyright FileCatalyst, 2013
Platform specific gotchas
Common file attributes for all platforms:
File name, Date Created, Date Modified, Owner
Operating System
File Attributes
MacOS
Extended attributes (xattr), Resource Forks, Symlinks, Permissions
(Read Write Execute), Access Control Levels (ACL’s)
Windows
Permissions, Attributes: Read Only – Archive - Hidden – System,
Security Attributes
Linux
Permissions (group, user, other)
9 | © Copyright FileCatalyst, 2013
Data replication across operating
systems
• Extended File Attributes are LOST when moving a file between
operating systems.
• But it’s OK because extended attributes may not be always
needed.
10 | © Copyright FileCatalyst, 2013
File system events
The file system events provides a way for an application to ask
for notification when the contents of a directory hierarchy are
modified.
For example, an application can use this to quickly detect
when the user modifies a file within a folder using another
application.
•
Avoid scanning large folders to detect changes! Use file system events
•
File system events only work on local drives (no SMB or other network drive
technology)
•
Transfer the initial file set without file system events, THEN turn it on.
11 | © Copyright FileCatalyst, 2013
HotFolder to HotFolder replication
• 2 Hotfolders connected to the same FileCatalyst Server
• Replication can be done between both HotFolders
• All the data will go through the server, however no data will
be stored on the server
• This reduces the amount of disk space required at the server
12 | © Copyright FileCatalyst, 2013
Demo: Acceleration with remote data
replication using FileCatalyst HotFolder
30 day free trial and licensing cost
To request free trial of FileCatalyst Direct visit
http://www.filecatalyst.com/trial
Licensing Options:
• Perpetual license is about $10,000 USD (one-time fee)
• Leased license is about $500/month
• Hosted License on a shared server is about $200 per month
14 | © Copyright FileCatalyst, 2013
Thank you for attending our February webinar!
Questions? Please type in your questions.