Download Crew Scheduler – External Data Updater

Crew Scheduler - External Data Updater
5/7/2017
p 1 of 9
Crew Scheduler – External Data Updater
The External Data Updater (EDU) is a small stand-alone program which imports and updates Resource data into the
CrewScheduler. The EDU also handles payroll codes, locations, and location mileage entries, and can also perform bulk
updates of shift status. Updates can be performed interactively, or run automatically as a windows scheduled task.
This document is current as of EDU version 1.0.61. RD 1/12/07.
Contents
CREW SCHEDULER – EXTERNAL DATA UPDATER..................................................................................................... 1
CONTENTS ................................................................................................................................................................................ 1
OVERVIEW................................................................................................................................................................................ 1
Features ............................................................................................................................................................................... 1
MAIN SCREEN........................................................................................................................................................................... 2
To import or update resources ............................................................................................................................................. 2
Setup Buttons (blue) ............................................................................................................................................................. 3
Action Buttons (Green/Red) ................................................................................................................................................. 3
Field Map............................................................................................................................................................................. 3
Built-in Attributes ................................................................................................................................................................ 4
OTHER UPDATES ...................................................................................................................................................................... 4
Payroll Codes ...................................................................................................................................................................... 4
Locations ............................................................................................................................................................................. 4
Location Mileage ................................................................................................................................................................. 5
SHIFT STATUS SCREEN ............................................................................................................................................................. 5
To update shift status ........................................................................................................................................................... 5
LOG FILE .................................................................................................................................................................................. 6
RUNNING IN BATCH MODE: ....................................................................................................................................................... 6
Running automatically using the Windows Task Scheduler: ............................................................................................... 6
NOTES AND CAVEATS ............................................................................................................................................................... 7
APPENDIX 1 – EXAMPLE DRIVER FILE ...................................................................................................................................... 7
Explanation of driver file items............................................................................................................................................ 8
APPENDIX 2 – DATABASE CONNECTION STRINGS ..................................................................................................................... 8
Example connection strings ................................................................................................................................................. 8
APPENDIX 3 - EXAMPLE OF EXTERNAL DATA SOURCE (EXCEL) ................................................................................................ 9
Overview
The EDU reads data from a wide variety of external data sources: SQL Server databases, Access MDB databases, Excel
spreadsheets, etc. This external data is imported and updated into the CrewScheduler database
The update is controlled by a text-based driver file that provides the EDU with all needed parameters. The key to the update
is a mapping of data fields (External field name -to- CrewScheduler field Name), along with the designation of a single Join
Field used to join the data together.
For example, if the external data has a field called “EmpNo” which corresponds to a CrewScheduler Resource Attribute
called “Employee Number”, then the driver file would map the external field “EmpNo” to the internal field “Employee
Number”. Furthermore, if the Employee Number serves as the primary key, or unique identifier, of each employee in the
database, then the “Employee Number” field would be designated as the Join Field.
During an update, the EDU takes each record from the External data, and looks for a record in the Internal database with a
matching Join Field. If a matching record is found, then the fields are copied from the External to the Internal database. If a
matching record is not found, a new Resource is created in the Internal database to store the information.
Features

Data Import/Update: From external sources, the following data can be imported and updated into CrewScheduler
o Employees and their attributes
Crew Scheduler - External Data Updater


5/7/2017
p 2 of 9
o Vehicles and their attributes
o Payroll codes: Department, Earnings, Project, Assignment, etc.
o Locations
o Location Mileage (distances from Location to Location)
Shift Status Update: the EDU can refresh shift and slot status, to keep schedules coordinated with certifications or
other constraints that have a temporal expiration.
Company Logo: the EDU provides an interface to set or update the company logo image
Main Screen
The EDU can be run interactively, or in batch mode. To run it interactively, simply double click on the executable file
“CrewSkedDataUpdater.exe”, and you’ll see the main screen, which handles updates for resources, payroll codes,
locations, and location mileage items.
The above example shows the main EDU screen, with settings loaded from the driver file “CrewSkedExternalUpdate.ini”.
This screen can be used to load, edit, and save various update scenarios as different driver files.
To import or update resources
1.
2.
3.
4.
5.
Run the EDU, bringing up the main screen.
Set the “Internal Data Source” to point to your CS database. (Double-click the textbox for a helper dialog).
Use [Connection] or [Excel] to point to your external data source.
a. If Excel, type the datasheet name in the “Table/View” box.
b. If not excel, type the table, query, or view name in the “Table/View” box.
Type the desired resource type in the “Update Resources of Type” box. This should be a name from your CS
“Resource Type” list, found in the screen: Main > Admin > Attributes. Typically this value is “Personnel”.
In the “Field Map” box, type in the fields you want to update. Each line in the box should contain three items,
separated by commas:
Crew Scheduler - External Data Updater
5/7/2017
p 3 of 9
a.
b.
c.
6.
7.
8.
Field name in the EXTERNAL data source.
Attribute name in the CS database. (From screen: Main > Admin > Attributes)
“1” if this field is the Join field, otherwise “0”. The Join field is the unique identifier used to match items
between the internal and external data sources.
Click [Validate Settings], and make any needed corrections to your settings.
Once the settings validate successfully, click [Update the Database] to begin the Import/Update job.
Before you close the EDU, click [Save] to save your settings as a new, or existing, driver file.
Setup Buttons (blue)
The blue buttons perform various setup tasks:
Blue Button
Load (Driver File)
Refresh
Save
Edit
Connection
Excel
Load (SQL Script)
Description
Browses to and loads a driver file into the main screen. All
current screen settings will be discarded.
Refreshes the main screen, using the latest saved version of
the driver file. This is used when editing and saving the
driver file in another application such as Notepad.
Saves all current settings to a new or an existing INI file.
Brings up the current driver file in Notepad for editing.
Brings up a dialog for creating a connection to databases like
SQL Server, MS Access, Oracle, etc.
Browses to and selects a Excel spreadsheet file.
Browses to and selects a text file for SQL post-processing.
Action Buttons (Green/Red)
Once the desired settings are loaded (either manually or from a driver file), there are two actions that can be performed:
Action Button
Validate Settings
Update the Database
Description
Checks the current settings for errors and reports any
problems. No updating will occur.
This performs an update, using the current settings. The bar
at the bottom of the screen will show the update’s progress.
Field Map
The “Field Map” box states which external data fields correspond with CrewScheduler attributes. Each line in the Field Map
box contains three items, separated by commas:
1. Field name in the EXTERNAL data source.
2. Attribute name in the CS database. (From screen: Main > Admin > Attributes), or a built-in attribute keyword.
(Explained in next section)
3. “1” if this field is the Join field, otherwise “0”.
The Join field is the unique identifier used to match items between the internal and external data sources. There must be
one, and only one, Join Field in the field map list .
Crew Scheduler - External Data Updater
5/7/2017
p 4 of 9
Built-in Attributes
If you want to update some of the built-in resource attributes, there are special keywords to be used in the Field Map. The
keyword should be used in the second position, where you would normally use the CS attribute name (AKA the “Internal
Field”).
Built-in Attribute
Keyword
Description
This is a 1/0 (or true/false) value that corresponds to the resource’s Active
status. Setting this value to FALSE is analogous to pressing the [Set Inactive]
button on screen Main > Admin > Resources.
Besides 1/0 and True/False, the following values are also accepted by the
EDU:
Active/Inactive status
IsActive
Values that correspond to Active:

"1", "a", "A", "L", "l", "TRUE", "True", "true"
Values that correspond to NOT-active:

"0", "T"*, "t"*, "NULL", "null", "FALSE", "False", "false", <blank
values>
*Many HR systems use “T” for status “Terminated”.
Department Code
DeptCodeID
Assignment Code
AssignCodeID
Payscale Code
PayScaleCodeID
The built-in attribute “Department Code”. Data values should correspond to
the “Codes” column of screen Main > Admin > Payroll Codes > [Code type:
Department]
The built-in attribute “Assignment Code”. Data values should correspond to
the “Codes” column of screen Main > Admin > Payroll Codes > [Code type:
Assignment ]
The built-in attribute “Payscale Code”. Data values should correspond to the
“Codes” column of screen Main > Admin > Payroll Codes > [Code type:
Payscale ]
Other Updates
Payroll Codes
To import/update payroll codes, select the “Other Update” radio button, and choose the desired code type from the dropdown
list. The CS fields that can be updated are: Code, Name, and Description.
Below is an example field mapping for payroll codes:
External name
Code
Name
Desc
Internal Name
Abbrev*
Name
Description
Join field?
1
0
0
*NOTE: For the payroll “code” value, you need to use “Abbrev” for the Internal Name, even though the Payroll Codes
screen uses the term “Code”.
Locations
To import/update locations, select the “Other Update” radio button, and choose “Locations” from the dropdown list. The CS
fields that can be updated are: Code, Name, Address, City, State, PostalCode.
Below is an example field mapping for locations:
External name
Code
Internal Name
Code
Join field?
1
Crew Scheduler - External Data Updater
Name
Address
City
State
Zip
Name
Address
City
State
PostalCode
5/7/2017
p 5 of 9
0
0
0
0
0
NOTE: When new locations are imported to CS, they are not tied to any Division or Region, and will not appear in the CS
GUI. You must do a manual database update to set a DivisonID for the newly-added locations. Here is the procedure:
1.
2.
Look in table Division and find the DivisionID for your newly-added locations.
Run the following SQL script:
Update Location set DivisionID = <YourDivisionID> Where DivisionID is null
Location Mileage
To import/update location mileage, select the “Other Update” radio button, and choose “Location Mileage” from the
dropdown list. The only CS field that can be updated is “Miles”. Note that the external data source must have both a “From”
and a “To” location. These should both map to the same Internal Name, either “Code” or “Name”.
Below is an example field mapping for locations:
External name
FromCode
ToCode
Miles
Internal Name
Code
Code
Miles
Join field?
1
0
0
Shift Status Screen
The Shift Status screen performs shift status updates. This is desirable when schedules contain employee certifications or
other date-based information that can expire or otherwise require re-evaluation. Also, the “Sync/Repair” option corrects the
issue where slot constraints are modified after shifts have already been scheduled.
To update shift status
1.
2.
3.
Open the EDU with an INI file that points to your desired CS database.
From the main screen, click the Utilities menu, then “Update Shift Status”.
From the Shift Status screen:
a. Select start and end dates. The end date can be left blank, to indicate “forever”.
b. Select a region, if desired.
c. Select a pause interval. Higher values take longer, but have less impact on the server.
d. Select maximum minutes to run. The update will continue until ONE of the following has occurred:
i. The max. minutes of processing has been reached.
Crew Scheduler - External Data Updater
e.
f.
5/7/2017
p 6 of 9
ii. All shifts in the date range have been processed.
Click [Count Shifts] to see how many shifts are in the set to be processed.
Click [Update Shift Status] to begin processing. You can cancel the operation with [Cancel]
Log File
After an update or a validation, a log file is created which contains all relevant status information. The format of the filename
is: “CS_EDULog_” + <name of driver file> + “.txt”. For example, if the driver file “Employees.ini” was run, the resulting
log file would be called CS_EDULog_Employees.txt.
NOTE: The EDU always appends to an existing log file. If you want to empty out a log file, use the [Purge Log] button.
Running in batch mode:
To run a resource update in batch mode, you simply provide the driver file name as a command-line argument. At the DOS
prompt, you would type:
C:\> CrewSkedDataUpdater Employees.ini
To run a shift status update in batch mode, you need to provide the keyword “shift”, followed by the driver file name and the
number of minutes to process. The example below will run a shift status update for 10 minutes, or until it runs out of shifts.
Below, the “Employees.ini” file is only used to get the CS database connection. The other settings in that file have no
bearing on a shift-status update, and are ignored:
C:\> CrewSkedDataUpdater SHIFT, Employees.ini, 10
If you want finer control of the shift status job, you can pass additional arguments, in this format:
CrewSkedDataUpdater SHIFT, <driverFile>, <runMinutes>, [startDate], [endDate], [regionName], [PauseInterval]
Required items are in <angle brackets>, optional items are in [square brackets]. Use “null” to skip an optional item and use
its default value.
Running automatically using the Windows Task Scheduler:
1.
Create a DOS batch file containing your desired call or calls to the EDU. The example below performs three jobs: a
resource update for employees, a resource update for vehicles, and a shift-update for 10 minutes.
Example batch file, called RunEDU.bat:
CrewSkedDataUpdater Employees.ini
CrewSkedDataUpdater Vehicles.ini
CrewSkedDataUpdater SHIFT, Employees.ini, 10
2.
In Windows, go to Start > Programs > Accessories > System Tools > Scheduled Tasks:
3.
Double click on "Add scheduled task" and use the wizard to schedule the program “RunEDU.bat” at the desired
time and periodicity (daily, weekly, etc.).
Crew Scheduler - External Data Updater
5/7/2017
p 7 of 9
Notes and Caveats

When new resources are added to CS, they will have a null Home Region. To assign them a region, go to Main >
Admin > Regions, and locate the new resources in the “Unassigned Resources” grid. If you have another existing
attribute that maps to region (e.g. Department Code), you can create a post-processing script that automatically sets
or updates the home region at each run of the EDU. Your post-processing script will make calls to the provided
utility script SetRegionForAttribValue.sql or SetRegionForAttribValueBuiltIn.sql.

The Driver file identifies CS Resource Types and Attributes by name. If these names are changed in the CS DB, the
field-mapping logic will fail.

Join Field values must be unique for each Resource. If more than one Resource share a join field value, then only
one of those Resources (chosen arbitrarily) will be updated.

Changing a join field value within CS will break that record’s connection to external data updates.

If Resource attributes are edited directly in the CrewScheduler application (via the Resources screen), they may be
overwritten by an EDU update. The intent of the EDU is to serve as a replacement for manual editing of Resource
Attributes.

Most actions performed by the EDU are logged in the CS Audit log, as a message of type “Update”. The “Login”
column for these messages is set to “ExternalUpdater”.
Appendix 1 – Example Driver File
Below is an example driver file for the EDU. This is a simple text file, where each line contains a keyword, followed by a
comma, followed by one or more parameters. Blank lines and lines beginning with a “#” symbol are ignored.
# CrewScheduler External Data Updater (EDU) driver file; created: 12/8/2005 6:38:06 PM
# Blank lines and lines starting with '#' are ignored.
# Line items should be separated by commas.
#==================================================================
# Internal DB to update (CrewScheduler):
IntDataSource, Provider=SQLOLEDB.1;Persist Security Info=False;User ID=sa;Initial
Catalog=Crewsked_dev;Data Source=LOCALHOST
# External DB (non-CrewScheduler):
ExtDataSource, C:\CrewScheduler\EDU\SmallTestData.xls
# Table/View name of External data source:
ExtTable, Sheet1$
# Resource Type to update:
ResourceType, Personnel
# Other update type (if not resource-mode):
OtherUpdateType,
# Field mapping:
#-----------------------------------------------------------------#
[External Field],
[Internal Field],
[IsJoinField]
#
------------------------------------------FieldMap,
EmpNum,
Employee Number,
1
FieldMap,
Last,
Last Name,
0
FieldMap,
First,
First Name,
0
FieldMap,
Phone,
Primary Phone,
0
FieldMap,
PTOTotal,
PTO Balance,
0
FieldMap,
LicenseExpiration,
License Expires,
0
FieldMap,
Status,
IsActive,
0
FieldMap,
Department,
DeptCodeID,
0
FieldMap,
Payscale,
PayScaleCodeID,
0
# Post-update SQL script file:
PostSQL, C:\scripts\YourScriptToPutNewPersonnelIntoRegions.sql
Crew Scheduler - External Data Updater
5/7/2017
p 8 of 9
Explanation of driver file items

IntDataSource, Provider=SQLOLEDB.1;Password=…
This is a connection string that specifies the CrewScheduler database to be updated. See the appendix for
information on creating connection strings.

ExtDataSource, Provider=SQLOLEDB.1;Password=…
This is a connection string that specifies the external database; or a path to an XLS file.

ExtTable, Sheet1$
This is the name of the table/view/datasheet within the external database. For a conventional database, this will be
the name of a table, query, or view. For an excel file, the datasheet name plus a “$” is used, e.g. “Sheet1$”.

ResourceType, Personnel
This is the CrewScheduler Resource Type that will be updated.

OtherUpdateType,
For non-resource updates, this is one of: Locations, Location Mileage, Payroll Codes, Assignment Codes,
Department Codes, Company Codes, Earnings Codes, Deduction Codes, Payscale Codes..

FieldMap, EmpNum, Employee Number, 1
This is a field mapping. The above example maps the external field “EmpNum” to the internal field “Employee
Number”. The “1” at the end of the line specifies this field as the join field. There must be one, and only one, Join
Field in the field mapping list

PostSQL, C:\SomeScript.sql
This is an optional SQL script to be run on the CrewScheduler database, after the update has completed. This
feature is designed to perform open-ended housekeeping chores, such as setting the Home Region for newly-added
Resources based on a Resource Attribute.
Appendix 2 – Database Connection Strings
A Connection String is a single line of information providing details on connecting to a variety of data sources. To create a
Connection String in Windows, do the following:
1.
2.
3.
4.
5.
6.
7.
Create a new (empty) text file called “FOO.UDL”.
Double-click on FOO.UDL.
A screen called “Data Link Properties” will pop up.
On the “Provider” tab, select your desired data provider.
On the “Connection” tab, select the appropriate properties.
Close the “Data Link Properties” screen.
Open FOO.UDL in Notepad. The Connection String is the last line of this file.
Example connection strings
Below are some example connection strings. Note that a connection string must be on one line of text. Do not place any line
breaks or carriage returns in a connection string.

SQL Server database:
Provider=SQLOLEDB.1;Password=abc123;Persist Security Info=True;User ID=sa;Initial
Catalog=CrewScheduler;Data Source=MyServer

Access MDB database:
Provider=Microsoft.Jet.OLEDB.4.0;Data
Source=C:\1rusty\Projects\CrewScheduler\Database\MetaDB_CrewSked.mdb;Persist Security Info=False
Crew Scheduler - External Data Updater
5/7/2017
p 9 of 9
Appendix 3 - Example of external data source (Excel)
Below is an example of how a typical Excel file would look, for each type of upate.
Resources (Employees):
EmpNum
Last
First
Phone
Status
Department
PayScale
PTOTotal
1619
Doe
Jane
111-2222
LicenseExpiration
3/12/2009
Hired
1/23/1995
1
EMS
3
12.5
1097
Doe
John
222-3333
4/10/2007
6/26/1999
1
Disp
2
39.2
117
Jones
Bob
444-5555
11/2/2006
1/9/2000
1
EMS
2
107.35
4302
Smith
Shirley
777-8888
5/22/2010
7/10/2001
0
HR
1
19.22
1811
Thompson
Tony
999-1111
12/9/2008
3/30/1990
1
Admin
4
74.88
Department Codes:
abbrev
FLD
Admin
Disp
name
Field
Administration
Dispatch
desc
Field Operations
Administration Dept; Salaried workers
Dispatchers
Earnings Codes:
abbrev
VA
SI
HO
ST
CP
EQ
JD
name
Vacation
Sick
Holiday
Scheduled Time
Cell Phone
Equipment
Jury
desc
Vacation PTO
Sick Time
Holiday pay
Regular Scheduled Time (Default code for Shifts)
Cell Phone Reimbursement
Misc Equipment Reimbursement
Jury Duty
Locations:
Code
HQ
S1
S2
Name
Headquarters
Station 1
Station 2
Address
123 Main St
333 Elm St.
9234 Calle Cerveza
Location Mileage:
FromCode
HQ
HQ
S1
ToCode
S1
S2
S2
Miles
12.5
6.3
18
City
Los Angeles
Yuba City
Santa Barbara
State
CA
AZ
CA
Zip
90274
80221
93101