Download SEDS Developer*s Manual - Michigan State University

Facility for Rare Isotope Beams
DEVELOPER’S MANUAL
Project Name
Project Code
Account
Department
Project Leader
Project Coordinator
Serialization of EPICS Datatype Standards
SEDS
Controls and Computing
Sheng Peng
Vasu Vuppala
Name
Prepared By
Reviewed By
Approved By
Aaron Barber
Signature
Date
7/14/14
SEDS Developer’s Manual
Page 2 of 13
Revision History
Version
1.0.0
Date
7/14/14
Author
Aaron Barber
Description
Initial document
Facility for Rare Isotope Beams
U.S. Department of Energy Office of Science | Michigan State University
East Lansing, MI 48824-1321 • Ph.: (517) 355-9672 • Fax: (517) 353-5967
www.frib.msu.edu
SEDS Developer’s Manual
Page 3 of 13
Table of Contents
1
2
3
4
5
INTRODUCTION ............................................................................................................................................. 5
1.1
PURPOSE .........................................................................................................................................................5
1.2
DEFINITIONS, ACRONYMS, AND ABBREVIATIONS .....................................................................................................5
1.3
REFERENCES .....................................................................................................................................................6
OVERVIEW ..................................................................................................................................................... 6
2.1
MOTIVATION ...................................................................................................................................................6
2.2
BASIC DESIGN REQUIREMENTS AND FEATURES ........................................................................................................6
2.2.1
Self-Describing Data ............................................................................................................................6
2.2.2
Persistence ..........................................................................................................................................6
2.2.3
Main Functions ....................................................................................................................................6
2.3
ARCHITECTURE DESIGN ......................................................................................................................................7
2.3.1
Part I: Java Application ........................................................................................................................7
2.3.2
Part II: JSON Schemas ..........................................................................................................................7
2.3.3
Control System Studio .........................................................................................................................7
2.3.4
Database ..............................................................................................................................................7
JAVA APPLICATION ........................................................................................................................................ 7
3.1
INTERFACE SERVICES ..........................................................................................................................................7
3.2
PACKAGE SUMMARY ..........................................................................................................................................8
3.3
JAVA EE – JSON ..............................................................................................................................................9
3.4
JSON VALIDATOR .............................................................................................................................................9
3.5
VTYPE.............................................................................................................................................................9
JSON SCHEMA ............................................................................................................................................. 10
4.1
VALIDATORS...................................................................................................................................................10
4.2
SEDS SCHEMA LAYOUT ....................................................................................................................................10
MODIFYING SEDS DATATYPES ..................................................................................................................... 12
5.1
MODIFYING THE PROTOCOL ..............................................................................................................................12
5.2
MODIFYING THE JAVA IMPLEMENTATION .............................................................................................................12
Facility for Rare Isotope Beams
U.S. Department of Energy Office of Science | Michigan State University
East Lansing, MI 48824-1321 • Ph.: (517) 355-9672 • Fax: (517) 353-5967
www.frib.msu.edu
SEDS Developer’s Manual
Page 4 of 13
List of Figures and Tables
Table 1 Definition, Acronyms, and Abbreviations .........................................................................................................5
Facility for Rare Isotope Beams
U.S. Department of Energy Office of Science | Michigan State University
East Lansing, MI 48824-1321 • Ph.: (517) 355-9672 • Fax: (517) 353-5967
www.frib.msu.edu
SEDS Developer’s Manual
Page 5 of 13
1 Introduction
National Superconducting Cyclotron Laboratory (NSCL) is a world leader in rare isotope research and
nuclear science education. NSCL scientists and researchers conduct advanced research in fundamental
nuclear science, nuclear astrophysics, and accelerator physics. To facilitate the research, NSCL operates
multiple particle accelerators. The control system at NSCL enables operators to control the various
devices in the lab: ion sources, cyclotrons, beam lines, experimental devices, and auxiliary equipment. It
is based on the Experimental Physics and Industrial Control Systems (EPICS) standard. EPICS is a
distributed soft real-time control system for large scientific experiments.
SEDS is a protocol for data processing. SEDS provides a set of software designs for high level composite
data types suitable for the application level data exchange between EPICS V4 network endpoints. SEDS
provides protocol documentation and datatype schemas. The intent is to serialize and deserialize the
datatypes as self-describing data for persistence. SEDS protocol specifies JSON as the self-describing
data language used to represent all SEDS data structures. This project will provide users the ability to
transmit standardized data between programming languages and to store standardized data in a
database.
This manual details the development progress of the SEDS project of FRIB so far and how to continue
the development in future.
1.1 Purpose
The purpose of this documentation is to detail the development progress of the traveler system of FRIB
so far and how to continue the development in future.
It is intended for the following audience:


Members of the Controls department that are overseeing the project (the Customer); they need
to make sure that the design fulfills the requirements;
Members of the design team at Controls department;
1.2 Definitions, Acronyms, and Abbreviations
Table 1 Definition, Acronyms, and Abbreviations
Item
SEDS
Controls
EPICS
NSCL
FRIB
CSV
Description
Serialization of EPICS Datatype Standards
FRIB’s Controls and Computing Department
Experimental Physics and Industrial Control System
National Superconducting Cyclotron Laboratory
Facility for Rare Isotope Beams
Comma Separated Values
Facility for Rare Isotope Beams
U.S. Department of Energy Office of Science | Michigan State University
East Lansing, MI 48824-1321 • Ph.: (517) 355-9672 • Fax: (517) 353-5967
www.frib.msu.edu
SEDS Developer’s Manual
JSON
PV
Page 6 of 13
JavaScript Object Notation
Process Variable. Identifier for a data item in EPICS.
1.3 References
1. SEDS Software Requirement Specifications
2. SEDS Software Architecture
2 Overview
2.1 Motivation
There are several standard well-known data types within FRIB and EPICS. These types are the scalars,
arrays, and tables. Each of these types can contain additional composite data such as alarm or
timestamp data.
A serialization protocol to represent these data types in a self-describing data format is necessary. JSON
is a universal structure to represent data and can be interpreted by any programming language, can be
easily processed using web sockets, and stored in a database.
SEDS is a protocol that defines the composite data associated with the standard well-known data types.
SEDS provides a Java implementation to serialize data in Java memory into JSON grammar, allowing the
benefits of JSON to extend to the standard data types of FRIB and EPICS.
2.2 Basic Design Requirements and Features
This section describes the basic requirements and features of this software product. For more details
about these basic and more advanced ones, please refer to the SEDS Software Requirement
Specifications.
2.2.1 Self-Describing Data
The standard datatypes must be self-describing. The data must contain sufficient metadata relevant to
the standard type as well as the raw data associated with each piece of metadata. JSON is lightweight
and is self-describing.
2.2.2 Persistence
The standard datatypes must be able to exist beyond the memory of a program. The data should be
prepared to be stored by formatting the data as a JSON string.
2.2.3 Main Functions
The following are the main functions of the implementation of the SEDS protocol:
1. Construct standard datatypes in memory
Facility for Rare Isotope Beams
U.S. Department of Energy Office of Science | Michigan State University
East Lansing, MI 48824-1321 • Ph.: (517) 355-9672 • Fax: (517) 353-5967
www.frib.msu.edu
SEDS Developer’s Manual
Page 7 of 13
2. Serialize the datatypes to a JSON string
3. Deserialize a JSON string into the standard datatype in memory
2.3 Architecture Design
The architecture of the system is composed of two parts: the Java application, and the JSON schemas.
For more details, please refer to the SEDS Software Architecture document.
2.3.1 Part I: Java Application
This is the implementation that follows the specifications of the SEDS protocol. The Java application
provides the utilities to construct SEDS datatypes and format the datatypes according to the SEDS
protocol.
The Java application contains the processing objects to work with EPICS VType data structures and CSV
files to construct SEDS datatypes.
The Java application validates all data according to the JSON schemas.
2.3.2 Part II: JSON Schemas
These are the definitions provided by the SEDS protocol. The JSON schemas are the datatype definitions
that all SEDS data is expected to be transmitted and processed as.
The schemas are the process in which a user is able to determine whether JSON data matches a SEDS
datatype.
2.3.3 Control System Studio
The Java implementation provides the capability of taking a JSON string (that matches a SEDS datatype)
into an EPICS VType data structure which is compatible with Control System Studio.
2.3.4 Database
Preparing data into a JSON string according to the SEDS protocol provides the capability to store the
data in a database. SEDS does not provide an interface to store SEDS datatypes in a database. SEDS does
provide a standardized format to describe data that can be stored to a database and later processed
from that database because the data is self-describing.
3 Java Application
3.1 Interface Services
The Java implementation provides the primary services:


Construct a SEDS datatype in memory
Serialize a SEDS datatype in memory into a JsonObject in memory
Facility for Rare Isotope Beams
U.S. Department of Energy Office of Science | Michigan State University
East Lansing, MI 48824-1321 • Ph.: (517) 355-9672 • Fax: (517) 353-5967
www.frib.msu.edu
SEDS Developer’s Manual





Page 8 of 13
Deserialize a JsonObject in memory into a SEDS datatype
Write a JsonObject in memory to an output source (such as a string or file)
Read an input source (such as a string or file) into a JsonObject in memory
Convert a CSV file to a SedsTable and vice versa
Convert a VType in memory to a SedsType in memory and vice versa
3.2 Package Summary
In general, the Java application is separated into three packages:



org.openepics.seds.api
o Interfaces for functionality and datatypes
org.openepics.seds.core
o Implementations of functionality and datatypes
org.openepics.seds.util
o Useful objects that do not fall into API or CORE
The details of each package and sub-package are discussed below:





org.openepics.seds.api
o Provides interface definitions of all functionality associated with the implementation
o Splits the functionality logic into processing objects
 Serialization, Deserialization, and Validation
 Read and Write (from input and to output sources)
 Datatype Factory
 Datatype Conversion
org.openepics.seds.api.datatypes
o Provides interface definitions of all data structures supported by SEDS
o Contains a generic interface in which all SEDS data structures fall under in the class
hierarchy
org.openepics.seds.core
o Implementation of the org.openepics.seds.api interfaces
o Contains the converter object for SEDS and JSON
o Contains the factory object to create all processing objects from the API package
org.openepics.seds.core.datatypes
o Implementation of the org.openepics.seds.api.datatypes interfaces
o All implementations are immutable data structures
o Factory objects to create these data structures
org.openepics.seds.core.io
Facility for Rare Isotope Beams
U.S. Department of Energy Office of Science | Michigan State University
East Lansing, MI 48824-1321 • Ph.: (517) 355-9672 • Fax: (517) 353-5967
www.frib.msu.edu
SEDS Developer’s Manual
Page 9 of 13
o


Processing objects focusing on input/output beyond what was specified in
org.openepics.seds.api
o CSV converter to and from SEDS datatypes
org.openepics.seds.core.vtype
o Processing objects focusing on EPICS VType datatypes
o VType datatype converter to and from SEDS datatypes
org.openepics.seds.util
o Useful objects that provide additional functionality beyond the API and CORE
o Contains enumerations and constant values
o Majority of these utilities do not need to be used or understood by the client
3.3 Java EE – JSON
Java EE is the standard in community-driven enterprise software in Java. Java EE provides an API to
parse, transform, and query JSON data.
Java EE is selected for the SEDS Java implementation. It is one of the standard framework
implementations of JSON for Java. A lot of sources can be found on the Internet to learn and use JSON
processing with Java EE.
Please go to the Oracle Java EE docs at http://docs.oracle.com/javaee/7/tutorial/doc/jsonp.htm to learn
more.
3.4 JSON Validator
JSON schemas are JSON strings following the JSON Schema Core protocol. JSON schemas provide
metadata to describe a given datatype in JSON. JSON schemas can validate that a JSON string contains
expected name-value pairs, and therefore validate if a JSON string matches an expected JSON data
structure.
Please go to the JSON Schema homepage at http://json-schema.org/ to learn more about JSON
schemas.
The FGE JSON Schema Validator is a Java API that verifies that a JSON string is valid according to a valid
JSON schema.
Please go to the FGE JSON Schema Validator homepage at https://github.com/fge/json-schemavalidator to learn more about the Java JSON schema validator.
3.5 VType
VType is a Java API developed to support standard well-known datatypes for EPICS and Control System
Studio.
Facility for Rare Isotope Beams
U.S. Department of Energy Office of Science | Michigan State University
East Lansing, MI 48824-1321 • Ph.: (517) 355-9672 • Fax: (517) 353-5967
www.frib.msu.edu
SEDS Developer’s Manual
Page 10 of 13
The VType module is included in the SEDS Java implementation. The Java implementation uses the
VType module to provide mapping between SEDS datatypes and VType datatypes.
Please go to the pvManager home page at http://pvmanager.diirt.org/index.html to learn more.
4 JSON Schema
Schemas define the structure of JSON data. The schema itself is written in JSON, with name-value pairs
that correspond to the json-schema-core specifications. The schema details every field (name-value pair)
that must be contained with JSON data to be considered valid.
Please go to the JSON Schema homepage at http://json-schema.org/ to learn more about JSON
schemas.
4.1 Validators
Validators are APIs that check whether JSON data matches a schema. The JSON Schema specification
references many validators for multiple programming languages.
The FGE JSON Schema Validator is a Java API that validates JSON data in Java from a provided schema.
Please go to the FGE JSON Schema Validator homepage at https://github.com/fge/json-schemavalidator to learn more about the Java JSON schema validator.
4.2 SEDS Schema Layout
The JSON Schema Core provides a way to link schemas. For example, a SedsScalar contains a SedsAlarm
field. A SedsScalar should be linked to a SedsAlarm.
To perform the linking, the referenced datatype must be included in the definitions portion of the
schema. To prevent duplicating and copying referenced types into other schemas, three overall schemas
are provided to determine whether JSON data fits to ANY SEDS datatype.
First, the prototype schema is provided for each specific SEDS datatype:



protocol/schema/prototype/SedsAlarm.json
protocol/schema/prototype/SedsScalar.json
...
Here is a snippet from the SedsScalar.json file:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "SedsScalar",
"type": "object",
"required": ["value"],
Facility for Rare Isotope Beams
U.S. Department of Energy Office of Science | Michigan State University
East Lansing, MI 48824-1321 • Ph.: (517) 355-9672 • Fax: (517) 353-5967
www.frib.msu.edu
SEDS Developer’s Manual
Page 11 of 13
"description": "A singe value with optional metadata.",
"additionalProperties": false,
"properties": {
"value": ...,
"alarm": {
"$ref": "#/definitions/SedsAlarm"
},
...
"time": {
"$ref": "#/definitions/SedsTime"
}
}
}
Note that the “#/definitions/” links to other datatypes. In the overall schemas those references are
resolved.
Second, the prototype schema for each general SEDS datatype is provided (these files are just stubs and
contain very little schema rules):



protocol/schema/prototype/SEDS_meta_prototype.json – SEDS data containing only meta data
protocol/schema/prototype/SEDS_raw_prototype.json – SEDS data containing only raw data
protocol/schema/prototype/SEDS_type_prototype.json – SEDS data containing meta and raw
data
Third, a contents file describes everything in the “protocol/schema/prototype/” directory:

contents.txt
The prototypes for individual datatypes, prototype stubs for general SEDS datatypes, and the contents
files can be combined to form the primary schemas. The three primary schemas are included (these
schemas are the only schemas that are required for a user/implementation to work with):



protocol/schema/SEDS_meta.json – SEDS data containing only meta data
protocol/schema/SEDS_raw.json – SEDS data containing only raw data
protocol/schema/SEDS_type.json – SEDS data containing meta and raw data
A utility, “protocol/resources/schema-updater” is provided to automatically link the prototype schemas
to the primary schemas. See the documentation within this directory for additional details and
instructions.
In summary, a user/implementation only needs to use the three JSON schemas in the directory
“protocol/schema/” to validate data to SEDS protocol.
Facility for Rare Isotope Beams
U.S. Department of Energy Office of Science | Michigan State University
East Lansing, MI 48824-1321 • Ph.: (517) 355-9672 • Fax: (517) 353-5967
www.frib.msu.edu
SEDS Developer’s Manual
Page 12 of 13
5 Modifying SEDS Datatypes
The following documents how to change a SEDS type. Change implies a change in the name of the SEDS
type, a change in the names of the composite fields, or a change in the composite fields (added or
removed a field). The following section refers to the type being modified as SedsX.
5.1 Modifying the Protocol
The following are the locations in the SEDS protocol directory that need to be modified when changing a
SEDS type:




Web Documentation
o index.html
 Introduction [id = “seds_types”]
 Ensure the List of SEDS Types contains SedsX
 SEDS Definitions [id = “definitions”]
 Ensure the title of SedsX is correct
 Ensure the definition of SedsX (and composite data) is correct
 Ensure the description of SedsX (and composite data) is correct
 Ensure the JSON example of SedsX is correct
UML
o uml/datatype.png
 UML for SEDS datatypes
 Ensure the UML accurately represents the datatype
o uml/datatype-detail.png
 UML for SEDS datatypes with extensive details
 Ensure the UML accurately represents the datatype
JSON Schema
o schema/prototype/SedsX.json
 Schema for SedsX
o schema/SEDS_type.json, schema/SEDS_raw.json, schema/SEDS_meta.json
 These are the general schemas that reference SedsX.json
 Use the schema-updater utility to automatically update these files
Version
o version.json
 Version (if the SEDS protocol is changed, the version should be changed)
5.2 Modifying the Java Implementation
The following are the locations in the SEDS Java implementation that need to be modified when
changing a SEDS type:
Facility for Rare Isotope Beams
U.S. Department of Energy Office of Science | Michigan State University
East Lansing, MI 48824-1321 • Ph.: (517) 355-9672 • Fax: (517) 353-5967
www.frib.msu.edu
SEDS Developer’s Manual





Page 13 of 13
Datatype Interface
o org.openepics.seds.api.datatypes.SedsX.java
 Interface definition of the SedsX datatype
 Ensure the interface is accurate (contains a getter for all serializable fields)
o org.openepics.seds.core.datatypes.SedsX.java
 Implementation of the SedsX datatype interface
 Ensure the implementation is accurate
Factory
o org.openepics.seds.api.SedsFactory.java
 Interface containing a method to create a SedsX in memory
 Ensure the factory method to create a SedsX in memory is accurate
o org.openepics.seds.core.datatypes.ImmutableSedsFactory.java
 Implementation of the SedsFactory.java
 Ensure the factory method to create a SedsX in memory is accurate
Mappers
o org.openepics.seds.core.vtype.VTypeMapper.java
 Update the function that maps SedsX to the corresponding VType
 Update the function that maps the VType to SedsX
o org.openepics.seds.core.JsonMapper.java
 Update the function that maps SedsX to a JsonObject
 Update the function that maps a JsonObject to SedsX
Resources
o src/main/resources/schema
 Update the JSON schemas that are used for validation (these files can be located
in the protocol module)
Version
o org.openepics.seds.util.ValueUtil.java
 Version (if the SEDS protocol is changed, the version should be changed)
Facility for Rare Isotope Beams
U.S. Department of Energy Office of Science | Michigan State University
East Lansing, MI 48824-1321 • Ph.: (517) 355-9672 • Fax: (517) 353-5967
www.frib.msu.edu