Download 1 Introduction - LHCb Computing

Simulation for LHCb and
Integrated Control Environment (DIRAC)
Job Monitoring and Control (Prototype & Discussion)
LHCB Technical Note
Issue:
Revision:
Draft
1
Reference:
Created:
Last modified:
GridPP-LHCb
14 November 2002
03 May 2017
Prepared By:
Gennady. G. Kuznetsov
Editor: Glenn N. Patrick
Simulation for LHCb and Integrated Control Environment (DIRAC)
Job Monitoring and Control (Prototype & Discussion)
LHCB Technical Note
Issue:
Draft
Table of Contents
Reference:
Revision:
Last modified:
GridPP-LHCb
1
03 May 2017
Abstract
DIRAC is the name given to the LHCb distributed Monte Carlo production environment and its monitoring and control
system.
This document is based on the following documents: “Job Configuration, Data Production, Bookkeeping” and “Data
Management”. It describes requirements for job monitoring and control services, gives a description and analysis of
currently implemented system and continues discussion about implementation of a new prototype of monitoring and
control service with provision for a smooth transition from a batch based submission system to a fully Grid enabled,
GANGA based, platform independent distributed application.
Document Status Sheet
Table 1 Document Status Sheet
1. Document Title: [Project Name Qualification] User Requirements Document
2. Document Reference Number: [Document Reference Number]
3. Issue
Draft
page i
4. Revision
1
5. Date
6. Reason for change
26 November 2002
First version
Simulation for LHCb and Integrated Control Environment (DIRAC)
Job Monitoring and Control (Prototype & Discussion)
LHCB Technical Note
Issue:
Draft
Table of Contents
Reference:
Revision:
Last modified:
GridPP-LHCb
1
03 May 2017
Table of Contents
1
INTRODUCTION .................................................................................................................................................... 1
1.1
1.2
1.3
PURPOSE OF THE DOCUMENT ............................................................................................................................... 1
DEFINITIONS, ACRONYMS AND ABBREVIATIONS .................................................................................................. 1
REFERENCES ........................................................................................................................................................ 2
2
OVERVIEW OF THE MONTE-CARLO SIMULATION ................................................................................... 3
3
BRIEF EXPLANATION OF REQUIREMENTS.................................................................................................. 4
3.1
3.2
3.3
3.4
4
REVIEW OF THE CURRENT PVSSII BASED CONTROL SYSTEM AND WP3 MIDDLEWARE ............ 8
4.1
4.2
4.3
4.4
4.5
5
GATEWAY SIDE .................................................................................................................................................... 4
SERVER SIDE ........................................................................................................................................................ 7
CLIENT SIDE (DIRAC DESKTOP) ......................................................................................................................... 7
CLIENT SIDE (JOB) ............................................................................................................................................... 7
PVSSII INTRODUCTION. ...................................................................................................................................... 8
LHCB SPECIFIC LAYER (DIM) ............................................................................................................................. 9
PVSS AND DIM TOGETHER AS AN INTEGRATED CONTROL ENVIRONMENT. ...................................................... 10
R-GMA: A GRID INFORMATION AND MONITORING SYSTEM (WP3) ................................................................. 11
JAVA 2 PLATFORM, ENTERPRISE EDITION (J2EE) .............................................................................................. 13
CONCLUSION. ...................................................................................................................................................... 15
page ii
Simulation for LHCb and Integrated Control Environment (DIRAC)Job Monitoring and Control (Prototype & Discussion)Reference:
LHCB Technical Note
Revision:
1
Issue:
Draft
Last modified:
03 May 2017
Introduction
List of Figures
Figure 1. Communication restrictions between client, server, batch front end and grid
gateway. ................................................................................................................... 5
Figure 2. Message transport and object extraction. ......................................................... 6
Figure 3. PVSS Managers overview (Picture taken from ItCoBe Cern website
http://itcobe.web.cern.ch). ......................................................................................... 8
Figure 4. DIM service (picture taken from DIM web site http://dim.web.cern.ch). ............. 9
Figure 5. Structure of DIRAC Monitoring and Control System ....................................... 10
Figure 6. Grid Monitoring Architecture (picture taken from “Publication” section at
http://hepunx.rl.ac.uk/edg/wp3/ “Time, Information and the Grid”). .......................... 11
Figure 7. More detailed architecture of R-GMA. ............................................................. 12
Figure 8. Relationship between Table (defined in Global Schema) and several
Producers. .............................................................................................................. 12
page iii
GridPP-L
Simulation for LHCb and Integrated Control Environment (DIRAC)Job Monitoring and Control (Prototype & Discussion)Reference:
LHCB Technical Note
Revision:
1
Issue:
Draft
Last modified:
03 May 2017
Introduction
1 Introduction
1.1 Purpose of the document
The purpose of this document is to:
Explain new requirements for further development of the existing monitoring and control system.
Provide an overview of existing system.
Discuss range of possible solutions.
Design architecture of the prototype of monitoring and control application.
1.2 Definitions, acronyms and abbreviations
1.2.1 Definitions
Architecture
The software architecture of a program or computing system is the structure or structures of the system, which
comprises software components, the externally visible properties of those components, and the relationships
among them.
Framework
A framework represents a collection of classes that provide a set of services for a particular domain; a framework exports a number of individual classes and mechanisms that clients can use or adapt. A framework realizes the architecture.
Component
A software component is a re-usable piece of software that has a well-specified public interface and it implements a limited functionality. Software components achieve reuse by following standard conventions.
Server (Hardware)
A computer provides resources during a considerable period of time to run services available for others (computer, program, users) within network.
Server (Software)
An application running on one or more servers and provides services available for others clients within network.
Client (Software)
A program uses services provided by one or more servers.
page 1
GridPP-L
Simulation for LHCb and Integrated Control Environment (DIRAC)Job Monitoring and Control (Prototype & Discussion)
Reference:
GridPP-LHCb
LHCB Technical Note
Revision:
1
Issue:
Draft
Last modified:
03 May 2017
Introduction
User
A person uses applications. In this document we are not making ANY difference between different types of
users. For example between production manager and physicist (see Data management document)
Grid Gateway
Grid enabled computer with Grid gateway installed and activated. It allows Grid job submission for local accounts.
Batch frond end
A computer configured to submit jobs from local account to the batch system.
Gateway
A computer, which is a GRID gateway or batch frond end.
Node
A computer which is part of a batch farm.
Step, Job, Production, Workflow
See “Data Management” document.
Job Wrapper
A program submitted with the job, provides additional capability for monitoring and executes logic defined in
the workflow.
Message proxy server
Program rerouting messages received from node to external collection point. It implements Point To Point
(PTP) protocol.
1.2.2 Acronyms
CERN
RAL
SSH
UML
WWW
European Organization for Nuclear Research
Rutherford Appleton Laboratory
Secure Shell
Unified Modelling Language
World Wide Web
1.3 References
2

Data Management: Job Configuration, Bookkeeping, Data Production, LHCb Technical Note,
LHCb COMP 02-nn, 22 May 2002

Gaudi Framework - http://proj-gaudi.web.cern.ch/proj-gaudi/

R-GMA: A GRID Information and Monitoring system. http://hepunx.rl.ac.uk/edg/wp3/

Interfacing GAUDI with GRID: GANGA - http://ganga.web.cern.ch/ganga
page 2
Simulation for LHCb and Integrated Control Environment (DIRAC)Job Monitoring and Control (Prototype & Discussion)Reference:
LHCB Technical Note
Revision:
1
Issue:
Draft
Last modified:
03 May 2017
Overview of the Monte-Carlo simulation
2 Overview of the Monte-Carlo simulation
The LHCb Monte Carlo simulation system is an application designed to simulate particle collisions in the LHCb detector. The main purpose of the simulations, at the current stage, is to provide guidance for the hardware and software design of detector subsystems. The process of simulation includes several steps: event generation, particle tracking, simulation of data acquisition and event reconstruction. Due to the statistical nature of the process, there is a requirement to
produce many millions of events. In order to comply with the production demand, it is essential to use a distributed
computer environment. The architecture of this distributed application can be easily simplified due to the event independency. It gives a freedom to escape from the needs of any process communication during the event calculation, and
to postpone publication of the results until a reasonable number of events are processed.
The process described above can be split into several steps and multiplied by number of available processors.
1. Job preparation.
2. Job submission.
3. Job execution and monitoring.
4. Collecting processed data and finalising
page 3
GridPP-L
Simulation for LHCb and Integrated Control Environment (DIRAC)Job Monitoring and Control (Prototype & Discussion)
Reference:
GridPP-LHCb
LHCB Technical Note
Revision:
1
Issue:
Draft
Last modified:
03 May 2017
Brief explanation of requirements
3 Brief explanation of requirements
3.1 Gateway side
3.1.1 Job preparation
This step requires: definition of the logic for the calculation, preparation of packages and applications to be used, extraction and preparation of all data needed during calculation process (see workflow definition “Data Management”
document). At this time any type of communication with any services is allowed and can be performed by client or
server. The result of this step should be a shell script with data bundled into file of archive. Use of any external persistent storage for keeping data specifically required by this single submitted job must be discouraged. Only data produced from previous steps or generated during the experiment can be taken from persistent remote storage.
3.1.2 Job submission
Job submission can be split into four steps: login into a computer that is the batch front end or GRID gateway, checking
installed software, if required installation of software using software cache storage, transmission of prepared shell script
and associated bundle, submission of script into execution queue.
Direct login from client into batch front end or GRID gateway could be performed via SSH using password or/and secure key (RSA). Indirect login also can be performed via dedicated server using common account (for example “lhcbprod”) but it must require login on server. Here it is strictly prohibited to use ANY kind of “backdoor” between server/client and batch front end (custom, RPC, RMI, …), only SSH, SSL protocols are allowed. The communication between client and server can be based on any protocol (see Figure 1).
After establishing a connection, but before any submission, the presence of the required software and availability of
network resources has to be checked. It will result in the submission of a Pilot application. Due to the fact that jobs are
grouped into productions (see Data Management document), the software check needs to be done once per production.
4
page 4
Simulation for LHCb and Integrated Control Environment (DIRAC)Job Monitoring and Control (Prototype & Discussion)Reference:
LHCB Technical Note
Revision:
1
Issue:
Draft
Last modified:
03 May 2017
Brief explanation of requirements
Client
Server
Grid Gateway
Batch Front End
Batch Front End
SSH Protocol
Any Protocol
Figure 1. Communication restrictions between client, server, batch front end and grid gateway.
At this point, we have to make the following assumptions:
For the batch system:
Gateway has a network connection with the outside world.
Nodes may not have access to the outside world but have full access to gateway resources.
Local network file system exists, and it is common for all nodes and gateway.
Local file space is assigned for a particular job, and it may not be accessible from gateway.
Local network space is available for writing and spit into three areas: for software installation, for
job submission, for data collection.
For EDG Grid:
Grid gateway has a network connection with the outside world.
Local file space is available for preparing jobs for submission.
Grid nodes have only grid services available and do not have access to any other network resources.
(Due to the incomplete status of the GRID project this statement is not strictly assumed.)
For Globus Grid:
Requirements for the Globus tools are not clear at this moment, and it could be a mixture of the two
previous examples.
page 5
GridPP-L
Simulation for LHCb and Integrated Control Environment (DIRAC)Job Monitoring and Control (Prototype & Discussion)
Reference:
GridPP-LHCb
LHCB Technical Note
Revision:
1
Issue:
Draft
Last modified:
03 May 2017
Brief explanation of requirements
If software installation is required, it can be installed from external resources or from the distribution existing on a client computer. The last option allows user to submit jobs based on locally prepared packages.
It is worth noting that the file space cache area can be cleaned at any time and it is very helpful to provide a time stamp
during the check procedure for each package.
Jobs are prepared in the form of a shell script and archived bundle transferred using the SFTP protocol and submitted
into different queues using a submission command. There is no specific requirement for production to be executed on
one single queue, but one production cannot be split between several gateways.
3.1.3 Job execution, control and monitoring
The submitted shell script (job) executes a job wrapper. The job wrapper is a program providing execution of applications according to a workflow specification and encapsulates additional functions required for large-scale data production. For example, all grid or specific LHCb services can be available to the application via communication with the
job wrapper. In this case, applications can be developed and executed independently from any specific infrastructure.
Ganga is the LHCb candidate for interfacing grid services.
M1
Batch
Front End
M2
M3
Message
Proxy
Server
LHCb
Server
O3
O2
O1
NDS
M1 Batch nodes
Job
M2
Mn – message
On – object
M3
Job
Job
Figure 2. Message transport and object extraction.
The monitoring task collects information about the internal status of objects inside a running job. Information on each
stage of calculation can be requested and collected by the job wrapper and sent to a collection point in the form of messages. Generally, a server program implementing Point To Point (PTP) protocol will be the collection point. For sites
with tight network restrictions, a message proxy will used to redirect messages via the gateway (Figure 2).
The original objects will be extracted from all received messages and stored in a Naming and Directory Service on the
server.
6
page 6
Simulation for LHCb and Integrated Control Environment (DIRAC)Job Monitoring and Control (Prototype & Discussion)Reference:
LHCB Technical Note
Revision:
1
Issue:
Draft
Last modified:
03 May 2017
Brief explanation of requirements
In our case, job control is limited by one action only – “kill job”. This command has to be performed by the client via
the batch frond end or via the grid user interface. No other routes, like issuing remote command to the job wrapper, are
considered.
3.1.4 Collecting processed data and finalising
At the final stage, data files have to be moved to the desired destination. Due to the wide area network limitations, this
process cannot be carried out by the batch node. It has to be done by a dedicated server inside the domain. On the grid
node, data will be transferred into the storage element.
When data are transferred, the bookkeeping database has to be updated. It is not clear at this moment what is the best
solution, but obviously it can be done by sending XML objects, to the server in the form of messages.
3.2 Server side
The server program provides two types of service.
The first type is a remote login gateway. It keeps a collection of secure keys and allows remote login for the verified
users into gateways registered for LHCb production. The service is not essential and mainly designed for the production
manager.
The second type is Message, Directory and Naming services. These services will provide Point-to-Point messaging domain with the hierarchical directory service for storing received messages. Publish – subscribe messaging is not required but can be easily implemented.
The clients (SLICE Desktop) will be connecting with the server using RMI over IIOP technology.
3.3 Client side (DIRAC Desktop)
The client desktop will be an interactive front end for GRID and LHCb services. It MUST be platform and OS independent. It also has to be built from independent modules. Due to lack of time and human resources, the Java platform
seems to be the best solution. Java Beans or Netbeans are good candidates.
3.4 Client side (Job)
The job wrapper is prepared inside the DIRAC Desktop by the Workflow Editor (“Data Management” document). At
this moment, this is assumed to be a Java program.
page 7
GridPP-L
Simulation for LHCb and Integrated Control Environment (DIRAC)Job Monitoring and Control (Prototype & Discussion)
Reference:
GridPP-LHCb
LHCB Technical Note
Revision:
1
Issue:
Draft
Last modified:
03 May 2017
Review of the current PVSSII based control system and WP3 middleware
4 Review of the current PVSSII based control system and WP3
middleware
4.1 PVSSII introduction.
PVSS stands for Process Visualization and Control System. It is an industrial two-platform (Linux and Windows with
some restrictions, server designed for Linux, graphical designing tools for Windows.) SCADA product from the Austrian company ETM. It is designed for local control and can be run in a distributed manner. It is device oriented with a
flexible data point concept. It also has advanced scripting capabilities and flexible API allowing access to all features of
PVSS from an external application.
Because PVSS is designed to control real time equipment it has a very advanced object orientated data point concept.
Each data point can be a structure of more simple data points with their own commands and status. Very robust provision for alert and range handling is implemented. The option of archiving process and data point status exists but not as
part of data point interface. The data point can be linked with a graphical object representing its status.
UIM
UIM
Ctrl
DM
UIM
API
D
Processing
Layer
Layer
Communication and
Memory Layer
EV
D
User Interface
D
Driver Layer
Figure 3. PVSS Managers overview (Picture taken from ItCoBe Cern website http://itcobe.web.cern.ch).
Each PVSS project must have one and only one Event Manager (EV). Its event-messaging system is based on the
TCP/IP protocol. It requires establishing and keeping connection during monitoring and is obviously designed for Local
Area Networks. The Event Manager administers data points, administers user authorisation, contain and administers
current process image, receives and evaluates messages, distributes messages to other messages.
8
page 8
Simulation for LHCb and Integrated Control Environment (DIRAC)Job Monitoring and Control (Prototype & Discussion)Reference:
LHCB Technical Note
Revision:
1
Issue:
Draft
Last modified:
03 May 2017
Review of the current PVSSII based control system and WP3 middleware
The Event Manager is directly linked with the Data Manager (DM), which provides a high-speed database for archiving
process status.
The Control Manager (Ctrl) provides an Interpreter with large scope of functions, complex processing of data points
and multithreading capability.
The API Manager (API) provides a C++ class library, integration of existing software and integration of PVSS functions.
User Interface Manager (UIM) provides graphics visualisation of messages received by EV, forwarding of user entries
to EV, animation of graphic components, User Interface Builder and Run-time module for process visualisation.
4.2 LHCb specific layer (DIM)
In the LHCb case of distributed Monte Carlo simulation, the Driver Layer (D) does not exist. It has been replaced by
DIM – Distributed Information Management system http://dim.web.cern.ch, which is a communication system for distributed / mixed environments.
It is based on the client/server paradigm. (Figure 4)
Figure 4. DIM service (picture taken from DIM web site http://dim.web.cern.ch).
The basic concept of the DIM approach is the concept of "service". Servers provide services to clients. A service is
normally a set of data (of any type or size) and it is recognized by a name - "named services". The client normally requests services once (at startup) and they are subsequently automatically updated by the server either at regular time
intervals or whenever the conditions change.
The client updating mechanism can be of two types, either by executing a callback routine or by updating a client buffer
with the new set of data, or both. In fact, this last type works as if the clients maintain a copy of the server's data in
cache, the cache coherence being assured by the server.
In order to allow dynamic discovery of the Server by a client a Name Server was introduced. Servers "publish" their
services by registering them with the Name Server (normally once, at startup). Clients "subscribe" to services by asking
the name server which server provides the service and then contacting the server directly, providing the type of service
and the type of update as parameters. The name server keeps an up-to-date directory of all the servers and services
available in the system.
page 9
GridPP-L
Simulation for LHCb and Integrated Control Environment (DIRAC)Job Monitoring and Control (Prototype & Discussion)
Reference:
GridPP-LHCb
LHCB Technical Note
Revision:
1
Issue:
Draft
Last modified:
03 May 2017
Review of the current PVSSII based control system and WP3 middleware
A special type of service “DimCommand” is implemented. It allows execution of commands by the client on the server
computer.
4.3 PVSS and DIM together as an Integrated Control Environment.
In the LHCb project, PVSS and DIM have been used to implement control and monitoring functions. The core of PVSS
has been used as an undistributed application where the driver layer is replaced by the DIM service. This schema extended PVSS to a Wide Area Network Application and provided the remote user with a PVSS remote user interface.
The server itself runs at CERN on the lbnts2.cern.ch Linux box. It consists of a PVSSII server, DIM Name Server and
DIM Client (See Figure 5)
CERN lbnts2 Server
Remote Linux Cluster
Name
Server
Publishing
Client
PVSS
Farm
Front-End
Nodes
Client
Client
Data
Server
Client
“mcsend”
Client
“mcsend”
“mcserver”
“mcsend”
Commands
Send status “mcsend”
User Computer
PVSS
EV
User
Interface
Figure 5. Structure of DIRAC Monitoring and Control System
As the DIM service was written before the introduction of hierarchical data points in PVSS, the DIM messaging system
adopts a flat data structure, where each variable is a separate service without any provision for message filtering. Messages have not been implemented as an object but as a set of methods belonging to Client and Server and therefore cannot be nested, modified or stored.
No security mechanism is available. All Unix commands can be issued via the DIM service including the launch of
remote Xterm sessions.
10
page 10
Simulation for LHCb and Integrated Control Environment (DIRAC)Job Monitoring and Control (Prototype & Discussion)Reference:
LHCB Technical Note
Revision:
1
Issue:
Draft
Last modified:
03 May 2017
Review of the current PVSSII based control system and WP3 middleware
4.4 R-GMA: A GRID information and monitoring system (WP3)
The Grid Monitoring Architecture as shown in Figure 6 consists of three components: consumers, producers and a directory service. Producers register themselves with the registry and describe the type and structure of information they
want to make available to the Grid. Consumers consult the registry to find a suitable source of information and once
this information is known the consumer connects directly to the Producer and receives data at once or have data
streamed to him.
Producer
subscribe
Registry
Consumer
lookup
Figure 6. Grid Monitoring Architecture (picture taken from “Publication” section at
http://hepunx.rl.ac.uk/edg/wp3/ “Time, Information and the Grid”).
In fact, this simplified “official” description does not explain anything at all. Obviously, the Producer and Consumer
can have different lifetimes and therefore require persistent storage. A more precise description follows (See Figure 7)

All clients communicate with the server where a “Tomcat” servlet runner executes several servlets and
MySQL database is installed and configured.

The biggest and independent unit of information is a Database Table. The table must be defined via a Schema
servlet and registered via the Registry servlet. It is implemented via the SQL CREATE TABLE command. It
must be done once before any client will produce or request the data. (See Figure 8)

The second and smallest unit of information is a Database Table Row. The Producer client calls the Producer
servlet and adds or updates a row in the table via SQL INSERT command.

The Consumer client defines the required SQL Table via the Registry servlet and requests information in the
form of a query expressed as an SQL SELECT statement.
There are APIs developed for Java, C and C++ languages. It supports three types of column: VARCHAR, INT and
REAL. Currently the streaming option is not implemented.
page 11
GridPP-L
Simulation for LHCb and Integrated Control Environment (DIRAC)Job Monitoring and Control (Prototype & Discussion)
Reference:
GridPP-LHCb
LHCB Technical Note
Revision:
1
Issue:
Draft
Last modified:
03 May 2017
Review of the current PVSSII based control system and WP3 middleware
Client: Application
Server
Producer
Servlet
Producer
API
Registry
Servlet
My SQL
Schema
Servlet
Table
Client: Monitor
Consumer
API
Consumer
Servlet
Figure 7. More detailed architecture of R-GMA.
Producer 1
Table_Name
Value
Value
Value
Time
Value
Value
Value
Time
Producer 2
Value
Value
Value
Time
Value
Value
Value
Time
Producer 3
Value
Value
Value
Time
Value
Value
Value
Time
Column1
Column2
Column3
Time stamp
Value
Value
Value
Time
Value
Value
Value
Time
Value
Value
Value
Time
Value
Value
Value
Time
Value
Value
Value
Time
Value
Value
Value
Time
Figure 8. Relationship between Table (defined in Global Schema) and several Producers.
12
page 12
Simulation for LHCb and Integrated Control Environment (DIRAC)Job Monitoring and Control (Prototype & Discussion)Reference:
LHCB Technical Note
Revision:
1
Issue:
Draft
Last modified:
03 May 2017
Review of the current PVSSII based control system and WP3 middleware
4.5 Java 2 Platform, Enterprise Edition (J2EE)
The Java 2 Platform, Enterprise Edition (J2EE) defines the standard for developing multitier client - server applications.
J2EE simplifies applications by basing them on standardized, modular components, by providing a complete set of services to those components, and by handling many details of application behavior automatically, without complex programming.
The J2EE, takes advantage of many features of the Java 2 Platform, Standard Edition, such as Java portability, JDBC
API for database access, CORBA technology for interaction with existing enterprise resources, and a security model
that protects data even in internet applications. Building on this base, J2EE adds full support for Enterprise JavaBeans
components, Java Servlets API, JavaServer Pages and XML technology.
4.5.1 JDBC API
JDBC technology is an API that lets you access virtually any tabular data source from the Java programming language.
It provides cross-DBMS connectivity to a wide range of SQL databases, and now, with the new JDBC API, it also provides access to other tabular data sources, such as spreadsheets or flat files
4.5.2 Java Servlets
Java Servlet technology provides Web developers with a simple, consistent mechanism for extending the functionality
of a Web server. A servlet can almost be thought of as an applet that runs on the server side.
Servlets are the Java platform technology of choice for extending and enhancing Web servers. Servlets provide a component-based, platform-independent method for building Web-based applications, without the performance limitations
of CGI programs.
Servlets have access to the entire family of Java APIs, including the JDBCTM API to access enterprise databases.
Servlets can also access a library of HTTP-specific calls and receive all the benefits of the mature Java language, including portability, performance, reusability, and crash protection.
4.5.3 Common Object Request Broker Architecture (CORBA)
CORBA (Common Object Request Broker Architecture) technology is the open standard for heterogeneous computing.
CORBA complements the Java platform by providing a distributed objects framework, services to support that framework, and interoperability with other languages. CORBA standards provide the proven, interoperable infrastructure to
Java 2 Platform, Enterprise Edition. CORBA technology is an integral part of the Java 2 platform through Enterprise
JavaBeans, RMI over IIOP, Java IDL, and Java Transaction Service.
RMI over IIOP is part of the Java 2 Platform, Enterprise Edition and Java 2 Platform, Standard Edition version 1.3.
RMI over IIOP provides the ability to write CORBA applications for the Java platform without learning CORBA Interface Definition Language (IDL). To work with CORBA applications in other languages, IDL can be generated from
Java programming language interfaces. RMI over IIOP includes the full functionality of a CORBA Object Request
Broker.
page 13
GridPP-L
Simulation for LHCb and Integrated Control Environment (DIRAC)Job Monitoring and Control (Prototype & Discussion)
Reference:
GridPP-LHCb
LHCB Technical Note
Revision:
1
Issue:
Draft
Last modified:
03 May 2017
Review of the current PVSSII based control system and WP3 middleware
4.5.4 Enterprise JavaBeans (EJB)
Enterprise JavaBeans is part of the Java 2 Platform, Enterprise Edition. The EJB server-side component model simplifies development of middleware components that are transactional, scalable, and portable. EJB servers reduce the complexity of developing middleware by providing automatic support for middleware services such as transactions, security, database connectivity, and more.
EJBs use the RMI/IDL CORBA subset for their distributed object model, and use the Java Transaction Service (JTS)
for their distributed transaction model. When Enterprise JavaBeans are implemented using the RMI-IIOP protocol for
EJB interoperability in heterogeneous server environments, the standard mapping of the EJB architecture to CORBA
enables the following interoperability:

A client using an ORB from one vendor can access enterprise beans residing on an EJB server provided by another vendor.

Enterprise beans in one EJB server can access enterprise beans in another EJB server.

A non-Java platform CORBA client can access any enterprise bean object.
4.5.5 Java XML Pack
SUN Java XML Pack brings together several of the key industry standards for XML -- such as SAX, DOM, XSLT,
SOAP, UDDI, ebXML, and WSDL. There ensure Java developers have a quick and easy development cycle for integration of XML functionality and standards support their applications.
The Java API for XML Messaging (JAXM) enables developers to send and receive XML messages based on the Simple
Object Access Protocol (SOAP) 1.1 with attachments specification and contains profiles for ebXML TR&P and SOAPRP.
The Java API for XML Processing (JAXP) enables developers to process XML documents by providing support for the
XML processing standards SAX, DOM, and XSLT.
The Java API for XML Registries (JAXR) provides a uniform and standard Java API for interacting with XML registries such as UDDI and ebXML Registry/Repository.
The Java API for XML-based RPC (JAX-RPC) enables Java technology developers to build Web applications and Web
services incorporating XML-based remote procedure call (RPC) functionality according to the Simple Object Access
Protocol (SOAP) 1.1 specification. By using JAX-RPC, Java developers can rapidly achieve web services interoperability based on widely adopted standards and protocols.
The SOAP with Attachments API for Java (SAAJ) enables developers to produce and consume messages conforming to
the SOAP 1.1 specification and SOAP with Attachments note.
Java Architecture for XML Binding (JAXB) provides an API and tools that automate the mapping between XML documents and Java objects.
14
page 14
Simulation for LHCb and Integrated Control Environment (DIRAC)Job Monitoring and Control (Prototype & Discussion)Reference:
LHCB Technical Note
Revision:
1
Issue:
Draft
Last modified:
03 May 2017
Conclusion.
5 Conclusion.
From the point of view of application monitoring, R-GMA technology is the least useful approach, probably because it
has been designed for the GRID monitoring rather than for application monitoring. The difference is that application
monitoring does not require ANY stable and long living infrastructure. It requires short-lived and very flexible persistent storage.
The R-GMA information system is based on a dedicated server with a database (MySQL), where the functionality of
the SQL database is reduced by the Servlet used as a gateway for data access. Database tables have to be predefined
and support only 3 types of object (String, Real and Integer). There is no planned provision for any object-orientated
container or hierarchical storage or for any type of component model or for any graphical user interface. A simple and
standard XML-RPC interface would be much more flexible (it has been use intensively in the LHCb project for communication with databases).
The current system, based on PVSS and DIM, has potentially much better functionality. The new version of the PVSS
system supports very complex object storage and provides a GIU for client visualization. It also supports two operating
systems on I386 platform: Windows and Linux. Unfortunately, it is a commercial product and requires licenses for
server and client. From the developer point of view, nothing from the existing system can be used. DIM must be redeveloped completely to support object encapsulation and Point-to-Point messaging protocol. Security also needs to be
introduced.
It worth noting, that none of the above systems fully support modern software development technology. Object orientated technology is completely really in use (R-GMA and DIM have classes but does not have any support for objects,
PVSS has objects but it just has a scripting language for the GUI). Neither component programming model nor distributed method nor object invocation are mentioned.
From this perspective, the best available tool is Java 2 SDK Enterprise Edition. It provides all the functionality mentioned above plus an enormous library of freely available code. The comparison table below shows the differences between all three available options.
Functionality
R-GMA
PVSS+DIM
J2EE
Table based storage
Yes
Yes
Yes
Hierarchical persistent storage
No
Yes
Dynamic persistent storage
No
Yes, no objects delivery, no
PTP protocol.
No
Data base type
JDBC driver
Internal
Any
Message service
No
Needs to be developed
Yes
Remote GUI
No
Yes
Yes
License
Not required
Required
Not required
Security
GRID (not ready?)
Not used
Yes
Cross platform
Yes
I386 Win/Linux
Yes
Distributed method invocation
No
No
Yes, based on CORBA
Component Model
No
No
Yes, server and client
Multi-language support
C, C++, Java
C++, C
Any
Yes
page 15
GridPP-L
Simulation for LHCb and Integrated Control Environment (DIRAC)Job Monitoring and Control (Prototype & Discussion)
Reference:
GridPP-LHCb
LHCB Technical Note
Revision:
1
Issue:
Draft
Last modified:
03 May 2017
Conclusion.
16
page 16