Download pyCGNS - ONERA

pyCGNS
Poinot Marc
ONERA - DSNA/ELSA
/ELSA/PRS-02036
[email protected]
http://www.onera.fr
Overview
CGNS
pyCGNS
Goals
History
Actors/ Components
Examples
Future/ ISO process
Goals/ Status
Interoperability architecture
Examples
/ELSA/PRS-02036
ISO data format and semantics
Proprietary data format and semantics
CFD (Computational Fluid Dynamics) General Notation System
CFD standard
/ELSA/PRS-02036
We want to use a common data representation for interoperability
• Data semantics
• Physical representation
Grid creation/ Solvers/ Visualizers
Pre/Post processing of very large amount of data
We want to ensure our data a long life format
• Consistency
• Timelessness
Provide a general,
portable and extensible
standard for storage and
retrieval of CFD analysis
data.
/ELSA/PRS-02036
Steering Committee
Airbus
Boeing commercial
Boeing Phantom Works
Boeing Space & Communications
CD ADAPCO Group
Fluent, Inc.
ICEM CFD Engineering
Intelligent Light
NASA Ames
NASA Langley
NASA Glenn
NUMECA International
ONERA
Pointwise, Inc.
Pratt & Whitney Aircraft
Rolls-Royce Allison
US Air Force
Mission
Ensure the continuation of CGNS
Responsabilities
Software devt., maintenance, test…
Documentation
Insure a soft. and doc. free distribution
Provide expertise and extensions
proposals
Promote acceptance of the standard
Organization
Main meeting once per year
One chairperson per organization
Governs by consensus
Welcomes participation of all parties,
members or not
Overall architecture
/ELSA/PRS-02036
CGNSLib
Data semantics and Syntax
Specific Sub-trees API
Node API
File format
Data exchange example - An informal view
WB234
body
wingZones
wing
solution#1
coordX
bodyConnectivity
/ELSA/PRS-02036
Mesh generator
density
pressure
Fluid solver
Communication layer
drag
Post processor
Example - A CGNS tree
wing
WB234
CGNSBase_t
Solution#1
Description
Wing and body test case 234
Descriptor_t
Date
13/03/2002
Descriptor_t
wing
Zone_t
Structured
ZoneType_t
CoordinateX
GridCoordinates_t
/ELSA/PRS-02036
WB234
coordX
density
solution#1
FlowSolution_t
Density
DataArray_t
Drag
UserDefined_t
DimensionalUnits
DimensionalUnits_t
dragCoef
DataArray_t
drag
Example - An ADF node
WB234
wing
Solution#1
coordX
Name:
Label:
DataType:
Dimensions:
DimensionValues:
Data:
Cardinality:
Parameters:
/ELSA/PRS-02036
Children:
Density
DataArray_t
DataType
IndexDimension
DataSize[]
[values]
0,N
DataType,
IndexDimension,
DataSize
[nodes]
density
drag
/ELSA/PRS-02036
ONERA - Office National D'Etudes et de Recherches Aérospatiales
The French aeronautics and space research center
From basic research to flight testing
All disciplines involved in aircraft, spacecraft and missile design
DSNA
Numerical Simulation and Aeroacoustics dept.
ELSA: Software unit in charge of elsA software
/ELSA/PRS-02036
elsA
New generation CFD solver
• Industrial tool
• Framework for research
• Gather 20 years of CFD methods and software expertise
Technologies leading edge
• OO design and implementation (C++, Python)
• Efficiency (scalar/vector, space/time)
• High level QA
CGNS related developments at ONERA
elsA
Main I/O data representation
• SIDS compliant
• Interoperability / Multi-processing
– In-memory and disk tree low level
• Validation resources
/ELSA/PRS-02036
pyCGNS
A Python binding to CGNS
Used as gluing language for application framework
• e.g. translators to CGNS trees
– From/To Tecplot and Aerospace actors data format
Open source
elsA Software
Interoperability
Multi-platform/ Multi-applications
• Fluid/ Structures/ Optimization/ Thermics/
pre/post processing know how
/ELSA/PRS-02036
Technologies
OO design/ implementation
• C++ kernel
• Fortran for computation leaves
• Python user interface
A single representation for I/O
• XDT trees + parsers for subset translation from/to
 CGNS/ XML/ Python
Using Python for extensions
Perfect mapping
Easy to map
Easy to deliver
Easy to maintain
Easy to use
Thanks
to a good design, to
Hard to cross compile (and static libs…)
a process and a set of tools
Re-use
/ELSA/PRS-02036
Numerical packages
DBMS
Parsers
OS, string, re...
Create your own tree class,
But... no tree
DAG, graph, b-tree, etc...
Ease of use example
a=1.4+0.2
b=[1.2, 0.3E-10, 1.4]
import Numeric
c=Numeric.ones([3])*a+b
print "Values :", c
/ELSA/PRS-02036
import math
for d in c:
print math.cos(d)
Values : array([2.8, 1.6, 3.])
-0.942222340669
-0.0291995223313
-0.9899924966
In-memory trees
Goal
Have a in-memory representation of trees
• Communicate semantics using trees
• Have a contiguous memory zone
– share
– transfert
• Use memory buffer based systems
– MPI/ SHM/ RPC/
• Fast dump to disk/ recovery
/ELSA/PRS-02036
Implementation
ADF API unchanged
• New implementation
• Thanks the two layers design of CGNS libraries
Disk dump
• Native ADF
• Plan to extend to HDF5
Architecture with Python modules
/ELSA/PRS-02036
Tree in-memory: Python interpreter
RPC
MPI
SHM
TCP/IP
Tree on disk: File system/DBMS
Interoperability example
Two solvers and glue
Structure solver
Glue FFT+CGNS tree
Fluid solver
Visualization
/ELSA/PRS-02036
Main loop
Structure computation
Translator HOST/ CGNS
• SPARC parser
• Numerical Python (arrays+FFT)
• pyCGNS tree
Fluid computation
• Full CGNS
Tecplot viewer
• Read CGNS
/ELSA/PRS-02036
Scenario
Helicopter blade deformation
Xdt classes and parsers
XdtParse
XdtTree *mytree;
XdtParse w;
w.dump(mytree);
XdtPython *xpy;
XdtTree *gridtree;
xpy=XdtPython::lookUp("__main__","grid_tree");
gridtree=xpy->load();
XdtCGNS db("/tmp/mytree.cgns",OPENWRITE);
db.dump(mytree);
/ELSA/PRS-02036
XdtMemory mbuff;
mbuff.dump(mytree);
MPI_Send(mbuff.data, mbuff.size, MPI_UNSIGNED_CHAR,n,id,
MPI_COMM_WORLD);
XdtPython
import xdt
import CRAB
p=CRAB.readTecplot("rotor23x.tp")
grid_tree=xdt.py2xdt(p)
/ELSA/PRS-02036
Python tree
import xdt
# --- Part 1 - etat file
from HOST import hfile
nh =['nh',hfile.data['Nh'],[]]
t0 =['t0',hfile.data['T0'],[]]
dcos=['dCos',hfile.data['dCos'],[]]
dsin=['dSin',hfile.data['dSin'],[]]
gval=['General','',[nh,t0,dcos,dsin]]
val=[]
for nbm in range(hfile.data['nbmodes']):
ms=hfile.data['Mode%.2d'%nbm]
nh =['Nh',ms['Nh'], []]
q0 =['Q0',ms['Q0'], []]
qhc=['Qhc',Numeric.array(ms['Qhc']), []]
qhs=['Qhs',Numeric.array(ms['Qhs']), []]
val.append(['mode%.2d'%nbm, 'Data for mode %d'%nbm, [nh, q0, qhc, qhs]])
mval=['Values','',val]
tree=['HOST',hfile.data['TITLE'], [gval, mval]]
xdt_host_tree=xdt.py2xdt(tree)
xdt_host_tree.dump()
Code example
XdtPython p1("__main__","host_data");
XdtPython p2("__main__","elsa_result");
...
XdtTree *host_tree;
host_tree=XdtTree::getSubTree("HOST");
host_tree->getByPath(ShaftAngle_s,&alp0);
host_tree->getByPath(FlapAngle_s,&_penBat);
host_tree->getByPath(LagAngle_s,&_penTra);
XdtTree *pbData;
p1.load(pbData);
…
p2.dump(pbData);
host_tree->getByPath(PreConAngle_s,&_preCone);
host_tree->getByPath(PreLagAngle_s,&_preTra);
host_tree=host_tree->getSubTree("Harmonics");
host_tree->getByPath("StateModeNumber",&_nbModEtat);
host_tree->getByPath("HarmonicNumber",&nbHarmonax);
host_tree->getByPath("Harmonics",&_nbHarmo.begin());
host_tree->getByPath(GeneralizedCoordConstant_s,&_qi0.begin());
host_tree->getByPath(GeneralizedCoordSin,&_qis.begin());
/ELSA/PRS-02036
XdtMemory p1(ptr);
XdtCGNS
...
p2("Rotor466.adf",MODE_WRITE);
host_tree->getByPath(GeneralizedCoordCos,&_qic.begin());
Archival example
Validation database
/ELSA/PRS-02036
RDBMS+ proprietary tool
• Store CGNS file as binary large object (blob)
• Add attributes
Extracted from tree
• Hidden to MLL
RDBMS extra features
• Access control
• Query
• Centralized archival
Code example - pyCGNS
#!/usr/bin/env python
# CFD General Notation System - CGNS lib wrapper
# ONERA/DSNA/ELSA - [email protected]
# User's Guide to CGNS - C.L.Rumsey et al. examples translation
#
from CGNS
import midlevel
from CGNS.wrap import *
from Numeric
import *
#
#
open CGNS file for read-only
file=pyCGNS('grid.cgns',midlevel.MODE_READ)
index_base=1
index_zone=1
index_flow=1
isize=file.zoneread(index_base,index_zone)[3]
irmin=[1,1,1]
irmax=[isize[0],isize[1],isize[2]]
(loc,name)=file.solinfo(index_base,index_zone,index_flow)
/ELSA/PRS-02036
if (loc != midlevel.Vertex):
print "Error, GridLocation must be Vertex!
print midlevel.GridLocation[loc]
Currently:",
#
read flow solution
r=file.fieldread(index_base,index_zone,index_flow,
'Density',midlevel.RealSingle,irmin,irmax)
p=file.fieldread(index_base,index_zone,index_flow,
'Pressure',midlevel.RealSingle,irmin,irmax)
#
close CGNS file
del file
print "Successfully read flow solution from file 'grid.cgns'"
print "For example, r,p(21,17,9)=",r[20,16,8],p[20,16,8]
print "Program successful... ending now"
pyCGNS Status
/ELSA/PRS-02036
v0.4
Binding
• Full ADF wrapper + 75% MLL wrapper
• Experimental OOL layer
Python + Numerical Python
• Standard package
• Documentation+ test suite
In-memory representation
• Python/C/C++/Fortran in-memory tree sharing
Tools
• SIFT - SIDS instance description check
• ParseTree - Tree dump/display as XML tree
• StampNode - Add sub-nodes with specific stamps (date, ownership…)
pyCGNS v1.0 commitments
/ELSA/PRS-02036
v1.0 (4Q2002)
Binding
• Full ADF+MLL wrapper
• Full OOL layer
Documentation
• Features 100% documented + tutorial + design
• C/Fortran examples
Test suite
• Complete test suite for OOL/MLL/ADF
Tools/extensions
• ADFM embedded
• DBMS interface for blobs
• Translators
Actual use
Companies
Most of our customers
All universities/ research centers working with/for elsA
Python
Lot of translators
Legacy shell scripts translation
Monolithic tools
Promote use of Distutils
Numerical Python
/ELSA/PRS-02036
Small arrays transformations
In-memory on the fly operations
Handle copies with care
pyCGNS
Translators
Tree manipulation for built and re-use
CGNS and pyCGNS references summary
CGNS
http://www.cgns.org
AIAA 2002-0752 - CFD General Notation System (CGNS): Status and future directions S.Legensky, D.Edwards, R.Bush, D.Poirier, C.Rumsey, R.Cosner, C.Towne
ONERA DSNA/ELSA
http://www.onera.fr
AIAA 2002-0108 - An efficient Object-Oriented solution to CFD complexity - L.Cambier, M.Gazaix
pyCGNS
/ELSA/PRS-02036
http://elsa.onera.fr/CGNS/releases
Python
http://www.python.org