Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
pyCGNS Poinot Marc ONERA - DSNA/ELSA /ELSA/PRS-02036 [email protected] http://www.onera.fr Overview CGNS pyCGNS Goals History Actors/ Components Examples Future/ ISO process Goals/ Status Interoperability architecture Examples /ELSA/PRS-02036 ISO data format and semantics Proprietary data format and semantics CFD (Computational Fluid Dynamics) General Notation System CFD standard /ELSA/PRS-02036 We want to use a common data representation for interoperability • Data semantics • Physical representation Grid creation/ Solvers/ Visualizers Pre/Post processing of very large amount of data We want to ensure our data a long life format • Consistency • Timelessness Provide a general, portable and extensible standard for storage and retrieval of CFD analysis data. /ELSA/PRS-02036 Steering Committee Airbus Boeing commercial Boeing Phantom Works Boeing Space & Communications CD ADAPCO Group Fluent, Inc. ICEM CFD Engineering Intelligent Light NASA Ames NASA Langley NASA Glenn NUMECA International ONERA Pointwise, Inc. Pratt & Whitney Aircraft Rolls-Royce Allison US Air Force Mission Ensure the continuation of CGNS Responsabilities Software devt., maintenance, test… Documentation Insure a soft. and doc. free distribution Provide expertise and extensions proposals Promote acceptance of the standard Organization Main meeting once per year One chairperson per organization Governs by consensus Welcomes participation of all parties, members or not Overall architecture /ELSA/PRS-02036 CGNSLib Data semantics and Syntax Specific Sub-trees API Node API File format Data exchange example - An informal view WB234 body wingZones wing solution#1 coordX bodyConnectivity /ELSA/PRS-02036 Mesh generator density pressure Fluid solver Communication layer drag Post processor Example - A CGNS tree wing WB234 CGNSBase_t Solution#1 Description Wing and body test case 234 Descriptor_t Date 13/03/2002 Descriptor_t wing Zone_t Structured ZoneType_t CoordinateX GridCoordinates_t /ELSA/PRS-02036 WB234 coordX density solution#1 FlowSolution_t Density DataArray_t Drag UserDefined_t DimensionalUnits DimensionalUnits_t dragCoef DataArray_t drag Example - An ADF node WB234 wing Solution#1 coordX Name: Label: DataType: Dimensions: DimensionValues: Data: Cardinality: Parameters: /ELSA/PRS-02036 Children: Density DataArray_t DataType IndexDimension DataSize[] [values] 0,N DataType, IndexDimension, DataSize [nodes] density drag /ELSA/PRS-02036 ONERA - Office National D'Etudes et de Recherches Aérospatiales The French aeronautics and space research center From basic research to flight testing All disciplines involved in aircraft, spacecraft and missile design DSNA Numerical Simulation and Aeroacoustics dept. ELSA: Software unit in charge of elsA software /ELSA/PRS-02036 elsA New generation CFD solver • Industrial tool • Framework for research • Gather 20 years of CFD methods and software expertise Technologies leading edge • OO design and implementation (C++, Python) • Efficiency (scalar/vector, space/time) • High level QA CGNS related developments at ONERA elsA Main I/O data representation • SIDS compliant • Interoperability / Multi-processing – In-memory and disk tree low level • Validation resources /ELSA/PRS-02036 pyCGNS A Python binding to CGNS Used as gluing language for application framework • e.g. translators to CGNS trees – From/To Tecplot and Aerospace actors data format Open source elsA Software Interoperability Multi-platform/ Multi-applications • Fluid/ Structures/ Optimization/ Thermics/ pre/post processing know how /ELSA/PRS-02036 Technologies OO design/ implementation • C++ kernel • Fortran for computation leaves • Python user interface A single representation for I/O • XDT trees + parsers for subset translation from/to CGNS/ XML/ Python Using Python for extensions Perfect mapping Easy to map Easy to deliver Easy to maintain Easy to use Thanks to a good design, to Hard to cross compile (and static libs…) a process and a set of tools Re-use /ELSA/PRS-02036 Numerical packages DBMS Parsers OS, string, re... Create your own tree class, But... no tree DAG, graph, b-tree, etc... Ease of use example a=1.4+0.2 b=[1.2, 0.3E-10, 1.4] import Numeric c=Numeric.ones([3])*a+b print "Values :", c /ELSA/PRS-02036 import math for d in c: print math.cos(d) Values : array([2.8, 1.6, 3.]) -0.942222340669 -0.0291995223313 -0.9899924966 In-memory trees Goal Have a in-memory representation of trees • Communicate semantics using trees • Have a contiguous memory zone – share – transfert • Use memory buffer based systems – MPI/ SHM/ RPC/ • Fast dump to disk/ recovery /ELSA/PRS-02036 Implementation ADF API unchanged • New implementation • Thanks the two layers design of CGNS libraries Disk dump • Native ADF • Plan to extend to HDF5 Architecture with Python modules /ELSA/PRS-02036 Tree in-memory: Python interpreter RPC MPI SHM TCP/IP Tree on disk: File system/DBMS Interoperability example Two solvers and glue Structure solver Glue FFT+CGNS tree Fluid solver Visualization /ELSA/PRS-02036 Main loop Structure computation Translator HOST/ CGNS • SPARC parser • Numerical Python (arrays+FFT) • pyCGNS tree Fluid computation • Full CGNS Tecplot viewer • Read CGNS /ELSA/PRS-02036 Scenario Helicopter blade deformation Xdt classes and parsers XdtParse XdtTree *mytree; XdtParse w; w.dump(mytree); XdtPython *xpy; XdtTree *gridtree; xpy=XdtPython::lookUp("__main__","grid_tree"); gridtree=xpy->load(); XdtCGNS db("/tmp/mytree.cgns",OPENWRITE); db.dump(mytree); /ELSA/PRS-02036 XdtMemory mbuff; mbuff.dump(mytree); MPI_Send(mbuff.data, mbuff.size, MPI_UNSIGNED_CHAR,n,id, MPI_COMM_WORLD); XdtPython import xdt import CRAB p=CRAB.readTecplot("rotor23x.tp") grid_tree=xdt.py2xdt(p) /ELSA/PRS-02036 Python tree import xdt # --- Part 1 - etat file from HOST import hfile nh =['nh',hfile.data['Nh'],[]] t0 =['t0',hfile.data['T0'],[]] dcos=['dCos',hfile.data['dCos'],[]] dsin=['dSin',hfile.data['dSin'],[]] gval=['General','',[nh,t0,dcos,dsin]] val=[] for nbm in range(hfile.data['nbmodes']): ms=hfile.data['Mode%.2d'%nbm] nh =['Nh',ms['Nh'], []] q0 =['Q0',ms['Q0'], []] qhc=['Qhc',Numeric.array(ms['Qhc']), []] qhs=['Qhs',Numeric.array(ms['Qhs']), []] val.append(['mode%.2d'%nbm, 'Data for mode %d'%nbm, [nh, q0, qhc, qhs]]) mval=['Values','',val] tree=['HOST',hfile.data['TITLE'], [gval, mval]] xdt_host_tree=xdt.py2xdt(tree) xdt_host_tree.dump() Code example XdtPython p1("__main__","host_data"); XdtPython p2("__main__","elsa_result"); ... XdtTree *host_tree; host_tree=XdtTree::getSubTree("HOST"); host_tree->getByPath(ShaftAngle_s,&alp0); host_tree->getByPath(FlapAngle_s,&_penBat); host_tree->getByPath(LagAngle_s,&_penTra); XdtTree *pbData; p1.load(pbData); … p2.dump(pbData); host_tree->getByPath(PreConAngle_s,&_preCone); host_tree->getByPath(PreLagAngle_s,&_preTra); host_tree=host_tree->getSubTree("Harmonics"); host_tree->getByPath("StateModeNumber",&_nbModEtat); host_tree->getByPath("HarmonicNumber",&nbHarmonax); host_tree->getByPath("Harmonics",&_nbHarmo.begin()); host_tree->getByPath(GeneralizedCoordConstant_s,&_qi0.begin()); host_tree->getByPath(GeneralizedCoordSin,&_qis.begin()); /ELSA/PRS-02036 XdtMemory p1(ptr); XdtCGNS ... p2("Rotor466.adf",MODE_WRITE); host_tree->getByPath(GeneralizedCoordCos,&_qic.begin()); Archival example Validation database /ELSA/PRS-02036 RDBMS+ proprietary tool • Store CGNS file as binary large object (blob) • Add attributes Extracted from tree • Hidden to MLL RDBMS extra features • Access control • Query • Centralized archival Code example - pyCGNS #!/usr/bin/env python # CFD General Notation System - CGNS lib wrapper # ONERA/DSNA/ELSA - [email protected] # User's Guide to CGNS - C.L.Rumsey et al. examples translation # from CGNS import midlevel from CGNS.wrap import * from Numeric import * # # open CGNS file for read-only file=pyCGNS('grid.cgns',midlevel.MODE_READ) index_base=1 index_zone=1 index_flow=1 isize=file.zoneread(index_base,index_zone)[3] irmin=[1,1,1] irmax=[isize[0],isize[1],isize[2]] (loc,name)=file.solinfo(index_base,index_zone,index_flow) /ELSA/PRS-02036 if (loc != midlevel.Vertex): print "Error, GridLocation must be Vertex! print midlevel.GridLocation[loc] Currently:", # read flow solution r=file.fieldread(index_base,index_zone,index_flow, 'Density',midlevel.RealSingle,irmin,irmax) p=file.fieldread(index_base,index_zone,index_flow, 'Pressure',midlevel.RealSingle,irmin,irmax) # close CGNS file del file print "Successfully read flow solution from file 'grid.cgns'" print "For example, r,p(21,17,9)=",r[20,16,8],p[20,16,8] print "Program successful... ending now" pyCGNS Status /ELSA/PRS-02036 v0.4 Binding • Full ADF wrapper + 75% MLL wrapper • Experimental OOL layer Python + Numerical Python • Standard package • Documentation+ test suite In-memory representation • Python/C/C++/Fortran in-memory tree sharing Tools • SIFT - SIDS instance description check • ParseTree - Tree dump/display as XML tree • StampNode - Add sub-nodes with specific stamps (date, ownership…) pyCGNS v1.0 commitments /ELSA/PRS-02036 v1.0 (4Q2002) Binding • Full ADF+MLL wrapper • Full OOL layer Documentation • Features 100% documented + tutorial + design • C/Fortran examples Test suite • Complete test suite for OOL/MLL/ADF Tools/extensions • ADFM embedded • DBMS interface for blobs • Translators Actual use Companies Most of our customers All universities/ research centers working with/for elsA Python Lot of translators Legacy shell scripts translation Monolithic tools Promote use of Distutils Numerical Python /ELSA/PRS-02036 Small arrays transformations In-memory on the fly operations Handle copies with care pyCGNS Translators Tree manipulation for built and re-use CGNS and pyCGNS references summary CGNS http://www.cgns.org AIAA 2002-0752 - CFD General Notation System (CGNS): Status and future directions S.Legensky, D.Edwards, R.Bush, D.Poirier, C.Rumsey, R.Cosner, C.Towne ONERA DSNA/ELSA http://www.onera.fr AIAA 2002-0108 - An efficient Object-Oriented solution to CFD complexity - L.Cambier, M.Gazaix pyCGNS /ELSA/PRS-02036 http://elsa.onera.fr/CGNS/releases Python http://www.python.org