Download Python bindings in BornAgain

Python bindings in BornAgain Gennady Pospelov Scien8fic Compu8ng at MLZ
October, 24 2013, ILL, Grenoble, France Scope
BornAgain is a a free and open source soJware package to simulate and fit small angle scaMering at grazing incidence. o  see talk of Walter Van Herck October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 02 Basic software architecture
Main features: Why C++? o  Performance. o  Type safety. o  External dependencies. o  Prospec8ve use of GPU, parallel compu8ng. o  Developers background. o  Programming language C++ o  Core libraries consist of 30k lines of code o  Python bindings o  Mac, Linux, Windows o  Depend on 4 well established libraries gsl, boost, eigen, Ww3 BornAgain Python bindings libCore Samples and algorithms GSL Ww3 Eigen Python bindings libFit minimizers ROOT October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 03 Basic software architecture
Working with BornAgain. o  Running user C++ program External graphics User C++ program BornAgain Python bindings libCore Samples and algorithms GSL Ww3 Eigen Python bindings libFit minimizers ROOT October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 04 Basic software architecture
Working with BornAgain. o  Running user Python script External graphics User Python script BornAgain Python bindings libCore Samples and algorithms GSL Ww3 Eigen Python bindings libFit Matplotlib minimizers (Py)ROOT October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 05 Motivation
o  C++ is a complex programming language to master. o  Several people find themselves more comfortable with higher-­‐level interpreted environments. Objec8ves o  create sample descrip8on and run the simula8on from Python o  extension of the framework func8onality with Python code new formfactors, pair-­‐correla8on func8ons etc. October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 06 C++ and Python integration
There are two basic methods for integra8ng C++ with Python. o  Extension wri8ng Python access to C++ o  Embedding C++ access to the Python interpreter Python extending embedding C++ We are primarily concerned with extension wri8ng See h%p://www.swig.org/papers/PyTutorial98/PyTutorial98.pdf for nice introduc=on to the problem October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 07 Extension example using Python-C-API
Python is wriMen on C Any third-­‐party C/C++ code can be interfaced to Python using wrapper func8ons func8on.c int fact(int n)
{
if(n<=1) return1;
else return n*fact(n-1);
}
Wrapper func8on o  converts func8on arguments from Python to C o  Returns results in Python expected form Ini8aliza8on func8on o  Register new methods with the Python interpreter wrapper.c #include <Python.h>
PyObject *wrap_fact(PyObject *self, PyObject *args) {
int n, result;
if (!PyArg_ParseTuple(args,”i:fact”,&n))
return NULL;
result = fact(n);
return Py_BuildValue(“i”,result);
}
static PyMethodDef exampleMethods[] = {
{ "fact", wrap_fact, 1 },
{ NULL, NULL }
};
void initexample() {
PyObject *m;
m = Py_InitModule("example", exampleMethods);
}
October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 08 Extension example using Python-C-API
o  Func8on and wrappers code might be compiled into single shared library. g++ –I /usr/local/python2.7 function.c wrapper.c
g++ -shared function.o wrapper.o –o mymodule.so
o  Resul8ng shared library can be imported in the python module import mymodule
Wrapping a C/C++ applica8on o  Write a Python wrapping func8on for every func8on, variable, structure and class you want to access. o  Write ini8aliza8on func8on. o  Compile everything into the module. Huge amount of work! October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 09 List of technologies
Python C API Strong candidate to wrap only a couple of func8ons Boost::python Comprehensive mapping of C++ features to corresponding Python features Stl containers, excep8ons, proper8es Swig Can create wrappers for other languages besides Python (Ruby, Java) SIP Created for PyQt4 package Shiboken Created for PySide package Ctypes Provides C compa8ble data types Cython Python like language to write C extensions for Python October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 10 How to select the one
o  Different technologies looks preMy easy while wrapping void hello_world()
{
std::cout << “Hello World!” << std::endl;
}
o  They are gekng preMy tricky to master while exposing classes class IBase
C++ class Derived(IBase):
{
def __init__(self):
? IBase(float x, float y);
IBase.__init__(self)
virtual ~IBase();
def init(self):
virtual init() = 0;
self.x = 0
}
o  And what about object ownership, shared_ptr, templates? Python It is hard to say if given exposing technology fits your C++ project before you actually try to apply it to the whole project October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 11 How to select the one
Disicussion of Python-­‐C-­‐API, Ctypes, SWIG, Cython from NumPy community hMp://scipy-­‐lectures.github.io/advanced/interfacing_with_c/interfacing_with_c.html Detailed evalua8on of boost-­‐python vs SWIG (2003) hMp://seal.web.cern.ch/seal/snapshot/work-­‐packages/scrip8ng/evalua8on-­‐report.html SWIG tutorial hMp://www.swig.org/papers/PyTutorial98/PyTutorial98.pdf StackOverflow discussions hMp://stackoverflow.com/ques8ons/1492755/python-­‐c-­‐binding-­‐library-­‐comparison hMp://stackoverflow.com/ques8ons/135834/python-­‐swig-­‐vs-­‐ctypes hMp://stackoverflow.com/ques8ons/456884/extending-­‐python-­‐to-­‐swig-­‐not-­‐to-­‐swig-­‐or-­‐cython October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 12 How to select the one
o 
o 
o 
o 
o 
o 
o 
o 
What is the performance? Build system integra8on? Is wrapping code on Python side or on on C++ side? How much code should be wriMen addi8onally? Should I affect or duplicate exis8ng C++ code? How big is a community? Is it possible to fully automa8ze wrappers genera8on? Do I need bindings with another languages? We have decided to go with boost::python providing the higher level of integra8on between C++ and Python among all compe8tors. October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 13 Boost::python
C++ template based library which enables interoperability between C++ and Python programming languages. o  References and pointers. o  Automa8c cross module type conversions. o  Efficient func8on overloading. o  C++ to Python excep8on transla8on. o  Default arguments. o  Keyword arguments. o  Expor8ng C++ iterators as Python iterators. o  Documenta8on strings. o  Manipula8ng Python objects in C++. October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 14 Boost::python
Example I o  Exposing class and its access func8ons to Python Class to expose Wrapper to create class TwoNumbers {
C++ public:
Num();
float getA() const;
void setA(float value);
float getB() const;
void setB(float value);
private:
double a;
double b;
};
C++ boost::python::class_<TwoNumbers>(”TwoNumbers")
.add_property(”A", &TwoNumbers::getA, &TwoNumbers::setA)
.add_property(”B", &TwoNumbers::getB);
Use from Python >>> x = TwoNumbers()
>>> x.A = 3.14
>>> x.A, x.B
(3.14, 3.14)
>>> x.B = 2.17 # error!
Python Class property “B” is read only since seMer member func8on not passed in October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 15 Boost::python
Example II o  Inheritance and virtual func8ons Class to expose class Base {
virtual ~Base() {}
virtual int f()
{
return 0;
}
};
Wrapper to create C++ struct BaseWrap : Base, wrapper<Base>
{
int f()
{
if (override f = this->get_override("f")) return f();
return Base::f();
}
int default_f() { return this->Base::f(); }
};
C++ class_<BaseWrap, boost::noncopyable>("Base")
.def("f", &Base::f, &BaseWrap::default_f) ;
Use from Python >>>
>>>
...
...
...
>>>
>>>
0
>>>
42
base = Base()
class Derived(Base):
def f(self):
return 42
Python derived = Derived()
base.f()
derived.f()
October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 16 pyplusplus
o  Boost::python allows to expose to Python almost any C++ construc8on. o  However, this requires wri8ng of a large amount of wrapping code. o  Fortunately, this can be tackled with Pyplusplus Pyplusplus package An object oriented framework for crea8ng a code generator for boost::python library BornAgain Pyplusplus Boost::python When When C++ required required codegenerator wrappers source code compiler o  We regenerate boost::python API only when public interface has changed October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 17 pyplusplus
codegenerator.py o  Provides automa8c genera8on of boost::python code for the whole project o  Contains a number of sekngs to adjust ownership policies, list of classes to expose, etc o  About 300 lines of python code October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 18 Current status
o  BornAgain core consist of 30k lines of code o  PythonAPI is generated automa8cally using boost::python + pyplusplus o  Same shared library can be used from C++ and from Python Learning curve o  10% of 8me was devoted to Python bindings during the first year o  <1% of 8me during the second Achieved Python/C++ interoperability o  Inheritance and overload o  Transfer of ownership o  Templates exposing o  Python lists to std::vector and back o  Export C++ data structures into NumPy arrays o  shared_ptr to Python and back Number of lines of code in BornAgain git repository October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 19 Current status
C++ Python October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 20 Summary
boost::python + pyplusplus Advantages o 
o 
o 
o 
It is possible to to establish complete interoperability between C++ and python. Actually no limita8ons have been found in exposing C++ construc8ons into Python. No C++ code should be adjusted. Single script regenerates PythonAPI for the whole project if needed. Disadvantages o  boost::python wrappers double the amount of code. o  It is heavy templated code which double the compila8on 8me. o  pyplusplus and gccxml are not developed anymore, no C++11 support will follow. Future plans. o  Wait for ROOT version 6.0 which will use technology llvm + PyPy and have a look on it October, 24, 2013, ILL, Grenoble Python bindings in BornAgain, Gennady Pospelov 21