Download dynamicsEngine.tref

Integration and development
within the CCPN framework
Expanded from presentation,
Paris Oct 2006
• Overview
• ARIA 2 integration
• DASH integration
• CLOUDS integration
Integration options
■ Integration requires lossless exchange of data
with the CCPN data standard :
● Direct API access. Application interacts directly with
CCPN data structures. Easiest for new applications.
● In-memory conversion. Application loads CCPN
data and converts them to its own data structures in
memory.
● Format conversion. Application input/output files are
converted to data standard via file conversion. Less
well integrated than the alternatives
■ Interoperability, or combining applications into a
pipeline, requires further coordination.
Integration options 2
The integration options in this presentation will work for
applications in any language that has a CCPN
implementation.
The CCPN applications are written mainly in Python, as
are the utility function libraries and GUI widgets
presented here. Only Python applications can be
developed ‘inside the CCPN framework’ in the manner
presented here. Applications in other languages will
have only the generated APIs and I/O libraries
available.
CCPN Interface Schemes
Via FormatConverter
Application
Proprietary
Memory
Formatted
File
Format
Converter
CCPN
XML/SQL
In-memory conversion
Application
Proprietary
Model
Custom conversion
CCPN
Data Model
Direct API access
Application
CCPN Data Model
CcpNmr Functions
CCPN
XML/SQL
CCPN
XML/SQL
Development inside the CCPN framework
■ CcpNmr Macros
● Small home-use Python
functions
■ Additions to function library
● Functions incorporated in
software release
● Community sharing
■ Embedded options
● Extension to CcpNmr application
■ Stand-alone applications
● Built on CCPN libraries and API
CcpNmr Analysis Function Library
■ Assignment
■ Constraints
■ Data Analysis
■ Experiments & Spectra
■ Peaks
■ Structure
■ Spectrum Windows
assignResToDim(peakDim, resonance)
Assign a resonance to peak dimension
Checks:
• Any atoms are of a valid molecule
• Isotopes match dimension
• Shift is within tolerances
• Whether the aliasing is changed
Creates:
• Covalent links between resonances
• Peak annotation label
• Updated chemical shift value
• Data Model objects
Ccp Stand-Alone AppTemplate
■ Menu System
■ Project handling
●
●
●
●
New
Load
Save
Backup
■ Popup template
● Widgets
● Geometry
● Plumbing
CcpNmr Graphical Widgets
■ A library for any developer to use
ColorList
PulldownMenu
ScrolledMatrix
LabelFrame
CheckButton
Button
Label
Entry
ButtonList
CcpNmr Mega Widgets
■ Build them into your own code!
● ScrolledMatrix
● ScrolledGraph
● StructureFrame
Example Applications
■ ARIA 2
● Structure calculation by iterative assignment of ambiguous restraints
● In memory data model to data model conversion: Python API code
● Constraints, Violations, Structures, Molecules
■ DASH
●
●
●
●
●
Predict dihedral angles from chemical shifts
GUI based on application template
Write input and read output of C program
Collate reference chemical shifts with structure: FormatConverter
Molecules, Dihedral constraints
■ CLOUDS
● Assignment free structure determinations
● Python and C modules: Python-C interface
● Interpret proprietary files
• Overview
• ARIA 2 integration
• DASH integration
• CLOUDS integration
ARIA2-CCPN interface
Python-Python in-memory conversion
Application
ARIA
XML
ARIA
Data Model
Custom conversion
■ ARIA imports
● Peak Lists
● Constraints
■ Distance
■ Dihedral
■ RDC
● Sequences
● Chemical shifts
CCPN
Data Model
■ ARIA exports
●
●
●
●
Peak Assignments
Constraints
Violations
Structures
CCPN
XML/SQL
ARIA2-CCPN interface
Python
ARIA
XML
ARIA
Data Model
ARIA functions &
CCPN functions
CCPN
Data Model
CCPN
XML/SQL
CCPN object selection
CCPN conversion in ARIA2
CCPN Nmr.Shift
ARIA ChemicalShift
def getAriaChemicalShift(self, ccpn_object):
from Datum import ChemicalShift
if ccpn_object is None:
val = None
err = None
else:
val = ccpn_object.value
if hasattr(ccpn_object, 'valueError'):
err = ccpn_object.valueError
else:
err = ccpn_object.error
return ChemicalShift(val, err)
ARIA 2 functions in CCPN
■ Bundled with CCPN distribution
■ This side of the interface maintained by CCPN
■ Gives CcpNmr Applications access to ARIA projects
importAria2RunData(dirName, constraintSet=None,
project=None, loadStructures=False)
getAria2ObjectsFromPickle(fileName)
getChainFromAria2(restraints, project)
getStructuresFromAria2Dir(dirName, chain)
getConstraintsFromAria2(ariaRestriaints, chain, constraintSet)
getConstraintFromAria2(ariaRestraint, constraintList)
getFixedResonanceFromAria2Atom(ariaAtom, chain, constraintSet)
getAria2AtomSetPairs(ariaRestraint)
ARIA 2 conversion in CCPN
ARIA Violation
CCPN NmrConstraints.Violation
def getViolationFromAria2(ariaViolation, constraint, violationList):
violation = None
if ariaViolation.isViolated():
calcDist = ariaViolation.getAverageDistance().getValue()
calcError = ariaViolation.getAverageDistance().getError()
fracViols = max(0.0,min(ariaViolation.getDegreeOfViolation(),1.0))
violValue = ariaViolation.getUpperBoundViolation().getValue()
violation = violationList.newViolation(violation=violValue,
calcValue=calcDist,
calcValueError=calcError,
constraint=constraint,
fractionViolated=fracViols)
return violation
ARIA 2 calling CCPN functions
import ccpnmr.analysis.ConstraintBasic as CB
self.message('CCPN export: Accessing data model...')
ccpn_project = self.open_ccpn_data_model(project)
export = 1
chain = CB.getChainfromAria2(restraints, ccpn_project)
if not chain:
self.warning(StandardError,
'CCPN export: No molecular system found.')
return
structures = CB.getStructuresFromAria2Dir(path, chain)
if not structures:
self.warning('CCPN export: Unable to load any structures
from iteration directory %s' % path)
else:
self.message('CCPN export: PDB files exported.')
struct_gen = CB.makeStructureGeneration(structures, constraintSet)
• Overview
• ARIA 2 integration
• DASH integration
• CLOUDS integration
DASH Overview
■ Predicts peptide dihedral angles from chemical shifts
● Similar to TALOS
■ Uses CCPN Python API directly in code
● No real CCPN conversion needed
■ Scientific C code separate from CCPN
DASH database
CCPN XML
DASH Python
Tkinter GUI
Input File
DASH C
Output File
DASH Overview
■ Imports
●
●
●
●
■ Exports
Chemical shifts
Reference chemical shifts
Reference structures
Molecular sequence
● Dihedral angle constraints
■ Values
■ Errors
Chemical shifts
Molecular sequence
Dihedral angle
constraints
BMRB chemical shifts
NMR Star format
PDB protein structures
PDB format
CcpNmr
FormatConverter &
Analysis functions
DASH
Python
DASH
C
CCPN API use in DASH
phiResonances = (fixedResonances[0], fixedResonances[1],
fixedResonances[2], fixedResonances[3])
phiConstraint = constraintList.newDihedralConstraint(resonances=phiResonances)
psiResonances = (fixedResonances[1], fixedResonances[2],
fixedResonances[3], fixedResonances[4])
psiConstraint = constraintList.newDihedralConstraint(resonances=psiResonances)
# make items
DASHres = self.DASHmol.getResidue(i + first_resNum)
phi = DASHres.pred_phi[DASHres.index]
psi = DASHres.pred_psi[DASHres.index]
phi_limit = DASHres.sd_phi[DASHres.index] * 2
psi_limit = DASHres.sd_psi[DASHres.index] * 2
phiConstraint.newDihedralConstraintItem(targetValue=phi,
upperLimit =phi+phi_limit,
lowerLimit =phi-phi_limit,
error=phi_limit)
psiConstraint.newDihedralConstraintItem(targetValue=psi,
upperLimit =psi+psi_limit,
lowerLimit =psi-psi_limit,
error=psi_limit)
F, Y angle prediction
CCPN DihedralConstraint
DASH – Python GUI
■ Uses the CcpNmr application template
■ Uses the CcpNmr Python Tkinter graphics objects
(widgets)
● AppTemplate provides menu
● ScrolledMatrix provides tables
● ScrolledCanvas used custom graphics
DASH – Python GUI
■ DASH Ramachandran plot used as basis for CcpNmr equivalent
● Development goes both ways!
• Overview
• ARIA 2 integration
• DASH integration
• CLOUDS integration
The CLOUDS Protocol
■ Automated assignment &
structure determination
● Miguel Llinas, Alex Grishaev, et al.
● Spatial distribution of anonymous
resonances generated with NOEs
H
H
■ Integrated within CCPN
●
●
●
●
An Analysis module
Data Model glues modules
Functional platform
Distribution network
H
H
A network of distance constraints between
anonymous atoms is sufficient to generate
a low resolution protein structure.
The CLOUDS Protocol
A family of Clouds
CcpNmr Clouds
Spectra
■ Implementation of existing software
● Interface existing code
● Re-written code modules
● Embed in CcpNmr
LinkPeakLists
Pick Peaks,
Link Shifts &
Combine
Spin Systems
MIDGE
BACUS
NOE matrix
Relaxation Matrix
Optimisation
Distance Constraints
HCloudsMD
Hydrogen Atom
Molecular Dynamics
Proton Clouds
CloudThreader
Chain Fitting &
Molecular Replacement
Chain Assignment
ARIA 2.1
Full Structure
Calculation
Protein Structure
CcpNmr Clouds: BACUS
■
■
■
■
Preserve C code
Generate input files
Interpret output files
Really simple GUI!
Resonances
ShiftList
NOE PeakLists
Assigned
PeakLists
Input Files
BACUS
Output Files
CcpNmr Clouds: HCloudsMD
■ FORTRAN -> C
■ Python wrapper
● Control distance constraints
● Cooling scheme parameters
● Dynamics engine
Distance Constraints
Parameters
Resonances
HCloudsMD
Family of Clouds
CCPN Python using CLOUDS objects
def runDynamicsProtocol(atomCoordList, noeConstrList, dynamicsEngine,
noeForceField, coolingScheme, rp_force_const=1):
for (i,temp_i,temp_f,ncooling,nsteps,tau,rp_scale) in coolingScheme:
tref = temp_i
rpf = rp_force_const * rp_scale
if (ncooling > 1):
dtemp = (temp_f - temp_i) / float(ncooling - 1)
else:
dtemp = 0
dynamicsEngine.rp_force_const
dynamicsEngine.tref
dynamicsEngine.tau
dynamicsEngine.nsteps
=
=
=
=
rpf
tref
tau
nsteps
for j in range(ncooling):
dynamicsEngine.run(atomCoordList, noeConstrList, noeForceField)
dynamicsEngine.tref = dynamicsEngine.tref + dtemp
return atomCoordList
Python-C interface
#include "python_util.h"
static PyObject *init_Py_Dynamics(PyObject *self, PyObject *args,
PyObject *keywds)
{
float rp_force_const = 1, beta = 10, rmin = 2.25, drzap = 2;
float tref = 1000, tau = 0.001, elapsed_time = 0;
int nsteps = 1000, nprint = 3000;
static char *kwlist[] = { "rp_force_const", "beta", "rmin",
"drzap", "tref", "tau", "elapsed_time", "nsteps", "nprint", NULL
};
if (!PyArg_ParseTupleAndKeywords(args, keywds, "|fffffffii", kwlist,
&rp_force_const, &beta, &rmin, &drzap, &tref, &tau,
&elapsed_time, &nsteps, &nprint))
RETURN_OBJ_ERROR("need arguments: [ rp_force_const, beta, rmin,
drzap, tref, tau, elapsed_time, nsteps, nprint ]");
return (PyObject *) new_py_dynamics(rp_force_const, beta, rmin,
drzap, tref, tau, elapsed_time, nsteps, nprint);
}
Python callable functions in C
static PyObject *run(PyObject *self, PyObject *args) {
Py_Dynamics py_dynamics = (Py_Dynamics) self;
PyObject *atom_coord_list_obj, *noe_list_obj, *noe_force_obj;
Py_Atom_coord_list py_atom_coord_list;
Py_Dist_constraint_list py_noe_list;
Py_Dist_force py_noe_force;
Atom_coord_list atom_coord_list;
Dist_constraint_list noe_list;
Line error_msg;
CcpnStatus status;
if (!PyArg_ParseTuple(args, "OOO", &atom_coord_list_obj,
&noe_list_obj, &noe_force_obj))
RETURN_OBJ_ERROR("must have three arguments: atomCoordList,
noeList, noeForce");
… …
py_atom_coord_list = (Py_Atom_coord_list) atom_coord_list_obj;
py_noe_list = (Py_Dist_constraint_list) noe_list_obj;
py_noe_force = (Py_Dist_force) noe_force_obj;
atom_coord_list = get_atom_coord_list(py_atom_coord_list);
noe_list = get_dist_constraint_list(py_noe_list);
status = run_dynamics(py_dynamics->dynamics, atom_coord_list,
noe_list, py_noe_force->dist_force,error_msg);
Popup Constructors and Notifiers
■ Init
● Setup local variables
● Subclass popup window
Initialisation
■ Body
● Arrange Graphical elements
● Set up Data Model notifiers
● Set initial state
■ Update
● Process updated values
● Redraw widgets based on status
■ Widget callback
● From entry, buttons etc
● User functions
● Data Model change
User
Influence
Widgets
Body
Notifiers
Update Filter
Update
External
Influence
Data
Model