Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Windows System Programming using Python Mark Hammond [email protected] OReilly Open Source Python Conference August 1999, Monterey, CA About this Presentation Most content taken directly from upcoming book for O’Reilly Python Programming on Win32 By Mark Hammond and Andy Robinson http://www.ora.com/catalog/pythonwin32/ O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 2 Who this talk is for? Existing Python programmers – Even those without any Windows experience should follow this without problem. Although there is not enough time to explain the relevant Windows APIs Existing Windows Programmers – Even without Python experience, you should immediately see the similarities between your existing language. O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 3 What is Python Interpreted, dynamic high-level language Obviously open source – Hence we are here. Often used in a similar problem domain to Perl/Tcl Proponents consider readability and maintainability a big strong-point http://www.python.org O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 4 Python and Windows Each Python version comes with an installer package for Windows. Standard Python port contains all crossplatform Python features, but very few Windows specific features. Python for Windows extensions contains many useful Windows extensions for Python. http://www.python.org/windows O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 5 Python Windows Extensions Includes: – Pythonwin: MFC based GUI environment and IDE/Debugger – win32com: Interfaces Python and COM – win32 Extensions: Interfaces to native Win32 API. Official releases can be found at http://www.python.org/windows Extensions home is at http://starship.python.net/crew/mhammond O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 6 System Level Programming? For this talk, we define system level programming as working with low-level features of Windows Files, Pipes, Processes, Threads, Services, Event Log and so forth. Python and similar languages really not suitable for device-driver type development, and other more system-like Systems Programming! O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 7 Why not just use Python? Python has excellent native support for files, processes, threads etc. These features are typically limited to those defined by ANSI C. – Many advanced Windows features are not exposed using these interfaces. – Standard implementation of some of the standard library functions leaves a little to be desired in a Windows environment. O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 8 Portable Process Control (1 of 3) Standard Python functions all work – Just often not quite how we would like! os.system() import os os.system(“notepad C:\\autoexec.bat”) Problems – Creates a new console window when run from a GUI. – Waits for process to terminate. O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 9 Portable Process Control (2 of 3) os.execv family – Doesn’t search system path, and doesn’t parse command lines os.execv("c:\\Winnt\\notepad.exe", \ ("c:\\autoexec.bat",) ) – Does clobber your existing process - the call to os.execv() never returns! O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 10 Portable Process Control (3 of 3) os.popen() >>> file = os.popen("echo Hello") >>> file.read() 'Hello\012' – Works fine from Windows NT console programs, but fails miserably from a GUI! – win32pipe module in the Win32 extensions provides a working replacement. O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 11 Better Process Control (1 of ) win32api module provides some highlevel, Windows specific functions. win32api.WinExec() – Very similar to os.system(), but overcomes limitations. – >>> import win32api – >>> win32api.WinExec("notepad") – Optional parameter allows you to specify the Window’s initial state (eg, minimized) O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 12 Better Process Control (2 of 2) win32api.ShellExecute() – Typically opens “documents” - eg, execute “foo.doc”, and (typically) Word will open. – Finer control over the new process. – Can also execute arbitrary executables - not limited to documents. – For example, to print a specific document: win32api.ShellExecute(0, "print", \ "MyDocument.doc", None, "", 1) O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 13 Ultimate Process Control (1 of 2) win32process module exposes the low level Win32 API. Full support for CreateProcess, CreateProcessAsUser, CreateThread etc. Full support for Windows Handles – Files can be passed as stdin/out/err – Process and thread handles are waitable using the win32event module. O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 14 Ultimate Process Control (2 of 2) Able to set thread and process priority and affinity levels – Very handy for bugs that only appear in multiprocessor machines. Able to do all this for both existing and new processes. Process usage samples included in distribution. O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 15 Introduction to our Sample Full sample Windows NT Service – Provides event log and performance monitor information. Clients connect using Named Pipes – Less than 75 lines of code for all this functionality. – Still too big to present in one hit Selected excerpts included in slides Full code on CD, and at http://starship.python.net/crew/mhammond/conferences/ ora99 O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 16 Portable Files and Pipes Python has excellent built-in file support – Inherits platform stdio support - wont bother discussing them here. Native Windows files only useful for highly-advanced features, such as: – Overlapped IO – IO Completion Ports – Named Pipes – NT Security requirements O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 17 Native Files (1 of 4) win32file.CreateFile() used for most file operations – Create and open regular files, memory mapped files, etc. – Takes seven parameters - c.f. open()’s two! – Returns a PyHANDLE object Can be passed to any Python API wrapper expecting a handle. Auto-closed when last reference removed. O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 18 Native Files (2 of 4) Overlapped IO for asynchronous operations – File opened for overlapped IO requires a Windows event object. – All IO operations return immediately. – Event object signalled when IO complete Great for high-performance servers – Simple to support multiple concurrent file operations per thread. O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 19 Native Files (3 of 4) NT Completion ports for even better asynchronous control – Windows manages associating the completed IO operation with a connection – More complex to use. – Often requires a state-machine implementation. – Offers excellent performance - Microsoft’s recommended architecture for scalable, highperformance servers. O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 20 Native Files (4 of 4) Full support for NT Security – Default security can be used by passing None – Many real-world applications require explicit security configuration Full support for Windows Impersonation – Processes can automatically impersonate the remote pipe client. – Impersonate any user given their password. O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 21 File Sample (1 of 2) Overlapped IO is used self.overlapped = \ pywintypes.OVERLAPPED() # create the event to be used. self.overlapped.hEvent = \ win32event.CreateEvent(None,0,0,None) Special security for pipes is needed for services sa = win32security.SECURITY_ATTRIBUTES() # Allow full access! sa.SetSecurityDescriptorDacl ( 1, None, 0 ) O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 22 File Sample (2 of 2) Create pipe and connect to client pipeHandle = \ win32pipe.CreateNamedPipe(pipeName, openMode, pipeMode, win32pipe.PIPE_UNLIMITED_INSTANCES, 0, 0, 6000, # 6 second timeout. sa) ... hr = win32pipe.ConnectNamedPipe(pipeHandle,\ self.overlapped) O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 23 Windows NT Services Similar concept to a Unix daemon A few special requirements – Must respond to asynchronous commands from NT to (e.g.) Shutdown – Must be capable of reporting status to NT NT has built-in UI for configuring and controlling services. O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 24 Python controlling Services Full exposure of the Windows NT Service Control Manager API – Start, Stop, Pause Services, Install or Remove services, etc win32serviceutil module makes it simple >>> win32serviceutil.StopService("Messenger") (32, 3, 0, 0, 0, 6, 20000) >>> win32serviceutil.StartService(...) >>> O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 25 Implementing Services Simple to implement Services in Python – Smallest service is around 16 lines of code! Includes debug support, self-installation, and fully controllable from Windows NT Few more lines needed to make it something useful :-) Simply sub-class Service base-class, and implement control features – Minimum required is StopService – Trivial to implement most controls O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 26 Windows NT Event Log Central place for programs to log information. – Ideal for Services - can not present effective GUIs Benefits to programmer – Built in transaction and thread safety, maximum size ability, etc. Benefits to Administrator – Central log of messages, and third party tools to help analysis. O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 27 Python reading the Event Log Complex native API from win32evtlog Simpler interface from win32evtlogutil – Define Feeder function def DumpRecord(record): print ”Got event ID”, record.EventID – And feed it! win32evtlogutil.FeedEventLogRecords( \ DumpRecord) Got Event ID -2147483645 Got Event ID -2147483645 O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 28 Python writing the Event Log More complex than reading - event sources must be registered. win32evtlog and win32evtlogutil used here too Python Service framework also supports simple event log writing O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 29 Writing the Event Log Sample Our sample uses the Service Framework functions which makes it trivial Events are logged by ID, rather than explicit text. – Our sample uses a built-in ID to log a “service starting” message import servicemanager servicemanager.LogMsg( servicemanager.EVENTLOG_INFORMATION_TYPE, servicemanager.PYS_SERVICE_STARTED, (self._svc_name_, '')) O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 30 NT Performance Monitor Built-in NT tool for monitoring an applications performance. Application must be written to provide this data. API designed for smallest impact on program supplying data – Moves the burden to the collecting application. – Not trivial to work with O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 31 Python reading from PerfMon Useful for overcoming limitations in builtin tool – For example, sample the data hourly and log to a longer-term database. – Useful for reading standard information about another process. Process ID, Memory usage, etc. win32pdhutil module for reading data – Good sample code in the source file O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 32 Python supplying PerfMon Complex installation and setup procedure. – C .h file and custom .ini file necessary at application installation – Supported by the Python Service framework Quite trivial to use once running – Simply increment a counter - Performance Monitor handles conversions to required units. O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 33 PerfMon Sample Installation more than we can cover here – See InitPerfMon() method Working with counters trivial – Increment our connections counter each connection self.counterConnections.Increment() Perfmon manages converting to connections-per-second automatically. O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 34 Our Sample in Detail (1 of 8) PipeService2.py – The service implementation. PipeService2_install.h PipeService2_install.ini – Required for performance monitor installation – Most services don’t need this! PipeServiceClient.py – Sample client to connect to our service. O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 35 Our Sample in Detail (2 of 8) Service functionality in PipeService class – Base class handles most of the grunt – We simply supply service name and other optional attributes class PipeService(\ win32serviceutil.ServiceFramework): _svc_name_ = "PythonPipeService" _svc_display_name_ = "A sample Python ... " O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 36 Our Sample in Detail (3 of 8) We respond to an NT ServiceStop request by telling NT we are stopping, and setting a Windows Event def SvcStop(self): # Before we do anything, tell the # SCM we are starting the stop process. self.ReportServiceStatus( \ win32service.SERVICE_STOP_PENDING) # And set my event. win32event.SetEvent(self.hWaitStop) O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 37 Our Sample in Detail (4 of 8) Overlapped IO means we can wait for either a connection, or our Stop request win32pipe.ConnectNamedPipe(pipeHandle,\ self.overlapped) ... # Wait for either a connection, or # a service stop request. waitHandles = self.hWaitStop, \ self.overlapped.hEvent rc = win32event.WaitForMultipleObjects(\ waitHandles, 0, timeout) O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 38 Our Sample in Detail (5 of 8) Install our service simply by executing the service script – If not for Performance Monitor, the command-line would be simple: C:\Scripts> PipeService2.py install – But PerfMon needs an extra arg C:\Scripts> PipeService2.py \ --perfmonini=PipeService2_install.ini install Installing service PythonPipeService to ... Service installed O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 39 Our Sample in Detail (6 of 8) Once installed, we can start the service – Start from Control Panel, or using the script itself C:\Scripts> python.exe PipeService2.py start And start a client test session C:\Scripts> python.exe PipeServiceClient.py \ Hi there The service sent back: You sent me:Hi there O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 40 Our Sample in Detail (7 of 8) We will have Event Log records... O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 41 Our Sample in Detail (8 of 8) And Performance Monitor Data. O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 42 Summary Python library flexible, rich and portable enough for many tasks Low-level Windows programming achieved by using extensions that expose the raw Windows API This talk should have shown: – An overview of how this programming is done in Python. – How simple it is to do complex Windows tasks. O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 43 More Information Python and Python Documentation – http://www.python.org – http://www.python.org/doc Python for Windows Extensions – http://starship.python.net/crew/mhammond – http://www.python.org/windows – Reference Manuals and Samples included with the distributions O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 44 Thanks for coming Mark Hammond – [email protected] – http://starship.python.net/crew/mhammond O’Reilly Python Conference Aug 24, 1999 Windows System Programming using Python Slide 45