Download Windows System Programming using Python

Windows System
Programming using
Python
Mark Hammond
[email protected]
OReilly Open Source Python Conference
August 1999, Monterey, CA
About this Presentation

Most content taken directly from
upcoming book for O’Reilly
Python Programming on Win32
By Mark Hammond and Andy Robinson

http://www.ora.com/catalog/pythonwin32/
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 2
Who this talk is for?

Existing Python programmers
– Even those without any Windows experience
should follow this without problem.
 Although
there is not enough time to explain the
relevant Windows APIs

Existing Windows Programmers
– Even without Python experience, you should
immediately see the similarities between your
existing language.
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 3
What is Python
Interpreted, dynamic high-level language
 Obviously open source

– Hence we are here.
Often used in a similar problem domain to
Perl/Tcl
 Proponents consider readability and
maintainability a big strong-point


http://www.python.org
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 4
Python and Windows
Each Python version comes with an
installer package for Windows.
 Standard Python port contains all crossplatform Python features, but very few
Windows specific features.
 Python for Windows extensions contains
many useful Windows extensions for
Python.
 http://www.python.org/windows

O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 5
Python Windows Extensions

Includes:
– Pythonwin: MFC based GUI environment and
IDE/Debugger
– win32com: Interfaces Python and COM
– win32 Extensions: Interfaces to native Win32 API.

Official releases can be found at
http://www.python.org/windows

Extensions home is at
http://starship.python.net/crew/mhammond
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 6
System Level Programming?
For this talk, we define system level
programming as working with low-level
features of Windows Files, Pipes,
Processes, Threads, Services, Event Log
and so forth.
 Python and similar languages really not
suitable for device-driver type
development, and other more system-like
Systems Programming!

O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 7
Why not just use Python?
Python has excellent native support for
files, processes, threads etc.
 These features are typically limited to
those defined by ANSI C.

– Many advanced Windows features are not
exposed using these interfaces.
– Standard implementation of some of the standard
library functions leaves a little to be desired in a
Windows environment.
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 8
Portable Process Control (1 of 3)

Standard Python functions all work
– Just often not quite how we would like!

os.system()
import os
os.system(“notepad C:\\autoexec.bat”)
Problems
– Creates a new console window when run from a
GUI.
– Waits for process to terminate.
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 9
Portable Process Control (2 of 3)

os.execv family
– Doesn’t search system path, and doesn’t parse
command lines
os.execv("c:\\Winnt\\notepad.exe", \
("c:\\autoexec.bat",) )
– Does clobber your existing process - the call to
os.execv() never returns!
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 10
Portable Process Control (3 of 3)

os.popen()
>>> file = os.popen("echo Hello")
>>> file.read()
'Hello\012'
– Works fine from Windows NT console programs,
but fails miserably from a GUI!
– win32pipe module in the Win32 extensions
provides a working replacement.
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 11
Better Process Control (1 of )
win32api module provides some highlevel, Windows specific functions.
 win32api.WinExec()

– Very similar to os.system(), but overcomes
limitations.
– >>> import win32api
– >>> win32api.WinExec("notepad")
– Optional parameter allows you to specify the
Window’s initial state (eg, minimized)
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 12
Better Process Control (2 of 2)

win32api.ShellExecute()
– Typically opens “documents” - eg, execute
“foo.doc”, and (typically) Word will open.
– Finer control over the new process.
– Can also execute arbitrary executables - not
limited to documents.
– For example, to print a specific document:
win32api.ShellExecute(0, "print", \
"MyDocument.doc", None, "", 1)
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 13
Ultimate Process Control (1 of 2)
win32process module exposes the low
level Win32 API.
 Full support for CreateProcess,
CreateProcessAsUser, CreateThread etc.
 Full support for Windows Handles

– Files can be passed as stdin/out/err
– Process and thread handles are waitable using
the win32event module.
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 14
Ultimate Process Control (2 of 2)

Able to set thread and process priority and
affinity levels
– Very handy for bugs that only appear in multiprocessor machines.
Able to do all this for both existing and
new processes.
 Process usage samples included in
distribution.

O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 15
Introduction to our Sample

Full sample Windows NT Service
– Provides event log and performance monitor
information. Clients connect using Named Pipes
– Less than 75 lines of code for all this functionality.
– Still too big to present in one hit
Selected excerpts included in slides
 Full code on CD, and at

http://starship.python.net/crew/mhammond/conferences/
ora99
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 16
Portable Files and Pipes

Python has excellent built-in file support
– Inherits platform stdio support - wont bother
discussing them here.

Native Windows files only useful for
highly-advanced features, such as:
– Overlapped IO
– IO Completion Ports
– Named Pipes
– NT Security requirements
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 17
Native Files (1 of 4)

win32file.CreateFile() used for most file
operations
– Create and open regular files, memory mapped
files, etc.
– Takes seven parameters - c.f. open()’s two!
– Returns a PyHANDLE object
 Can
be passed to any Python API wrapper expecting a
handle.
 Auto-closed when last reference removed.
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 18
Native Files (2 of 4)

Overlapped IO for asynchronous
operations
– File opened for overlapped IO requires a
Windows event object.
– All IO operations return immediately.
– Event object signalled when IO complete

Great for high-performance servers
– Simple to support multiple concurrent file
operations per thread.
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 19
Native Files (3 of 4)

NT Completion ports for even better
asynchronous control
– Windows manages associating the completed IO
operation with a connection
– More complex to use.
– Often requires a state-machine implementation.
– Offers excellent performance - Microsoft’s
recommended architecture for scalable, highperformance servers.
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 20
Native Files (4 of 4)

Full support for NT Security
– Default security can be used by passing None
– Many real-world applications require explicit
security configuration

Full support for Windows Impersonation
– Processes can automatically impersonate the
remote pipe client.
– Impersonate any user given their password.
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 21
File Sample (1 of 2)

Overlapped IO is used
self.overlapped = \
pywintypes.OVERLAPPED()
# create the event to be used.
self.overlapped.hEvent = \
win32event.CreateEvent(None,0,0,None)

Special security for pipes is needed for
services
sa = win32security.SECURITY_ATTRIBUTES()
# Allow full access!
sa.SetSecurityDescriptorDacl ( 1, None, 0 )
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 22
File Sample (2 of 2)

Create pipe and connect to client
pipeHandle = \
win32pipe.CreateNamedPipe(pipeName,
openMode,
pipeMode,
win32pipe.PIPE_UNLIMITED_INSTANCES,
0, 0, 6000, # 6 second timeout.
sa)
...
hr = win32pipe.ConnectNamedPipe(pipeHandle,\
self.overlapped)
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 23
Windows NT Services
Similar concept to a Unix daemon
 A few special requirements

– Must respond to asynchronous commands from
NT to (e.g.) Shutdown
– Must be capable of reporting status to NT

NT has built-in UI for configuring and
controlling services.
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 24
Python controlling Services

Full exposure of the Windows NT Service
Control Manager API
– Start, Stop, Pause Services, Install or Remove
services, etc

win32serviceutil module makes it simple
>>> win32serviceutil.StopService("Messenger")
(32, 3, 0, 0, 0, 6, 20000)
>>> win32serviceutil.StartService(...)
>>>
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 25
Implementing Services

Simple to implement Services in Python
– Smallest service is around 16 lines of code!
 Includes
debug support, self-installation, and fully
controllable from Windows NT
 Few more lines needed to make it something useful :-)

Simply sub-class Service base-class, and
implement control features
– Minimum required is StopService
– Trivial to implement most controls
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 26
Windows NT Event Log

Central place for programs to log
information.
– Ideal for Services - can not present effective GUIs

Benefits to programmer
– Built in transaction and thread safety, maximum
size ability, etc.

Benefits to Administrator
– Central log of messages, and third party tools to
help analysis.
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 27
Python reading the Event Log
Complex native API from win32evtlog
 Simpler interface from win32evtlogutil

– Define Feeder function
def DumpRecord(record):
print ”Got event ID”, record.EventID
– And feed it!
win32evtlogutil.FeedEventLogRecords( \
DumpRecord)
Got Event ID -2147483645
Got Event ID -2147483645
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 28
Python writing the Event Log
More complex than reading - event
sources must be registered.
 win32evtlog and win32evtlogutil used here
too
 Python Service framework also supports
simple event log writing

O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 29
Writing the Event Log Sample
Our sample uses the Service Framework
functions which makes it trivial
 Events are logged by ID, rather than
explicit text.

– Our sample uses a built-in ID to log a “service
starting” message
import servicemanager
servicemanager.LogMsg(
servicemanager.EVENTLOG_INFORMATION_TYPE,
servicemanager.PYS_SERVICE_STARTED,
(self._svc_name_, ''))
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 30
NT Performance Monitor
Built-in NT tool for monitoring an
applications performance.
 Application must be written to provide this
data.
 API designed for smallest impact on
program supplying data

– Moves the burden to the collecting application.
– Not trivial to work with
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 31
Python reading from PerfMon

Useful for overcoming limitations in builtin tool
– For example, sample the data hourly and log to a
longer-term database.
– Useful for reading standard information about
another process.
 Process

ID, Memory usage, etc.
win32pdhutil module for reading data
– Good sample code in the source file
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 32
Python supplying PerfMon

Complex installation and setup procedure.
– C .h file and custom .ini file necessary at
application installation
– Supported by the Python Service framework

Quite trivial to use once running
– Simply increment a counter - Performance
Monitor handles conversions to required units.
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 33
PerfMon Sample

Installation more than we can cover here
– See InitPerfMon() method

Working with counters trivial
– Increment our connections counter each
connection
self.counterConnections.Increment()

Perfmon manages converting to
connections-per-second automatically.
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 34
Our Sample in Detail (1 of 8)

PipeService2.py
– The service implementation.

PipeService2_install.h
PipeService2_install.ini
– Required for performance monitor installation
– Most services don’t need this!

PipeServiceClient.py
– Sample client to connect to our service.
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 35
Our Sample in Detail (2 of 8)

Service functionality in PipeService class
– Base class handles most of the grunt
– We simply supply service name and other
optional attributes
class PipeService(\
win32serviceutil.ServiceFramework):
_svc_name_ = "PythonPipeService"
_svc_display_name_ = "A sample Python ... "
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 36
Our Sample in Detail (3 of 8)

We respond to an NT ServiceStop request
by telling NT we are stopping, and setting
a Windows Event
def SvcStop(self):
# Before we do anything, tell the
# SCM we are starting the stop process.
self.ReportServiceStatus( \
win32service.SERVICE_STOP_PENDING)
# And set my event.
win32event.SetEvent(self.hWaitStop)
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 37
Our Sample in Detail (4 of 8)

Overlapped IO means we can wait for
either a connection, or our Stop request
win32pipe.ConnectNamedPipe(pipeHandle,\
self.overlapped)
...
# Wait for either a connection, or
# a service stop request.
waitHandles = self.hWaitStop, \
self.overlapped.hEvent
rc = win32event.WaitForMultipleObjects(\
waitHandles, 0, timeout)
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 38
Our Sample in Detail (5 of 8)

Install our service simply by executing the
service script
– If not for Performance Monitor, the command-line
would be simple:
C:\Scripts> PipeService2.py install
– But PerfMon needs an extra arg
C:\Scripts> PipeService2.py \
--perfmonini=PipeService2_install.ini install
Installing service PythonPipeService to ...
Service installed
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 39
Our Sample in Detail (6 of 8)

Once installed, we can start the service
– Start from Control Panel, or using the script itself
C:\Scripts> python.exe PipeService2.py start

And start a client test session
C:\Scripts> python.exe PipeServiceClient.py \
Hi there
The service sent back:
You sent me:Hi there
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 40
Our Sample in Detail (7 of 8)

We will have Event
Log records...
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 41
Our Sample in Detail (8 of 8)

And Performance
Monitor Data.
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 42
Summary
Python library flexible, rich and portable
enough for many tasks
 Low-level Windows programming
achieved by using extensions that expose
the raw Windows API
 This talk should have shown:

– An overview of how this programming is done in
Python.
– How simple it is to do complex Windows tasks.
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 43
More Information

Python and Python Documentation
– http://www.python.org
– http://www.python.org/doc

Python for Windows Extensions
– http://starship.python.net/crew/mhammond
– http://www.python.org/windows
– Reference Manuals and Samples included with
the distributions
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 44
Thanks for coming

Mark Hammond
– [email protected]
– http://starship.python.net/crew/mhammond
O’Reilly Python Conference
Aug 24, 1999
Windows System Programming using Python
Slide 45