* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Windows
Survey
Document related concepts
Library (computing) wikipedia , lookup
Windows Mobile wikipedia , lookup
Microsoft Windows wikipedia , lookup
Windows Phone wikipedia , lookup
Criticism of Windows Vista wikipedia , lookup
Burroughs MCP wikipedia , lookup
Spring (operating system) wikipedia , lookup
Unix security wikipedia , lookup
Windows Phone 8.1 wikipedia , lookup
Transcript
Windows OS Principles Instructor: Hengming Zou, Ph.D. In Pursuit of Absolute Simplicity 求于至简,归于永恒 Content Outline 2.1 Structuring of the Windows Operating System 2.2 Windows Core System Mechanisms 2.3 Windows on Windows - OS Personalities 2.4 The Windows API - Naming, Conventions, Types 2 2.1 Structuring of Windows OS Architecture Overview Program Execution Environment Kernel Mode Architecture System Threads System Processes / Services 3 Simplified OS Architecture System support processes User Mode Kernel Mode Service processes User applications Environment subsystems Subsystem DLLs Executive Kernel Device drivers Windowing and graphics Hardware Abstraction Layer (HAL) 4 OS Architecture Multiple personality OS design – user applications don't call the native Windows operating system services directly Subsystem DLLs is to translate a documented function into the appropriate internal (and undocumented) Windows system service calls 5 OS Architecture Environment subsystem processes – Manage client processes in their world – Impose semantics such as process model, security Windows 2000 shipped with three environment subsystems: Win32, POSIX, and OS/2 – XP has only Windows subsystem – Win32 only one that matters – DOS, Win 3.1 emulated by a Win32 program – Enhanced POSIX subsystem available (Interix) 6 OS Architecture 7 Kernel-Mode Components: Core OS Core OS contains two components: – Executive and Kernel – both contained in file Ntoskrnl.exe Executive – base operating system services, – memory management, process and thread management, – security, I/O, inter-process communication 8 Kernel-Mode Components: Core OS Kernel – low-level operating system functions, – thread scheduling, interrupt and exception dispatching, – multiprocessor synchronization. Kernel provides a set of routines & basic objects – that the rest of the executive uses to implement higherlevel constructs 9 Kernel-Mode Components: Drivers Device drivers – hardware device drivers translate user I/O function calls into specific hardware device I/O requests – virtual devices - system volumes and network protocols – Contained in files ending in .Sys 10 Kernel-Mode Components: Drivers Windowing and Graphics System – graphical user interface (GUI) functions (Win32 USER and GDI), – dealing with windows, user interface controls, and drawing – [Contained in file Win32k.sys Hardware Abstraction Layer (HAL) – isolates kernel, device drivers, and executive from hardware – Hides platform-specific hardware differences (motherboards) – Contained in file Hal.dll 11 Background System Processes Core system processes, – logon process, the session manager, etc. – not started by the service control manager Service processes – host Win32 services – i.e.; Task Scheduler and Spooler services – Many Windows server apps, such as SQL Server and Exchange Server, also include components that run as services 12 Portability When NT was designed, there was no dominant processor architecture – Therefore it was designed to be portable How achieved? 13 Portability Most NT OS code and device drivers is written in C – HAL and kernel contain some assembly language Some components are written in C++: – Win32 windowing/graphics subsystem driver – Win2K volume manager Hardware-specific code is isolated in low level layers of the OS (such as Kernel and the HAL) – Provides portable interface 14 Portability NT 4.0 had support for: – x86, MIPS, PowerPC, Digital Alpha AXP PowerPC and MIPS dropped soon after NT 4 release Alpha AXP dropped in 1999 (supported through SP6) 15 Reentrant and Asynchronous Operation Windows kernel is fully reentrant – Kernel functions can be invoked by multiple threads simultaneously – No serialization of user threads when performing sys calls I/O system works fully asynchronously – Asynchronous I/O improves application’s throughput – Synchronous wrapper functions provide ease-of-programming 16 Key Windows System Files Core OS components: – NTOSKRNL.EXE**: Executive and kernel – HAL.DLL: Hardware abstraction layer – NTDLL.DLL: Internal support functions and system service dispatch stubs to executive functions Core system processes: – SMSS.EXE: Session manager process – WINLOGON.EXE: Logon process – SERVICES.EXE: Service controller process – LSASS.EXE : Local Security Authority Subsystem 17 Key Windows System Files Windowing subsystem: – CSRSS.EXE*: Windows subsystem process – WIN32K.SYS: in32 and GDI kernel-mode components – KERNEL32/USER32/GDI32.DLL: Win32 subsystem DLLs 18 Key System Components Environment Subsystems System & Service Processes User Application Subsystem DLL User Mode Kernel Mode OS/2 Windows Windows Executive Device Drivers Hardware Abstraction Layer (HAL) POSIX Kernel Windows User/GDI Device Driver 19 Multiple OS Personalities NT was designed to support multiple “personalities”, called environment subsystems – Programming interface – File system syntax – Process semantics 20 Multiple OS Personalities Environment subsystems provide exposed, documented interface between application and Windows native API Each subsystem defines a different set of APIs and semantics Subsystems implement these by invoking native APIs Example: – Windows CreateFile in Kernel32.Dll calls native NtCreateFile 21 Multiple OS Personalities .exes and .dlls you write are associated with a subsystem – Specified by LINK /SUBSYSTEM option – Cannot mix calls between subsystems 22 Environment Subsystems 3 environment subsystems originally provided with NT: Windows –Windows API – (originally 32-bit, now also 64-bit) OS/2 - 1.x character-mode applications only – Removed in Windows 2000 Posix - only Posix 1003.1 – bare minimum Unix services -no networking, windowing, threads – Removed in XP/Server 2003 – enhanced version ships with Services For Unix 3.0 23 Environment Subsystems Of the three, Windows provides access to the majority of NT native functions Of the three, Windows is required to be running – System crashes if Windows subsystem process exits – POSIX and OS/2 subsystems are actually Windows applications – POSIX & OS/2 start on demand (first time an app is run) Stay running until system shutdown 24 App calls Subsystem Function is entirely implemented in user mode – No message sent to environment subsystem process – No Win NT executive system service called – Examples: PtInRect(), IsRectEmpty() Function requires one/more calls to NT executive – Examples: Windows ReadFile()/WriteFile() implemented using I/O system services NtReadFile()/NtWriteFile() 25 App calls Subsystem Function requires some work in environment subsystem process (maintain state of client app) – Client/server request (message) to env. Subsystem (LPC facility) – Subsystem DLL waits for reply before returning to caller Combinations of 2/3: CreateProcess() / CreateThread() 26 Windows Subsystem Environment subsystem process (CSRSS.EXE): – Console (text) windows – Creating and deleting processes and threads – Portions of the support for 16-bit virtual DOS machine (VDM) – Other func: GetTempFile, DefineDosDevice, ExitWindowsEx kernel-mode device driver (WIN32K.SYS): – Window manager: manages screen output; – input from keyboard, mouse, and other devices – user messages to applications. – Graphical Device Interface (GDI) 27 Windows Subsystem (contd.) Subsystem DLLs (such as USER32.DLL, ADVAPI32.DLL, GDI32.DLL, and KERNEL32.DLL) – Translate Windows API functions into calls to NTOSKRNL.EXE and WIN32K.SYS. Graphics device drivers – graphics display drivers, printer drivers, video miniport drivers Prior to NT 4.0, window manager and graphics services were part of the user-mode Win32 subsystem process Is NT Less Stable with Win32 USER & GDI in Kernel Mode? 28 32-bit x86 Address Space Default 2 GB User process space 2 GB System Space 3 GB user space 3 GB User process space 1 GB System Space 29 Kernel vs User-Mode: QuickSlice (qslice.exe) Fastest way to find CPU hogs Red=Kernel, Blue=User mode Double-click on a process to see a per-thread display for that process Sum of threads’ bars for a process represents all of the process’s time, not all CPU time Screen snapshot from: Resource Kit | QuckSlice 30 Task Manager: Processes vs Applications Tabs Processes tab: List of processes Applications tab: List of top level visible windows “Running” means waiting for window messages Right-click on a window and select “Go to process” 31 Windows NT/2000/XP/2003 Architecture System Processes Services Environment Subsystems Applications Service Control Mgr. Task Manager WinMgt.Exe WinLogon User Mode Windows SvcHost.Exe LSASS Explorer SpoolSv.Exe Session Manager OS/2 User Application Services.Exe POSIX Subsystem DLLs Windows DLLs NTDLL.DLL System Threads Kernel Mode System Service Dispatcher Windows USER, GDI (kernel mode callable interfaces) I/O Mgr Local Procedure Call Configuration Mgr (registry) Processes & Threads Virtual Memory Security Reference Monitor Power Mgr. Plug and Play Mgr. Object Mgr. File System Cache Device & File Sys. Drivers Graphics Drivers Kernel Hardware Abstraction Layer (HAL) hardware interfaces (buses, I/O devices, interrupts, interval timers, DMA, memory cache control, etc., etc.) 32 Microkernel OS? Is NT/2000/XP/Server 2003 a microkernel-based OS? No – not using the academic definition – OS components and drivers run in their own private address spaces, layered on a primitive microkernel All kernel components live in a shared address space – Therefore no protection between OS and drivers 33 Microkernel OS? Why not pure microkernel? Performance – separate address spaces would mean context switching to call basic OS services Most other commercial OS (Unix, Linux, VMS etc.) have the same design 34 Microkernel OS? But it does have some attributes of a microkernel OS OS personalities running in user space as separate processes Kernel-mode components don't reach into one another’s data structures – Use formal interfaces to pass parameters and access and/or modify data structures Therefore the term “modified microkernel” 35 Symmetric Multiprocessing (SMP) No master processor – All the processors share just one memory space – Interrupts can be serviced on any processor – Any processor can cause another processor to reschedule what it’s running Maximum # of CPUs stored in registry – HKLM\System\CurrentControlSet\Control\Session Manager\LicensedProcessors 36 Symmetric Multiprocessing (SMP) Current implementation limit is # of bits in a native word – 32 processors for 32-bit systems – 64 processors for 64-bit systems – Not an architectural limit CPUs – just implementation L2 Cache Memory I/O SMP 37 Hyperthreading New technology in newer Xeon & Pentium 4 processors – Makes a single processor appear as a dual processor to OS – Also called simultaneous multithreading technology (SMT) Chip maintains two separate CPU states – “logical processors” – Execution engine & onboard cache is shared 38 Hyperthreading Works with Windows 2000, but only XP & Server 2003 are “hyperthreading aware” Logical processors don’t count against physical processor limits Scheduling algorithms take into account logical vs physical processors – Applications can also optimize for it (new Windows function in Server 2003) 39 Hyperthreading Four logical processors 1 2 3 4 Two physical processors 40 NUMA NUMA (non uniform memory architecture) systems Groups of physical processors with local memory – Connected to the larger system through a cache-coherent interconnect bus Still an SMP system – e.g. any processor can access all of memory – But node-local memory is faster 41 NUMA Scheduling algorithms take this into account – Tries to schedule threads on processors within the same node – Tries to allocate memory from local memory for processes with threads on the node New Windows APIs to allow applications to optimize 42 SMP Scalability Scalability is a function of: – parallelization and resource contention Can’t make a general statement Depends on what you are doing and – if the code involved scales well 43 SMP Scalability Kernel is scalable OS can run on any available processor and on multiple processors at the same time Fine-grained synchronization within the kernel as well as within device drivers allows more components to run concurrently on multiple processors Concurrency has improved with every release 44 SMP Scalability Applications can be scalable Threads can be scheduled on any available CPU Processes can contain multiple threads that can execute simultaneously on multiple processors Programming mechanisms provided to facilitate scalable server applications – Most important is I/O completion ports 45 Multiple Product Packages… Windows XP Home Edition – Licensed for 1 CPU, 4GB RAM Windows 2000 & XP Professional – Desktop version (but also is a fully functional server system) – Licensed for 2 CPUs, 4GB RAM (16GB on Itanium; 32GB on x64) Windows Server 2003, Web Server – Reduced functionality Standard Server (no domain controller) – Licensed for 2 CPUs, 2GB RAM 46 Multiple Product Packages… Windows Server 2003, Standard Edition (formerly Windows 2000 Server) – Adds server and networking features (active directory-based domains, host-based mirroring and RAID 5, NetWare gateway, DHCP server, WINS, DNS, …) – Licensed for 4 CPUs, 4GB RAM (32GB on x64) 47 Multiple Product Packages… Windows Server 2003, Enterprise Edition (formerly Windows 2000 Advanced Server ) – 3GB per-process address space option, Clusters (8 nodes) – Licensed for 8 CPUs, 32GB RAM (64GB on 64-bit editions) Windows 2000 Datacenter Server & Windows 2003 Server, Datacenter Edition – Process Control Manager – Licensed for 32 CPUs, 64GB RAM (64 CPUs & 512GB on Itanium—in the future, 1TB!) 48 …Built On the Same Core OS Through W2K, core OS executables were identical – NTOSKRNL.EXE, HAL.DLL, xxxDRIVER.SYS, etc. – but XP & Server 2003 have different kernel versions Registry indicates system type (set at install time) – HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\ProductOpt ions – ProductType: WinNT=Workstation, ServerNT=Server not a domain controller, LanManNT=Server that is a Domain Controller – ProductSuite: indicates type of Server (Advanced, Datacenter, or for NT4: Enterprise Edition, Terminal Server, …) 49 …Built On the Same Core OS Code in the operating system tests these values and behaves slightly differently in a few places – Licensing limits (number of processors, number of network connections, etc.) – Boot-time calculations (mostly in the memory manager) – Default length of time slice – See DDK: MmIsThisAnNtasSystem 50 NTOSKRNL.EXE Core operating system image Contains Executive and Kernel Also includes entry points for routines actually implemented in Hal.Dll Many functions are exposed to user mode via NtDll.Dll and the environment subsystems – To be discussed 51 NTOSKRNL.EXE Four retail variations: NTOSKRNL.EXE Uniprocessor NTKRNLMP.EXE Multiprocessor W2K adds PAE (page address extension) versions – must boot /PAE (32-bit Windows only) NTKRNLPA.EXE Uniprocessor w/extended addressing support NTKRPAMP.EXE Multiprocessor w/extended addressing support 52 NTOSKRNL.EXE Two checked build (debug) variations: NTOSKRNL.EXE, NTKRNLMP.EXE Debug multiprocessor NTKRNLPA.EXE, NTKRPAMP.EXE Debug multiprocessor w/extended addressing 53 UP vs MP File Differences These files are updated when moving from UP to MP Everything else is the same (drivers, EXEs, DLLs) NT4: Win32k.sys, Ntdll.dll, and Kernel32.dll had uniprocessor versions Name of file on Name of uniprocessor on system disk version on CD-ROM Name of version on CD-ROM NTOSKRNL.EX \I386\NTOSKRNL.EXE \I386\NTKRNLMP.EXE NTKRNLPA.EX \I386\NTKRNLMP.EXE \I386\NTKRPAMP.EXE HAL.DLL Depends on system Depends on system 54 Debug Version (“Checked Build”) Special debug version called “Checked Build” Multiprocessor versions only (runs on UP systems) – helps catch synchronization bugs that are more visible on MP Primarily for driver testing, but can be useful for catching timing bugs in multithreaded applications 55 Debug Version (“Checked Build”) Built from same source files as “free build” – aka “retail build” – But with “DBG” compile-time symbol defined This enables: – error tests for “can’t happen” conditions in kernel mode (ASSERTs) – validity checks on arguments passed from one kernel mode routine to another 56 Debug Version (“Checked Build”) #ifdef DBG if (something that should never happen has happened) KeBugCheckEx(…) #endif 57 Debug Version (“Checked Build”) Since no checked Windows 2000 Server provided, can copy checked NTOSKRNL, HAL, to a normal Server system – Select debug kernel & HAL with Boot.ini /KERNEL=, /HAL= switches Windows Server 2003 has its own checked build See Knowledge base article 314743 HOWTO: – Enable Verbose Debug Tracing in Various Drivers & Subsystems 58 Executive Upper layer of the operating system Provides “generic OS” functions (“services”) – Process Manager, Object Manager, Cache Manager – LPC (local procedure call) Facility – Configuration Manager, Memory Manager – Security Reference Monitor – I/O Manager, Power Manager – Plug-and-Play Manager 59 Executive Almost completely portable C code Runs in kernel (“privileged”, ring 0) mode Most interfaces to executive services not documented 60 Kernel Lower layers of the operating system Implements processor-dependent functions – x86 vs. Itanium etc. implements many processor-independent functions that – are closely associated with processor-dependent functions 61 Kernel Main services – Thread waiting, scheduling & context switching – Exception and interrupt dispatching – Operating system synchronization primitives (different for MP vs. UP) – A few of these are exposed to user mode Not a classic “microkernel” – shares address space with rest of kernel-mode components 62 HAL-Hardware Abstraction Layer Responsible for a small part of hardware abstraction – Components on the motherboard not handled by drivers System timers, Cache coherency, and flushing SMP support, Hardware interrupt priorities Subroutine library for the kernel & device drivers – Isolates Kernel and Executive from platform-specific details – Presents uniform model of I/O hardware interface to drivers Sample HAL routines: HalGetInterruptVector HalGetAdapter WRITE_PORT_UCHAR 63 HAL-Hardware Abstraction Layer Reduced role as of Windows 2000 – Bus support moved to bus drivers – Majority of HALs are vendor-independent HAL also implements some functions that appear to be in the Executive and Kernel 64 HAL - Hardware Abstraction Layer Selected at installation time – See \windows\repair\setup.log to find out which one – Can select manually at boot time with /HAL= in boot.ini HAL kit – Special kit only for vendors that must write custom HALs – requires approval from Microsoft – see http://www.microsoft.com/whdc/ddk/HALkit/default.mspx 65 Kernel-Mode Device Drivers Separate loadable modules (drivername.SYS) – Linked like .EXEs – Typically linked against NTOSKRNL.EXE and HAL.DLL – Only one version of each driver binary for both uniprocessor (UP) and multiprocessor (MP) systems… – … but drivers call routines in the kernel that behave differently for UP vs. MP Versions Defined in registry – Same area as Windows services (t.b.d.) - differentiated by Type value 66 Kernel-Mode Device Drivers Several types: – “ordinary”, file system, NDIS miniport, SCSI miniport (linked against port drivers), bus drivers – More information in I/O subsystem section To view loaded drivers, run drivers.exe – Also see list at end of output from pstat.exe – includes addresses of each driver To update & control: – System properties->Hardware Tab->Device Manager – Computer Management->Software Environment->Drivers 67 System Threads Functions in OS and some drivers that need to run as real threads – E.g., need to run concurrently with other system activity, wait on timers, perform background “housekeeping” work – Always run in kernel mode – Not non-preemptible (unless they raise IRQL to 2 or above) – For details, see DDK documentation on PsCreateSystemThread What process do they appear in? – “System” process (NT4: PID 2, W2K: PID 8, XP: PID 4) – In Windows 2000 & later, windowing system threads (from Win32k.sys) appear in “csrss.exe” (Windows subsystem process) 68 Examples of System Threads Memory Manager – Modified Page Writer for mapped files – Modified Page Writer for paging files – Balance Set Manager – Swapper (kernel stack, working sets) – Zero page thread (thread 0, priority 0) Security Reference Monitor – Command Server Thread 69 Examples of System Threads Network – Redirector and Server Worker Threads Threads created by drivers for their exclusive use – Examples: Floppy driver, parallel port driver Pool of Executive Worker Threads – Used by drivers, file systems, … – Work queued using ExQueueWorkItem – System thread (ExpWorkerThreadBalanceManager) manages pool 70 Identifying System Threads: Process Explorer With Process Explorer: – Double click on System process – Go to Threads tab – sort by CPU time As explained before, threads run between clock ticks (or at high IRQL) and thus don’t appear to run – Sort by context switch delta column 71 Identifying System Threads: Process Explorer 72 Process-Based Code OS components that run in separate executables (.exe’s), in their own processes – Started by system – Not tied to a user logon Three types: – Environment Subsystems (already described) – System startup processes “system startup processes” is not an official MS-defined name – Windows Services 73 Process-Based Code Let’s examine the system process “tree” – Use Tlist /T or Process Explorer 74 Process-Based NT Code: System Startup Processes First two processes aren’t real processes not running a user mode .EXE no user-mode address space different utilities report them with different names data structures for these processes (and their initial threads) are “pre-created” in NtosKrnl.Exe – and loaded along with the code 75 Process-Based NT Code: Startup Processes (Idle) Process id 0 – Part of the loaded system image – Home for idle thread(s) (not a real process nor real threads) – Called “System Process” in many displays (System) Process id 2 (8 in Windows 2000; 4 in XP) – Part of the loaded system image – Home for kernel-defined threads (not a real process) – Thread 0 (routine name Phase1Initialization) launches the first “real” process, running smss.exe...and then becomes the zero page thread 76 Process-Based NT Code: Startup Processes smss.exe Session Manager The first “created” process Takes parameters from \HKEY_LOCAL_MACHINE\System\CurrentControlSet \Control\Session Manager Launches required subsystems (csrss) and then winlogon csrss.exe Windows subsystem winlogon.exe Logon process: Launches services.exe & lsass.exe; presents first login prompt When someone logs in, launches apps in \Software\Microsoft\Windows NT\WinLogon\Userinit services.exe Service Controller; also, home for many NT-supplied services Starts processes for services not part of services.exe (driven by \Registry\Machine\System\CurrentControlSet\Services ) lsass.exe Local Security Authentication Server userinit.exe Started after logon; starts Explorer.exe (see \Software\Microsoft\Windows NT\CurrentVersion\WinLogon\Shell) and exits (hence Explorer appears to be an orphan) explorer.exe and its children are the creators of all interactive apps 77 Where are Services Defined? Defined in the registry: – HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services – one key per installed service Mandatory information kept on each service: – Type of service (Windows, Driver, ...) – Image name of service .EXE Note: some .EXEs contain more than one service – Start type (automatic, manual, or disabled) 78 Where are Services Defined? Optional information: – Display Name – New in W2K: Description – Dependencies – Account & password to run under Can store app-specific configuration parameters – “Parameters” subkey under service key 79 Life of a Service Install time – Setup app tells Service Controller about service Setup Application Registry CreateService System boot/initialization – SCM reads registry, starts services as directed Management/maintenance Service Controller/ Manager (Services.Exe) – Control panel can start/stop services & change startup parameters Service Processes Control Panel 80 Process Explorer: Service Information Process Explorer identifies Service Processes – Click on Options->Highlight Services 81 Service Processes A process created & managed by the Service Control Manager (Services.exe) – Similar in concept to Unix daemon processes – Typically configured to start at boot time (if started while logged on, survive logoff) – Typically do not interact with the desktop Note: Prior to Windows 2000 this was the only way to start a process on a remote machine – Now you can do it with WMI 82 Mapping Services to Service Processes Tlist /S (Debugging Tools) or Tasklist /svc (XP/2003) list internal name of services inside service processes Process Explorer shows more: external display name and description 83 Mapping Services to Service Processes 84 Service Control Tools Net start/stop – local system only Sc.exe (built in to XP/2003; also in W2K Resource Kit) – Command line interface to all service control/configuration functions – Works on local or remote systems Psservice (Sysinternals) – similar to SC 85 Service Control Tools Other tools in Resource Kit – Instsrv.exe – install/remove services (command line) – Srvinstw.exe – install/remove services (GUI) Why are service creation tools included in Reskit? – Because Reskit comes with several services that are not installed as services when you install the Reskit 86 Services Infrastructure Windows 2000 introduced generic Svchost.exe – Groups services into fewer processes Improves system startup time Conserves system virtual memory – Not user-configurable as to which services go in which processes – 3rd parties cannot add services to Svchost.exe processes 87 Services Infrastructure XP/2003 have more Svchost processes due to two new less privileged accounts for built-in services – LOCAL SERVICE, NETWORK SERVICE – Less rights than SYSTEM account Reduces possibility of damage if system compromised On XP/2003, four Svchost processes (at least): – SYSTEM – SYSTEM (2nd instance – for RPC) – LOCAL SERVICE – NETWORK SERVICE 88 2.2 Windows Core System Mechanisms Object Manager & Handles Local Procedure Calls Exception Handling Memory Pools 89 Objects and Handles Many Windows APIs take arguments that are handles to system-defined data structures, or “objects” App calls CreateXxx, which creates an object and returns a handle to it App then uses the handle value in API calls that operate on that object 90 Three Types of Windows Objects/Handles Windows “kernel objects” – events, mutexes, files, processes, threads, etc. – Objects are managed by the NT “Object Manager”, – and represent data structures in system address space – Handle values are private to each process 91 Three Types of Windows Objects/Handles Windows “GDI objects” – pens, brushes, fonts, etc. – Objects are managed by the Windows subsystem – Handle values are valid system-wide / session-wide Windows “User objects” (windows, menus, etc.) – Objects are managed by the Windows subsystem – Handle values are valid system-wide / session-wide 92 Handles and Security Process handle table – Unique for each process – In system address space (hence, is trusted) – Hence cannot be modified from user mode Security checks are made when handle table entry is created – i.e. at CreateXxx time – Handle table entry indicates the “validated” access rights to the object Read, Write, Delete, Terminate, etc. 93 Handles and Security APIs that take an “already-opened” handle look in the handle table entry before performing the function For example: – TerminateProcess checks to see if the handle was opened for Terminate access No need to check file ACL, process or thread access token, etc., on every write request – checking is done at file handle creation, i.e. “file open”, time 94 Handles, Pointers, and Objects Process A handles System Space Event Object Handle Table index HandleCount = 1 ReferenceCount = 1 Handle to a kernel object is an index into the process handle table, and is invalid in any other process Handle table entry contains the Process B Handle Table system-space address (8xxxxxxx or above) of the data structure; this address is the same regardless of process context Although handle table is per-process, it is actually in system address space (hence protected) 95 Handles, Pointers, and Reference Count Process A handles Handle Table index System Space Event Object HandleCount = 3 1 2 ReferenceCount = 4 0 DuplicateHandle Process B Thread (in a wait state for the event) Handle Table Note: there is actually another data structure, a “wait block”, “between” the thread and the object it’s waiting for 96 Object Manager Executive component For managing system-defined “objects” Objects are data structures with optional names Objects managed here include – Windows Kernel objects, but not Windows User or GDI objects Object manager implements user-mode handles and the process handle table 97 Object Manager OM is not used for all NT data structures Generally, only those types that need to be shared, named, or exported to user mode Some data structures are called “objects” but are not managed by the object manager – e.g. “DPC objects” In part, a heap manager… – Allocates memory for data structure from system-wide, kernel space heaps (pageable or nonpageable) 98 Object Manager … with a few extra functions: – Assigns name to data structure (optional) – Allows lookup by name – Objects can be protected by ACL-based security – Provides uniform naming, sharing, and protection scheme Simplifies C2 security certification by centralizing all object protection in one place – Maintains counts of handles and references (stored pointers in kernel space) to each object Object cannot be freed back to the heap until all handles and references are gone 99 Executive Objects Object type Represents Object directory Container object for other objects: implement hierarchical namespace to store other object types Symbolic link Mechanism for referring to an object name indirectly Process Virtual address space and control information necessary for execution of thread objects Thread Executable entity within a process Section Region of shared memory (file mapping object in Windows API) File Instance of an opened file or I/O device Port Mechanism to pass messages between processes Access token Security profile (security ID, user rights) of a process or process or thread 100 Executive Objects (contd.) Object type Represents Event Object with persistent state (signaled or not) usable for for synchronization or notification Semaphore Counter and resource gate for critical section Mutant Synchronization construct to serialize resource access Timer Mechanism to notify a thread when a fixed period of time time elapses Queue Method for threads to enqueue/dequeue notifications of I/O completions (Windows I/O completion port) Key Reference to registry data – visible in object manager namespace Profile Mechanism for measuring execution time for a process within an address range 101 Object Structure Object header Object body Object name Object name Object directory Object name Object directory Security descriptor Object directory Security descriptor Quota chargesdescriptor Security Quota charges OpenQuota handle count charges Open handle count OpenOpen handles list handle count Opentype handles list Object Opentype handles list Object Reference Objectcount type Reference count Reference count Object-specific data Object-specific data Object-specific data Type object contains static, object-type specific data: - shared among all instances of an object type - link all instances together (enumeration) Process 1 Process 2 Process 3 Type object Type name Access types Synchronizable? (Y/N) Pageable? (Y/N) Methods: open, close, delete parse, security, query name 102 Object Methods Method When method is called Open When an object handle is opened Close When an object handle is closed Delete Before the object manager deletes an object Query name When a thread requests the name of an object, such as a file, that that exists in a secondary object domain Parse When the object manager is searching for an object name that exists in a secondary object domain Security When a process reads/changes protection of an objects, such as a a file, that exists in a secondary object domain Process opens handle to object \Device\Floppy0\docs\resume.doc Object manager traverses name tree until it reaches Floppy0 Calls parse method for object Floppy0 with arg \docs\resume.doc 103 Winobj from Platform SDK \mssdk\bin\winnt\winobj.exe (no shortcut exists by default) 104 Object Manager Namespace System and session-wide internal namespace View with Winobj from www.sysinternal s.com 105 Interesting Objects(visible in WinObj) in \ObjectTypes – objects that define types of objects in \BaseNamedObjects these will appear when Windows programs use CreateEvent, etc. – mutant (Windows mutex) – queue (Windows I/O completion port) – section (Windows file mapping object) – Event, Semaphore In \GLOBAL?? – DOS device name mappings for console session 106 Object Manager Namespace Hierarchical directory structure – based on file system model System-wide (not per-process) – With Terminal Services, Windows objects are per-session by default – Can override this with “global\” prefix on object names Volatile (not preserved across boots) – As of Server 2003, requires SeCreateGlobalPrivilege can be extended by secondary object managers – e.g. file system – Hook mechanism to call external parse routine (method) 107 Object Manager Namespace Supports case sensitive or case blind Supports symbolic links – used to implement drive letters, etc. Lookup done on object creation or access by name – Not on access by handle Not all objects managed by object manager are named – e.g. file objects are not named – un-named objects are not visible in WinObj 108 Kernel Memory Pools (System-Space Heaps) Nonpaged pool Has initial size and upper limit – can be grown dynamically, up to the max 32-bit upper limit: 256 MB on x86 (NT4: 128MB) – 64-bit: 128 GB Performance counter displays current total size – allocated + free Max size stored in kernel variable – MmMaximumNonPagedPoolInBytes 109 Kernel Memory Pools (System-Space Heaps) Paged pool 32-bit upper limit: – 650MB (Windows Server 2003), – 470MB (Windows 2000), – 192MB (Windows NT 4.0) 64-bit: 128 GB Max size stored in MmSizeOfPagedPoolInBytes Pool size performance counters display – current size, not max – To display maximums, use “!vm” kernel debugger command 110 Invoking Kernel-Mode Routines Code is run in kernel mode for one of three reasons: Requests from user mode – Via the system service dispatch mechanism – Kernel-mode code runs in the context of the requesting thread 111 Invoking Kernel-Mode Routines Interrupts from external devices – Interrupts (like all traps) are handled in kernel mode – NT-supplied interrupt dispatcher invokes the interrupt service routine – ISR runs in the context of the interrupted thread (so-called “arbitrary thread context”) – ISR often requests the execution of a “DPC routine”, which also runs in kernel mode 112 Invoking Kernel-Mode Routines Dedicated kernel-mode threads – Some threads in the system stay in kernel mode at all times (mostly in the “System” process) – Scheduled, preempted, etc., like any other threads 113 Accounting for Kernel-Mode Time Screen snapshot from: Programs | Administrative Tools | Performance Monitor click on “+” button, or select Edit | Add to chart... 114 Accounting for Kernel-Mode Time “Processor Time” = total busy time of processor – equal to elapsed real time - idle time “Processor Time” = “User Time” + “Privileged Time” “Privileged Time” = time spent in kernel mode “Privileged Time” includes: – Interrupt Time – DPC Time – other kernel-mode time (no separate counter for this) 115 Kernel Memory Pools (System-Space Heaps) NT provides two system memory pools: Nonpaged Pool and Paged Pool Used for systemwide persistent data – visible from any process context Nonpaged pool required for memory accessed from – DPC/dispatch IRQL or above – Page faults at DPC/dispatch IRQL or above cause a system crash 116 Kernel Memory Pools (System-Space Heaps) Pool sizes are a function of memory size & Server vs. Workstation – Can be overidden in Registry: HKLM\System\CurrentControlSet\Control\Session Manager \Memory Management – But are limited by implementation limits (next slide 117 Lookaside Lists Instead of frequently allocating and freeing pool… …Use lookaside lists instead Implements a driver private list of preallocated blocks of pool Routines automatically extend the lookaside list from the systemwide pool Avoids hitting systemwide pool spinlock for every allocation/release 118 Lookaside Lists Optimizes CPU/memory cache behavior (the same driver will use the same list items over and over) See: – ExInitialize[N]PagedLookasideList – ExAllocateFrom[N]PagedLookasideList – ExFreeTo[N]PagedLookasideList – ExDelete[N]PagedLookasideList 119 Increased System Memory Limits Key system memory limits raised in XP & Server 2003 W2K limit of 200 GB of mapped file data eliminated – Previously limited size of files that could be backed up Maximum System Page Table Entries (PTEs) increased – Can now describe 1.3 GB of system space (960 MB contiguous) – Windows 2000 limit was 660 MB (220 MB contiguous) – Increases number of users on Terminal Servers – Also means maximum device driver size is now 960MB (was 220) 120 Local Procedure Calls (LPCs) IPC – high-speed message passing Not available through Windows API – Windows OS internal Application scenarios: – RPCs on the same machine are implemented as LPCs – Some Windows APIs result in sending messages to Windows subsyst. proc. – WinLogon uses LPC to communicate with local security authentication server process (LSASS) – Security reference monitor uses LPC to communicate with LSASS 121 Local Procedure Calls (LPCs) LPC communication: – Short messages < 256 bytes are copied from sender to receiver – Larger messages are exchanged via shared memory segment – Server (kernel) may write directly in client‘s address space 122 Port Objects LPC exports port objects to maintain state of communication: – Server connection port: named port, server connection request point – Server communication port: unnamed port, one per active client, used for communication – Client communication port: unnamed port a particular client thread uses to communicate with a particular server – Unnamed communication port: unnamed port created for use by two threads in the same process 123 Port Objects Typical scenario: – Server creates named connection port – Client makes connection request – Two unnamed ports are created, client gets handle to server port, server gets handle to client port 124 Use of LPC ports Client address space Kernel address space Server address space Connection port Client process Message queue Server process Handle Handle Client view of section Client communication port Server communication port Handle Server view of section Shared section 125 Exception Dispatching Exceptions are conditions that result directly from the execution of the program that is running Windows introduced a facility known as structured exception handling, which allows applications to gain control when exceptions occur 126 Exception Dispatching The application can then fix the condition and return to the place the exception occurred, – unwind the stack (thus terminating execution of the subroutine that raised the exception), or – declare back to the system that the exception isn’t recognized and the system should continue searching for an exception handler that might process the exception. 127 Exception Dispatching (contd.) Structured exception handling; – Accessible from MS VC++ language: __try, __except, __finally – See Jeffrey Richter, „Advanced Windows“, MS Press – See Johnson M.Hart,Win32 System Programming, AddisonWesley LPC to exception Trap handler debugger port (Exception frame, client thread ID) Exception dispatcher) Unhandled exceptions are passed to next handler Exception dispatcher sends debug message to Debugger via LPC/excepion port & session manager process Debugger (first chance) Frame-based handlers Debugger (second chance) Environment subsystem Kernel default handler 128 Internal Windows API exception handler Processes unhandled exceptions – At top of stack, declared in StartOfProcess()/StartOfThread() void Win32StartOfProcess(LPTHREAD_START_ROUTINE lpStartAddr, LPVOID lpvThreadParm) { __try { DWORD dwThreadExitCode = lpStartAddr(lpvThreadParm); ExitThread(dwThreadExitCode); } __except(UnhandledExceptionFilter( GetExceptionInformation())) { ExitProcess(GetExceptionCode()); } } 129 2.3 Windows on Windows - OS Personalities Environment Subsystems System Service Dispatching Windows on Windows - 16bit Windows on Windows - 64bit 130 Multiple OS Personalities Environment Subsystems System & Service Processes User Application Subsystem DLL User Mode Kernel Mode OS/2 Windows Windows Executive Device Drivers Hardware Abstraction Layer (HAL) POSIX Kernel Windows User/GDI Device Driver 131 What about .NET and WinFX? WinFX is the .NET Framework that will ship with Longhorn Both .NET and WinFX are built on standard Windows APIs – They are not a subsystem – They do not call undocumented NT system calls .NET/WinFX Application .NET/WinFX Framework User Mode Windows API DLLs Kernel Mode Windows Kernel 132 Subsystem Information in Registry Subsystems configuration and startup information is in: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control \Session Manager\SubSystems Values: – Required - list of value names for subsystems to load at boot time – Optional - list of value names for subsystems to load when needed – Windows - value giving filespec of Windows subsystem (csrss.exe) csrss.exe Windows APIs required - always started when NT boots – Kmode - value giving filespec of Win32K.Sys (kernel-mode driver portion of Windows subsystem) – Posix - file name of POSIX subsystem psxss.exe Posix APIs optional - started when first Posix app is run 133 Subsystem Information in Registry Some Windows API DLLs are in “known DLLs” registry entry: HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\KnownDLLs – Files are opened as mapped files – Improves process creation/image startup time 134 Subsystem Components 1:API DLLs – for Windows: Kernel32.DLL, Gdi32.DLL, User32.DLL, etc. 2:Subsystem process – for Windows: CSRSS.EXE (Client Server Runtime SubSystem) 3:For Windows only: kernel-mode GDI code – Win32K.SYS - (this code was formerly part of CSRSS) 135 Subsystem Components Environment Subsystems 2 System & Service Processes User Application 1 Subsystem DLL User Mode POSIX OS/2 Windows 3 Kernel Mode Executive Device Drivers Kernel Windows User/GDI Hardware Abstraction Layer (HAL) 136 Role of Subsystem Components 1:API DLLs:Export the APIs defined by the subsystem – Implement them by calling NT “native” services, or by asking the subsystem process to do the work 3:Win32K.Sys – Implements Windows User & GDI functions; calls routines in GDI drivers – Also used by Posix and OS/2 subsystems to access the display 137 Role of Subsystem Components 2: Subsystem process: – Maintains global state of subsystem – Implements a few APIs that require subsystem-wide state changes Processes and threads created under a subsystem Drive letters Window management for apps with no window code of their own (character-mode apps) Handle and object tables for subsystem-specific objects 138 Simplified Architecture (3.51 and earlier) Environment Subsystems System & Service Processes User Mode Kernel Mode User Application Subsystem DLL NtDll.Dll 1 OS/2 Windows POSIX 2 KiSystemService Executive LPC Device Drivers Kernel Hardware Abstraction Layer (HAL) most Windows Kernel APIs all other Windows APIs, including User and GDI APIs 139 NT4/2000/XP/2003 Simplified Architecture Environment Subsystems System & Service Processes User Mode Kernel Mode User Application OS/2 Subsystem DLL NtDll.Dll 1 3 Windows POSIX 2 KiSystemService Executive LPC Device Drivers Kernel Windows User/GDI Hardware Abstraction Layer (HAL) 1: most Windows Kernel APIs 2: most Windows User and GDI APIs (these were formerly part of CSRSS) 3: a few Windows APIs 140 Role of CSRSS.EXE Windows Subsystem Process A few APIs are implemented in this separate process In 3.51 and earlier: – Nearly all User and GDI APIs were implemented in CSRSS – CSRSS had a thread for every app thread that created a window – GDI drivers (video, printer) were user mode, mapped into this process – This was done for protection, esp. to keep GDI drivers in user mode 141 Role of CSRSS.EXE CSRSS in NT 4.0: role is greatly diminished – Maintains system-wide state information for all Windows “client” processes – Several Windows services LPC to CSRSS for “setup and teardown” functions Process and thread creation and deletion Get temporary file name Drive letters – Security checks for file system redirector – Window management for console (character cell) applications … – … including NTVDM.EXE 142 Header of Executable File Specifies Subsystem Type Subsystem for each .exe specified in image header – see winnt.h IMAGE_SUBSYSTEM_UNKNOWN IMAGE_SUBSYSTEM_NATIVE – IMAGE_SUBSYSTEM_WINDOWS_GUI IMAGE_SUBSYSTEM_WINDOWS_CUI IMAGE_SUBSYSTEM_OS2_CUI IMAGE_SUBSYSTEM_POSIX_CUI 0 1 2 3 5 7 // // // // // // Unknown subsystem Image doesn't require a subsystem Windows subsystem (graphical app) Windows subsystem (character cell) OS/2 subsystem Posix subsystem 143 Header of Executable File Specifies Subsystem Type – see Explorer / QuickView (right-click on .exe or .dll file) – or exetype image.exe (Resource Kit) 144 Native Images .EXEs not linked against any subsystem – Interface to NT executive routines directly via NTDLL.DLL Two examples: – smss.exe (Session Manager -- starts before subsystems start) – csrss.exe (Windows subsystem) 145 Lab: Subsytems & Images Look at subsystem startup information in registry Using EXETYPE, look at subsystem types for: – \windows\system32\notepad.exe, – cmd.exe, csrss.exe 146 POSIX Subsystem Windows 2000 implements POSIX.1 – ISO/IEC 9945-1:1990 or IEEE POSIX standard 1003.1-1990 – POSIX.1 compliance as specified in Federal Information Processing Standard (FIPS) 151-2 (NIST) – POSIX Conformance Document in \HELP in Platform SDK support for impl.of POSIX.1 subsystem was mandatory for NT – fork service in NT executive – hard file links in NTFS 147 POSIX Subsystem limited set of services – such as process control, IPC, simple character cell I/O – POSIX subsystem alone is not a complete programming environment POSIX executable cannot – create a thread or a window – use remote procedure calls (RPCs) or sockets Microsoft supplies a full POSIX subsystem for Windows 2000 under the Interix product name 148 Porting UNIX Apps to NT UNIX-to-Win32 porting libraries – DataFocus (http://www.datafocus.com/) – ConsenSys (http://www.consensys.com/) – Cygnus - CygWin GNU tools (http://www.cygnus.com/) POSIX subsystem with complete UNIX system service and utilities environment – Interix from Microsoft (used to be SoftWay (http://www.opennt.com/) – Bought by Microsoft as of Sept. 99 149 Porting UNIX Apps to NT NT Resource Kit includes optional set of POSIX utilities POSIX executables are linked against PSXDLL.DLL – Header files in platform SDK 150 Watching the POSIX Subsystem Start POSIX subsystem starts on demand: 1. Type tlist /t, check that POSIX subsyst. is not running 2. Run \ntreskit\POSIX\LS.EXE 3. Run tlist /t again, PSXSS.EXE is child of SMSS.EXE 151 Invoking (a few) Windows Services Some Windows services are still implemented in the Windows subsystem process Application Process Subsystem DLL LPC is an undocumented system service in NtDll.Dll (not exposed directly in API) CSRSS.EXE (Windows subsystem process) U Executive K Local Procedure Call 152 System Call Dispatching NTDLL.DLL provides interface for NT native system calls 153 Example: Invoking a Windows Kernel API Windows application WriteFile in Kernel32.Dll NtWriteFile in NtDll.Dll call WriteFile(…) call NtWriteFile return to caller Int 2E or SYSCALL or SYSENTER return to caller software interrupt Windowsspecific used by all subsystems U K KiSystemService in NtosKrnl.Exe call NtWriteFile dismiss interrupt NtWriteFile in NtosKrnl.Exe do the operation return to caller 154 Invoking System Functions from User Mode Kernel-mode functions (“services”) are invoked from user mode via a protected mechanism – x86: INT 2E (as of XP, SYSENTER is used where available); AMD32 and x64: SYSCALL – i.e., on a call to an OS service from user mode, the last thing that happens in user mode is this “change mode to kernel” instruction – Causes an exception or interrupt, handled by the system service dispatcher (KiSystemService) in kernel mode – Return to user mode is done by dismissing the interrupt or exception 155 Invoking System Functions from User Mode The desired system function is selected by the “system service number” – Every NT function exported to user mode has a unique number – This number is stored in a register just before the “change mode” instruction (after pushing the arguments to the service) – This number is an index into the system service dispatch table – Table gives kernel-mode entry point address and argument list length for each exported function 156 Invoking System Functions from User Mode All validity checks are done after the user to kernel transition – KiSystemService probes argument list, copies it to kernel-mode stack, and calls the executive or kernel routine pointed to by the table – Service-specific routine checks argument values, probes pointed-to buffers, etc. – Once past that point, everything is “trusted” 157 Invoking System Functions from User Mode This is safe, because: – The system service table is in kernel-protected memory; and – The kernel mode routines pointed to by the system service table are in kernel-protected memory; therefore: – User mode code can’t supply the code to be run in kernel mode; it can only select from among a predefined list – Arguments are copied to the kernel mode stack before validation; therefore: – Other threads in the process can’t corrupt the arguments “out from under” the service 158 Example: Invoking a Posix API Posix application call write(…) write in psxdll.dll call NtWriteFile return to caller Posixspecific NtWriteFile in NtDll.Dll Int 2E return to caller used by all subsystems software interrupt U K KiSystemService in NtosKrnl.Exe call NtWriteFile dismiss interrupt NtWriteFile in NtosKrnl.Exe do the operation return to caller 159 Ntdll.dll Interface to NT system calls (285 calls starting with “Nt”) These user-mode routines have the same function names and arguments as the kernel mode routines they invoke – e.g. NtWriteFile in NtDll.Dll invokes NtWriteFile in NtosKrnl.Exe Majority are not supported or documented 160 Ntdll.dll 7 are (partially) documented in the Platform SDK: – NtQuerySystemInformation, NtQuerySystemTime, – NtQueryInformationProcess, NtQueryInformationThread, – NtCreateFile, NtOpenFile, NtWaitForSingleObject The DDK describes 25 of them as “Zw” routines – such as ZwReadFile – These entry points call the corresponding “Nt” interface via the system call interface Thus, “previous mode” is kernel mode (means no security checks) – Kernel mode code could also call NtReadFile directly 161 Ntdll.dll Other user-mode support routines – Image loader (“Ldr”) – Debug infrastructure (“Dbg”) – Csrss support routines (“Csr”) – RTL routines (“Rtl”) – Tracing routines (“Etw”) [new as of Windows Server 2003] 162 Calling a “Native” API from User Mode native application (smss.exe, csrss.exe) NtWriteFile in NtDll.Dll call NtWriteFile(…) Int 2E return to caller software interrupt used by all subsystems U K KiSystemService in NtosKrnl.Exe call NtWriteFile dismiss interrupt NtWriteFile in NtosKrnl.Exe do the operation return to caller 163 Invoking (most) User and GDI Services application Gdi32.Dll or User32.Dll call user or GDI service(…) Int 2E return to caller software interrupt KiSystemService in NtosKrnl.Exe service entry point in Win32K.Sys Windowsspecific U K call Windows routine dismiss interrupt do the operation return to caller 164 16-bit Applications on 32-bit Windows NT runs NTVDM.EXE MS-DOS application – NT Virtual Dos Machine 0 640K 16-bit MS-DOS emulation NTVDM is a Windows 1MB MS-DOS ext. memory image 32-bit MS-DOS emulation – No “DOS subsystem” or “Win16 subsystem” Virtual device drivers (COM, LPT, keyboard) Windows Subsystem DLLs 16-bit 16MB 32-bit 7FFFFFFF Environment Subsystems System & Service Processes User Mode Kernel Mode User Application Subsystem DLL OS/2 Windows Executive Device Drivers Kernel POSIX Windows User/GDI Hardware Abstraction Layer (HAL) 165 DOS 16-bit Applications e.g. command.com, edit.com (NT4 had qbasic.exe) NT runs NTVDM.EXE (NT Virtual DOS Machine) – See \System\CurrentControlSet\Control\WOW\cmdline MS-DOS application 0 640K 16-bit MS-DOS emulation 16-bit 1MB extended memory Example: three DOS apps running in three NTVDM processes 32-bit MS-DOS emulation 16MB Windows 32-bit Virtual device drivers COM, LPT, keyboard Windows Subsystem DLLs 7FFFFFFF 166 DOS 16-bit Applications Each DOS app has a separate process running NTVDM – DOS & Windows 16-bit drivers not supported Note: NT command prompt is not a “DOS box”, despite icon; – it’s a Windows console application (CMD.EXE) 167 Windows 16-Bit Applications e.g. sysedit.exe, winhelp.exe NT also runs NTVDM.EXE – See \CurrentControlSet\Control\WOW\wowcmdline NTVDM loads wowexec.exe – WOW = “Windows on Windows” – Win16 calls are translated to Win32 168 Windows 16-Bit Applications example: three Win16 apps (and wowexec.exe) running in one NTVDM process Win16 app Win16 app 0 16-bit Win16 app Windows 16MB Win16 API 32-bit Virtual Device Drivers Windows Subsystem DLLs 7FFFFFFF 169 Windows 16-bit Apps Multitasking Details Each Win16 app runs in a separate thread in the common NTVDM process They cooperatively multitask among themselves – Win16 Yield API... …and the one (if any) that wants to run, preemptively multitasks with all other threads on NT necessary to meet serialization assumptions of some Win16 apps 170 Windows 16-bit Apps Multitasking Details NT4 only: 171 Windows 16-bit Apps Multitasking Details Option to run Win16 apps in separate VDMs – Run in Separate Memory Space = run in separate process – default set by \CurrentControlSet\Control\WOW\DefaultSeparateVDM – Win16 apps run this way preemptively multitask with all other threads, including the un-Yield’ed thread in a shared Win16 NTVDM (if any) 172 Monitoring 16-bit Applications To most of NT, an NTVDM process is just another process Task Manager – tasks are simply the names of top-level windows Win16 windows included – processes display identifies Win16 apps within NTVDM processes by reading the NTVDM process’s private memory (undocumented interface) – does not identify the DOS apps within each NTVDM process 173 Monitoring 16-bit Applications TLIST (resource kit) – identify DOS apps within each NTVDM process (by window title) – for a shared Win16 NTVDM process, only shows one window title QuickView, exetype – identifies DOS, Win16, etc., application .exe’s 174 Lab: 16-bit Applications DOS applications: Run qbasic.exe and then edit.com look at process list in Task Manager Process tab – cannot differentiate which NTVDM.EXE is which From Applications tab, right click on window – goto process (now can map which NTVDM.EXE process is which) 175 Lab: 16-bit Applications Windows 3.1 applications: Run winhelp.exe twice – do not check “run in separate memory space” Run winhelp.exe once and – check “run in separate memory space” Bring up Task Manager Process tab – make sure “Show 16-bit Tasks” is checked on the View menu Look at Task Manager Process tab and see 16-bit applications identified inside the two NTVDMs 176 Wow64 Allows execution of Win32 binaries on 64-bit Windows – Wow64 intercepts system calls from the 32-bit application Converts 32-bit data structure into 64-bit aligned structures Issues the native 64-bit system call Returns any data from the 64-bit system call IsWow64Process() function can tell a 32-bit process if it is running under Wow64 177 Wow64 Performance – On x64, instructions executed by hardware – On IA64, instructions have to be emulated New Intel IA-32 EL (Execution Layer) does binary translation of Itanium to x86 to speed performance Downloadable now – bundled with Server 2003 SP1 178 Wow64 Components Wow64.dll: – provides core emulation infrastructure and thunks for Ntoskrnl.exe entry-point functions; exception dispatching Wow64win.dll: – provides thunks for Win32k.sys entry-point functions Wow64cpu.dll: – manages thread contexts, supports mode-switch instructions 179 Wow64 Components 32-bit EXE, DLLs Kernel Mode User Mode 32-bit ntdll.dll Wow64cpu.dll Wow64.dll Wow64win.dll 64-bit ntdll.dll Executive Win32k.sys 180 Wow64 Limitations Cannot load 32-bit DLLs in 64-bit process and vice versa Does not support 32-bit kernel mode device drivers – Drivers must be ported to 64-bits – Special support required to support 32-bit applications using DeviceIoControl to driver Driver must convert 32-bit structures to 64-bit 181 Wow64 Limitations Platforms IA64 x64 16-bit Virtual DOS Machine (VDM) support N/A N/A Physical Address Extension (PAE) APIs N/A Yes GetWriteWatch() API N/A Yes Scatter/Gather I/O APIs N/A Yes SoftwareEmulation Only Yes Wow64 Feature Support on 64-bit Windows Hardware accelerated with DirectX version 7,8 and 9 182 Wow64 File Locations Location of system files – 64-bit system files are in \windows\system32 – 32-bit system files are in \windows\syswow64 – 32-bit applications live in “\Program Files (x86)” – 64-bit applications live in “\Program Files” File access to %windir%\system32 redirected to %windir%\syswow64 Two areas of the registry redirected (see next slide) 183 Wow64 Registry Redirection Two registry keys have 32-bit sections: – HKEY_LOCAL_MACHINE\Softwar e – HKEY_CLASSES_ROOT – Everything else is shared 32-bit data lives under \Wow6432Node – When a Wow64 process opens/creates a key, it is redirected to be under Wow6432Node 184 Example: Cmd.exe on 64-bit System 32-bit Cmd.exe process: 185 Example: Cmd.exe on 64-bit System 64-bit Cmd.exe process: 186 Four Contexts for Executing Code Full process and thread context: User applications Windows Services Environment subsystem processes System startup processes Have thread context but no “real” process: Threads in “System” process 187 Four Contexts for Executing Code Routines called by other threads/processes: Subsystem DLLs Executive system services (NtReadFile, etc.) GDI32 and User32 APIs implemented in Win32K.Sys (and graphics drivers) No process or thread context (“arbitrary thread context”) Interrupt dispatching Device drivers 188 Windows NT/2000/XP/2003 Architecture System Processes Service Control Mgr. LSASS WinLogon User Mode Session Manager Services Environment Subsystems Applications Windows SvcHost.Exe WinMgt.Exe SpoolSv.Exe Task Manager Explorer OS/2 User Application Services.Exe Subsystem DLLs POSIX Windows DLLs NTDLL.DLL System Threads Kernel Mode System Service Dispatcher (kernel mode callable interfaces) Kernel Hardware Abstraction Layer (HAL) hardware interfaces (buses, I/O devices, interrupts, interval timers, DMA, memory cache control, etc., etc.) Local Procedure Call Processes & Threads Virtual Memory Security Reference Monitor Power Mgr. Plug and Play Mgr. Object Mgr. File System Cache Device & File Sys. Drivers Configuration Mgr (registry) I/O Mgr Windows USER, GDI Graphics Drivers Where is the Code? Kernel32.Dll, Gdi32.Dll, User32.Dll – export Windows entry points NtDll.Dll provides user-mode access to system-space routines also contains heap manager, image loader, thread startup routine NtosKrnl.Exe (or NtkrnlMp.Exe) executive and kernel includes most routines that run as threads in “system” process 190 Where is the Code? Win32K.Sys – the loadable module that includes the now-kernel-mode Windows code (formerly in csrss.exe) Hal.Dll – Hardware Abstraction Library drivername.Sys – loadable kernel drivers 191 The Windows API - Naming, Conventions, Types Windows API principles Portable programming - the standard C library Comparing UNIX and Windows programming styles: – an example program 192 Windows API - Overview APIs to Windows systems evolved over time: – Win16 - introduced with Windows 2.0 – Win32 - introduced with Windows NT, Windows 95 – Win64 (?) - some additions to Win32 introduced with Windows XP 64bit Edition “Windows API” summarizes all of the above 193 Windows API - major functionality File System and Character I/O Direct File Access and File Attributes Structured Exception Handling Memory Management and Memory-Mapped Files Security Process Management Inter-process Communication Threads and Scheduling, Windows Synchronization 194 Windows API Principles System resources are kernel objects referenced by a handle (handle vs. UNIX file descriptors & PIDs) Kernel objects must be manipulated via Windows API Objects – files, processes, threads, IPC pipes, memory mappings, events – have security attributes Windows API is rich & flexible: – convenience functions often combine common sequences of function calls Windows API offers numerous synchronization and communication mechanisms 195 Windows API principles (contd.) Thread is unit of executions (instead of UNIX process) – A process can contain one or more threads Function names are long and descriptive (as in VMS) - WaitForSingleObject() - WaitForMultipleObjects() 196 Windows API Naming Conventions Predefined data types are in uppercase – BOOL (32 bit object to store single logical value) – HANDLE – DWORD (32 bit unsigned integer) – LPTSTR – LPSECURITY_ATTRIBUTE 197 Windows API Naming Conventions Prefix to identify pointer & const pointer – LPTSTR (defined as TCHAR *) – LPCTSTR(defined as const TCHAR *) (Unicode: TCHAR may be 1-byte char or 2-byte wchar_t) – See \$MSDEV\INCLUDE\WINDOWS.H, WINNT.H, WINBASE.H (MSDEV=C:\Program Files\Microsoft Visual Studio\VC98\) 198 Differences from UNIX HANDLEs are opaque (no short integers) – No analogy to file descriptors 0,1,2 in Windows No distinctions between HANDLE and process ID – Most functions treat file, process, event, pipe identically Windows API processes have no parent-child relationship – Although the NT kernel maintains this relationship Windows text files have CR-LF instead of LF (UNIX) Anachronisms: “long pointer“ (32 bit) – LPSTR, LPVOID 199 Portability: The Standard C Library Included in the Windows API C library contains functions with limited capability to manage OS resources (e.g.; files) Often adequate for simple programs Possible to write portable programs Include files: – <stdlib.h>, <stdio.h>, <string.h> 200 Example Application Sequential file copy: – The simplest, most common, and most essential capability of any file system – Common form of sequential processing Comparing programs: – Quick way to introduce Windows API essentials – Contrast different approaches – Minimal error processing 201 Sequential File Copy UNIX: File descriptors are integers; error value: -1 read()/write() return number of bytes processed, – 0 indicates EOF – Positive return value indicates success close() works only for I/O objects I/O is synchronous Error processing depends on perror() & errno (global) 202 CP Program: UNIX Implementation #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> #include <stdio.h> #include <errno.h> #define BUF_SIZE 256 int main (int argc, char *argv []) { int input_fd, output_fd; ssize_t bytes_in, bytes_out; char rec [BUF_SIZE]; if (argc != 3) { printf ("Usage: cp file1 file2\n"); return 1; } input_fd = open (argv [1], O_RDONLY); if (input_fd == -1) { perror (argv [1]); return 2; } output_fd = open(argv[2],O_WRONLY|O_CREAT,0666); if (output_fd == -1) { perror (argv [2]); return 3; } /* Process the input file a record at atime. */ while ((bytes_in = read (input_fd, &rec, BUF_SIZE)) > 0) { bytes_out = write (output_fd, &rec, bytes_in); if (bytes_out != bytes_in) { perror ("Fatal write error."); return 4; } } close (input_fd); close (output_fd); return 0; } 203 File Copy with Standard C Library Open files identified by pointers to FILE structures – NULL indicates invalid value – Pointers are „handles“ to open file objects Call to fopen() specifies whether file is text or binary Errors are diagnosed with perror() of ferror() Portable between UNIX and Windows Competitive performance Still constrained to synchronous I/O No control of file security via C library 204 CP Program: C library Implementation #include <windows.h> #include <stdio.h> #include <errno.h> #define BUF_SIZE 256 int main (int argc, char *argv []) { FILE *in_file, *out_file; char rec [BUF_SIZE]; size_t bytes_in, bytes_out; if (argc != 3) { printf ("Usage: cp file1 file2\n"); return 1; } in_file = fopen (argv [1], "rb"); if (in_file == NULL) { perror (argv [1]); return 2; } out_file = fopen (argv [2], "wb"); if (out_file == NULL) { perror (argv [2]); return 3; } /* Process the input file a record at a time. */ while ((bytes_in = fread (rec,1,BUF_SIZE,in_file)) > 0) { bytes_out = fwrite (rec, 1, bytes_in, out_file); if (bytes_out != bytes_in) { perror ("Fatal write error."); return 4; } } fclose (in_file); fclose (out_file); return 0; } 205 File Copying with Windows API <windows.h> imports all Windows API function definitions and data types Access Windows objects via variables of type HANDLE Generic CloseHandle() function works for most objects Symbolic constants and flags – INVALID_HANDLE_VALUE, GENERIC_READ Functions return boolean values System error codes obtained via GetLastError() Windows security is complex and difficult to program 206 CP Program - Windows API Implementation hOut = CreateFile (argv [2], GENERIC_WRITE, 0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL); if (hOut == INVALID_HANDLE_VALUE) { printf("Output file error: %x\n", GetLastError ()); return 3; } while (ReadFile (hIn, Buffer, BUF_SIZE, &nIn, NULL) && nIn > 0) { WriteFile (hOut, Buffer,nIn,&nOut,NULL); if (nIn != nOut) { printf ("Fatal write error: %x\n", GetLastError ()); return 4; } } CloseHandle (hIn); CloseHandle (hOut); return 0; #include <windows.h> #include <stdio.h> #define BUF_SIZE 256 int main (int argc, LPTSTR argv []) { HANDLE hIn, hOut; DWORD nIn, nOut; CHAR Buffer [BUF_SIZE]; if (argc != 3) { printf("Usage: cp file1 file2\n"); return 1; } hIn = CreateFile (argv [1], GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL); if (hIn == INVALID_HANDLE_VALUE) { printf ("Input file error:%x\n", GetLastError ()); return 2; } } 207 CP with Windows API Convenience Functions Convenience functions may improve performance – Programmer does not need to be concerned about arbitrary buffer sizes – OS manages speed vs. space tradeoffs at runtime #include <windows.h> #include <stdio.h> int main (int argc, LPTSTR argv []) { if (argc != 3) { printf ("Usage: cp file1 file2\n"); return 1; } if (!CopyFile (argv [1], argv [2], FALSE)) { printf ("CopyFile Error: %x\n", GetLastError ()); return 2; } return 0; } 208 Thoughts Change Life 意念改变生活