Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
OpenEdge Release 10.1B and higher RDBMS Shared Memory February 13, 2008 By Keith Sanborn and Gus Bjorklund Overview A set of shared memory enhancements were introduced in OpenEdge Release 10.1B that enable increased shared memory addressability and better management of the shared memory allocation process. Increased shared memory addressability allows larger pools of shared memory for databases, which can reduce operating system resource requirements. The ability to customize shared memory allocation allows user specified shared memory segment sizing to meet the needs of specific environments. This paper describes the extent of the changes concentrating on 32-bit platforms, and describes how the changes will affect OpenEdge in your particular environment. OpenEdge Shared Memory Changes The shared-memory architecture for the OpenEdge RDBMS has changed in two important ways: First, the method used by the OpenEdge RDBMS to address shared memory is enhanced so that OpenEdge can now address larger amounts of allocated shared memory on both 32 and 64-bit operating systems. Second, you can now specify the maximum shared-memory segment size that the OpenEdge RDBMS will allocate. Previously, the maximum segment size was fixed at 128 megabytes for 32-bit systems, and 1 gigabyte for 64-bit systems. These enhancements enable the OpenEdge RDBMS to allocate large shared memory pools using the segment sizes you specify. However, each operating system imposes additional limitations that the OpenEdge RDBMS must follow. OpenEdge RDBMS Release 10 can now address larger amounts of allocated shared memory than previous releases. On 32-bit systems, the addressable limit is 4 gigabytes and on 64-bit systems, the addressable limit is 8 terabytes. These limits are architectural limits and are often reduced in practice by limitations imposed by the underlying operating system and other factors. This is particularly true on 32-bit systems because the entire process address space is limited to 4 gigabytes and you cannot use all of it for shared memory. In practice, most 32-bit OpenEdge releases are limited to a maximum of approximately 2 gigabytes of shared memory. In addition to providing for increased shared memory addressability, OpenEdge Release 10 also introduces the database configuration parameter, –shmsegsize, or Shared Memory Segment Size. This parameter allows the setting the maximum size of shared memory segments for each database at database startup time. If –shmsegsize is omitted during database startup, OpenEdge will size shared memory segments automatically based on the shared memory requirements of the database. No discussion of shared memory can begin without first considering some operating system principles, and one of the most important to understand is the concept of the Process Address Space. The 32-bit Process Address Space Many factors affect the total amount of shared memory that the OpenEdge RDBMS can allocate, including program size, stack space required, shared libraries, memory-mapped files, and memory allocated from the heap. The address space and memory allocation policies used by different operating systems are complex and vary considerably from one operating system to another. Explaining the details of each memory model for each operating system is beyond the scope of this paper, but the following diagram provides a sample layout for a generic 32-bit process address space. Generic 32-bit Operating System Process Address Space Kernel Program & Program Data Most 32-bit operating systems divide the 4 GB address space into different address ranges or sections. Each section has explicit purposes. For example, the operating system must provide a section of the address space for the program or the code that is running, and additional sections for the program’s stack and static data. Stack Heap Global Data Including, Shared Memory & Shared Libraries or DLL’s 4 GB Process Address Space Consider the diagram to the left and notice that the entire 4 GB process address space is divided into multiple sections including, the kernel, the program and its data, stack space, heap space used for dynamic memory allocation, and global data that includes shared memory and shared libraries or DLL’s. The size of each section varies depending on operating system and does affect the size of allocated shared memory segments. 2 The following table summarizes some of the key details of the process address space for Solaris, Linux, Windows, AIX, and HP-UX. The table shows the total memory allocation per process, the memory required for the kernel, and the memory available in the user address space. Operating System Total Memory per Process Solaris 4 GB Linux Reserved Kernel Memory User Address Space Available ~4 GB shared between program, its data & stack, and shared memory 4 GB Kernel and user space are separated into their own 4 GB address spaces 1 GB Windows 4 GB 2 GB 2 GB shared between program, its data & stack, and shared memory AIX 4 GB divided into sixteen 256 MB segments One 256 MB segment One 256 MB segment for the program One 256 MB segment for the program’s data & stack Twelve 256 MB segments available for shared memory HP-UX 4 GB divided into four 1 GB quadrants 3 GB shared between program, its data & stack, and shared memory The four quadrants are organized as follows: 1. 1 GB Program 2. 1 GB program data 3. 1 GB global space 4. .75 GB global space A 32-bit program can allocate and manage a maximum 1.75 GB of shared memory. Testing the Limits Progress ran a series of tests to determine the maximum amount of shared memory that OpenEdge Release 10.1B and Release 10.1C are able to allocate for a single database. Testing took place on Solaris, Linux, Windows, AIX, HP-UX, and Unixware using the appropriate 32-bit versions of OpenEdge. A test was considered successful after the database started, a simple ABL query ran and returned the correct records, PROMON ran and reported correct metrics, and the database shutdown cleanly. The table below describes the conditions that define the tests and the resulting maximum-shared memory size that the 32-bit OpenEdge RDBMS is able to allocate. The test variables include: The platform including its architecture and operating system release The OpenEdge release information The database block size The startup configuration parameters used when starting the database The only two database configuration parameters used during testing were –B (Database Buffer Pool) and –shmsegsize (Shared Memory Segment Size). Defaults were accepted for all other parameters. –B provides a quick and efficient method to increase the shared memory 3 requirements for the database and the buffer pool is initialized so all of shared memory is “touched”. The –shmsegsize parameter provides a more efficient way for the operating system to fit smaller shared memory segments into the address space. By manipulating these two parameters, the amount of allocated shared memory was adjusted up and down. The maximum allocated shared memory achieved is shown in the following table. Operating System and Hardware OpenEdge Release Solaris 5.10 Sun Sparc - 64-bit 32 GB Ram Shared Memory Allocated (as reported by PROMON R&D/Status: Shared Resources) 8K database block size 32-bit OE 10.1C for Solaris –B 460000 –shmsegsize 512 Total shared memory = 3,884 MB 32-bit OE 10.1B for Solaris –B 480000 –shmsegsize 512 Total shared memory = 3,892 MB Linux RH EL 4 U4 Intel x86 - 32-bit 1,010 MB Ram Database Information & Startup Arguments 8K database block size 32-bit OE 10.1C for Linux –B 346000 –shmsegsize 512 Total shared memory = 2,807 MB 32-bit OE 10.1B for Linux –B 347000 –shmsegsize 512 Total shared memory = 2,810 MB Windows 2003 Server 4K database block size Intel x86 - 32-bit 32 GB Ram 32-bit OE 10.1C for Windows(1,2) –B 406000 –shmsegsize 512 Total shared memory = 1,711 MB Client/server ABL connection 32-bit OE 10.1B for Windows –B 420000 –shmsegsize 512 Total shared memory = 1,746 MB Windows 2003 Server 4K database block size Intel x86 - 32-bit 32 GB Ram 32-bit OE 10.1C for Windows –B 350000 –shmsegsize 256 Total shared memory 1,477 MB Self-Service ABL Connection3 32-bit OE 10.1B for Windows –B 350000 –shmsegsize 256 Total shared memory 1,474 MB 1 OpenEdge for Windows does not support the Windows /3Gb parameter. On 32-bit versions of Windows, the /3GB parameter enables 4 GT RAM Tuning, a feature that enlarges the usermode virtual address space to 3 GB and restricts the kernel-mode components to the remaining 1 GB. ( http://msdn2.microsoft.com/en-us/library/ms791558.aspx ) 3 The OpenEdge ABL client is a larger program, loads multiple DLLs and has larger data and heap requirements than the database broker or server. The additional resource requirements reduce the amount of address space available for shared memory mapping. 2 4 Operating System and Hardware OpenEdge Release AIX 5.2 Database Information & Startup Arguments Shared Memory Allocated (as reported by PROMON R&D/Status: Shared Resources) 8K database block size IBM Power PC - 64-bit & IBM Power PC – 32-bit 32 GB Ram 32-bit OE 10.1C for AIX –B 340000 –shmsegsize 512 Total shared memory = 2,763 MB 32-bit OE 10.1B for AIX –B 345000 –shmsegsize 512 Total shared memory = 2,801 MB HP-UX 11.11 HP hp9000 – 64-bit PA_RISC 4 GB Ram 8K database block size 32-bit OE 10.1C for HP-UX –B 200000 –shmsegsize 256 Total shared memory = 1,669 MB 32-bit OE 10.1B for HP-UX –B 202000 –shmsegsize 256 Total shared memory = 1,684 MB Unixware 7.1.4 Intel x86 Pentium XEON 2 GB Ram 8K database block size 32-bit OE 10.1C for Unixware –B 350000 –shmsegsize 512 Total shared memory = 2,839 MB 32-bit OE 10.1B for Unixware –B 360000 –shmsegsize 512 Total shared memory = 2,915 MB Shared Memory Segment Size (–shmsegsize) Configuration Parameter The new database configuration parameter, –shmsegsize, or Shared Memory Segment Size, was introduced to allow tuning of the shared memory segment sizes. The valid values for the – shmsegsize parameter are: 32-bit systems 128, 256, 512, 1024, 2048, 4096 megabytes or 1g, 2g, 4g where g is gigabytes The 4096 and 4g values are allowed on 32-bits platform, but in reality a 4 GB, shared memory segment cannot be allocated on 32-bit platforms. Specifying a –shmsegsize of 4096 or 4g will allow the OpenEdge creation of shared memory segments greater than 2 GB on systems that support segments greater than 2 GB. 64-bit systems 1024, 2048, 4096, 8192, 16383, 32767 megabytes or 1g, 2g, 4g, 8g, 16g, 32g where g is gigabytes 5 The –shmsegsize configuration parameter allows control over the maximum segment size used by the OpenEdge RDBMS. The actual segment sizes created may be smaller than this due to a variety of factors such as: The actual amount of shared memory needed may be less than the requested –shmsegsize. The operating system's maximum segment size may less than the requested –shmsegsize. The amount of available unused address space may be less than the requested –shmsegsize. The value of SHMMAX and other kernel tunables may impose lower limits. The initial maximum-shared segment size is calculated based on your shared memory requirements and the support provided by your operating system. If –shmsegsize is omitted, OpenEdge will attempt to fit all the shared memory required into the fewest shared memory segments possible. Previously the fixed maximum was 128 megabytes for 32-bit systems and 1 gigabyte for 64-bit systems. When supported by the operating system OpenEdge creates fewer but larger segments. For some operating systems, this enables use of slightly larger database buffer pools with 32-bit versions of the OpenEdge RDBMS. The Impact on Your Database The database data structures placed into shared memory are larger in Release 101B and higher than they were in prior releases. In particular, shared pointers are now 64 bits wide for both 64bit and 32-bit versions of the OpenEdge RDBMS. Previously they were 32 bits wide on 32-bit versions. As a result, some data structures that are quite numerous, such as database buffer-pool headers and lock table entries, consume noticeably more memory than before. While the changes in release 10.1B and later have definite advantages, they are also not without some drawbacks. Thus, if you have a 32-bit system and you upgrade from an earlier release of OpenEdge, you might encounter errors when you start the database server or when clients try to connect to it in self-serving mode because the larger size shared segments cannot be allocated within the available address space. The following table identifies the potential problems that could occur and presents their solutions: Problem You cannot start the server because data structures do not fit and they did before. OpenEdge Message(s) in the database log file or standard output: Solution Lower the value of –B by 10 percent to overcome this problem if you are close to the maximum amount of shared memory for the system and the value of –B is large. Continue to reduce the shared memory requirements for the database until it starts successfully. The data space of the process is not enough for the shm segment (1176) 6 Problem Solution The server starts, but clients cannot connect in self-serving mode. This may be caused by lack of sufficient contiguous free address space in combination with the larger default segment size used by the server. Lower the value of –shmsegsize. When the address space has several smaller holes, the operating system may be able to locate the segments in them. This is likely to happen in Windows, where the system has mapped .dll's in a manner such that there are no large chunks of free address space. The problem is similar to filling a box with variable size blocks of wood. Small blocks are easier to fit into the box than larger ones. Or A client that connects to several databases in self-serving mode may no longer be able to do so. This can happen due to larger segments. OpenEdge Message(s) in the database log file or standard output: Unable to attach shared memory “segment name” error ##. (1720) There may be a conflict between large shared ABL procedure libraries (that are mapped in a single contiguous memory segment) and large shared memory segments created by the server. Lower the value of –shmsegsize. 7 Summary The shared memory addressing enhancements made in OpenEdge Release 10.1B and higher increases the shared memory addressability on 32-bit and 64-bit platforms and increases the flexibility of shared memory allocation when the –shmsegsize configuration parameter is used. On 32-bit platforms, the OpenEdge RDBMS will allocate more than 2 GB of shared memory if the operating systems and machine supports the increased shared memory allocations. Even though the OpenEdge RDBMS can address larger shared memory pools than it did in prior releases, it can still only allocate and utilize as much shared memory as the operating system allows. Understanding the basic principles used by your operating system to “build” a process address space will help you understand how to set your configuration parameters when you start your databases. References AIX 5L Version 5.3 General Programming Concepts: Writing and Debugging Programs SC23-4896-04, Fifth Edition (November 2007) Solaris Internals, Jim Mauro and Richard McDougall, Sun Microsystems, Inc. 2001 Inside Microsoft Windows Internals, Covering Windows 2000, Windows XP, Windows Server 2003, Mark E. Russinovich, David A. Solomon, Jim Allchin, - Microsoft Press, 2005 8