On many UNIX operating systems (including Linux and AIX) and on Windows, it is recommended that you configure HugePages for databases with a large SGA (such as greater than 2GB), and accompanying ASM instances. HugePages is also known as “Large Pages” for AIX and Windows. We will confine the HugePages discussion—and the steps to enable it—to Linux, as this is the most popular OS for an Oracle database. If you are using another platform, please consult your operating system documentation for the equivalent steps to set up HugePages (or Large Pages).
HugePages is a feature integrated into the Linux kernel from Release 2.6 that allows large SGA memory to be utilized with reduced overhead by ensuring that the memory is not paged to disk. HugePages allows you to use much larger page sizes (for example, 2MB or 4MB) than the default 4K page size, which can be crucial for faster Oracle database performance on Linux, and increasingly so for systems with large RAM and database SGA sizes, as is the case even for small CC deployments.
One possible drawback of using HugePages is that you cannot use it in conjunction with Automatic Memory Management (AMM); you must use Automatic Shared Memory Management (ASMM) instead. Therefore, before you decide to implement HugePages, you may want to weigh its advantages against any disadvantages of not being able to enable AMM.
For those unfamiliar with AMM and ASMM, ASMM was introduced in Oracle 10g with the SGA_TARGET and SGA_MAX_SIZE parameters that dynamically resize many SGA components on demand. With Oracle 11g, Oracle developed this feature further to include the PGA as well, and called it Automatic Memory Management (AMM), which is enabled by setting the parameter MEMORY_TARGET. AMM makes use of the SHMF—a pseudo file system like /proc. Every file created in the SHMFS is also created in memory.
ASMM works well and has been extensively tested in the field. By contrast, AMM has several bugs that you may encounter, including non-public Bug 6325878, which affects 11.1 databases, and whose fix is to disable native compilation by setting JAVA_JIT_ENABLED=FALSE. That said, both AMM and ASMM are supported for the Cloud Control Repository database.
While there are several MOS bulletins on HugePages (notably MOS ID 361323.1) and how to configure it (MOS ID 361468.1), we have not found a bulletin that covers all steps to our satisfaction. So in the following steps, we provide a detailed procedure on how to configure HugePages for an Oracle 11gdatabase on Linux. All steps should be performed as the owner of the Oracle Database software (for example, the oracle user), unless otherwise indicated as needing to be run as the root user.
1. If you’re not sure whether HugePages has already been configured, you can check the HugePages state in /proc/meminfo:
All HugePages states except AnonHugePages and Hugepagesize will be 0 when HugePages is not configured:
2. Determine the current value for memlock (reported in KB) for the oracle user:
The memlock value should be slightly smaller than the amount of RAM installed on the database server. On UNIX, you can determine the RAM on your system with the following command:
3. If the current value for memlock is not set as required, log in as root to set the memlock user limit, which must be expressed in KB (enter only the number of KB for <memlock_value>, not the units themselves), in /etc/security/limits.conf:
For guidance on what would be considered a slightly smaller memlock_value than the amount of RAM installed, see the following table, which lists memlock values appropriate for the more common amounts of RAM found on database servers:
| RAM (GB) on Database Server | Memlock Value (KB) to Use |
| 64 | 60397977 |
| 32 | 30198988 |
| 16 | 15099494 |
| 8 | 7549748 |
You can also set memlock to “unlimited,” though this is not recommended.
4. Exit the oracle account, log in again, and confirm that the new memlock limit has been set as intended:
5. Ensure that the Automatic Memory Management (AMM) feature is disabled, because AMM is incompatible with HugePages. AMM is set by default for 11g Database, but can be disabled by unsetting (setting to “0”) the following initialization parameters:
MEMORY_TARGET
MEMORY_MAX_TARGET
First, check the setting for these parameters with:
If you’re using an SPFILE, and these parameters are explicitly set to non-zero values, run the following commands to unset them (that is, set them to “0”) in the SPFILE:
If these parameters are not explicitly set (that is, set to 0), you do not need to unset them, and the preceding commands will accordingly error out with “ORA-32010: cannot find entry to delete in SPFILE.”
6. Confirm that Automatic Shared Memory Management (ASMM) is being used instead of AMM, as evidenced by the following initialization parameters being set to nonzero values:
SGA_TARGET
SGA_MAX_SIZE
If using an SPFILE, the following commands will set these parameters to 2G and 2.4GB respectively (as an example, using the recommended values for a small CC site) in the SPFILE, to take effect on database restart:
7. If the database is Oracle 11.2.0.2 or higher, set the initialization parameter USE_LARGE_PAGES=ONLY to ensure that the database instance will start only when it can get all of its memory for SGA from HugePages, and to avoid out-of-memory errors. Use this setting only for database instances; for ASM instances leave USE_LARGE_PAGES=TRUE (the default value). This ensures that HugePages are used when available, but that ASM will start when HugePages are not sufficiently configured.
If using an SPFILE, execute the following command to set USE_LARGE_PAGES=ONLY, made effective when the database is restarted:
Note that the USE_LARGE_PAGES parameter is not currently documented in the Oracle documentation and cannot be set up during database creation with DBCA. For more information on USE_LARGE_PAGES, see MOS ID 1392497.1.
8. As root, edit /etc/sysctl.conf and set the kernel parameter vm.nr_hugepages as recommended by the Oracle-supplied script hugepages_settings.sh, contained in MOS ID 401749.1:
9. As root, dynamically enable this new kernel parameter setting to ensure it does not error:
10. Stop all databases and database listeners on the server, and as root reboot the server, such as with the
reboot command.11. Validate the HugePages Configuration. After the system is rebooted, ensure that all database instances on the host are started, then check the HugePages state from /proc/meminfo:
The following type of output is expected (values will vary), now that HugePages has been configured:
To verify that the configuration is valid, ensure that HugePages_Total is set as follows, and note that the remaining values will be set as indicated:
- The value for HugePages_Total is that set for vm.nr_hugepages in Step 8.
- The value for HugePages_Free will be smaller than that for HugePages_Total.
- HugePages_Rsvd will be set to a nonzero value.
- The sum of HugePages_Free and HugePages_Rsvd, multiplied by the Hugepagesize, will be somewhat smaller than the total combined database SGA sizes of all databases on the system. This is expected because database instances allocate pages dynamically and proactively as needed.
- The value for HugePages_Surp will be 0 or thereabouts.
12. Restart the database listener on the database tier:
This concludes the configuration process for HugePages on Linux.
Is the reboot of server is necessary?
how can you check at database level if the database is really using huge pages after configuring the OS as required.
This is a great article and very helpful information on how to deploy hugepages. However, I am not finding any performance gains in implementing this, and also Oracle support is not saying we should definitely use it. The additional complexity of using ASSM instead of AMM also does not seem to be worthwhile; I’ve been tweaking PGA, SGA, and other parameters and still coming up with weird memory errors. I never had any problems with this when using AMM, so am going to return to that instead. But thanks for the well-written and detailed information on how to implement this, it was great.
Thanks Amy for sharing your experience with ASSM and huge pages. As with most recommendations for database performance, the environment, workload and configuration always determine the final outcome and/or the benefits of a recommendation. My motto is to always ‘test, test, test and then test again’ before implementing them. 🙂 – Janis
You shouldn’t use AMM for databases on servers with more than 4GB of memory
https://blog.dbi-services.com/12cr2-dbca-automatic-memory-management-and-databasetype/