Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Assistance to calculate number of hugepages Post 303046210 by sand1234 on Tuesday 28th of April 2020 01:34:09 AM
Old 04-28-2020
Assistance to calculate number of hugepages

Hi,

I am trying to count the number of hugepages & total usage being used by applications..

INPUT FILE
==========

Code:
[root@xxx~]# grep huge /proc/*/numa_maps 2>/dev/null
/proc/47173/numa_maps:140000000 default file=/anon_hugepage\040(deleted) huge anon=1 dirty=1 N0=1 kernelpagesize_kB=1048576
/proc/47173/numa_maps:1000000000 default file=/buffers-numa-0 huge dirty=313 N0=313 kernelpagesize_kB=2048
/proc/47173/numa_maps:1027200000 default file=/buffers-numa-1 huge dirty=313 N1=313 kernelpagesize_kB=2048
/proc/47173/numa_maps:2d00000000 default file=/anon_hugepage\040(deleted) huge anon=1 dirty=1 N1=1 kernelpagesize_kB=1048576
/proc/47173/numa_maps:4dc0200000 default file=/anon_hugepage\040(deleted) huge anon=1 dirty=1 N0=1 kernelpagesize_kB=2048
/proc/47173/numa_maps:4dc0400000 default file=/anon_hugepage\040(deleted) huge anon=1 dirty=1 N0=1 kernelpagesize_kB=2048
/proc/47173/numa_maps:4dc0600000 default file=/anon_hugepage\040(deleted) huge anon=1 dirty=1 N0=1 kernelpagesize_kB=2048
/proc/47173/numa_maps:2aaac0000000 default file=/mnt/hugepages1G/libvirt/qemu/2-instance-0000001f/qemu_back_mem._objects_ram-node0.su05tD\040(deleted) huge dirty=2 mapmax=2 N0=2 kernelpagesize_kB=1048576
/proc/488519/numa_maps:7f6e80000000 bind:0 file=/mnt/hugepages1G/libvirt/qemu/2-instance-0000001f/qemu_back_mem._objects_ram-node0.su05tD\040(deleted) huge dirty=2 mapmax=2 N0=2 kernelpagesize_kB=1048576

SCRIPT
==========

Code:
#!/bin/bash

total_num_hugepages=0

for id in $(grep huge /proc/*/numa_maps 2>/dev/null | awk -F'/' '{proc[$3]=1} END {for (i in proc) print i}'); do
        for hugepagesize in $(find /sys/devices/system/node/node[0-9]*/hugepages/ -type d -name "hugepages-*" | sed 's/.*hugepages-\([0-9]\+\).*/\1/'| sort | uniq);
        do
        ps $id;
        total_num_hugepages+=$(grep huge /proc/$id/numa_maps 2>/dev/null | grep "kernelpagesize.*=$hugepagesize" | sed 's/.*dirty=\([0-9]\+\).*/\1/')
        echo $total_num_hugepages
        done
done

Code:
[root@xxx ~]# ./hugepage_usage.sh
    PID TTY      STAT   TIME COMMAND
  47173 ?        SLl  333416:10 /usr/bin/vpp -c /etc/vpp/vpp.conf
01 1 2
    PID TTY      STAT   TIME COMMAND
  47173 ?        SLl  333416:10 /usr/bin/vpp -c /etc/vpp/vpp.conf
01 1 2313 313 1 1 1
    PID TTY      STAT   TIME COMMAND
 488519 ?        Sl    27:46 /usr/libexec/qemu-kvm -name guest=instance-0000001f,debug-threads=on -S -ob
01 1 2313 313 1 1 12
    PID TTY      STAT   TIME COMMAND
 488519 ?        Sl    27:46 /usr/libexec/qemu-kvm -name guest=instance-0000001f,debug-threads=on -S -ob
01 1 2313 313 1 1 12

Expected sample output
======================
PID TTY STAT TIME COMMAND
47173 ? SLl 333416:10 /usr/bin/vpp -c /etc/vpp/vpp.conf

/usr/bin/vpp
10 hugepages x 1048576 kB = xx kB
2 hugepages x 2048 kB = xx kB
Total hugepage usage = xx kB

Please give me some pointers on how I can improve my code, and achieve the output.

Thanks.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

calculate the number of days left in a month

does any one have any ideas how i would go about calculating the number of days left in the month from a bash script ?. I want to do some operations on a csv file according to the result (8 Replies)
Discussion started by: dunryc
8 Replies

2. Shell Programming and Scripting

calculate number of strings in a variable

Hi all I have a variable called "variable" and is of the form variable ="AAA BBB CCC DDD" {basically it has values separated by spaces} What is the simplest way to check if "variable" has more that one value in its list? Thanks. (9 Replies)
Discussion started by: felixmat1
9 Replies

3. Solaris

Calculate Network Number in /etc/netmasks file.

Hi All, I have an issue with calculating the network number that needs to be put in /etc/netmasks file in my Sol-9 server. The IP of my server in 10.164.114.135 Default Gateway - 10.164.114.130 Netmask - 255.255.255.240 If I set "10.164.114.130 255.255.255.240" in netmask file, after... (2 Replies)
Discussion started by: Hari_Ganesh
2 Replies

4. Programming

perl - calculate the number of lines from particular files

Hi, plz see the below code. here my aim is to calculate the number of lines in unprocessedData.out if this file contains 40 lines then lastly $linenum should print 40.(except blank lines) i have tried below code but it giving me the output only one. can anyone help me how to do ? ... (9 Replies)
Discussion started by: pspriyanka
9 Replies

5. Web Development

Calculate the number of days between 2 dates - PHP

Is this code good for this purpose? <?php $date1 = mktime(0,0,0,01,01,1991); $date2 = mktime(0,0,0,03,22,2012); $diff = $date2 - $date1; $days = $diff / (60*60*24); echo ($days . "<br />"); ?> (3 Replies)
Discussion started by: kovacsakos
3 Replies

6. Shell Programming and Scripting

Calculate the number of days between 2 dates - bash script

I wrote the day calculator also in bash. I would like to now, that is it good so? #!/bin/bash datum1=`date -d "1991/1/1" "+%s"` datum2=`date "+%s"` diff=$(($datum2-$datum1)) days=$(($diff/(60*60*24))) echo $days Thanks in advance for your help! (3 Replies)
Discussion started by: kovacsakos
3 Replies

7. Shell Programming and Scripting

How to calculate the total number of weeks from a specify year?

Hi anyone can help? How to calculate total number of weeks from a specify date, for example, 01 Jan 2012. Thx! (2 Replies)
Discussion started by: rayray2013
2 Replies

8. Solaris

How to calculate the number of TPS on Solaris server?

Hi, Please help me to find how to calculate the number of TPS supported by any solaris server for example one server with below configuration . Sun Blade X6270 with two 4-core processors 2 x 300 GB internal disk drives (2 Replies)
Discussion started by: manojkakkar
2 Replies

9. Shell Programming and Scripting

Sum and calculate number in table

Hello everyone, I have some problem in calculation number using awk. input file format : S1 1 : 0.003 0.031 S2 1 : 0.020 0.095 S3 4 : 0.088 0.012 S4 2 : 0.010 0.090 S5 2 : 0.244 0.066 S6 3 : 0.249 0.751 S7 3 : 0.010 0.990 S8 3 : 0.230... (4 Replies)
Discussion started by: awil
4 Replies

10. Solaris

How to calculate total number of cores on my servers ?

Hi, I want to get total number of cores on my all non-global zones on Solaris 10. I got two methods and both are giving different results. Below link is a script, which tells me that total cores are 8 Mandalika's scratchpad: Oracle Solaris: Show Me the CPU, vCPU, Core Counts and the... (4 Replies)
Discussion started by: ron323232
4 Replies
HUGECTL(8)						      System Manager's Manual							HUGECTL(8)

NAME
hugectl - Control policy for backing text, data and malloc() with hugepages SYNOPSIS
hugectl [options] command {arguments} DESCRIPTION
hugectl runs processes with a specific policy for backing memory regions with hugepages. The use of hugepages benefit applications that use large amounts of address space and suffer a performance hit due to TLB misses. Policy is enforced by libhugetlbfs and hugectl configures the environment based on the options provided. Wall-clock time or oprofile can be used to determine if there is a performance benefit from using hugepages or not. To effectively back text/data, the target process must be relinked to align the ELF segments on a hugepage boundary. The library also sup- ports more options for the control of memory regions than are exposed by the hugectl utility. See the libhugetlbfs manual page for more details. The following options affect what memory regions are backed by hugepages. --text[=<size>],--data[=<size>],--bss[=<size>] Back the text, data or BSS segments with hugepages, optionally with pages of the specified size. To be effective, the process must be relinked as described in the HOWTO to align the ELF segments. It is possible to partially back segments using the HUGETLB_FORCE_ELMAP environment variable as described in the libhugetlbfs manual page. --heap[=<size>] Use the glibc morecore hook to back malloc() with hugepages, optionally with pages of the specified size. Note that this does not affect brk() segments and applications that use custom allocators potentially do not use hugepages for their heap even with this option specified. --shm This option overrides shmget() to back shared memory regions with hugepages if possible. Segment size requests will be aligned to fit to the default hugepage size region. --share-text Request that multiple application instances share text segments that are backed with huge pages. This option sets the environment variable HUGETLB_SHARE to 1. --thp Align heap regions to huge page size for promotion by khugepaged. For more information on transparent huge pages see linux-2.6/Doc- umentation/transhuge.txt The following options affect how hugectl behaves. --no-preload Disable any pre-loading of the libhugetlbfs library. This may be necessary if only the heap is being backed by hugepages and the application is already linked against the library. hugectl may pre-load the library by mistake and this option prevents that. --force-preload Force pre-loading of the libhugetlbfs library. This option is used when the segments of the binary are aligned to the hugepage boundary of interest but the binary is not linked against libhugetlbfs. This is useful on PPC64 where binaries are aligned to 64K as required by the ABI and the kernel is using a 4K base pagesize. --no-reserve By default, huge pages are reserved at mmap() time so future faults will succeed. This avoids unexpected application but some appli- cations depend on memory overcommit to create large sparse mappings. For this type of application, this switch will create huge page backed mappings without a reservation if the kernel is recent enough to make this operation safe. Use this option with extreme care as in the event huge pages are not available when the mapping is faulted, the application will be killed. --dry-run Instead of running the process, the hugectl utility will describe what environment variables it set for libhugetlbfs. This is useful if additional environment variables are to be set and a launcher shell script is being developed. --library-use-path By default, hugectl will use the version of libhugetlbfs it was installed with, even if this is not in the LD_LIBRARY_PATH environ- ment. Using this option forces hugectl to use the version of libhugetlbfs installed in the library system path. --library-path <path> This option forces hugectl to use the libhugetlbfs libraries within the given prefix. The following options affect the verbosity of libhugetlbfs. --verbose <level>, -v The default value for the verbosity level is 1 and the range of the value can be set with --verbose from 0 to 99. The higher the value, the more verbose the library will be. 0 is quiet and 3 will output much debugging information. The verbosity level is increased by one each time -v is specified. -q The -q option will drecease the verbosity level by 1 each time it is specified to a minimum of 0. SEE ALSO
oprofile(1), hugeadm(7), libhugetlbfs(7) AUTHORS
libhugetlbfs was written by various people on the libhugetlbfs-devel mailing list. October 10, 2008 HUGECTL(8)
All times are GMT -4. The time now is 05:54 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy