Sponsored Content
Operating Systems Linux Underutilization of Linux memory Post 303044206 by Neo on Sunday 16th of February 2020 09:40:41 PM
Old 02-16-2020
Quote:
Originally Posted by veeresh_15
Thanks Neo for your comments and taking out time to perform those tests. I will also try checking my system's performance by turning off swap.
But if you or anyone else can help me understanding my current system behaviour, it will help me understand Linux Memory management better.
As part of my python application, we are making some rest api calls to Azure and using various azure services. Please let me know what other information i can provide for commenting on this behaviour?

Regards,
Veeresham
Dear Veeresham,

I think you may have completely missed the point in my first post. MadeInGermany obviously did as well.

Let's take the price of this RAM on AliExpress:

Samsung server memory DDR3 8GB 16GB 1333MHz 1600MHz 1866 ECC REG DDR3 PC3-12800R Register DIMM RAM 240pin 12800 8G 2RX4 X58 X79

In the link above, the price of server RAM is about $20 USD for 8 GB.

Now, I am not sure how much time you have spent working on this issue and how much you think your time is worth to you; but I can tell you that already I have spend time and effort on answering your question for you with a value of at least $40 USD, including replying to you and others, doing some quick Internet searches, and logging into a remote system and turning off swap (to illustrate a point for the "naysayers" who remain in love with swap from the yesterdays of expensive RAM nostalgia).

Let's say that the time you and others have spent on the problem (on your end) is also work about $40 USD. This means that $80 USD total human capital has already been spent on this problem.

$80 USD divided by $20 USD turns out to be roughly the same in terms of resources, as purchasing 32 GB of Samsung server RAM for your application. Of course, this is different in a VM service, but that is a different topic; but the same idea applies (just differently, and I tend NOT to use VMs and cloud services for security, and command and control reasons)

Let's say you try tweaking swappiness (which I have done many times before on Linux systems) and playing around with more Linux system parameters (like cache clearing, refresh rates, etc) to try to further optimize. You are spending more resources, all of which could be used to add RAM to you system, if you value your time (in the least). Only in an "academic" context, doing a study for the sake of learning (not operations) is all this optimization an efficient use of time and energy. You have indicated you are working on a real-world application and are not in academics.

This is my point.

RAM is very cheap (and it gets cheaper every year). If your system is swapping, that is an indicator, and a very strong indicator, that you need more RAM. Tweaking and tweaking your system parameters and encouraging others to engage in this kind of tweaking with you, is an inefficient use of human resources and capital. It's more "academic" in nature, versus practical, in operational terms.

In answering you in this reply, this reply is at least work 8 GB of server RAM in 2020; but let's be very conservative, and say all this thinking and typing and error checking (my in-browser spell checker seems to be in a plot against me, LOL) is only worth 4GB of RAM, either way; you are talking about your system which you say has 8GB of RAM installed. That RAM is worth about $20 USD, today (in hardware terms, not inflated, artificial VM terms).

If you want better performance, go out and spend $20 more USD and buy 8GB more and you will have 16GB of RAM.

This is a better use of your time and your energy. Your application will run faster and better. The world will be a better place.

The reason I have turned off swap on one server (yesterday), and am posting the results tonight in this thread, is to prove a simple point to others who are "swap nostalgic". This is not "a test" for me. I have been working with Linux since before Slackware 0.8, which was released before 1993. This translates to using Linux for around 27 years, continuously. I know the results before I turn off swap. There will be no change in system performance because the server and the applications I am running on that server are (1) not constrained by RAM and (2) there are no memory leaks. Linux runs well and I always configure all my Linux systems to have plenty of RAM and upgrade if RAM becomes an issue.

There are a lot of people who like to talk about the virtue of swap. This is like saying that when the glass of milk is full, I will go to the shelf and get another glass to store my milk until the main glass is less full, as I pour into the main glass. While this might be good if you are pouring milk very fast into your glass; most people would get a bigger glass.

The same in true for computer RAM.

RAM is so cheap these days (and has been for many years), you are better off getting more RAM versus spending a lot of time and energy trying to tweak a system to squeeze more performance from your system, which is asking you for more RAM because it is starting to swap.

You asked for ideas; so here is mine, put very simply. I would never run a Linux server application with 8GB of RAM and 2GB of swap because I do not want my systems to swap, ever, for any reason. I run 64 GB RAM on my desktop. I only use 8 GB VMs for "playgrounds", not production apps. If I built an app in a 8GB sandbox and it then became memory constrained, I would move it to a more robust server or upgrade it (if in VM, but I tend to move real-world apps to dedicated hardware, but that is just me because I like having more control over the OS and the upgrade cycles, etc). This is because time is worth more than money. We cannot gain back time. The arrow of time runs only in one direction (in this universe we are in now).

What I do is to always make sure my computer systems, all of them, remote servers, desktops all of them; have more than enough RAM to do the job because RAM is so cheap and I have better things to do with my time, which I consider valuable and a limited resource which I have little control.

Tonight I will post the graph of my MQTT / Node-RED server with swap off for 24 hours. You will see that there is no issue at all because there is enough RAM. If your system is memory constrained and you turn off swap, you will more-than-likely have some performance issues; but my point is not "turn off swap".. my point is "upgrade and add more RAM". For me "swap" is a "big red, flashing LED" telling me "please upgrade".

Now, in the VM case, RAM is a bit more expensive because when running in a VM and we are paying more for the RAM; but that is an entirely different discussion (that is a business issue, not a technology issue); but the point is still the same.

If you system is swapping, install more RAM.

You don't need to turn swap on and off. I was trying to make a point. If your system is memory bound (constrained) it will perform poorly even if you use swap; although it will perform better with swap than without swap (generally speaking) when you are really constrained by RAM. Like in my milk metaphor, it is OK to swap glasses when you are pouring milk to fast into a small glass; but it's even better (and smarter) to get a bigger glass.

The best choice, as I said in my first post, is to upgrade your RAM and move on to other tasks, unless (1) you are doing this as an academic exercise or (2) your time has very little value.

This is why I have replied this way. Time is an arrow which only moves in one directly. Use your time and energy wisely.
This User Gave Thanks to Neo For This Post:
 

10 More Discussions You Might Find Interesting

1. Programming

memory layout in C on linux

Hi, Does any one know what tool to use to visualize how is memory layed out for C on linux systems. I mean how much stack portion is used in functional call. Where exactly does the argument to function sit in memory ? I have written small program pasted below. But I am not able to infer... (3 Replies)
Discussion started by: parasa
3 Replies

2. Programming

Determining free(available) memory in MV linux

HI I'm a rookie in C programming and I'm working in Monta Vista Linux. I have to write a program that displays free memory. I have memtester(allready written by someone else) and now I have to type how much amount of memory tester will test and I want that memtester finds out himself how much of... (11 Replies)
Discussion started by: trancedeejay
11 Replies

3. Red Hat

Linux memory usage

What's the best way to find out how much memory is being used/available? I tried using free, but I didn't quite understand the output. Can someone explain it? $ free total used free shared buffers cached Mem: 16304536 16256376 48160 0 ... (6 Replies)
Discussion started by: junkmail426
6 Replies

4. UNIX for Advanced & Expert Users

Memory managment - linux

Hi, I having problem with my linux machine it have 6Gb physical memory and somehow it always almost coming to the bottom neck and than it start writing to the swap memory you can see that there is more than 4G in cahce, is there any way to clean the cache or to limit it to 2Gb? host1... (6 Replies)
Discussion started by: Igal Malka
6 Replies

5. Red Hat

share memory on linux

how to list the orphaned shared memory? how to kill them so that shared mem is free again. thanks (9 Replies)
Discussion started by: melanie_pfefer
9 Replies

6. Linux

Linux Memory Track

Hi All, We are using the linux servers and need to track the memory utilization of the box. Could anyone advice how the same can be achived. :) (1 Reply)
Discussion started by: haitorajesh
1 Replies

7. Programming

Memory sniffing in linux

I am trying to create an application that will be able to sniff memory of other applications. I am not completely new to systems programming but I am not sure how to go about this task. I understand that accomplishing this mainly require these steps. 1: Get a list of processes 2: Find the... (2 Replies)
Discussion started by: mosey
2 Replies

8. Linux

Memory Usage on Linux server

Hi , As I am new to Linux server, I am facing some doubts like: On linux server virtual memory usage goes to 99%, but I have Threshold limit of 95%. So it crossed the threshold limit and alarmd. Yesterday I moniterd the server using TOP command, and found some of Tibco process was consuming... (4 Replies)
Discussion started by: Jaywantmm
4 Replies

9. Red Hat

about shared memory space in Linux !

Dear Friends , I am confused about shared memory in Linux . plz look @ the below output : $ df -h Filesystem Size Used Avail Use% Mounted on /dev/sda1 126G 61G 59G 52% / tmpfs 1.7G 0 1.7G 0% /dev/shm Here , I see that every time... (1 Reply)
Discussion started by: shipon_97
1 Replies

10. Red Hat

Shared memory in linux

Hello, I am using Linux os. $ df -k /dev/shm Filesystem 1K-blocks Used Available Use% Mounted on tmpfs 2023256 1065000 958256 53% /dev/shm $ Based on my google this, it is shared memory. What is this shared memory and where exactly it is used? Can you... (5 Replies)
Discussion started by: govindts
5 Replies
All times are GMT -4. The time now is 08:57 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy