Quote:
Originally Posted by
veeresh_15
Thanks Neo for your comments and taking out time to perform those tests. I will also try checking my system's performance by turning off swap.
But if you or anyone else can help me understanding my current system behaviour, it will help me understand Linux Memory management better.
As part of my python application, we are making some rest api calls to Azure and using various azure services. Please let me know what other information i can provide for commenting on this behaviour?
Regards,
Veeresham
Dear Veeresham,
I think you may have completely missed the point in my first post. MadeInGermany obviously did as well.
Let's take the price of this RAM on AliExpress:
Samsung server memory DDR3 8GB 16GB 1333MHz 1600MHz 1866 ECC REG DDR3 PC3-12800R Register DIMM RAM 240pin 12800 8G 2RX4 X58 X79
In the link above, the price of server RAM is about $20 USD for 8 GB.
Now, I am not sure how much time you have spent working on this issue and how much you think your time is worth to you; but I can tell you that already I have spend time and effort on answering your question for you with a value of at least $40 USD, including replying to you and others, doing some quick Internet searches, and logging into a remote system and turning off swap (to illustrate a point for the "naysayers" who remain in love with swap from the yesterdays of expensive RAM nostalgia).
Let's say that the time you and others have spent on the problem (on your end) is also work about $40 USD. This means that $80 USD total human capital has already been spent on this problem.
$80 USD divided by $20 USD turns out to be roughly the same in terms of resources, as purchasing 32 GB of Samsung server RAM for your application. Of course, this is different in a VM service, but that is a different topic; but the same idea applies (just differently, and I tend NOT to use VMs and cloud services for security, and command and control reasons)
Let's say you try tweaking
swappiness (which I have done many times before on Linux systems) and playing around with more Linux system parameters (like cache clearing, refresh rates, etc) to try to further optimize. You are spending more resources, all of which could be used to add RAM to you system, if you value your time (in the least). Only in an "academic" context, doing a study for the sake of learning (not operations) is all this optimization an efficient use of time and energy. You have indicated you are working on a real-world application and are not in academics.
This is my point.
RAM is very cheap (and it gets cheaper every year). If your system is swapping, that is an indicator, and a very strong indicator, that you need more RAM. Tweaking and tweaking your system parameters and encouraging others to engage in this kind of tweaking with you, is an inefficient use of human resources and capital. It's more "academic" in nature, versus practical, in operational terms.
In answering you in this reply, this reply is at least work 8 GB of server RAM in 2020; but let's be very conservative, and say all this thinking and typing and error checking (my in-browser spell checker seems to be in a plot against me, LOL) is only worth 4GB of RAM, either way; you are talking about
your system which you say has
8GB of RAM installed. That RAM is worth about $20 USD, today (in hardware terms, not inflated, artificial VM terms).
If you want better performance, go out and spend $20 more USD and buy 8GB more and you will have 16GB of RAM.
This is a better use of your time and your energy. Your application will run faster and better. The world will be a better place.
The reason I have turned off swap on one server (yesterday), and am posting the results tonight in this thread, is to prove a simple point to others who are "swap nostalgic". This is not "a test" for me. I have been working with Linux since before Slackware 0.8, which was released before 1993. This translates to using Linux for around 27 years, continuously. I know the results before I turn off swap. There will be no change in system performance because the server and the applications I am running on that server are (1) not constrained by RAM and (2) there are no memory leaks. Linux runs well and I always configure all my Linux systems to have plenty of RAM and upgrade if RAM becomes an issue.
There are a lot of people who like to talk about the virtue of swap. This is like saying that when the glass of milk is full, I will go to the shelf and get another glass to store my milk until the main glass is less full, as I pour into the main glass. While this might be good if you are pouring milk very fast into your glass; most people would get a bigger glass.
The same in true for computer RAM.
RAM is so cheap these days (and has been for many years), you are better off getting more RAM versus spending a lot of time and energy trying to tweak a system to squeeze more performance from your system, which is asking you for more RAM because it is starting to swap.
You asked for ideas; so here is mine, put very simply. I would never run a Linux server application with 8GB of RAM and 2GB of swap because I do not want my systems to swap, ever, for any reason. I run 64 GB RAM on my desktop. I only use 8 GB VMs for "playgrounds", not production apps. If I built an app in a 8GB sandbox and it then became memory constrained, I would move it to a more robust server or upgrade it (if in VM, but I tend to move real-world apps to dedicated hardware, but that is just me because I like having more control over the OS and the upgrade cycles, etc). This is because time is worth more than money. We cannot gain back time. The arrow of time runs only in one direction (in this universe we are in now).
What I do is to always make sure my computer systems, all of them, remote servers, desktops all of them; have more than enough RAM to do the job because RAM is so cheap and I have better things to do with my time, which I consider valuable and a limited resource which I have little control.
Tonight I will post the graph of my MQTT / Node-RED server with swap off for 24 hours. You will see that there is no issue at all because there is enough RAM. If your system is memory constrained and you turn off swap, you will more-than-likely have some performance issues; but my point is not "turn off swap".. my point is "upgrade and add more RAM". For me "swap" is a "big red, flashing LED" telling me "please upgrade".
Now, in the VM case, RAM is a bit more expensive because when running in a VM and we are paying more for the RAM; but that is an entirely different discussion (that is a business issue, not a technology issue); but the point is still the same.
If you system is swapping, install more RAM.
You don't need to turn swap on and off. I was trying to make a point. If your system is memory bound (constrained) it will perform poorly even if you use swap; although it will perform better with swap than without swap (generally speaking) when you are really constrained by RAM. Like in my milk metaphor, it is OK to swap glasses when you are pouring milk to fast into a small glass; but it's even better (and smarter) to get a bigger glass.
The best choice, as I said in my first post, is to upgrade your RAM and move on to other tasks, unless (1) you are doing this as an academic exercise or (2) your time has very little value.
This is why I have replied this way. Time is an arrow which only moves in one directly. Use your time and energy wisely.