I am in the process of building a workstation and have a question related to performance. I am a scientist who deals with big data (average file size 30-50gb). My OS is ubuntu 14.04 and so far I have a 128gb dual xeon E5-2630 with 6 cores each. I/O buffering is an issue so I am adding a 256/512? PCIe card and either 2 SSD or SAS drives for the OS and software. Since the PCIe will be separate its main purpose will be for file transfer, so would a SAS or SSD be a better fit for the OS? I am leaning towards SAS for the buffering issue, but wanted to ask more knowledgeable users. I forgot to mention that there will be a separate 1 or 2TB drive. Any recommendations for the size of the SAS or SSD? Thanks .
SSD is more than an order of magnitude (or much) faster than SAS high-rpm disks.
SSD is limited - usually to 1-2 TB of storage. With 128GB of memory, you could easily use SSD disks to load whatever file you want into memory - e.g., usual term is a RAMDISK. Ubuntu supports this. It also caches files very effectively without much human intervention other than configuration.
Not directly related but i had a longer workshop yesterday about our new storage system (EMC VMax 200k). EMC claims that they had intended the 300GB 15k-SAS drives for high-performance, but phase them out now because (quoting from memory) with the development of Flash-SSDs its just not worth it any more. They also claim that, because they use SLC-based hardware, they have even lower rates of disk-replacement, even in heavy-duty transactional storage systems, than with rotational disks, to which a much lower energy consumption of the SSDs compared to the 15k-SAS disks contributes. There is simply less heat involved and that shows when you pack some ~2500 disks into a rack.
You haven't said where you are going to place the workstation, but in case it is going to be somewhere near your desk: 15k-disks are awefully LOUD in addition to be premier heating devices while SSDs are completely silent.
I hope this helps.
These 2 Users Gave Thanks to bakunin For This Post:
I recently received a new SSD that I am going to use for the purpose of Booting Virtual Machines. I use VMWare Player to boot Windows Guest Operating Systems onto my Linux Laptop.
I currently have a SSD drive that I use for this exact same purpose that is formatted as ext3 and I'm... (3 Replies)
Nowadays the fastest SSDs achieve read-speeds of between 1500 MB/s to 1900 MB/s. Let's say that two such SSDs in RAID 0 achieve roughly double the throughput, ie 3000 MB/s. That is only half an order of magnitude removed from RAM ((10)^(1/2) * 3000 = 10.000), very broadly speaking.
So for the... (6 Replies)
Hey every one!
I have a dataset like this :
1 100 1 0
5 100 1 8
7 50 1 0
7 100 2 0
10 20 1 8
10 30 1 8
10 100 3 8
15 50 5 0
20 90 1 0
20 99 9 0
I wanna check if the 4th column is 0 or 8
If it's zero write the 1st column itself, if it's 8 write sum of 1st and second
something... (2 Replies)
IHAC who complaint that his CentOS is getting performance issue.
I have to help him out of there.
Could you please tell me which tools is better to gathering the whole system performance data?
-- CPU/Memory/IO(disk & Network)/swap
I would like the tools could be... (6 Replies)
I posted the same message under 'Kaleidagraph like software for Ubuntu' thread. I guess there may not be many people familiar with Kaleidagraph.
So I post my message under another subject name.
I need a tool for Ubuntu 8.10,
-which is quick and easy to learn and use (as I am... (1 Reply)