Making things run faster Post: 302249129

Sponsored Content

Top Forums UNIX for Advanced & Expert Users Making things run faster Post 302249129 by shamrock on Monday 20th of October 2008 05:11:09 PM

10-20-2008

Registered User

Quote:

Originally Posted by Legend986

@matrixmadhan: Thanks a lot... I have used an almost similar approach from your script but slightly adapted for my own datasets. I will try timing both the approaches and will paste the result here.

And one more thing: I have found this really cool package called xjobs. Would you mind taking a look at it? It basically handles the master part from your logic and is very useful. Thought you might find some use out of it too. You can access it here: xjobs

@shamrock: Again, thank you for clarifying the issue. I just didn't know if it was really RAID not JBOD because the CPU is spending 88% of its time waiting (taken from the mpstat command) which seemed really weird to me.

@bakunin: Thank You for the advice. I actually agree with you as that was my experience too. I switched to PERL after a really bad experience with awk. Blame it on my lack of expertise in them. Other than that, I am still using awk and sed whenever things can be done easily.

How many mpus are there in your machine?
The reason the mpu is spending so much time waiting is because the terrabytes of data being processed...I/O wait.

shamrock

View Public Profile for shamrock

Find all posts by shamrock

8 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

making ssh run without password

Hello Everybody, Could anyone please tell me how to get ssh to work without asking for passwords? (i want to do a ssh <hostname> without getting a request for a password but getting connected straight away) I have attempted the following but to no avail :( ... I tried to generate a SSH...

2. Programming

Complicating things?

So basically what im trying to do is ... Open file, read that file, than try to find .. We or we and replace them with I, but not replace the cases where words contain We or we, such as Went, went, etc a and replace them with the, but not replace the cases where words contain a, such as...

3. Shell Programming and Scripting

When things doesn't run into crontab???

Could someone explain my problem? I've the following script... #! /bin/ksh ... vmquery -m $MediaID | awk ' BEGIN {FS=": " getline expdate <"ExpDate.txt" } $1 ~ /media ID/ {MediaNumber = $NF} ... $1 ~ /number of mounts/ { "date +%Y"|getline YearToday Year4 = YearToday - 4 if...

4. Shell Programming and Scripting

Can anyone make this script run faster?

One of our servers runs Solaris 8 and does not have "ls -lh" as a valid command. I wrote the following script to make the ls output easier to read and emulate "ls -lh" functionality. The script works, but it is slow when executed on a directory that contains a large number of files. Can anyone make...

5. UNIX for Dummies Questions & Answers

things root can't do

Hey all my co-workers and I are trying to put together a list of things root "Can't" do on any *NIX OS, so I wanted to come here and see what all we could come up with. Here are two to start this off: write to a read only mount FS kill a tape rewind Please add what you know. Thanks,...

6. Shell Programming and Scripting

Making script run faster

Can someone help me edit the below script to make it run faster? Shell: bash OS: Linux Red Hat The point of the script is to grab entire chunks of information that concerns the service "MEMORY_CHECK". For each chunk, the beginning starts with "service {", and ends with "}". I should...

7. Shell Programming and Scripting

Making a faster alternative to a slow awk command

Hi, I have a large number of input files with two columns of numbers. For example: 83 1453 99 3255 99 8482 99 7372 83 175 I only wish to retain lines where the numbers fullfil two requirements. E.g: =83 1000<=<=2000 To do this I use the following...

8. Shell Programming and Scripting

Optimize shell script to run faster

data.file: contact { contact_name=royce-rolls modified_attributes=0 modified_host_attributes=0 modified_service_attributes=0 host_notification_period=24x7 service_notification_period=24x7 last_host_notification=0 last_service_notification=0 host_notifications_enabled=1...

LEARN ABOUT NETBSD

ataraid

ATARAID(4)						   BSD Kernel Interfaces Manual 						ATARAID(4)

NAME

     ataraid -- software BIOS RAID

SYNOPSIS

     pseudo-device ataraid
     ld* at ataraid? vendtype ? unit ?

DESCRIPTION

     The ataraid driver provides support for BIOS-based software RAID controllers.  These are devices which have some simple support for several
     basic RAID levels (often RAID 0 and RAID 1), but which require software support to actually perform the RAID function.  The BIOS support is
     largely just to create and recognize the array so that it may be a boot device.

     The driver currently supports RAID formats from:
	   o   Adaptec HostRAID (found in Intel 6300ESB)
	   o   Intel MatrixRAID
	   o   JMicron RAID
	   o   nVidia MediaShield
	   o   Promise FastTrak
	   o   Via V-RAID (found in many VIA-based motherboards)

     Status of the logical disk as well as the disks associated with it, can be viewed through the bioctl(8) utility.

SEE ALSO

     ld(4), bioctl(8)

HISTORY

     The ataraid driver first appeared in NetBSD 2.0.

AUTHORS

     The ataraid driver was originally adapted from FreeBSD by Jason Thorpe <thorpej@NetBSD.org>.

BUGS

     Not all features of the software RAID are currently recognized or supported.  For example, the Adaptec support doesn't recognize when a RAID
     1 should be in a ``building'' state, and it does not do the right thing.

     At least part of the reason for this is that the publically-available information on these formats is quite limited.

BSD
								September 16, 2008							       BSD