More and more of what I'm doing is advanced manipulation of data, so I'm using awk/sed/grep/whatever to bend my data and do things with it, I just have this nagging feeling with bash that I'm reaching the end of it's capability
You're not, really. A lot of the places you're using awk and so forth you're doing things that could be done with bash builtins, and you're still making some beginner mistakes like cat instead of redirection. If you don't mind, I'll improve your script a little. Give me a bit.
---------- Post updated at 11:21 AM ---------- Previous update was at 10:55 AM ----------
You can do a lot with pipes. Bourne shell lets you put them wherever. The trick is to not to use them for one-liners if you can help it, they're inefficient that way, but excellent for manipulating arbitrary amounts of data -- connect long-running processes or code sections with them, not pipe a single word through sed. Here we're creating an entire list of insert operations and feeding it into one instance of mysql instead of running mysql N*M times. This is important since process creation times can be significant compared to other operations.
This is what I'd consider the biggest difference between shell scripting and perl, one of shell's most powerful features, and one of the last things people grasp about shells. To get a pipe in perl you have to make an explicit open call, it doesn't come naturally. Nor does it understand redirection and pipes on the statement or code block level and so forth.
Last edited by Corona688; 05-14-2010 at 03:16 PM..
Hi All,
I want to sort a flat file which will contain millions of records based on a key/field. For this I want to use unix sort command and before that I want to make sure that unix sort command has any file size limitations. And also please let me know whether I have to change any... (2 Replies)
I am trying to use the csplit file on a file that contains records that have more than 2048 characters on a line. The resultant split file seems to ignore the rest of the line and I lose the data.
Is there any way that csplit can handle record lengths greater than 2048?
Thanks (0 Replies)
I would like to set my minimum password length to on Linux and AIX.
However, doing this normally would only make it so newly added users will be affected by this. I would like for when I make this change, it either truncates everyone elses password, or prompts them to change it to 8+ characters.... (2 Replies)
Hi All,
I need to pass a variable to perl script from bash script, where in perl i am using if condition. Here is the cmd what i am using in perl
FROM_DATE="06/05/2008"
TO_DATE="07/05/2008"
"perl -ne ' print if ( $_ >="$FROM_DATE" && $_ <= "$TO_DATE" ) ' filename"
filename has... (10 Replies)
I'm running a simulation (programmed in C) which makes calls to gnuplot periodically to plot data I have stored.
First I open a pipe to gnuplot and set it to multiplot:
FILE * pipe = popen("gnuplot", "w");
fprintf(pipe, "set multiplot\n");
fflush(pipe);
(this pipe stays open until the... (0 Replies)
Hi,
I have noticed some performance issues on my RHEL5 server but the memory and CPU utilization on the box is fine.
I have a 1G full duplexed eth0 card and I am suspicious that this may be causing the problem. My eth0 settings are as follows:
Settings for eth0:
Supported ports: ... (12 Replies)
Hi,
I recently started working with Solaris, and what I noticed is that a lot of commands I used to regularly use don't work, like sed -i and grep -r. I have found work arounds for these problems though but it's a pain in the ass.
I'm just wondering why they decided not to include these handy... (4 Replies)
In recently reading an article on linux basics before I embark and my personal installation project I came across this passage -
IDE drives have three types of partition: primary, logical, and extended. The partition table is located in the master boot record (MBR) of a disk. The MBR is the... (12 Replies)