Sorry for the delay, it has been an unexpectedly busy end of the week.
... ... ...
Quote:
Originally Posted by Don Cragun
But, why have an array at all. Why not just process the files one at a time as they come out of awk?
It turns out that I didn't end up using an array. Everything that came out of the code you posted ended up in a single long string variable ($FILE_LIST). I just looped on the number of files I was expecting to find and parsed the long string to pull out the file name for each iteration of the loop.
When you assign a value to a shell variable using the syntax:
and your shell is a recent bash or ksh, you are defining var to be an array. So, the way you initialized FILE_LIST, it was an array containing only one element.
Quote:
Originally Posted by LMHmedchem
Quote:
Originally Posted by Don Cragun
But, why have an array at all. Why not just process the files one at a time as they come out of awk?
The simple explanation for this is that I don't know awk very well at all. I can more or less use it to parse things, but only in the simplest implementations. I did spend some time trying to take the output from awk and work with it, but I think I was using a redirect instead of a pipe.
So as I read this now, ls is passing all .out.txt to sort, sort is sorting on the key field and passes the sorted list to awk. Awk processes each item in the list and outputs the specified fields. Then it looks like the output of awk is passed to read, which dumps the output into the vars $FILE_NAME and $RAND_SET. Once you are there, the rest is straightforward. I am not familiar with read, so that is something new to me. I would not have known how to get awk to output two variables and get them into something that I could use with cp. I am also not quite clear about how awk knows when it has read in enough lines to get to $FILE_COUNT. Is "NR" an implicit running counter of some kind so that when NR > $FILE_COUNT awk quits (I see that you have passed $FILE_COUNT to awk as c)?
Yes, you correctly interpreted what ls, sort, awk, and read are doing.
I suggest that you look at the read(1) man page. The read utility built into your shell will probably have additional options, but the POSIX description in the link above is all you need to understand what is going on in the simple script suggestions I provided. You might also want to read the awk(1) man page; the awk command:
The -F_ sets the input field separator to the underscore character, -v c="$FILE_COUNT" sets the awk variable c to the expansion of the shell variable FILE_COUNT, NR > c {exit} exits awk if the current number of input records read from all input files is greater than the awk variable c, and {print $0, $5} prints the entire current input record followed by a space followed by the 5th field from the current input record followed by a newline character. And then the:
does read one line in a loop until the end-of-file is detected on the input pipe and sets the shell variables FILE_NAME and RAND_SET to the two values written by awk. And, as you said, the loop processes each line of output from awk to move the appropriate files into their desired places.
Quote:
Originally Posted by LMHmedchem
From time to time I think I am getting better at this, then I try to do something new and find out how much I still don't know. I do really appreciate that help that is available here.
LMHmedchem
That's what we're here for. Don't be afraid to experiment. If you have a loop like this and want to see what it will do without actually copying files, put an echo in front of the cp to have the script show you what it will do when you remove the echos.
Get used to using:
to surround segments of shell code that you don't understand so you can see what commands are being called and what operands are being passed to them.
Hello all
im using allot with the method of getting file list from misc place in unix and copy them into text file
and then doing misc action on this list of files using
foreach f (`cat file_list.txt`)
do something with $f
end
can I replace this file_list.txt with some place in memory? ... (1 Reply)
I have the files logged in the file system with names in the format of : filename_ordernumber_date_time
eg:
file_1_12012007_1101.txt
file_2_12022007_1101.txt
file_1_12032007_1101.txt
I need to find out all the files that are logged multiple times with same order number. In the above eg, I... (1 Reply)
Hi,
I have list of files as following:
/home/abc/x/23344.php
/home/axx/zz/ddddd/abc/7asda/2434.php
/home/zzz/7x/y/114.php
/home/assssc/x/yasyday/23664.php
( last part in each line is <somenumber.php>
I need to somehow get this from the above:
/home/abc/x/... (6 Replies)
Please see how can I do this:
File A (three columns):
X1,Y1,1.01
X2,Y2,2.02
X3,Y3,4.03
File B (three columns):
X1,Y1,1
X2,Y2,2
X3,Y3,4.0005
Now I have to compare file A and B based on the integer part of column 3. Means first 2 rows should be OK and the third row should not satisfy... (12 Replies)
Hi
i have a file which has mutiple line in it.
inside that i have a pattern similar to this
/abc/def/hij
i want to fine the pattern starting with "/" and get the first word in between the the symbols "/" i.e. "abc" in this case into a variable.
thanks in advance (13 Replies)
Hi Gurus,
I need to list only the files with out certain extension.
For eg from the following list of files:
I need to only list:
Thanks
Shash (7 Replies)
Hello,
I have a huge directory (with millions of files) and need to find out duplicates based on BOTH file name and File size.
I know fdupes but it calculates MD5 which is very time-consuming and especially it takes forever as I have millions of files.
Can anyone please suggest a script or... (7 Replies)
I am trying to modify the "corestat v1.1" code which is in Perl.The typical output of this code is below:
Core Utilization
CoreId %Usr %Sys %Total
------ ----- ----- ------
5 4.91 0.01 4.92
6 0.06 ... (0 Replies)
Hi,
I have a directory with a lot of files like this:
a.bam
b.bam
c.bam
I like to rename these files based on a list where the name of the files in the first column will be replasced by the names in the second column. Here is my list which is a tab-delimited text file:
a x
b y
c ... (4 Replies)