This is why I often don't post what I want to do in its entirety. It seems that most of the time when I post a long script, no one seems to want to wade into it (which is quite understandable). I guess I still need to work on making posts that are long enough to convey what I am asking and get a workable solution, but short enough that they will actually be read.
I modified the code that you posted and have something that gives me what I need. This is the modified code,
Code:
#!/bin/bash
# argument $1 is the field to sort on based on file names as below
# 53.96_E3000_50.19_E2200_35_ri_OA_f0_S1C_v17_52.26.1_4_ON_0.25lr.out.txt
# 1 2 3 4 5
# argument $2 is the file count, meaning the number of files to find and copy
# argument $3 is the set type EV/CV
# for the top 10 EV outcomes call as ./01_copy_top_outcomes.sh 1 10 EV
# for the top 10 CV outcomes call as ./01_copy_top_outcomes.sh 3 10 CV
USAGE="./script_name sort_field file_count set_type"
# field to sort on
KEY_FIELD=$1
# number of files to find
FILE_COUNT=$2
# processing set type EV/CV
SET_TYPE=$3
# make sure there are 3 arguments and the first two are numberss
if [ $# -ne 3 ] || [ "$1" != "${2%*[^0-9]*}" ] || [ "$2" != "${3%*[^0-9]*}" ]
then echo "$USAGE"
exit 1
fi
# loop on all folds
FOLDS=(f0 f1 f2 f3 f4 f5 f6 f7 f8 f9)
for FOLD in ${FOLDS[@]}
do
# check if the directory exists, this should never throw.
if [[ ! -d "$FOLD" ]]
then echo 'directory' $FOLD'/ does not exist, exit script'
exit 1
fi
# change directory to current fold
cd $FOLD
echo "processing" $FOLD
#re-initalize
OUTPUT=""
FILE_TEMP=""
FILE_NAME=""
RAND_SET=""
# sort the list of filenames and output the top number "n" as specified in argument $3
FILE_LIST=( $(df -h | ls *.out.txt | sort -t_ -k$KEY_FIELD,$KEY_FIELD'n' | awk -F_ -v f=$KEY_FIELD -v c=$FILE_COUNT 'NR > c {exit} {printf("%s", $0)}') )
# loop up to file count to parse output and copy files that were found by sort
for (( LOOP_CT=1; LOOP_CT<=$FILE_COUNT; LOOP_CT++ ))
do
# parse output string on .out.txt to locate individual files
FILE_TEMP=`echo $FILE_LIST | awk -v N=$LOOP_CT 'BEGIN {FS=".out.txt"} {print $N}'`
# restore file extension
FILE_NAME=$FILE_TEMP'.out.txt'
echo $FILE_NAME
# copy file and corresponding ini weight set to continue
# copy file to continue
cp -p './'$FILE_NAME './'$FOLD'_continue/'$SET_TYPE'/'$FILE_NAME
# find random ini set number
RAND_SET=`echo $FILE_NAME | awk 'BEGIN {FS="_"} {print $5}'`
# copy random ini weight file to continue
cp -p '../rnd_ini/'$FOLD'/ri_'$RAND_SET'_'*'.wts' './'$FOLD'_continue/'$SET_TYPE'/'
done
# return to start directory
cd ../
done
The entire list of files that is found ends up in the variable FILE_LIST, so that gets parsed into individual file names. Those files are copied to the proper location and an associated file is also located an copied. This loops through all sub folders f0-f9, so that is no longer an argument.
This seems reasonable and works, but I don't know awk well enough to see if there are any hidden problems. There is probably an easier way to copy the files I need, but I don't know how to copy in awk, so I needed to get the file names in bash variables that I know how to manipulate to some extent.
Do you see anything dreadfully wrong here? This does give me the ablity to specify the sort field and the number of files I want, which is a big improvement over what I first posted.
Hello all
im using allot with the method of getting file list from misc place in unix and copy them into text file
and then doing misc action on this list of files using
foreach f (`cat file_list.txt`)
do something with $f
end
can I replace this file_list.txt with some place in memory? ... (1 Reply)
I have the files logged in the file system with names in the format of : filename_ordernumber_date_time
eg:
file_1_12012007_1101.txt
file_2_12022007_1101.txt
file_1_12032007_1101.txt
I need to find out all the files that are logged multiple times with same order number. In the above eg, I... (1 Reply)
Hi,
I have list of files as following:
/home/abc/x/23344.php
/home/axx/zz/ddddd/abc/7asda/2434.php
/home/zzz/7x/y/114.php
/home/assssc/x/yasyday/23664.php
( last part in each line is <somenumber.php>
I need to somehow get this from the above:
/home/abc/x/... (6 Replies)
Please see how can I do this:
File A (three columns):
X1,Y1,1.01
X2,Y2,2.02
X3,Y3,4.03
File B (three columns):
X1,Y1,1
X2,Y2,2
X3,Y3,4.0005
Now I have to compare file A and B based on the integer part of column 3. Means first 2 rows should be OK and the third row should not satisfy... (12 Replies)
Hi
i have a file which has mutiple line in it.
inside that i have a pattern similar to this
/abc/def/hij
i want to fine the pattern starting with "/" and get the first word in between the the symbols "/" i.e. "abc" in this case into a variable.
thanks in advance (13 Replies)
Hi Gurus,
I need to list only the files with out certain extension.
For eg from the following list of files:
I need to only list:
Thanks
Shash (7 Replies)
Hello,
I have a huge directory (with millions of files) and need to find out duplicates based on BOTH file name and File size.
I know fdupes but it calculates MD5 which is very time-consuming and especially it takes forever as I have millions of files.
Can anyone please suggest a script or... (7 Replies)
I am trying to modify the "corestat v1.1" code which is in Perl.The typical output of this code is below:
Core Utilization
CoreId %Usr %Sys %Total
------ ----- ----- ------
5 4.91 0.01 4.92
6 0.06 ... (0 Replies)
Hi,
I have a directory with a lot of files like this:
a.bam
b.bam
c.bam
I like to rename these files based on a list where the name of the files in the first column will be replasced by the names in the second column. Here is my list which is a tab-delimited text file:
a x
b y
c ... (4 Replies)
I have a expect file like this
#!/opt/tools/unsupported/expect-5.39/bin/expect
spawn ssh -l user ip
expect_after eof {exit 0}
set timeout 10
log_file /report.txt
expect "Password:" { send "pasword\r" }
expect "$ " { send "date\r" }
expect "$ " { send "readlink /somelink\r" }
set... (7 Replies)
Discussion started by: bebehnaz
7 Replies
LEARN ABOUT CENTOS
fold
FOLD(1) User Commands FOLD(1)NAME
fold - wrap each input line to fit in specified width
SYNOPSIS
fold [OPTION]... [FILE]...
DESCRIPTION
Wrap input lines in each FILE (standard input by default), writing to standard output.
Mandatory arguments to long options are mandatory for short options too.
-b, --bytes
count bytes rather than columns
-c, --characters
count characters rather than columns
-s, --spaces
break at spaces
-w, --width=WIDTH
use WIDTH columns instead of 80
--help display this help and exit
--version
output version information and exit
GNU coreutils online help: <http://www.gnu.org/software/coreutils/> Report fold translation bugs to <http://translationproject.org/team/>
AUTHOR
Written by David MacKenzie.
COPYRIGHT
Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
SEE ALSO
The full documentation for fold is maintained as a Texinfo manual. If the info and fold programs are properly installed at your site, the
command
info coreutils 'fold invocation'
should give you access to the complete manual.
GNU coreutils 8.22 June 2014 FOLD(1)