Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Process a specific number of files ina list
Top Forums Shell Programming and Scripting Process a specific number of files ina list Post 302887306 by LMHmedchem on Thursday 6th of February 2014 06:02:30 PM
Old 02-06-2014
Process a specific number of files ina list
Hello,

I have a list of files that was created with,

Code:
FILES='./'$FOLD'/'$FOLD'_continue/'$OPTIMIZE_ON'/'*'out.txt'

I am doing a loop on this list

Code:
for INPUT in $FILES
do
...
done

but I may not want to process everything. Is there a simple way to just process the first 5,10,n, etc in this list.

I process each file name with awk to extract two numbers from each file name,
Code:
#remove path from filename
FILENAME=`echo $INPUT | awk 'BEGIN {FS="/"} {print $5}'`
echo $FILENAME

# store random ini set
RAND_SET=`echo $FILENAME | awk 'BEGIN {FS="_"} {print $5}'`
echo "RAND_SET"  $RAND_SET

# find MAE min epoch values
MAE_EPOCH=`echo $FILENAME | awk 'BEGIN {FS="_"} {print $2}'`

#remove leading E
MAE_EPOCH=${MAE_EPOCH#"E"}
#create value at -100 epochs
let "MAE_EPOCH= $MAE_EPOCH"
echo "MAE_EPOCH"  $MAE_EPOCH

I end up with the numbers $MAE_EPOCH and $RAND_SET for each file. What I would really like to do is to scan all the files, extract the MAE_EPOCH, and then process the best some number, like the top 5 (based on the lowest values for MAE_EPOCH). I need to know the value of RAND_SET associated with each MAE_EPOCH value.

File names look like,
Code:
108.72_E1300_101.62_E3000_39_ri_OA_f0_S4A_v2_42.41.1_ON_0.25lr.out.txt

The text in blue is what I am capturing. I would need both the MAE_EPOCH and the corresponding RAND_SET value for each file I am going to process. I guess I would loop on the file set and then store the data for the files I need to process, but I'm not so sure how to do that kind of thing in bash.

Help would be greatly appreciated,

LMHmedchem
Last edited by Scrutinizer; 02-06-2014 at 07:14 PM.. Reason: Additional code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk command to find the count of files ina directory

hi Gurus, can anyone provide a awk command to get teh count of number of file sin a specific directory. appreciate any kind of information.. thanks (11 Replies)
Discussion started by: sish78
11 Replies

2. Shell Programming and Scripting

Retreive content between specific lines ina file

Hi I have a text file which has two sets of lines repeating for "n" number of times.Some data is printed between the two lines.I want to retrieve all the data thats there in between those two set of lines.I have the string value of those two set of lines. To be much more clearer ... (4 Replies)
Discussion started by: chennaitomcruis
4 Replies

3. UNIX for Dummies Questions & Answers

How to reduce multiple files into a specific number of files

Can anyone please let me know how do I reduce files into a specific number of files by cat'ing files? For example: 15 files must be reduced to 1 or 5 or 9 (possible values 1 to 14) (5 Replies)
Discussion started by: aryanbelank
5 Replies

4. Shell Programming and Scripting

Creating large number of files of specific size

Hi I am new to shell scripting.I want to create a batch file which creates a desired number of files with a specific size say 1MB each to consume space.How can i go about it using for loop /any other loop condition using shell script? Thanks (3 Replies)
Discussion started by: swatideswal
3 Replies

5. UNIX for Dummies Questions & Answers

Unix command to count the number of files with specific characters in name

Hey all, I'm looking for a command that will search a directory (and all subdirectories) and give me a file count for the number of files that contain specific characters within its filename. e.g. I want to find the number of files that contain "-a.jpg" in their name. All the searching I've... (6 Replies)
Discussion started by: murphysm
6 Replies

6. Shell Programming and Scripting

highly specific search and replace for a large number of files

hey guys, I have a directory with about 600 files. I need to find a specific word inside a command and replace only that instance of the word in many files. For example, lets say I have a command called 'foo' in many files. One of the input arguments of the 'foo' call is 'bar'. The word 'bar'... (5 Replies)
Discussion started by: ksubrama
5 Replies

7. Shell Programming and Scripting

Need script to count specific word and iterate over number of files.

Hi Experts, I want to know the count of specific word in a file. I have almost 600+ files. So I want to loop thru each file and get the count of the specific word. Please help me on achieving this... Many thanks (2 Replies)
Discussion started by: elamurugu
2 Replies

8. UNIX for Dummies Questions & Answers

grep specific number from a list of numbers

Hello. I have 9060 files labelled File1 to File9060. They are in numerical order. When I grep a file eg. File90 it will show me all files that contain the pattern "File90", eg File901 or File9001. I can only get specific files for File1000 or higher. How can I resolve this problem? Is there a... (5 Replies)
Discussion started by: godzilla07
5 Replies

9. UNIX for Dummies Questions & Answers

[Solved] How to find particular files ina directory?

Hi, I am trying to write a script to find some files in a directory Example: if i have files like 2014-02-01_aaaa.txt 2014-02-01_bbbb.txt 2014-02-01_cccc.txt 2014-02-01_dddd.txt and some other files how can i just check to see if there four files exits or not i tried some thing like this... (5 Replies)
Discussion started by: vikatakavi
5 Replies

10. Shell Programming and Scripting

List files with number to select based on number

Hi experts, I am using KSH and I am need to display file with number in front of file names and user can select it by entering the number. I am trying to use following command to display list with numbers. but I do not know how to capture number and identify what file it is to be used for... (5 Replies)
Discussion started by: mysocks
5 Replies

LEARN ABOUT CENTOS

grub-render-label

GRUB-RENDER-LABEL(1)                                               User Commands                                              GRUB-RENDER-LABEL(1)

NAME

       grub-render-label - generate a .disk_label for Apple Macs.

SYNOPSIS

       grub-render-label [OPTION...] [OPTIONS]

DESCRIPTION

       Render Apple .disk_label.

       -b, --bgcolor=COLOR
              use COLOR for background

       -c, --color=COLOR
              use COLOR for text

       -f, --font=FILE
              use FILE as font (PF2).

       -i, --input=FILE
              read text from FILE.

       -o, --output=FILE
              set output filename. Default is STDOUT

       -t, --text=STRING
              set the label to render

       -v, --verbose
              print verbose messages.

       -?, --help
              give this help list

       --usage
              give a short usage message

       -V, --version
              print program version

       Mandatory or optional arguments to long options are also mandatory or optional for any corresponding short options.

REPORTING BUGS

       Report bugs to <bug-grub@gnu.org>.

SEE ALSO

       The  full  documentation  for grub-render-label is maintained as a Texinfo manual.  If the info and grub-render-label programs are properly
       installed at your site, the command

              info grub-render-label

       should give you access to the complete manual.

grub-render-label (GRUB) 2.02-2ubuntu8.3                             July 2018                                                GRUB-RENDER-LABEL(1)
All times are GMT -4. The time now is 10:39 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy