Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Getting information from various files
Top Forums Shell Programming and Scripting Getting information from various files Post 302489932 by kristinu on Saturday 22nd of January 2011 05:45:03 PM
Old 01-22-2011
Getting information from various files
I have got this piece of csh code that looks into various log files and outputs some parameters

For example, I might have 4 files and want to grep for the lines containing "Best Value"

npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw.log
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run2.log
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run3.log
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run4.log

Doing
Code:
grep "Best Value" *dc0p004-16x12drw*.log

gives

Code:
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw.log:0. Best Value   = 0.0348089
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw.log:1. Best Value   = 0.0348089
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw.log:2. Best Value   = 0.0347595
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw.log:3. Best Value   = 0.0347595
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw.log:4. Best Value   = 0.0343211
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw.log:5. Best Value   = 0.0343211
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw.log:6. Best Value   = 0.034262
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw.log:7. Best Value   = 0.034262
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw.log:8. Best Value   = 0.034262
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw.log:9. Best Value   = 0.0341526
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw.log:10. Best Value   = 0.0341407
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw.log:11. Best Value   = 0.0341101
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw.log:12. Best Value   = 0.0340129
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw.log:13. Best Value   = 0.0340129
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw.log:14. Best Value   = 0.0339073
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw.log:15. Best Value   = 0.0339073
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run2.log:0. Best Value   = 0.0339073
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run2.log:1. Best Value   = 0.0338745
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run2.log:2. Best Value   = 0.0338223
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run2.log:3. Best Value   = 0.0338054
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run2.log:4. Best Value   = 0.033714
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run2.log:5. Best Value   = 0.033714
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run2.log:6. Best Value   = 0.033714
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run2.log:7. Best Value   = 0.0336161
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run2.log:8. Best Value   = 0.0336161
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run2.log:9. Best Value   = 0.0336161
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run2.log:10. Best Value   = 0.0336161
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run2.log:11. Best Value   = 0.0336161
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run2.log:12. Best Value   = 0.0336161
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run2.log:13. Best Value   = 0.0336161
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run2.log:14. Best Value   = 0.0336161
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run3.log:0. Best Value   = 0.0336161
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run3.log:1. Best Value   = 0.0336147
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run3.log:2. Best Value   = 0.0334466
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run3.log:3. Best Value   = 0.0334466
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run3.log:4. Best Value   = 0.0334466
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run3.log:5. Best Value   = 0.0334184
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run3.log:6. Best Value   = 0.0333759
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run3.log:7. Best Value   = 0.0333759
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run3.log:8. Best Value   = 0.0333759
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run4.log:0. Best Value   = 0.0333759
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run4.log:1. Best Value   = 0.0333759
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run4.log:2. Best Value   = 0.0333759
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run4.log:3. Best Value   = 0.0333041
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run4.log:4. Best Value   = 0.0333041
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run4.log:5. Best Value   = 0.0333041
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run4.log:6. Best Value   = 0.0332733
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run4.log:7. Best Value   = 0.0332733
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run4.log:8. Best Value   = 0.0332733
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run4.log:9. Best Value   = 0.033229
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run4.log:10. Best Value   = 0.033229
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run4.log:11. Best Value   = 0.033229
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run4.log:12. Best Value   = 0.033229
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run4.log:13. Best Value   = 0.033229


Now I want to have something like this stored in a file

Code:
0. Best Value = 0.0348089 
1. Best Value = 0.0348089
2. Best Value = 0.0347595
3. Best Value = 0.0347595
4. Best Value = 0.0343211
5. Best Value = 0.0343211
6. Best Value = 0.034262
7. Best Value = 0.034262
8. Best Value = 0.034262
9. Best Value = 0.0341526
10. Best Value = 0.0341407
11. Best Value = 0.0341101
12. Best Value = 0.0340129
13. Best Value = 0.0340129
14. Best Value = 0.0339073
15. Best Value = 0.0339073
0. Best Value = 0.0339073
1. Best Value = 0.0338745
2. Best Value = 0.0338223
3. Best Value = 0.0338054
4. Best Value = 0.033714
5. Best Value = 0.033714
6. Best Value = 0.033714
7. Best Value = 0.0336161
8. Best Value = 0.0336161
9. Best Value = 0.0336161
10. Best Value = 0.0336161
11. Best Value = 0.0336161
12. Best Value = 0.0336161
13. Best Value = 0.0336161
14. Best Value = 0.0336161
0. Best Value = 0.0336161
1. Best Value = 0.0336147
2. Best Value = 0.0334466
3. Best Value = 0.0334466
4. Best Value = 0.0334466
5. Best Value = 0.0334184
6. Best Value = 0.0333759
7. Best Value = 0.0333759
8. Best Value = 0.0333759
0. Best Value = 0.0333759
1. Best Value = 0.0333759
2. Best Value = 0.0333759
3. Best Value = 0.0333041
4. Best Value = 0.0333041
5. Best Value = 0.0333041
6. Best Value = 0.0332733
7. Best Value = 0.0332733
8. Best Value = 0.0332733
9. Best Value = 0.033229
10. Best Value = 0.033229
11. Best Value = 0.033229
12. Best Value = 0.033229
13. Best Value = 0.033229

As one may notice the 0. value occurs multiple times. I need to remove the entry 0. except on the first occurance which is in the first line

The code below actually does that.

However if I have only one file, the script does not work. Also I want to change the file naming system to

Code:
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run1.log
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run2.log
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run3.log
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw-run4.log

which means that if the log files are a continuation of runs, the first run is tagged with -run1- as well. Of course, if only one run is done the lof file will still be
npt02-z30-sr65-rgdt0p50-dc0p004-16x12drw.log, i.e. without the -run1- tag.

I would also need to run the script on such a file and create a file with the parameters of interest.

The csh code is shown below, but I am getting confused how to change it so that I will pass multiple files all having a -run- tag. The code below ignored the -run- tag as currently the first run never has the -run1- tag.

Code:
    foreach f ($fnames)

      echo "(I) ${f}.log, (O) ${f}.mis"

      set fdrwtag = `echo $f | awk '/drw/'`
      set fsmptag = `echo $f | awk '/smp/'`

      if ($fdrwtag != "") then

        grep "Rms Value" $f*.log              \
          | awk '{ if ($0 !~ /run[0-9]+\.log:0\. /)  \
                   { sub(/[^:]*:[0-9]*\.[ \t]*/,x); print i++". " $0 }  \
                 }' > $f.mis

        grep "Best Value" $f*.log             \
          | awk '{ if ($0 !~ /run[0-9]+\.log:0\. /)  \
                   { sub(/[^:]*:[0-9]*\.[ \t]*/,x); print i++". " $0 }  \
                 }' >> $f.mis

      else if ($fsmptag != "") then

        grep "Best Value" $f*.log             \
          | awk '{ if ($0 !~ /run[0-9]+\.log:0\. /)  \
                   { sub(/[^:]*:[0-9]*\.[ \t]*/,x); print i++". " $0 }  \
                 }' >> $f.mis

    end

 

10 More Discussions You Might Find Interesting

1. Linux

Need to store information relating to certain files

I need to save information relating to certain files that are projected to deleted. I am using find $defPath/archive/sub/subchild -type f -mtime +365 > LOGFILE cat LOGFILE| while read line do ls -l $line | cut -d" " -f 10-15 done But, since cut is cutting columns on space (single... (3 Replies)
Discussion started by: ramu_indian
3 Replies

2. Solaris

To get the hidden files information only

Hi all, I want to get only hidden files(which are start with '.' or '..') information in a current directory. I tried the below command, $ find . -name "^." -exec ls -la '{}' \; but it's not working. Can anyone give me your outputs. Thanks in advance, Raghu. (5 Replies)
Discussion started by: raghu.iv85
5 Replies

3. UNIX for Dummies Questions & Answers

Prepending information from filenames into files

I would like to know how to take information from a filename and place it into the text of the same file. Let's say I have a file called height_2_width_1.txt containing data that is related to a height of 2 and a width of 1, and the text originally looks like this: where these two columns... (13 Replies)
Discussion started by: Scatterbrain26
13 Replies

4. Shell Programming and Scripting

Compare two files and add new information

Hi, I want to compare two fields in two different files and add a corresponding field in a third output file. Something similar to vlookup. Please see the attached input files and the example output file. I want to compare each entry in column 1 in file1 with column 5 in file2. If both the... (6 Replies)
Discussion started by: chabook
6 Replies

5. UNIX for Dummies Questions & Answers

Combining information from Excel files

Hi, I am looking for an AWK or grep script (join will not work here since the data is not sorted) to combine two Excel files that look lke this: Infile1: Georgia Atlanta 1234 1234 Georgia Marrieta 2134 2134 Georgia Scottdale 3414 3414 Georgia Clarkston 2321 2321 Infile2: ... (6 Replies)
Discussion started by: Xterra
6 Replies

6. UNIX for Dummies Questions & Answers

How to copy files with only certain information

I need to copy a list of files in directory and only files with content DUMMY_B should be copy over to a new directory. How can i do that ? Thanks a lot. (2 Replies)
Discussion started by: Alyssa
2 Replies

7. Shell Programming and Scripting

Updated files information

I am using the below script to remove the rows which contains null values in the 3rd column.My requirement here is want to get the filenames which row is removed .please help me. #!/usr/bin/sh Scripts=/ushhquest/data001/Scripts cd /ushhquest/data011/TgtFiles/MonthlyData ls CUSTADDR*.txt >... (4 Replies)
Discussion started by: katakamvivek
4 Replies

8. Shell Programming and Scripting

search information in multiple files and save in new files

hi everyone, im stuck in here with shell :) can you help me?? i have a directory with alot files (genbank files ... all ended in .gbk ) more than 1000 for sure ... and i want to read each one of them and search for some information and if i found the right one i save in new file with new... (6 Replies)
Discussion started by: andreia
6 Replies

9. UNIX for Beginners Questions & Answers

Combine information from 2 files

Hi there, I‘m a newbie in linux (ubuntu) working with several files, some of them containing hundred thousands of lines. I started to extract information out of 2 files, combining them by 1 column: I need a Vlookup-like command that reads sampleID (column 2)(line 2,..line by line) in file 1, looks... (1 Reply)
Discussion started by: Nika
1 Replies

10. UNIX for Beginners Questions & Answers

Matching information from different files

I have a dataset with coordinates of certain points with their names. file1: AAA 5 7 BBB 7 2 CCC 4 4 DDD 4 4 EEE 8 9 FFF 9 9 GGG 4 9 HHH 32 1 III 7 6 JJJ 12 4 KKK 12 3 MMM 15 3The letters represent names of points. In file1 they are unique. I also have file2 with a dataset with... (2 Replies)
Discussion started by: maya3
2 Replies
All times are GMT -4. The time now is 08:04 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy