grep/awk column or variable?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting grep/awk column or variable?
# 1  
Old 01-29-2011
grep/awk column or variable?

Hi, I'm running via PuTTY, in a BASH shell to do my work. I'm running calculations where steps are reported like this every 100 steps:



Code:
NSTEP =  249900   TIME(PS) = 249.900  TEMP(K) =   299.94  PRESS = 21.1
 Etot   =    -12912.5557  EKtot   =      4996.8780  EPtot      =    -17909.4336
 BOND   =      3502.4858  ANGLE   =        39.5581  DIHED      =        13.5982
 1-4 NB =         2.4191  1-4 EEL =        71.3522  VDWAALS    =      3805.6043
 EELEC  =    -25344.4513  EHBOND  =   0.0000  RESTRAINT  =         0.0000
 EKCMT  =      1624.0988  VIRIAL  =    1599.4088  VOLUME    = 54265.8517
                                                    Density    =         1.0280
 Ewald error estimate:   0.2435E-03
 ------------------------------------------------------------------------------


 NSTEP = 250000  TIME(PS) = 250.000  TEMP(K) = 300.29  PRESS =   193.8
 Etot   =    -12881.4528  EKtot   =      5002.6739  EPtot      =    -17884.1267
 BOND   =      3487.9960  ANGLE   =        41.0739  DIHED      =        11.4793
 1-4 NB =         1.8325  1-4 EEL =        68.9061  VDWAALS    =      3810.8825
 EELEC  =    -25306.2971  EHBOND  =   0.0000  RESTRAINT  =         0.0000
 EKCMT  =  1674.6162  VIRIAL  =  1447.5445  VOLUME     =     54268.4640
                                                    Density    =         1.0280
 Ewald error estimate:   0.2073E-03
 ------------------------------------------------------------------------------

I need to create data files showing the Etot (for example) over all 2500 entries. I've tried grep "Etot" filename > newfile but then I get something like this:
Code:
 Etot   =     -8020.5755  EKtot   =         0.0000  EPtot      =     -8020.5755
 Etot   =     -7327.1765  EKtot   =       359.4934  EPtot      =     -7686.6698
 Etot   =     -6736.4171  EKtot   =       625.2909  EPtot      =     -7361.7080
 Etot   =     -6290.5101  EKtot   =       865.9846  EPtot      =     -7156.4947
 Etot   =     -5896.3326  EKtot   =      1039.1907  EPtot      =     -6935.5234
 Etot   =     -5563.3851  EKtot   =      1213.5608  EPtot      =     -6776.9459
 Etot   =     -5307.6959  EKtot   =      1316.2096  EPtot      =     -6623.9055

So far, I've just been marking and killing the columns I don't need (ctrl+space at top, ctrl+x at bottom opposite corner of column,r,k), but it's slow when you have 2500 lines. I'm coming up on data that will have 10000 lines and I need to do this quicker. I desire something like this ( which would just be the data for only the Etot column, minus the Etot = portion):

Code:
-8020.5755
-7327.1765  
-6736.4171  
-6290.5101  
-5896.3326  
-5563.3851  
-5307.6959

The only reason I mention variables here is that it might be saving it as a variable when it reports it to the output file. I'm not sure though, since I'm still a newbie at Unix
Moderator's Comments:
Mod Comment
Please use code tags when posting data and code samples!

Last edited by vgersh99; 01-29-2011 at 12:01 PM.. Reason: code tags, please!
# 2  
Old 01-29-2011
Code:
nawk '/Etot/ {print $3}' myFile

# 3  
Old 01-29-2011
Alright thanks. Also, I apologize for not using code boxes.

The line that produced the results I wanted was:

Code:
awk '/Etot/ {print $3}' myFile > newfile.dat

---------- Post updated at 12:59 PM ---------- Previous update was at 09:40 AM ----------

Okay so while that command worked for Etot, I still don't understand how the command does what it does. I cannot make it work for VOLUME, EPtot, PRESS, or TEMP.
# 4  
Old 01-29-2011
Hi,

Try this 'perl' script:

Code:
$ cat script.pl
use strict;
use warnings;

die "Usage: $0 <word> <file>\n" if  @ARGV < 2; 

my $column = $ARGV[0]; 
open my $file, "<", $ARGV[1] or die "Cannot open file: $!\n";

while (<$file>) { 
    print "$1\n" if /\b${column}\s*=\s*(.*?)\s+/i;
}
$ perl script.pl 
Usage: script.pl <word> <file>
$ perl script.pl etot infile 
-12912.5557
-12881.4528

Regards,
Birei
# 5  
Old 01-29-2011
Code:
bash-2.05$ nawk 'match($0,str2find " *= *") {str=substr($0,RSTART+RLENGTH);print substr($0,RSTART+RLENGTH,index(str," ")-1)}' str2find=Etot myFile
-12912.5557
-12881.4528
bash-2.05$
bash-2.05$ nawk 'match($0,str2find " *= *") {str=substr($0,RSTART+RLENGTH);print substr($0,RSTART+RLENGTH,index(str," ")-1)}' str2find=EKtot myFile
4996.8780
5002.6739

# 6  
Old 01-29-2011
Short and Sweet

Okay so thank you for all the suggestions, but I figured out how to make the awk command work precisely as needed. I've always subscribed to the thought that simpler is better.

Here's what I got:

Code:
awk '/VOLUME/ {print $9}' file-to-look-in.file > datafile.dat

For any other newbies, what this does is says:
1) look for lines containing the word "VOLUME"
2) print to screen the 9th variable (text field). For some reason in my results, $1 would correspond to EKCMT, $2 is the = sign, $3 is 1674.6162, etc. to $9 which is the value of VOLUME that I'm wanting.
3) look for all of this data in file-to-look-in.file, then pipe (send to new file) it to a new file datafile.dat

If you wrote something like
Code:
awk < 'argument-here' FileToLookIn.file

it would print results straight to your screen, not saving it to a file.
# 7  
Old 01-30-2011
With this command you do not need to know the field number on a line:
Code:
awk '/Etot/&&getline{print $1}' RS== infile

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to extract value from column using variable

I am having trouble extracting the value in the columns declared in a variable. I have tried several different variation of awk but both give me the column number and not the actual value of that column. Any suggestions? Neither of the "extract" variables below are performing as desired ... (5 Replies)
Discussion started by: ncwxpanther
5 Replies

2. Shell Programming and Scripting

How to awk or grep the last column in file when date on column contains spaces?

Hi have a large spreadsheet which has 4 columns APM00111803814 server_2 96085 Corp IT Desktop and Apps APM00111803814 server_2 96085 Corp IT Desktop and Apps APM00111803814 server_2 96034 Storage Mgmt Team APM00111803814 server_2 96152 GWP... (6 Replies)
Discussion started by: kieranfoley
6 Replies

3. Shell Programming and Scripting

Grep/awk part of info of a column

I have a question that I am at a loss to solve. I have 3 column tab-separated data, such as: abs nmod+n+n-commitment-n 349.200023 abs nmod+n+n-a-commitment-n 333.306429 abs into+ns-j+vn-pass-rb-divide-v 295.57316 abs nmod+n+ns-commitment-n 182.085018 abs nmod+n+n-pledge-n ... (2 Replies)
Discussion started by: owwow14
2 Replies

4. Shell Programming and Scripting

Use grep/awk to remove part of column

hi all, how can i use grep or awk to clean the following input data: n<>the<>96427210 861521305 123257583 n<>obj<>79634223 861521305 79634223 n<>nmod<>68404733 861521305 68422718 where the desired results is to remove all non-numeric characters?: 96427210 861521305 123257583 ... (5 Replies)
Discussion started by: owwow14
5 Replies

5. Shell Programming and Scripting

awk or grep to search one column and output the other

Hello, it would be great if someone can help me with the following: I want to search for the rows from fileA in column 1 of fileB and output column 2 of fileB if found in fileC. In the moment I search within the complete file. How can I change the code so only column 1 is searched? cat fileA... (7 Replies)
Discussion started by: Manyaka
7 Replies

6. Shell Programming and Scripting

awk arrays - compare value in second column to variable

Hello, I am trying to redirect files to a directory by using a config file. The config files is as such: xxxxxx,ID,PathToDirectory xxxxxx,ID2,PathToDirectory2 and so on... I have a variable that should match one of these IDs. I want to load this config file into an awk array, and... (2 Replies)
Discussion started by: jrfiol
2 Replies

7. Shell Programming and Scripting

Printing a variable column using awk

Hi everyone, Ok here's the scenario. I have a control file like this. component1,file1,file2,file3,file4,file5 component2,file1,file2,file3,file4,file5I want to do a while loop here to read all files for each component. file_count=2 while ] do file_name=`cat list.txt | grep... (2 Replies)
Discussion started by: The Gamemaster
2 Replies

8. Shell Programming and Scripting

grep/awk on a single column in a for loop

Hi I'm trying to loop through a small list of id's and then pull out a few columns if the id matches that found in column 2 of the larger file. I managed to get one command to work awk -F " " '{if ($2 == '154080196') print $2,$3,$4}' tst.txt | less However, when I try it in a for loop I... (3 Replies)
Discussion started by: flotsam
3 Replies

9. Shell Programming and Scripting

Row to column converter using Awk or grep?

Hello, Can someone please help me on this.:confused: I have a file which has more than 1 million lines (XML file). What I need is: Search for "abcd" in the input file > output the result into a output.txt (colloum1) Search for "efghi" in the input file > output the result in to... (3 Replies)
Discussion started by: Needhelp2
3 Replies

10. Shell Programming and Scripting

can awk print column using a variable ??

i want to print the column file using awk or cut in dynamic manner like trmp=2;temp1=1;temp3=2 awk 'BEGIN{OFS=IFS="\t"} {print $temp,$temp1,$temp3}' client_data.txt or cut -f $temp1,$temp2,$temp3 -d"\t" file_name . but it is showing error , In awk can i use variable as in printing... (36 Replies)
Discussion started by: jambesh
36 Replies
Login or Register to Ask a Question