Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk- reading input file twice

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk- reading input file twice
# 8  
Old 04-26-2011
Quote:
Originally Posted by Franklin52
After the sort command this command should be suffice:
Code:
... | sort | 
awk '{a[++c]=$1; b[c]+=$2}END{for(i=1;i<=c;i++)print a[i], b[i]/b[c]}'


Doesn't work like it used to with the other awk before the pipe... it prints the first and second columns without any change.
# 9  
Old 04-26-2011
Can you post the output after the sort command?
Please use code tags.
# 10  
Old 04-26-2011
I´m actually using quite a huge amount of information, the output looks something like this:

Code:
0      18767
0.000999928      87183
0.00100017      38975
0.00199986      80390
0.00200009      124310
0.00299978      8133
0.00300002      84539
0.00399995      40803
0.00400019      11242
0.00499988      21893
0.00500011      23798
0.0059998      8121
0.00600004      37201
0.00699997      24423
0.00700021      3676
0.0079999      10935
0.00800014      8433
0.00899982      3501



I'm attaching a file with the whole output, if needed.
sort_output.txt (2.1 KB) 
# 11  
Old 04-26-2011
This is my output based on the snippet of the output after the last pipe:
Code:
$ cat file
0      18767
0.000999928      87183
0.00100017      38975
0.00199986      80390
0.00200009      124310
0.00299978      8133
0.00300002      84539
0.00399995      40803
0.00400019      11242
0.00499988      21893
0.00500011      23798
0.0059998      8121
0.00600004      37201
0.00699997      24423
0.00700021      3676
0.0079999      10935
0.00800014      8433
0.00899982      3501
$
$ cat file | awk '{a[++c]=$1; b[c]+=$2}END{for(i=1;i<=c;i++)print a[i], b[i]/b[c]}'
0 5.36047			# 18767/3501
0.000999928 24.9023		# 87183/3501
0.00100017 11.1325
0.00199986 22.962
0.00200009 35.507
0.00299978 2.32305
0.00300002 24.1471
0.00399995 11.6547
0.00400019 3.21108
0.00499988 6.25336
0.00500011 6.79749
0.0059998 2.31962
0.00600004 10.6258
0.00699997 6.97601
0.00700021 1.04999
0.0079999 3.12339
0.00800014 2.40874
0.00899982 1
$

# 12  
Old 04-26-2011
Hmm yeah but the point is that the original output after the awk that got deleted

awk '{print $1, p += $2}'
was as follows:

Code:
0 18767
0.000999928 105950
0.00100017 144925
0.00199986 225315
0.00200009 349625
0.00299978 357758
0.00300002 442297
0.00399995 483100
0.00400019 494342
0.00499988 516235
0.00500011 540033
0.0059998 548154



and the new awk is not doing this sum properly.... since each value on the second column is an accumulation.

The final step there would be only to divide each value by the last (so the final accumulation) value on the second column, and I can't seem to achieve that.

The output of the second column should be in a range from something to 1.... since well of course the last value would be divided by itself
# 13  
Old 04-26-2011
Try this...
Code:
##--get unique tags
for i in ` cat testfile.txt | awk  '{print $1}'|sort -u`
do
grep $i testfile.txt >temp.txt
cat temp.txt | sort -n |tail -1  >>finaldata.txt
done

# 14  
Old 04-26-2011
Try this one:
Code:
awk '{a[++c]=$1; b[c]=b[c-1]+$2;t+=$2}END{for(i=1;i<=c;i++)print a[i],b[i]/t}' file

This User Gave Thanks to Franklin52 For This Post:
acsg
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Open Source

Splitting files using awk and reading filename value from input data

I have a process that requires me to read data from huge log files and find the most recent entry on a per-user basis. The number of users may fluctuate wildly month to month, so I can't code for it with names or a set number of variables to capture the data, and the files are large so I don't... (7 Replies)
Discussion started by: rbatte1
7 Replies

2. UNIX for Dummies Questions & Answers

User input while reading from a file

I am not able to capture the user input in this script(bash).There is prompt for user input.Could some one help me capture user input while reading afile? while read line do echo "$i" path1=$line path2=`echo $line|sed s/new_dir/old_dir/` echo "Do you want to replace?";... (4 Replies)
Discussion started by: parijat guh
4 Replies

3. Shell Programming and Scripting

Help with reading two input files in awk

Hello, I'm trying to write an awk program that reads two files inputs. example, file 1: 0.00017835 0.000176738 0.00018811 0.000189504 0.000188155 0.000180065 0.000178991 0.000178252 0.000182513 file 2: 1.7871769E-05 1.5139576E-16 1.5140196E-16 1.5139874E-16 1.7827407E-04 ... (5 Replies)
Discussion started by: joseamck
5 Replies

4. UNIX for Dummies Questions & Answers

Help in reading the date from the input file name

Hi, I need to read the date from the input file. The format of the input file is as follows: a_b_c_yyyymmdd.txt I need to read the date(yyyymmdd) part from the name of the input file. Would really appreciate if someone can help me in this regard Thanks a lot. (1 Reply)
Discussion started by: Sunny_teotia
1 Replies

5. Shell Programming and Scripting

awk script - reading input lines

Can I do something like, if($0==/^int.*$/) { print "Declaration" } for an input like: int a=5; If the syntax is right, it is not working for me, but I am not sure about the syntax. Please help. Thanks, Prasanna (1 Reply)
Discussion started by: prasanna1157
1 Replies

6. Shell Programming and Scripting

Reading from standard input with awk

Hello, Could somebody please give me an awk example on how to read from the standard input. It means as the "read" function in Korn shell. Thx in advance ... (3 Replies)
Discussion started by: rany1
3 Replies

7. Shell Programming and Scripting

Reading specific contents from 1 input files and appending it to another input file

Hi guys, I am new to AWK and unix scripting. Please see below my problem and let me know if anyone you can help. I have 2 input files (example given below) Input file 2 is a standard file (it will not change) and we have to get the name (second column after comma) from it and append it... (5 Replies)
Discussion started by: sksahu
5 Replies

8. Shell Programming and Scripting

reading input from a file

I am trying to read input for a C program (that expects input from the user) from a file using the shell command: progname < filename but it seems that the program considers the char '<' as the first input, hence causing an "error" in my program. I checked it with another program and it... (2 Replies)
Discussion started by: nadbar
2 Replies

9. Shell Programming and Scripting

Reading an Input file and searching for occurrences WIHOUT SED or AWK

Hi people. I am new to shell scripting, so I need a little help. I want to create a script named that takes an argument as a file, Read the input file and look for occurrences of the current username (say abc.xyz) who is executing the script. On finding an occurrence of the username take that line... (2 Replies)
Discussion started by: kartikkumar84@g
2 Replies

10. Shell Programming and Scripting

awk reading 2 input files but not getting expected value

I'm reading 2 input files but not getting expected value. I should get an alpha value on file_1_data but not getting any. Please help. >cat test6.sh awk ' FILENAME==ARGV { file_1_data=$0; print "----- 1 Line " NR " -----" $1; next } FILENAME==ARGV { file_2_data=$0; print "----- 2... (1 Reply)
Discussion started by: pdtak
1 Replies
Login or Register to Ask a Question