Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How do I include the file being compared into calculation?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How do I include the file being compared into calculation?
# 1  
Old 12-26-2011
How do I include the file being compared into calculation?
Code:
nawk -F, 'NR==FNR{file=FILENAME;a[$1OFS$2OFS$3]++;next} a[$1OFS$2OFS$3]{if(FILENAME~file)next;b[$1OFS$2OFS$3]++;}
END{ for(i in a){if(a[i] && !b[i]){print "NEW: "i}} for(i in b){if(b[i])print i"\t\t"b[i]}}' OFS=, 123.csv *.csv

I need to include 123.csv into the equation for the total output currently it compares whatever is on 123.csv against everything else and prints the results but I need it to include itself in the total i.e


Code:
NE:20300468,SHELF:5,SLOT:5              21

currently the above output is excluding the 123.csv file

SmilieSmilieSmilie

---------- Post updated at 01:28 PM ---------- Previous update was at 05:59 AM ----------

Please help?
# 2  
Old 12-26-2011
Code:
nawk -F, 'NR==FNR{a[$1OFS$2OFS$3]++;next} a[$1OFS$2OFS$3]{b[$1OFS$2OFS$3]++;}
END{ for(i in a){if(a[i] && !b[i]){print "NEW: "i}} for(i in b){if(b[i])print i"\t\t"b[i]}}' OFS=, 123.csv *.csv

--ahamed
This User Gave Thanks to ahamed101 For This Post:
llcooljatt
# 3  
Old 12-26-2011
Quote:
Originally Posted by ahamed101
Code:
nawk -F, 'NR==FNR{a[$1OFS$2OFS$3]++;next} a[$1OFS$2OFS$3]{b[$1OFS$2OFS$3]++;}
END{ for(i in a){if(a[i] && !b[i]){print "NEW: "i}} for(i in b){if(b[i])print i"\t\t"b[i]}}' OFS=, 123.csv *.csv

--ahamed


Original Script

Code:
#nawk -F, 'NR==FNR{file=FILENAME;a[$1OFS$2OFS$3]++;next} a[$1OFS$2OFS$3]{if(FILENAME~file)next;b[$1OFS$2OFS$3]++;}
 END{ for(i in a){if(a[i] && !b[i]){print "NEW: "i}} for(i in b){if(b[i])print i"\t\t"b[i]}}' OFS=, 20111125.csv *.csv
NEW: NE:564867,SHELF:10,SLOT:1
NEW: NE:565229,SHELF:4,SLOT:3
NEW: NE:507423,SHELF:6,SLOT:5
NEW: NE:508089,SHELF:7,SLOT:6
NEW: NE:557688,SHELF:10,SLOT:1
NEW: NE:985068,SHELF:2,SLOT:6
NEW: NE:503703,SHELF:16,SLOT:2
NEW: NE:249454,SHELF:2,SLOT:2
NE:556416,SHELF:9,SLOT:1                4
NE:565229,SHELF:7,SLOT:6                2
NE:20173322,SHELF:3,SLOT:1              1
NE:572549,SHELF:8,SLOT:6                5
NE:600866,SHELF:8,SLOT:3                17
NE:508089,SHELF:1,SLOT:6                1
NE:991626,SHELF:4,SLOT:3                3
NE:20159466,SHELF:6,SLOT:2              3
NE:508539,SHELF:7,SLOT:4                2
NE:506443,SHELF:17,SLOT:4               2
NE:20173322,SHELF:2,SLOT:5              1
NE:230388,SHELF:10,SLOT:2               1
NE:557688,SHELF:3,SLOT:1                3
NE:556029,SHELF:5,SLOT:2                1
NE:503284,SHELF:6,SLOT:3                13
NE:597829,SHELF:8,SLOT:3                3



Second script


Code:
nawk -F, 'NR==FNR{a[$1OFS$2OFS$3]++;next} a[$1OFS$2OFS$3]{b[$1OFS$2OFS$3]++;}
> END{ for(i in a){if(a[i] && !b[i]){print "NEW: "i}} for(i in b){if(b[i])print i"\t\t"b[i]}}' OFS=, 20111125.csv *.csv
NE:985068,SHELF:2,SLOT:6                1
NE:556416,SHELF:9,SLOT:1                5
NE:565229,SHELF:7,SLOT:6                3
NE:20173322,SHELF:3,SLOT:1              2
NE:572549,SHELF:8,SLOT:6                7
NE:600866,SHELF:8,SLOT:3                18
NE:249454,SHELF:2,SLOT:2                1
NE:503703,SHELF:16,SLOT:2               1
NE:508089,SHELF:1,SLOT:6                2
NE:991626,SHELF:4,SLOT:3                4
NE:20159466,SHELF:6,SLOT:2              4
NE:508539,SHELF:7,SLOT:4                3
NE:506443,SHELF:17,SLOT:4               3
NE:20173322,SHELF:2,SLOT:5              2
NE:230388,SHELF:10,SLOT:2               2
NE:507423,SHELF:6,SLOT:5                1
NE:557688,SHELF:3,SLOT:1                4
NE:556029,SHELF:5,SLOT:2                3
NE:508089,SHELF:7,SLOT:6                1
NE:503284,SHELF:6,SLOT:3                14
NE:565229,SHELF:4,SLOT:3                1
NE:557688,SHELF:10,SLOT:1               1
NE:597829,SHELF:8,SLOT:3                5
NE:564867,SHELF:10,SLOT:1               1


Just need the 1's to be displayed as "NEW"
# 4  
Old 12-27-2011
Code:
nawk -F, 'NR==FNR{a[$1OFS$2OFS$3]++;next} a[$1OFS$2OFS$3]{b[$1OFS$2OFS$3]++}
 END{for(i in b){if(b[i]-1){print i"\t\t"b[i]-1}else{print "NEW :"i} } }' OFS=,20111125.csv *.csv | sort

--ahamed
Last edited by ahamed101; 12-27-2011 at 04:04 AM..
This User Gave Thanks to ahamed101 For This Post:
llcooljatt
# 5  
Old 12-28-2011
output


Code:
#nawk -F, 'NR==FNR{a[$1OFS$2OFS$3]++;next} a[$1OFS$2OFS$3]{b[$1OFS$2OFS$3]++}
 END{for(i in b){if(b[i]-1){print i"\t\t"b[i]-1}else{print "NEW :"i} } }' OFS=, 20111127.csv *.csv | sort -r
NEW :NE:571209,SHELF:18,SLOT:6
NEW :NE:566030,SHELF:2,SLOT:6
NEW :NE:564588,SHELF:5,SLOT:6
NEW :NE:556029,SHELF:8,SLOT:5
NEW :NE:510150,SHELF:9,SLOT:5
NEW :NE:508622,SHELF:10,SLOT:1
NEW :NE:20107650,SHELF:2,SLOT:4
NE:985068,SHELF:6,SLOT:4                1
NE:985068,SHELF:4,SLOT:1                2
NE:600866,SHELF:8,SLOT:3                20
NE:571209,SHELF:6,SLOT:1                51
NE:571209,SHELF:3,SLOT:3                14
NE:571209,SHELF:18,SLOT:2               1
NE:565808,SHELF:3,SLOT:4                1
NE:565229,SHELF:14,SLOT:1               2
NE:556029,SHELF:5,SLOT:2                3
NE:503284,SHELF:6,SLOT:3                14
NE:321636,SHELF:16,SLOT:1               4
NE:249314,SHELF:8,SLOT:1                6
NE:230388,SHELF:10,SLOT:2               2
NE:222268,SHELF:7,SLOT:6                34
NE:20173322,SHELF:5,SLOT:5              2
NE:20170632,SHELF:3,SLOT:3              12


it still shows the 1's it should be 2 does not look like it is including 20111127.csv file in calculation
# 6  
Old 12-28-2011
The entries with the count 1 are not new, they might be present in the other files and is present once. Please go through the code and experiment it.

--ahamed
# 7  
Old 12-28-2011
Quote:
Originally Posted by ahamed101
The entries with the count 1 are not new, they might be present in the other files and is present once. Please go through the code and experiment it.

--ahamed
ok here's a comparison


Code:
nawk -F, 'NR==FNR{a[$1OFS$2OFS$3]++;next} a[$1OFS$2OFS$3]{b[$1OFS$2OFS$3]++;}
 END{for(i in b){if(b[i]-1>0){print i"\t\t"b[i]-1}else{print "NEW :"i} } }' OFS=, 20111127.csv *.csv | sort -r
NEW :NE:571209,SHELF:18,SLOT:6
NEW :NE:566030,SHELF:2,SLOT:6
NEW :NE:564588,SHELF:5,SLOT:6
NEW :NE:556029,SHELF:8,SLOT:5
NEW :NE:510150,SHELF:9,SLOT:5
NEW :NE:508622,SHELF:10,SLOT:1
NEW :NE:20107650,SHELF:2,SLOT:4
NE:985068,SHELF:6,SLOT:4                1
NE:985068,SHELF:4,SLOT:1                2
NE:600866,SHELF:8,SLOT:3                20
NE:571209,SHELF:6,SLOT:1                51
NE:571209,SHELF:3,SLOT:3                14
NE:571209,SHELF:18,SLOT:2               1
NE:565808,SHELF:3,SLOT:4                1
NE:565229,SHELF:14,SLOT:1               2
NE:556029,SHELF:5,SLOT:2                3
NE:503284,SHELF:6,SLOT:3                14
NE:321636,SHELF:16,SLOT:1               4
NE:249314,SHELF:8,SLOT:1                6
NE:230388,SHELF:10,SLOT:2               2
NE:222268,SHELF:7,SLOT:6                34
NE:20173322,SHELF:5,SLOT:5              2
NE:20170632,SHELF:3,SLOT:3              12


nawk -F, 'NR==FNR{a[$1OFS$2OFS$3]++;next} a[$1OFS$2OFS$3]{b[$1OFS$2OFS$3]++;}
END{ for(i in a){if(a[i] && !b[i]){print "NEW: "i}} for(i in b){if(b[i])print i"\t\t"b[i]}}' OFS=, 20111127.csv *.csv
NE:556029,SHELF:8,SLOT:5                1
NE:566030,SHELF:2,SLOT:6                1
NE:222268,SHELF:7,SLOT:6                35
NE:564588,SHELF:5,SLOT:6                1
NE:20107650,SHELF:2,SLOT:4              1
NE:510150,SHELF:9,SLOT:5                1
NE:565808,SHELF:3,SLOT:4                2
NE:565229,SHELF:14,SLOT:1               3
NE:600866,SHELF:8,SLOT:3                21
NE:20170632,SHELF:3,SLOT:3              13
NE:985068,SHELF:4,SLOT:1                3
NE:571209,SHELF:3,SLOT:3                15
NE:571209,SHELF:6,SLOT:1                52
NE:321636,SHELF:16,SLOT:1               5
NE:571209,SHELF:18,SLOT:2               2
NE:508622,SHELF:10,SLOT:1               1
NE:230388,SHELF:10,SLOT:2               3
NE:571209,SHELF:18,SLOT:6               1
NE:556029,SHELF:5,SLOT:2                4
NE:503284,SHELF:6,SLOT:3                15
NE:20173322,SHELF:5,SLOT:5              3
NE:249314,SHELF:8,SLOT:1                7
NE:985068,SHELF:6,SLOT:4                2



the file 20111127.csv is not being included in the calculation for the first script, where as the second is perfect just need the counts showing as 1 to be called NEW as well as showing 1
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with awk percentage calculation from a file

i have a file say test with the below mentioned details Folder Name Total space Space used /test/test1 500.1GB 112.0 GB /test/test2 3.2 TB 5TB /test/test3 3TB 100GB i need to calculate percentage of each row based on total space and space used and copy... (9 Replies)
Discussion started by: venkitesh
9 Replies

2. Shell Programming and Scripting

Get file name which include the current date

Hi, I'm trying to get the name of a file that has the current date in the name. The filename is on the form A<current date>01.DC, for example A2012110501.DC I have this script so far date +%y%m%d D=$(date +%y%m%d) N=A20$D echo $N N2={$N}01.DC echo $N2 And gets the following... (12 Replies)
Discussion started by: Immelstorn
12 Replies

3. Shell Programming and Scripting

Need to include two more columns in the file using awk

Hi, I have a input file with many records as below: 1J4RR4GG0BC508200 68646 1 N M i want my output file to be like with columns included dgismdh and timestamp : Example: 1J4RR4GG0BC508200 68646 1 N M dgismdh 2012-02-21 07:22:25.98591 How to do it.can we do using awk? Pls help. (6 Replies)
Discussion started by: sonam273
6 Replies

4. Shell Programming and Scripting

math calculation for a txt file

Hi All, I have a text file which is only one column. I want to multiply all these values in file1.txt by 0.01 and get the output.txt file1.txt 65 85 90 ... output.txt 0.65 0.85 0.90 ... Thanks. Please use code tags when posting data and code samples! (2 Replies)
Discussion started by: senayasma
2 Replies

5. Shell Programming and Scripting

Calculation from a file

Hi , I have a file with below format 6/1/200618:00:011348797339524635352499218493964052971 6/1/200617:00:011348672070311735349025833693964052971 6/1/200616:00:011348546711070035345679259193964052971 I want a output as below 6/1/200618:00:011348797339524635352499218493964052971... (1 Reply)
Discussion started by: manojit123
1 Replies

6. Shell Programming and Scripting

include file name to extracted files

I've written the script below to merge only .txt files that exist in one directory into one huge .txt file and ignore other files with other extensions. now the result is one huge .txt file with all the contents of other .txt files how can i add a File Name as a comment before each file? ... (12 Replies)
Discussion started by: miss_dodi
12 Replies

7. Shell Programming and Scripting

File Size calculation with AWK

Hello Friends, Im calculating file sizes with below AWK script. I do this before some spesific files are transferred. I run the script it works but after several running it stuck with a limit of 2147483647 (2 Gbytes -1 byte) and cant exceed this. Something is wrong and I can't proceed, would... (1 Reply)
Discussion started by: EAGL€
1 Replies

8. UNIX for Dummies Questions & Answers

calculation on a large file

Trying to do some simple maths on a large file. Excel works fine but I have 1 million entries: If the difference between a number in column 2 and the one above it is more than 100 the insert a new line 8 4001 4100 8 4101 4200 8 4201 4300 8 15901 16000 8 15910 ... (1 Reply)
Discussion started by: dr_sabz
1 Replies

9. Programming

Not able to compile Pro*c file due - give errors and points to /usr/include/.. file

I am trying to compile the pro*C file but gives errors. It says it encountered "std" while it was expecting ; , = ( $ $ORACLE_HOME/bin/proc tradewind/dataaccess/Blob.pcc Pro*C/C++: Release 10.2.0.3.0 - Production on Fri May 9 11:10:54 2008 Copyright (c) 1982, 2005, Oracle. All rights... (0 Replies)
Discussion started by: shafi2all
0 Replies
Login or Register to Ask a Question