Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: How do I include the file being compared into calculation?
Top Forums Shell Programming and Scripting How do I include the file being compared into calculation? Post 302585622 by llcooljatt on Thursday 29th of December 2011 03:31:53 AM
Old 12-29-2011
another problem I noticed is that the script is not looking for say new lines that come in on 20111125.csv say NEW: NE:564867,SHELF:10,SLOT:1 line appeared 2 or 3 or multiple times then it should print

for example
Code:
NEW: NE:564867,SHELF:10,SLOT:1		                  11

so it looks like this bit
Code:
(b[i]-1)

needs changing to
Code:
(b[i]-*.csv)

?

---------- Post updated at 08:21 AM ---------- Previous update was at 07:19 AM ----------

Code:
nawk -F, 'NR==FNR{a[$1OFS$2OFS$3]++;next} a[$1OFS$2OFS$3]{b[$1OFS$2OFS$3]++}
END{for(i in b){if(b[i]-1){print i";\t\t"b[i]}else{print "NEW:"i";\t\t1"} } }' OFS=, 20111228.csv *.csv | sort
NE:221478,SHELF:13,SLOT:4;              2
NE:221726,SHELF:8,SLOT:1;               2
NE:222318,SHELF:14,SLOT:1;              9
NE:222799,SHELF:5,SLOT:2;               2
NE:230478,SHELF:10,SLOT:6;              2
NE:252848,SHELF:3,SLOT:5;               3
NE:267909,SHELF:1,SLOT:6;               2
NE:267909,SHELF:4,SLOT:4;               4
NE:509033,SHELF:2,SLOT:1;               11
NE:509622,SHELF:12,SLOT:2;              2
NE:541469,SHELF:11,SLOT:6;              31
NE:565538,SHELF:7,SLOT:4;               15
NE:571528,SHELF:8,SLOT:2;               12
NE:593848,SHELF:3,SLOT:1;               3
NE:594728,SHELF:3,SLOT:5;               32
NE:597708,SHELF:12,SLOT:6;              2
NE:602306,SHELF:18,SLOT:6;              4
NE:800189,SHELF:8,SLOT:3;               3
NE:801048,SHELF:3,SLOT:1;               2
NE:825890,SHELF:5,SLOT:1;               2
NE:852991,SHELF:11,SLOT:4;              20
NE:877073,SHELF:3,SLOT:3;               6
NE:984177,SHELF:1,SLOT:4;               6
NEW:NE:220072,SHELF:6,SLOT:3;           1
NEW:NE:221514,SHELF:14,SLOT:1;          1
NEW:NE:230293,SHELF:6,SLOT:2;           1
NEW:NE:239160,SHELF:15,SLOT:4;          1
NEW:NE:509860,SHELF:8,SLOT:3;           1
NEW:NE:602306,SHELF:6,SLOT:3;           1
NEW:NE:801048,SHELF:6,SLOT:2;           1
NEW:NE:825890,SHELF:2,SLOT:4;           1
NEW:NE:826030,SHELF:8,SLOT:3;           1
NEW:NE:841068,SHELF:8,SLOT:4;           1
NEW:NE:854479,SHELF:6,SLOT:5;           1
NEW:NE:895435,SHELF:5,SLOT:5;           1
NEW:NE:993180,SHELF:3,SLOT:1;           1

#cat 20111228.csv
NE:221726,SHELF:8,SLOT:1,01:00:02,Wed Dec 28 2011
NE:801048,SHELF:3,SLOT:1,01:30:02,Wed Dec 28 2011
NE:841068,SHELF:8,SLOT:4,02:00:03,Wed Dec 28 2011
NE:593848,SHELF:3,SLOT:1,02:30:09,Wed Dec 28 2011
NE:801048,SHELF:6,SLOT:2,04:00:01,Wed Dec 28 2011
NE:252848,SHELF:3,SLOT:5,04:30:03,Wed Dec 28 2011
NE:852991,SHELF:11,SLOT:4,05:30:01,Wed Dec 28 2011
NE:509033,SHELF:2,SLOT:1,06:30:01,Wed Dec 28 2011
NE:509033,SHELF:2,SLOT:1,09:00:01,Wed Dec 28 2011
NE:222318,SHELF:14,SLOT:1,10:00:01,Wed Dec 28 2011
NE:267909,SHELF:4,SLOT:4,12:30:02,Wed Dec 28 2011
NE:800189,SHELF:8,SLOT:3,13:30:02,Wed Dec 28 2011
NE:509622,SHELF:12,SLOT:2,14:30:01,Wed Dec 28 2011
NE:852991,SHELF:11,SLOT:4,14:30:01,Wed Dec 28 2011
NE:602306,SHELF:6,SLOT:3,15:00:01,Wed Dec 28 2011
NE:597708,SHELF:12,SLOT:6,15:30:01,Wed Dec 28 2011
NE:239160,SHELF:15,SLOT:4,15:30:01,Wed Dec 28 2011
NE:800189,SHELF:8,SLOT:3,16:00:01,Wed Dec 28 2011
NE:825890,SHELF:5,SLOT:1,17:00:02,Wed Dec 28 2011
NE:571528,SHELF:8,SLOT:2,17:30:01,Wed Dec 28 2011
NE:594728,SHELF:3,SLOT:5,17:30:02,Wed Dec 28 2011
NE:565538,SHELF:7,SLOT:4,18:30:03,Wed Dec 28 2011
NE:825890,SHELF:2,SLOT:4,18:30:04,Wed Dec 28 2011
NE:220072,SHELF:6,SLOT:3,19:00:03,Wed Dec 28 2011
NE:826030,SHELF:8,SLOT:3,19:00:03,Wed Dec 28 2011
NE:895435,SHELF:5,SLOT:5,19:00:06,Wed Dec 28 2011
NE:222799,SHELF:5,SLOT:2,19:00:06,Wed Dec 28 2011
NE:267909,SHELF:1,SLOT:6,19:30:01,Wed Dec 28 2011
NE:800189,SHELF:8,SLOT:3,19:30:01,Wed Dec 28 2011
NE:984177,SHELF:1,SLOT:4,20:00:02,Wed Dec 28 2011
NE:230293,SHELF:6,SLOT:2,20:30:02,Wed Dec 28 2011
NE:602306,SHELF:18,SLOT:6,20:30:02,Wed Dec 28 2011
NE:541469,SHELF:11,SLOT:6,20:30:02,Wed Dec 28 2011
NE:854479,SHELF:6,SLOT:5,20:30:02,Wed Dec 28 2011
NE:825890,SHELF:5,SLOT:1,20:30:02,Wed Dec 28 2011
NE:221478,SHELF:13,SLOT:4,21:00:01,Wed Dec 28 2011
NE:252848,SHELF:3,SLOT:5,21:00:01,Wed Dec 28 2011
NE:230478,SHELF:10,SLOT:6,21:00:01,Wed Dec 28 2011
NE:509860,SHELF:8,SLOT:3,21:00:02,Wed Dec 28 2011
NE:565538,SHELF:7,SLOT:4,21:30:02,Wed Dec 28 2011
NE:993180,SHELF:3,SLOT:1,22:00:01,Wed Dec 28 2011
NE:852991,SHELF:11,SLOT:4,22:00:02,Wed Dec 28 2011
NE:221514,SHELF:14,SLOT:1,22:00:02,Wed Dec 28 2011
NE:267909,SHELF:4,SLOT:4,22:30:01,Wed Dec 28 2011
NE:877073,SHELF:3,SLOT:3,23:00:02,Wed Dec 28 2011


In above example the file is new but resets twice, script is designed to pick this up as a repeat but because it has not been in previous files it is therefore a NEW with a 2 count, if you understand?

---------- Post updated at 08:28 AM ---------- Previous update was at 08:21 AM ----------

please amend this script as it latest one im using
Code:
nawk -F, 'NR==FNR{a[$1OFS$2OFS$3]++;next} a[$1OFS$2OFS$3]{b[$1OFS$2OFS$3]++}
END{for(i in b){if(b[i]-1){print i";\t\t"b[i]}else{print "NEW:"i";\t\t1"} } }' OFS=, 20111228.csv *.csv | sort

---------- Post updated at 08:31 AM ---------- Previous update was at 08:28 AM ----------

I think what we need looking at this is another if statement to declare if lines are not matched from 20111228.csv in *.csv then keep count in associate array & print as NEW with \t\t number increment ?
Last edited by llcooljatt; 12-29-2011 at 09:22 AM..
 

9 More Discussions You Might Find Interesting

1. Programming

Not able to compile Pro*c file due - give errors and points to /usr/include/.. file

I am trying to compile the pro*C file but gives errors. It says it encountered "std" while it was expecting ; , = ( $ $ORACLE_HOME/bin/proc tradewind/dataaccess/Blob.pcc Pro*C/C++: Release 10.2.0.3.0 - Production on Fri May 9 11:10:54 2008 Copyright (c) 1982, 2005, Oracle. All rights... (0 Replies)
Discussion started by: shafi2all
0 Replies

2. UNIX for Dummies Questions & Answers

calculation on a large file

Trying to do some simple maths on a large file. Excel works fine but I have 1 million entries: If the difference between a number in column 2 and the one above it is more than 100 the insert a new line 8 4001 4100 8 4101 4200 8 4201 4300 8 15901 16000 8 15910 ... (1 Reply)
Discussion started by: dr_sabz
1 Replies

3. Shell Programming and Scripting

File Size calculation with AWK

Hello Friends, Im calculating file sizes with below AWK script. I do this before some spesific files are transferred. I run the script it works but after several running it stuck with a limit of 2147483647 (2 Gbytes -1 byte) and cant exceed this. Something is wrong and I can't proceed, would... (1 Reply)
Discussion started by: EAGL€
1 Replies

4. Shell Programming and Scripting

include file name to extracted files

I've written the script below to merge only .txt files that exist in one directory into one huge .txt file and ignore other files with other extensions. now the result is one huge .txt file with all the contents of other .txt files how can i add a File Name as a comment before each file? ... (12 Replies)
Discussion started by: miss_dodi
12 Replies

5. Shell Programming and Scripting

Calculation from a file

Hi , I have a file with below format 6/1/200618:00:011348797339524635352499218493964052971 6/1/200617:00:011348672070311735349025833693964052971 6/1/200616:00:011348546711070035345679259193964052971 I want a output as below 6/1/200618:00:011348797339524635352499218493964052971... (1 Reply)
Discussion started by: manojit123
1 Replies

6. Shell Programming and Scripting

math calculation for a txt file

Hi All, I have a text file which is only one column. I want to multiply all these values in file1.txt by 0.01 and get the output.txt file1.txt 65 85 90 ... output.txt 0.65 0.85 0.90 ... Thanks. Please use code tags when posting data and code samples! (2 Replies)
Discussion started by: senayasma
2 Replies

7. Shell Programming and Scripting

Need to include two more columns in the file using awk

Hi, I have a input file with many records as below: 1J4RR4GG0BC508200 68646 1 N M i want my output file to be like with columns included dgismdh and timestamp : Example: 1J4RR4GG0BC508200 68646 1 N M dgismdh 2012-02-21 07:22:25.98591 How to do it.can we do using awk? Pls help. (6 Replies)
Discussion started by: sonam273
6 Replies

8. Shell Programming and Scripting

Get file name which include the current date

Hi, I'm trying to get the name of a file that has the current date in the name. The filename is on the form A<current date>01.DC, for example A2012110501.DC I have this script so far date +%y%m%d D=$(date +%y%m%d) N=A20$D echo $N N2={$N}01.DC echo $N2 And gets the following... (12 Replies)
Discussion started by: Immelstorn
12 Replies

9. Shell Programming and Scripting

Help with awk percentage calculation from a file

i have a file say test with the below mentioned details Folder Name Total space Space used /test/test1 500.1GB 112.0 GB /test/test2 3.2 TB 5TB /test/test3 3TB 100GB i need to calculate percentage of each row based on total space and space used and copy... (9 Replies)
Discussion started by: venkitesh
9 Replies
All times are GMT -4. The time now is 07:51 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy