Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Array & text file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Array & text file
# 1  
Old 11-20-2013
Array & text file
Hi all,
i have a text file such as:
Code:
10 17:54:47,213
10 17:54:47,214
10 17:54:49,338
10 17:54:49,399
10 17:54:50,402
10 17:54:50,403
11 17:54:47,213
11 17:54:47,213
11 17:54:49,362
11 17:54:49,422
11 17:54:50,429
11 17:54:50,429
11 17:54:50,429
11 17:54:50,429
11 17:54:51,510
12 17:54:47,213
12 17:54:47,214
12 17:54:49,311
12 17:54:49,383
12 17:54:50,379
12 17:54:50,380
12 17:54:50,380
12 17:54:50,380
12 17:54:51,460
24 18:30:01,405
24 18:30:01,406
24 18:30:03,122
24 18:30:03,122
24 18:30:03,122

My objective is to make the difference of time between first value of first digit column and last value of the same occurrency.
In fact:
First occurrency of 10: 10 17:54:47,213
Last occurrency of 10: 10 17:54:50,403

I make 17:54:50,403 - 17:54:47,213 = result (in a new text file)

This is for every valure of first column.

Regards
Last edited by Franklin52; 11-20-2013 at 06:43 AM.. Reason: Please use code tags
# 2  
Old 11-20-2013
Could this help you ?
Code:
awk 'BEGIN{lastH=0}
{ if(!a[$1]) {
if ( lastH != 0 ) {
print prev,lastH - fstH":"lastM - fstM":"lastS - fstS":"lastMS - fstMS }
fstH=substr($2,1,2);fstM=substr($2,4,2);fstS=substr($2,7,2);fstMS=substr($2,10);
a[$1]++
} else {
prev=$1
lastH=substr($2,1,2);lastM=substr($2,4,2);lastS=substr($2,7,2);lastMS=substr($2,10);
} } END { print prev,lastH - fstH":"lastM - fstM":"lastS - fstS":"lastMS - fstMS}' filename

# 3  
Old 11-20-2013
Quote:
Originally Posted by sbamap
Hi all,
i have a text file such as:

10 17:54:47,213
10 17:54:47,214
10 17:54:49,338
10 17:54:49,399
10 17:54:50,402
10 17:54:50,403
11 17:54:47,213
11 17:54:47,213
11 17:54:49,362
11 17:54:49,422
11 17:54:50,429
11 17:54:50,429
11 17:54:50,429
11 17:54:50,429
11 17:54:51,510
12 17:54:47,213
12 17:54:47,214
12 17:54:49,311
12 17:54:49,383
12 17:54:50,379
12 17:54:50,380
12 17:54:50,380
12 17:54:50,380
12 17:54:51,460
24 18:30:01,405
24 18:30:01,406
24 18:30:03,122
24 18:30:03,122
24 18:30:03,122

My objective is to make the difference of time between first value of first digit column and last value of the same occurrency.
In fact:
First occurrency of 10: 10 17:54:47,213
Last occurrency of 10: 10 17:54:50,403

I make 17:54:50,403 - 17:54:47,213 = result (in a new text file)

This is for every valure of first column.

Regards
for given input this will work
Code:
awk '
        BEGIN {
                    m1 = "First occurrency of"
                    m2 = "Last occurrency of"
              }
NR==1 || p!=$1{
                   print s ? m2 FS x FS ":" FS s RS m1 FS $1 ":" $0 : m1 FS $1 FS ":" FS $0
              }
              {
                p=$1;s=$0; x=$1
              }
           END{
                     print m2 FS $1 ":" $0
              }
   ' file

Resulting
Code:
First occurrency of 10 : 10 17:54:47,213
Last occurrency of 10 : 10 17:54:50,403
First occurrency of 11:11 17:54:47,213
Last occurrency of 11 : 11 17:54:51,510
First occurrency of 12:12 17:54:47,213
Last occurrency of 12 : 12 17:54:51,460
First occurrency of 24:24 18:30:01,405
Last occurrency of 24:24 18:30:03,122

# 4  
Old 11-20-2013
It's good....
but difference is not correct...the result is wrongSmilie

this is complete list of item:
Code:
10 17:54:47,213
10 17:54:47,214
10 17:54:49,338
10 17:54:49,399
10 17:54:50,402
10 17:54:50,403
11 17:54:47,213
11 17:54:47,213
11 17:54:49,362
11 17:54:49,422
11 17:54:50,429
11 17:54:50,429
11 17:54:50,429
11 17:54:50,429
11 17:54:51,510
12 17:54:47,213
12 17:54:47,214
12 17:54:49,311
12 17:54:49,383
12 17:54:50,379
12 17:54:50,380
12 17:54:50,380
12 17:54:50,380
12 17:54:51,460
24 18:30:01,405
24 18:30:01,406
24 18:30:03,122
24 18:30:03,122
24 18:30:03,122
24 18:30:03,122
24 18:30:04,213
25 18:30:01,406
25 18:30:01,406
25 18:30:03,124
25 18:30:03,124
25 18:30:03,124
25 18:30:03,124
25 18:30:04,214
26 18:30:01,406
26 18:30:01,406
26 18:30:03,122
26 18:30:03,122
26 18:30:03,122
26 18:30:03,122
26 18:30:04,202
29 18:30:01,407
29 18:30:01,407
29 18:30:03,142
29 18:30:03,142
29 18:30:03,142
29 18:30:03,142
29 18:30:04,223
31 18:30:01,407
31 18:30:01,407
31 18:30:03,072
31 18:30:03,072
31 18:30:03,072
31 18:30:03,072
31 18:30:04,153
40 21:19:31,946
40 21:19:31,946
40 21:19:33,297
40 21:19:33,297
40 21:19:33,297
40 21:19:33,297
40 21:19:34,388
41 21:19:31,946
41 21:19:31,946
41 21:19:33,297
41 21:19:33,297
41 21:19:33,297
41 21:19:33,297
41 21:19:34,388
42 21:19:31,947
42 21:19:31,947
42 21:19:33,296
42 21:19:33,296
42 21:19:33,296
42 21:19:33,296
42 21:19:34,376
45 21:19:31,947
45 21:19:31,947
45 21:19:33,315
45 21:19:33,315
45 21:19:33,315
45 21:19:33,315
45 21:19:34,395
46 21:19:31,947
46 21:19:31,947
46 21:19:34,108
46 21:19:34,108
46 21:19:34,108
46 21:19:34,108
46 21:19:35,188
56 21:49:32,076
56 21:49:32,076
56 21:49:33,868
56 21:49:33,868
56 21:49:33,868
56 21:49:33,868
56 21:49:34,958
57 21:49:32,076
57 21:49:32,076
57 21:49:33,869
57 21:49:33,869
57 21:49:33,869
57 21:49:33,869
57 21:49:34,959
58 21:49:32,076
58 21:49:32,076
58 21:49:33,865
58 21:49:33,865
58 21:49:33,865
58 21:49:33,865
58 21:49:34,946
59 21:49:32,077
59 21:49:32,077
59 21:49:33,886
59 21:49:33,886
59 21:49:33,886
59 21:49:33,886
59 21:49:35,476
61 21:49:32,077
61 21:49:32,077
61 21:49:33,820
61 21:49:33,820
61 21:49:33,820
61 21:49:33,820
61 21:49:34,900
8 17:54:47,213
8 17:54:47,213
8 17:54:49,340
8 17:54:49,410
8 17:54:55,803
8 17:54:55,803
9 17:54:47,213
9 17:54:47,213
9 17:54:49,341
9 17:54:49,430
9 17:54:55,869
9 17:54:55,869

---------- Post updated at 06:00 AM ---------- Previous update was at 05:43 AM ----------

With this code i receive this error:
Code:
awk: syntax error near line 1
awk: illegal statement near line 1
awk: syntax error near line 1
awk: illegal statement near line 1



Quote:
Originally Posted by Akshay Hegde
for given input this will work
Code:
awk '
        BEGIN {
                    m1 = "First occurrency of"
                    m2 = "Last occurrency of"
              }
NR==1 || p!=$1{
                   print s ? m2 FS x FS ":" FS s RS m1 FS $1 ":" $0 : m1 FS $1 FS ":" FS $0
              }
              {
                p=$1;s=$0; x=$1
              }
           END{
                     print m2 FS $1 ":" $0
              }
   ' file

Resulting
Code:
First occurrency of 10 : 10 17:54:47,213
Last occurrency of 10 : 10 17:54:50,403
First occurrency of 11:11 17:54:47,213
Last occurrency of 11 : 11 17:54:51,510
First occurrency of 12:12 17:54:47,213
Last occurrency of 12 : 12 17:54:51,460
First occurrency of 24:24 18:30:01,405
Last occurrency of 24:24 18:30:03,122

Last edited by Franklin52; 11-20-2013 at 07:10 AM.. Reason: Please use code tags
# 5  
Old 11-20-2013
Try : for difference

Code:
awk '
NR==1 || p!=$1{
                  Arr[++j] = s ? s RS $0 : $0
              }
              {
                p=$1;s=$0
              }
           END{
                   Arr[j+1]=$0
                   for(i=1;i<=j;i++){
                                         split(Arr[i],S1,RS);split(Arr[i+1],S2,RS)
                                         start = i == 1? S1[1] : S1[2]
                                         end   = i == 1? S2[1] : S2[1]
                                         split(start,St,"[:, ]");split(end,Se,"[:, ]")
                                         print start,end,((Se[2]*3600)+(Se[3]*60)+Se[4])-((St[2]*3600)+(St[3]*60)+St[4]),"Sec"
                                    }   
              }
   ' file

OR
Code:
awk  '

function sec(val){
                      gsub(/^.*[[:space:]]|,.*/,x,val)
                      split(val,Ar,":")
                      return ((Ar[1]*3600)+(Ar[2]*60)+Ar[3])
                 }
FNR==1 || p != $1{
                  if(s){
                         print t,s, sec(s)-sec(t),"sec"
                       }
                        t = $0 
                 }
                 {
                        p=$1;s=$0
                 }
             END{
                     print t,s,sec(s)-sec(t),"sec"
                }
           
   ' file

for input in post #1
Code:
10 17:54:47,213 10 17:54:50,403 3 Sec
11 17:54:47,213 11 17:54:51,510 4 Sec
12 17:54:47,213 12 17:54:51,460 4 Sec
24 18:30:01,405 24 18:30:03,122 2 Sec

for input in post #4
Code:
10 17:54:47,213 10 17:54:50,403 3 Sec
11 17:54:47,213 11 17:54:51,510 4 Sec
12 17:54:47,213 12 17:54:51,460 4 Sec
24 18:30:01,405 24 18:30:04,213 3 Sec
25 18:30:01,406 25 18:30:04,214 3 Sec
26 18:30:01,406 26 18:30:04,202 3 Sec
29 18:30:01,407 29 18:30:04,223 3 Sec
31 18:30:01,407 31 18:30:04,153 3 Sec
40 21:19:31,946 40 21:19:34,388 3 Sec
41 21:19:31,946 41 21:19:34,388 3 Sec
42 21:19:31,947 42 21:19:34,376 3 Sec
45 21:19:31,947 45 21:19:34,395 3 Sec
46 21:19:31,947 46 21:19:35,188 4 Sec
56 21:49:32,076 56 21:49:34,958 2 Sec
57 21:49:32,076 57 21:49:34,959 2 Sec
58 21:49:32,076 58 21:49:34,946 2 Sec
59 21:49:32,077 59 21:49:35,476 3 Sec
61 21:49:32,077 61 21:49:34,900 2 Sec
8 17:54:47,213 8 17:54:55,803 8 Sec
9 17:54:47,213 9 17:54:55,869 8 Sec

Last edited by Akshay Hegde; 11-20-2013 at 09:08 AM.. Reason: Some more code
# 6  
Old 11-20-2013
Another variation
Code:
awk '
		{split($2,a,":|,")
		t=a[1]*3600+a[2]*60+a[3]+a[4]/1000
		$0=$1 FS sprintf ("%.3f\n",t)}
	
	NR==1 {f=$1
		t1=$2} 
	f!=$1 {x=$2
		split(l,b," ")
		t=b[2]-t1
		print b[1],strftime("%T",int(t)-3600)"." (t-int(t))*1000
		f=$1}
	
		{l=$0}
	END	{t=$2-x
		print $1,strftime("%T",int(t)-3600)"." (t-int(t))*1000}
	' file

Gives for data in post #1:
Code:
10 00:00:03.190
11 00:00:04.297
12 00:00:04.247
24 00:00:01.717

# 7  
Old 11-20-2013
Jotne,
i received this error with your version of script:
awk: syntax error near line 11
awk: illegal statement near line 11
awk: syntax error near line 16
awk: illegal statement near line 16

Can you help me?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Efficient way to search array in text file by awk

I have one array SPLNO with approx 10k numbers.Now i want to search the subscriber number from MDN.TXT file (containing approx 1.5 lac record)from the array.if subscriber number found in array it will perform below operation.my issue is that it's taking more time because for one number it's search... (6 Replies)
Discussion started by: siramitsharma
6 Replies

2. Shell Programming and Scripting

Reading columns from a text file and to make an array for each column

Hi, I am not so familiar with bash scripting and would appreciate your help here. I have a text file 'input.txt' like this: 2 3 4 5 6 7 8 9 10 I want to store each column in an array like this a ={2 5 8}, b={3 6 9}, c={4 7 10} so that i can access any element, e.g b=6 for the later use. (1 Reply)
Discussion started by: Asif Siddique
1 Replies

3. Shell Programming and Scripting

Efficient population of array from text file

Hi, I am trying to populate an array with data from a text file. I have a working method using awk but it is too slow and inefficent. See below. The text file has 70,000 lines. As awk is a line editor it reads each line of the file until it gets to the required line and then processes it.... (3 Replies)
Discussion started by: carlr
3 Replies

4. Shell Programming and Scripting

Search & Replacing text within a file

Hi all! I'm a newbie and I'm writing a script which will ask a user for data to search. I will then search for this data using grep and displaying this data back to the screen for the user to see. The user will then enter new data to replace the data searched. Now how do I replace this data... (4 Replies)
Discussion started by: macastor
4 Replies

5. Shell Programming and Scripting

Text substitution & getting file name from url

hi, sorry if this seems trivial. i have a file url.txt which consists of a list of urls (it was supposed to be my wget -i file). however, since the server from which i am trying to download uses redirect, wget dows not remeber the filename of ther original url will save to a file name which is... (3 Replies)
Discussion started by: texttoolong
3 Replies

6. Shell Programming and Scripting

Parse text file in shell & store to variable

Hi, I need to parse a simple text file like below and store the word that starts with BR* to a variable say $BRno. I need to do this in sh script. NOTE: the length of the numbers following BR is in constant. And there is only 1 BRXXX in a file at a given time. .txt file: BR276828... (1 Reply)
Discussion started by: script2010
1 Replies

7. Shell Programming and Scripting

Pipe text from a file into an array

Hi Guys I have a question about filling up an array I have a file called USER_FILE.txt it contains the following: Real Name:Thomas A Username:THOMAS_A Real Name:Thomas B Username:THOMAS_B Real Name:Thomas C Username:THOMAS_C Real Name:Thomas D Username:THOMAS_D Real Name:Thomas E... (8 Replies)
Discussion started by: grahambo2005
8 Replies

8. Shell Programming and Scripting

Advanced Search & Delete Text File

I have a file in which email messages are stored in. Every email is separated by by a ^Z character (Control-Z). I need to extract all emails after the 65,00th one to another file and delete them from the original file. Any suggests on accomplishing this? (2 Replies)
Discussion started by: maxcell
2 Replies

9. Shell Programming and Scripting

input text from file into 2d array

Hi all I have a little brainscratcher here. I want to draw a pie chart from data in a text file. The drawing of the graph works fine, if I insert the data manually into a 2d array. Now I want to pull the data from a text file (which was created using a uniq -c command) see sample below.... (2 Replies)
Discussion started by: pietie
2 Replies

10. Shell Programming and Scripting

Overwrite & Delete in Text File

Dear All, I have text file like this: Header Record 1 Record 2 ....... Record n Tail This line of code : awk '{ if ( NR == 1 ) { head=substr($0,1,300);} else { last = substr($0,1,300);}END{printf "Header is : %-300s Trailer is : %-300s\n", head, last}' filename converted Header... (11 Replies)
Discussion started by: 33junaid
11 Replies
Login or Register to Ask a Question