Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

combining two files

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers combining two files
# 29  
Old 09-29-2008
Ok,
try to change the number of records:

Code:
awk 'END {
  while (++j < i-6)
    print l[j]FS r[l[j]~/Total/?c++:c]
  }
NR == FNR {
  sub(/[^ \t]* /,"")
  r[FNR] = $0
  next
  }
!t[$1]++ && FNR > 1 {
  while (++j < i-6)
    print l[j]FS r[l[j]~/Total/?c++:c?c:++c]
  i = j = 0
  split("",l)
  }
{ l[++i] = $0 }
' file1 file2

This is the output I get:

Code:
==> file1 <==
FILENAME1 xxxxx first, last Date: 09/01/08
FILENAME1 xxxxx first, last Date: 09/02/08
FILENAME1 xxxxx first, last Date: 09/03/08
FILENAME1 xxxxx first, last Date: 09/04/08
FILENAME1 xxxxx first, last Date: 09/05/08

==> file2 <==
FILENAME1 Open 3:45 3:40 218 7
FILENAME1 Closed 0:00 0:00 0 1
FILENAME1 Break 0:15 0:17 14 1
FILENAME1 Rest 0:00 0:04 0 0
FILENAME1 Total 4:00 4:01 232 9
FILENAME1 Open 3:45 3:56 212 13
FILENAME1 Closed 0:00 0:00 0 20
FILENAME1 Break 0:15 0:14 11 4
FILENAME1 Tech Issues 0:00 0:01 0
FILENAME1 Rest 0:00 0:09 0 0
FILENAME1 Total 4:00 4:20 223 37
FILENAME1 Open 33:00 35:51 1865 115
FILENAME1 Closed 20:15 16:25 977 272
FILENAME1 Break 2:15 2:31 94 41
FILENAME1 Coaching 0:45 0:54 45 0
FILENAME1 Tech Issues 0:00 0:09 0
FILENAME1 Rest 0:00 0:59 0 0
FILENAME1 Total 56:15 56:49 2981 428
FILENAME2 Open 3:45 3:40 218 7
FILENAME2 Closed 0:00 0:00 0 1
FILENAME2 Break 0:15 0:17 14 1
FILENAME2 Rest 0:00 0:04 0 0
FILENAME2 Total 4:00 4:01 232 9
FILENAME2 Open 3:45 3:56 212 13
FILENAME2 Closed 0:00 0:00 0 20
FILENAME2 Break 0:15 0:14 11 4
FILENAME2 Tech Issues 0:00 0:01 0
FILENAME2 Rest 0:00 0:09 0 0
FILENAME2 Total 4:00 4:20 223 37
FILENAME2 Open 33:00 35:51 1865 115
FILENAME2 Closed 20:15 16:25 977 272
FILENAME2 Break 2:15 2:31 94 41
FILENAME2 Coaching 0:45 0:54 45 0
FILENAME2 Tech Issues 0:00 0:09 0
FILENAME2 Rest 0:00 0:59 0 0
FILENAME2 Total 56:15 56:49 2981 428
% awk 'END {          
  while (++j < i-6)
    print l[j]FS r[l[j]~/Total/?c++:c]
  }
NR == FNR {
  sub(/[^ \t]* /,"")
  r[FNR] = $0
  next
  }
!t[$1]++ && FNR > 1 {
  while (++j < i-6)
    print l[j]FS r[l[j]~/Total/?c++:c?c:++c]
  i = j = 0
  split("",l)
  }
{ l[++i] = $0 }
' file1 file2
FILENAME1 Open 3:45 3:40 218 7 xxxxx first, last Date: 09/01/08
FILENAME1 Closed 0:00 0:00 0 1 xxxxx first, last Date: 09/01/08
FILENAME1 Break 0:15 0:17 14 1 xxxxx first, last Date: 09/01/08
FILENAME1 Rest 0:00 0:04 0 0 xxxxx first, last Date: 09/01/08
FILENAME1 Total 4:00 4:01 232 9 xxxxx first, last Date: 09/01/08
FILENAME1 Open 3:45 3:56 212 13 xxxxx first, last Date: 09/02/08
FILENAME1 Closed 0:00 0:00 0 20 xxxxx first, last Date: 09/02/08
FILENAME1 Break 0:15 0:14 11 4 xxxxx first, last Date: 09/02/08
FILENAME1 Tech Issues 0:00 0:01 0 xxxxx first, last Date: 09/02/08
FILENAME1 Rest 0:00 0:09 0 0 xxxxx first, last Date: 09/02/08
FILENAME1 Total 4:00 4:20 223 37 xxxxx first, last Date: 09/02/08
FILENAME2 Open 3:45 3:40 218 7 xxxxx first, last Date: 09/03/08
FILENAME2 Closed 0:00 0:00 0 1 xxxxx first, last Date: 09/03/08
FILENAME2 Break 0:15 0:17 14 1 xxxxx first, last Date: 09/03/08
FILENAME2 Rest 0:00 0:04 0 0 xxxxx first, last Date: 09/03/08
FILENAME2 Total 4:00 4:01 232 9 xxxxx first, last Date: 09/03/08
FILENAME2 Open 3:45 3:56 212 13 xxxxx first, last Date: 09/04/08
FILENAME2 Closed 0:00 0:00 0 20 xxxxx first, last Date: 09/04/08
FILENAME2 Break 0:15 0:14 11 4 xxxxx first, last Date: 09/04/08
FILENAME2 Tech Issues 0:00 0:01 0 xxxxx first, last Date: 09/04/08
FILENAME2 Rest 0:00 0:09 0 0 xxxxx first, last Date: 09/04/08
FILENAME2 Total 4:00 4:20 223 37 xxxxx first, last Date: 09/04/08

# 30  
Old 09-29-2008
thanks a ton!! Yes it correct now.. the code is a liitle difficult for my level of expertise Smilie I havent used arrays much.. If you can help me underatnad it a little i would appreciate it,

thanks
# 31  
Old 09-29-2008
the reason i ask is i must be able to modify it,... what if they want the summary as well i must be able to add those lines with some dummy date or with date blank..

thanks
# 32  
Old 09-29-2008
Here's the explanation:

Code:
NR == FNR {
  sub(/[^ \t]* /,"")
  r[FNR] = $0
  next
  }

While reading the first non-empty input file (NR == FNR) do:
- remove the first field (assuming a default FS)
- store the rest of the current record in an associative array named r,
keyed by FNR.
- mark the record as processed (no further actions will operate on those records).

Code:
!t[$1]++ && FNR > 1 {
  while (++j < i-6)
    print l[j]FS r[l[j]~/Total/?c++:c?c:++c]
  i = j = 0
  split("",l)
  }

When the expressions !t[$1]++ (a common AWK idiom that means "when the key value is seen for the first time", i. e. when the value of $1 changes [actually this part could be rewritten in a more efficient way given your file is already sorted - you don't need an array here] and (&& - logical AND) FNR is greater than 1 do:
- print all but the last 6 values of the array named l concatenating to them the values of the r array (see below) incrementing the index if the value of the l array matches the string Total and if the counter variable c is used for the first time. This is the part of the code that prints the above:

Code:
while (++j < i-6)
    print l[j]FS r[l[j]~/Total/?c++:c?c:++c]

- set i and j to 0/false
- delete the l array, if your AWK implementation supports the delete function to delete the entire array, use that function instead:

Code:
delete l

Code:
{ l[++i] = $0 }

Build an array l keyed by the auto-incremented variable i and store the entire record (here we are reading the second input file) as value.

Code:
END {
  while (++j < i-6)
    print l[j]FS r[l[j]~/Total/?c++:c]
  }

Because we actually output the lines when the value of the first field changes, we need to repeat the loop for the last section in the END block.

Hope this helps a bit.
# 33  
Old 09-29-2008
thank you so much.. i understand it much better now.. however i do have some basic questions..

NR == FNR this condition I am little confused..
While reading the first non-empty input file (NR == FNR)

What is being stored in the array r is the rest of the record minus the key (filename in this case) .. so what is
r[FNR] how is FNR pointing to the key?

agina here !t[$1]++ && FNR > 1

again here what is FNR.. i guess i do not know the basic FNR definition..

please help
# 34  
Old 09-29-2008
sorry for so many questions..

also i am not clear on this line:

print l[j]FS r[l[j]~/Total/?c++:c?c:++c]

print all but the last 6 values of the array named l concatenating to them the values of the r array -- this is clear

#incrementing the index if the value of the l array matches the string Total ----- this is clear

and if the counter variable c is used for the first time. ???
# 35  
Old 09-29-2008
Quote:
NR == FNR this condition I am little confused..
While reading the first non-empty input file (NR == FNR)[...]
If you run a simple:

Quote:
man awk|grep -E 'F?NR'
FNR The input record number in the current input file.
NR The total number of input records seen so far.
...
So, consider the following:

Code:
$ repeat 3 print line$((++i))>>file{1..2}
$ head file[12]
==> file1 <==
line1
line2
line3

==> file2 <==
line1
line2
line3
$ awk '{ 
  printf "filename is: %s, NR is %d,\
  FNR is %d,\
  $0 is %s,\
  NR == FNR: true/false --> %s\n",
  FILENAME,
  NR,
  FNR,
  $0,
  NR == FNR?"true":"false"
}' file[12]
filename is: file1, NR is 1,  FNR is 1,  $0 is line1,  NR == FNR: true/false --> true
filename is: file1, NR is 2,  FNR is 2,  $0 is line2,  NR == FNR: true/false --> true
filename is: file1, NR is 3,  FNR is 3,  $0 is line3,  NR == FNR: true/false --> true
filename is: file2, NR is 4,  FNR is 1,  $0 is line1,  NR == FNR: true/false --> false
filename is: file2, NR is 5,  FNR is 2,  $0 is line2,  NR == FNR: true/false --> false
filename is: file2, NR is 6,  FNR is 3,  $0 is line3,  NR == FNR: true/false --> false

 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Combining files from different directories

I have two directories, each have 27 files with same name and now I want to combine them one by one into another directory with same names. I dont know how to use "and" for the "for loop" so it will not go in the circle. so my code has a problem I dont know how to fix :wall::wall::wall::wall:... (15 Replies)
Discussion started by: A-V
15 Replies

2. Shell Programming and Scripting

Combining files

Hi I have about 108 files (text files) that end with .avg and each one of these files have a distinct name that describes what is in the file. In each file there is a set of 80 values that are tab separated. I want to combine all 108 files into ONE main file. So each file is named: 1.avg... (5 Replies)
Discussion started by: phil_heath
5 Replies

3. Shell Programming and Scripting

Combining 2 files

i am having 2 files like this file 1 1, 2, 3, 4, file2 5, 6, 7, 8, what i want do is like this i want to put all the contents for file 2 after file 1,means adding column in file1 (5 Replies)
Discussion started by: sagar_1986
5 Replies

4. UNIX for Dummies Questions & Answers

Need Help in reading N days files from a Directory & combining the files

Hi All, Request your expertise in tackling one requirement in my project,(i dont have much expertise in Shell Scripting). The requirement is as below, 1) We store the last run date of a process in a file. When the batch run the next time, it should read this file, get the last run date from... (1 Reply)
Discussion started by: dsfreddie
1 Replies

5. Shell Programming and Scripting

Matching and combining two files

Hi there, I have two files. What I want to do is search for the values in second field of file1 in the 6th field of the file2 and of they match to add the fields 1-5 of the file2 at the end of the line of file1 with a comma before. E.g File1 FWB,CHUAGT87HUMAS/BUD01,REUAIR08KLM... (3 Replies)
Discussion started by: sickboy
3 Replies

6. Shell Programming and Scripting

Combining two files

I have two files and I need to combine (not append - but combine a row to a row) eg: File1: apples grapes oranges lemons File2: red green orange yellow After combining, the file should look like: (the second column should start at a specific byte) apples red grapes green... (7 Replies)
Discussion started by: hemangjani
7 Replies

7. Shell Programming and Scripting

Combining Two Files

Could someone help me reduce the number of runs for a shell program I created? I have two text files below: $ more list1.txt 01 AAA 02 BBB 03 CCC 04 DDD $ more list2.txt 01 EEE 02 FFF 03 GGG I want to combine the lines with the same number to get the below: 01 AAA 01 EEE 02... (4 Replies)
Discussion started by: stevefox
4 Replies

8. Shell Programming and Scripting

Combining Two Files

I have two files which contain data from two different transactions in the same format: <Name> - <Count> My goal is to end up with data in this format after combining the two: <Name> - <Count1> - <Count2> Is this possible to do with awk, or is there something better? Thanks... (3 Replies)
Discussion started by: bat711
3 Replies

9. UNIX for Dummies Questions & Answers

Combining files

Hi, is there a way to combine 2 files together, joining line 1 from file A with line 1 from file B, line 2 from A with line 2 from B etc. File A File B 1 4 2 5 3 6 Combined result = File C 14 25 36 (2 Replies)
Discussion started by: Enda Martin
2 Replies

10. UNIX for Dummies Questions & Answers

combining files

how will i combine these 2 files below, with the desired output specified below: file1: one two three four file2: red blue yellow green file3: aaa bbb ccc ddd (3 Replies)
Discussion started by: apalex
3 Replies
Login or Register to Ask a Question