Combine incrimental line


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Combine incrimental line
# 8  
Old 10-16-2017
Try this - there may be smarter and more elegant ways to get what you want, but it may serve as a starting point:
Code:
awk '
/header|footer/ {next
                }

                {n = split ($1, T, "[][]")
                 X  = T[1] (n>1?"[XYZ]":_) FS $2 FS $3 FS $4
                 if (LAST != X) {sub (/XYZ/, MN ":" MX, LAST)
                                 if (LAST ~ /[^ ]/) print LAST
                                 MN = T[2]
                                }
                   else         {MX = T[2]
                                }
                 LAST = X
                }

' file
road[0:3] 100 300 500
road[4:5] 100 400 600
street[0:1] 400 200 700
path 200 100 900

This User Gave Thanks to RudiC For This Post:
# 9  
Old 10-17-2017
Quote:
Originally Posted by RudiC
Try this - there may be smarter and more elegant ways to get what you want, but it may serve as a starting point:
Code:
awk '
/header|footer/ {next
                }

                {n = split ($1, T, "[][]")
                 X  = T[1] (n>1?"[XYZ]":_) FS $2 FS $3 FS $4
                 if (LAST != X) {sub (/XYZ/, MN ":" MX, LAST)
                                 if (LAST ~ /[^ ]/) print LAST
                                 MN = T[2]
                                }
                   else         {MX = T[2]
                                }
                 LAST = X
                }

' file
road[0:3] 100 300 500
road[4:5] 100 400 600
street[0:1] 400 200 700
path 200 100 900

PERFECTLY NAILED IT !!! SmilieSmilieSmilieSmilieSmilieSmilie Thanks a lot..
i know it does everything that was given but i really wish there is some explanation on what is going on.. at least on what did you sort first in term of overall plan until getting such beautiful output..
# 10  
Old 10-17-2017
Quote:
Originally Posted by pedot
... .. at least on what did you sort first in term of overall plan until getting such beautiful output..
Thanks for the "beautiful output" - it has been defined by you, hasn't it? Not sure I understand that quote - are you asking for the heuristics applied?

First it is mandatory to look at the input data and the desired output to find out what needs to be eliminated, what propagated and mayhap transformed or converted. Seeing previous attempts (with part successes or failures) always helps. That's why I frequently comment

Quote:
Please become accustomed to provide decent context info of your problem.

It is always helpful to carefully and detailedly phrase a request, and to support it with system info like OS and shell, related environment (variables, options), preferred tools, adequate (representative) sample input and desired output data and the logics connecting the two including your own attempts at a solution, and, if existent, system (error) messages verbatim, to avoid ambiguities and keep people from guessing.
With above, the basis for a solution is established. Now, build a prototype to see if the logics is close to what is requested, then tweak the solution before finally polishing it... the former gets the logics right, the latter adds e.g. error handling or ergonomy / user friendlyness.
# 11  
Old 10-18-2017
Quote:
Originally Posted by RudiC
Thanks for the &quot;beautiful output&quot; - it has been defined by you, hasn't it? Not sure I understand that quote - are you asking for the heuristics applied?<br />
<br />
First it is mandatory to look at the input data and the desired output to find out what needs to be eliminated, what propagated and mayhap transformed or converted. Seeing previous attempts (with part successes or failures) always helps. That's why I frequently comment

That is why im interested on how do you plan out all the information given to produce such output.


Quote:
With above, the basis for a solution is established. Now, build a prototype to see if the logics is close to what is requested, then tweak the solution before finally polishing it... the former gets the logics right, the latter adds e.g. error handling or ergonomy / user friendlyness.
Actually im having hard time to sort things out to get the desired output.. my first attempt is to remove the first column and pipe out sort -u to get the uniq 2nd field.. since i dont know awk that deep to store the output somewhere i then output it to some temporary file.txt to call out later.. then i stuck in there to merge the uniq data back with the 1st column (that i have not yet know how to combine the bit )

Last edited by RudiC; 10-18-2017 at 09:04 AM.. Reason: Adapted the quote tags to reflect what I think pedot meant to say...
# 12  
Old 10-18-2017
Explaining heuristics (or a "plan to get from input to output") is not easy a task... First it helps a lot to know the ropes, i.e. what tools do you have at hand, their respective applicability, and what are the strengthes and weaknesses of each that might lean itself to the solution in this special, individual case. Here, awk stood out, but e.g. perl might fit even better - unfortunately, I don't have reasonable command of it.
Now, to the creative and - can I say - intuitive part, which can be a stepwise, repetitive, and approximating approach:
Check features and idiosyncrasies of the input, and what needs to be transported to the output. In this case, we can safely assume the input is sorted - should that not be the case, additional steps might need to be taken. We can identify two entities:


Code:
road[0] 100 300 500   \
road[1] 100 300 500    \  constant; go to road ... 100 300 500
road[2] 100 300 500    /  
road[3] 100 300 500   / 
     |
     +--> go to [0:3]

You see: $1 (which is field 1) needs to be split to get at its two elements that need different treatment. The constant part plus a placeholder plus the residual fields go to a working variable X, the numerical part is used to find MIN and MAX values, the latter in successive lines as we assume input is sorted. If the pattern in X changes, the output line is created from the LAST variable, by replacing the placeholder with the actual values, and printed out. At the end of each line's processing, X is saved in LAST.
Please bear in mind that above solution is taylored to the input sample given, so we stop here. Should additional conditions arise, they would need to be programmed in as well in - which may well happen - countless additional steps, conditional statements, loops, special cases, etc.

Last edited by RudiC; 10-18-2017 at 11:08 AM..
# 13  
Old 10-18-2017
Quote:
Originally Posted by RudiC
Explaining heuristics (or a "plan to get from input to output") is not easy a task... First it helps a lot to know the ropes, i.e. what tools do you have at hand, their respective applicability, and what are the strengthes and weaknesses of each that might lean itself to the solution in this special, individual case. Here, awk stood out, but e.g. perl might fit even better - unfortunately, I don't have reasonable command of it.
Now, to the creative and - can I say - intuitive part, which can be a stepwise, repetitive, and approximating approach:
Check features and idiosyncrasies of the input, and what needs to be transported to the output. In this case, we can safely assume the input is sorted - should that not be the case, additional steps might need to be taken. We can identify two entities:


Code:
road[0] 100 300 500   \
road[1] 100 300 500    \  constant; go to road ... 100 300 500
road[2] 100 300 500    /  
road[3] 100 300 500   / 
     |
     +--> go to [0:3]

You see: $1 (which is field 1) needs to be split to get at its two elements that need different treatment. The constant part plus a placeholder plus the residual fields go to a working variable X, the numerical part is used to find MIN and MAX values, the latter in successive lines as we assume input is sorted. If the pattern in X changes, the output line is created from the LAST variable, by replacing the placeholder with the actual values, and printed out. At the end of each line's processing, X is saved in LAST.
Please bear in mind that above solution is taylored to the input sample given, so we stop here. Should additional conditions arise, they would need to be programmed in as well in - which may well happen - countless additional steps, conditional statements, loops, special cases, etc.

@RudiC
thanks a lot for such explanation / information..
i got overall general idea but need to understand more on deep level of awk maybe..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to combine lines from line with pattern match to a line that ends in a pattern

I am trying to combine lines with these conditions: 1. First line starts with text of "libname VALUE db2 datasrc" where VALUE can be any text. 2. If condition1 is met then continue to combine lines through a line that ends with a semicolon. 3. Ignore case when matching patterns and remove any... (5 Replies)
Discussion started by: Wes Kem
5 Replies

2. Shell Programming and Scripting

Combine multiline to one line till a blank line

Hello, I have a file as :- ABC DEF GHI JKL <BlankLine> MNO PQR STU VWX <BlankLine> YZA I need it as below:- ABCDEFGHIJKL; MNOPQRSTUVWX; (3 Replies)
Discussion started by: jassi10781
3 Replies

3. Shell Programming and Scripting

Combine 2 line with space issue

Hello all, i am new to linux , and please need your help and suggestion on.... when vi 1.txt :set list, it looks like $ is displaying the end of line Filter: vlan1-BUM-1M $ BUM-1M 0 ... (4 Replies)
Discussion started by: samoptimus
4 Replies

4. Shell Programming and Scripting

Combine line before the pattern

Hi, I am not very familiar with sed and awk and i have a huge file to process which is impossible to do manually. I want to print out beginning from "Network" until end of line only (excluding the Version). AND. the decription to be all in one line. File: Version: 2.0 Network: xxx... (9 Replies)
Discussion started by: The One
9 Replies

5. Shell Programming and Scripting

Combine common line from 2 Huge files

Hi, I am having 2 huge files having line count more than 10million. The files look like: File 1 45905099 2059 942961505 3007 8450875165 7007 615565331 3015 9415586035 9012 9871573 5367 4415655 4011 44415539519 5361 3250659295 4001 5950718618 9367 File 2 44415539519 TQ03... (2 Replies)
Discussion started by: rochitsharma
2 Replies

6. UNIX for Dummies Questions & Answers

What is the command use to combine line using sed

Hi all, What is the sed command use to combine line? Example: Below is an output after extracted from few commands aaa bbb ccc ddd eee fff ggg and i would like to combine all the line as shown below, aaa,bbb,ccc,ddd,eee,fff (5 Replies)
Discussion started by: 793589
5 Replies

7. UNIX for Dummies Questions & Answers

combine files line by line

Hi all, I once knew of a simple unix command to do this, but I can't remember it and I can't find it by searching. I have two files. ### FILE A #### A1 A2 A3 A4 A5 ### FILE B #### B1 B2 B3 B4 B5 (2 Replies)
Discussion started by: Digby
2 Replies

8. Shell Programming and Scripting

How to combine text data into one line?

The following input needs to be manipulated as follows: INPUT from file or results of command: ============start: Medium identifier : a45c0213:47eb5485:0aec:0321 Medium label : SQL Disk_11516 Location : Protected : None ... (2 Replies)
Discussion started by: rcky_mntere
2 Replies

9. Shell Programming and Scripting

Combine output on same line

I am trying to get some data from a file and print it on the same line. I have a script that gets the date/time and calculates a DB query call time and sends to a file. I need to take this file and send it in a xcel like format with multiple data columns. example output file (fndbq.out) ... (3 Replies)
Discussion started by: theninja
3 Replies

10. Shell Programming and Scripting

Combine reports and append a line between each

I am new to Unix and have tried to write a ksh script to do the following without success: I have several reports in a directory (report1, report2, report3, etc). I would like to combine all of these reports into one file (REPORTS). I would like to append *** End of Report *** to each report so... (2 Replies)
Discussion started by: ldevito1
2 Replies
Login or Register to Ask a Question