Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Finding Similiarities and Merging Lines

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Finding Similiarities and Merging Lines
# 1  
Old 04-17-2012
Finding Similiarities and Merging Lines
Based on the following lines entries...how do we identify similiarities (up to a number of characters or up to a number of delimiters, or something else), and then say "these lines all have similar characters except for the last delimited characters, so we will merge all these lines into a single line of only the similiar characters)...i hope that makes sense.
Code:
http-myline-hostname38
http-myline-hostname58
http-myline
http-myline-secure-hostname37
http-myline-secure-hostname28
http-myline-person-hostnameBZ
http-myline-person-hostnameAB

resulting in the following entries:
Code:
http-myline
http-myline-secure
http-myline-person

Moderator's Comments:
Mod Comment How to use code tags
Last edited by Scrutinizer; 04-17-2012 at 06:05 PM..
# 2  
Old 04-17-2012
Try:
Code:
awk NF-=1 FS=- OFS=- infile | sort -u

Code:
sed 's/-[^-]*$//' infile | sort -u

This User Gave Thanks to Scrutinizer For This Post:
47shailesh
# 3  
Old 04-18-2012
Thank you.

I should have added more specific input lines, i will do so here.

From:
Code:
http-myline-hostname38
http-myline-hostname58
http-myline87
http-file
http-blob
http-para
http-lab
http-myline-secure-hostname37
http-myline-secure-hostname28
http-myline-person-hostnameBZ
http-myline-person-hostnameAB
http-may-server400
http-feb-server500-keep
http-april-server700-abcd-devel

And the desired output is to only merge and collapse lines that are similar up to a designated number of characters (say 10 or 20, depending).

So the desired output will be:

Code:
http-myline
http-myline87
http-file
http-blob
http-para
http-lab
http-myline-secure
http-myline-person
http-may-server400
http-feb-server500-keep
http-april-server700-abcd-devel

Last edited by Corona688; 04-18-2012 at 04:18 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merging multiple lines to columns with awk, while inserting commas for missing lines

Hello all, I have a large csv file where there are four types of rows I need to merge into one row per person, where there is a column for each possible code / type of row, even if that code/row isn't there for that person. In the csv, a person may be listed from one to four times... (9 Replies)
Discussion started by: RalphNY
9 Replies

2. Shell Programming and Scripting

Merging 2 lines together

I have a small problem, which due to my lack of knowledge, has left me unable to decipher some of the solutions that I looked at on these forums. So below is a piece of text, which I ran via cat -vet, which comes from within a program file. I have many such programs to process and repeatable,... (4 Replies)
Discussion started by: skarnm
4 Replies

3. Shell Programming and Scripting

Merging lines

Thanks it worked for me. I have one more question on top of that. We had few records which were splitted in 2 lines instead of one. Now i identified those lines. The file is too big to open via vi and edit it. How can i do it without opening the file. Suppose, I want line number 1001 & 1002 to... (2 Replies)
Discussion started by: Gangadhar Reddy
2 Replies

4. UNIX for Dummies Questions & Answers

Finding lines with a regular expression, replacing them with blank lines

So the tag for this forum says all newbies welcome... All I want to do is go through my file and find lines which contain a given string of characters then replace these with a blank line. I really tried to find a simple command to do this but failed. Here's what I did come up with though: ... (2 Replies)
Discussion started by: Golpette
2 Replies

5. Shell Programming and Scripting

merging two .txt files by alternating x lines from file 1 and y lines from file2

Hi everyone, I have two files (A and B) and want to combine them to one by always taking 10 rows from file A and subsequently 6 lines from file B. This process shall be repeated 40 times (file A = 400 lines; file B = 240 lines). Does anybody have an idea how to do that using perl, awk or sed?... (6 Replies)
Discussion started by: ink_LE
6 Replies

6. Shell Programming and Scripting

Merging lines

Hi folks. Could somebody help me write a script or command that will look through a file and for every line that doesn't contain a certain value, merge it with the one above? For example, the file contains: SCOTLAND|123|ABC|yes SCOTLAND|456|DEF|yes SCOTLAND|78 9|GHI|yes ... (3 Replies)
Discussion started by: MDM
3 Replies

7. Shell Programming and Scripting

Finding lines matching the Pattern and their previous lines in a file

Hi, I am trying to locate the occurences of certain pattern like 'Possible network disconnect' in a text file. I can get the actual lines matching the pttern using: grep -w 'Possible network disconnect' file_name. But I am more interested in getting the timing of these events which are... (7 Replies)
Discussion started by: sagarparadkar
7 Replies

8. Shell Programming and Scripting

Merging lines in a file

Hi, I want to merge the lines starting with a comma symbol with the previous line of the file. Input : cat file.txt name1,name2 ,name3,name4 emp1,emp2,emp3 ,emp4 ,emp5 user1,user2 ,user3 Output name1,name2,name3,name4 emp1,emp2,emp3,emp4,emp5 (9 Replies)
Discussion started by: mohan_tuty
9 Replies

9. Shell Programming and Scripting

Merging lines using AWK

Hi, Anybody help on this. :( I want to merge the line with previous line, if the line starts with 7. Otherwise No change in the line. Example file aa.txt is like below 122122 222222 333333 734834 702923 389898 790909 712345 999999 My output should be written in another file... (6 Replies)
Discussion started by: senthil_is
6 Replies

10. UNIX for Dummies Questions & Answers

Merging lines into one

Hello. I would be very pleased if sb. help me to solve my problem. I've got a file with many non blank lines and I want to merge all lines into one not destroy the informations on them. I've tryed it with split and paste, tr, sed , but everything I've done has been wrong. I know about crazy... (8 Replies)
Discussion started by: Foxgard
8 Replies
Login or Register to Ask a Question