Sponsored Content
Top Forums Shell Programming and Scripting Merge lines in text file based on pattern Post 302391330 by Bertik on Monday 1st of February 2010 03:36:28 AM
Old 02-01-2010
Question Merge lines in text file based on pattern

Hello,

I have searched forum trying to find a solution to my problem, but could not find anything or I did not understand the examples....

I should say, I am very inexperienced with text processing.

I have a text file with approx 60k lines in it.
I need to merge lines based on the number on the end of the "master line".

Example1:
Code:
Word|1
(1)|Wordel|One Word

So, here I need to delete one pipe character and number 1 after the word 'Word' and merge first line with second line. Delete number 1 and this "(" and this ")" characters. Result should look like this:
Code:
Word|Wordel|One Word

Example 2:
Code:
Eye|4
(1)|Human Eye|Animal Eye
(2)|My Eye|Your Eye|His Eye|Her Eye
(3)|Second Eye|Third Eye
(4)|So Much About Eye

Here I need to delete one pipe character and number 4 after the word 'Eye' and merge first line with four following lines. Delete number 1 and 2 and 3 and 4 and this "(" and this ")" characters. Result should look like this:
Code:
Eye|Human Eye|Animal Eye|My Eye|Your Eye|His Eye|Her Eye|Second Eye|Third Eye|So Much About Eye

So, if the txt file looks like this before processing:
Code:
Word|1
(1)|Wordel|One Word
Eye|4
(1)|Human Eye|Animal Eye
(2)|My Eye|Your Eye|His Eye|Her Eye
(3)|Second Eye|Third Eye
(4)|So Much About Eye

I need it look like this after processing:
Code:
Word|Wordel|One Word
Eye|Human Eye|Animal Eye|My Eye|Your Eye|His Eye|Her Eye|Second Eye|Third Eye|So Much About Eye

Could somebody help me with this please?
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merge lines in Flat file based on first 5 characters

Hi I have the fixed width flat file having the following data 12345aaaaaaaaaabbbbbbbbbb 12365sssssssssscccccccccc 12365sssss 12367ddddddddddvvvvvvvvvv 12367 vvvvv Here the first column is length 5 second is length 10 third is length 10 if the second or third column exceeds... (3 Replies)
Discussion started by: Brado
3 Replies

2. Shell Programming and Scripting

Merge two file data together based on specific pattern match

My input: File_1: 2000_t g1110.b1 abb.1 2001_t g1111.b1 abb.2 abb.2 g1112.b1 abb.3 2002_t . . File_2: 2000_t Ali england 135 abb.1 Zoe british 150 2001_t Ali england 305 g1111.b1 Lucy russia 126 (6 Replies)
Discussion started by: patrick87
6 Replies

3. Shell Programming and Scripting

Merge lines from one file if pattern matches

I have one comma separated file (a.txt) with two or more records all matching except for the last column. I would like to merge all matching lines into one and consolidate the last column, separated by ":". Does anyone know of a way to do this easily? I've searched the forum but most talked... (6 Replies)
Discussion started by: giannicello
6 Replies

4. UNIX for Dummies Questions & Answers

merge lines within a file that start with a similar pattern

Hello! i have a text file.. which contains the data as follows i want to merge the declarations lines pertaining to one datatype in to a single line as follows i've searched the forum for help.. but couldn't find much help.. how can i do this?? (1 Reply)
Discussion started by: a_ba
1 Replies

5. UNIX for Dummies Questions & Answers

print multiple lines from text file based on pattern list

I have a text file with a list of items/patterns: ConsensusfromCGX_alldays_trimmedcollapsedfilteredreadscontiglist(229095contigs)contig12238 ConsensusfromCGX_alldays_trimmedcollapsedfilteredreadscontiglist(229095contigs)contig34624... (1 Reply)
Discussion started by: Oyster
1 Replies

6. Shell Programming and Scripting

Merge file lines based off of keyword

Hello Everyone, I have two files I created in a format similar to the ones found below (character position is important): File 1: 21 Cat Y N S Y Y N N FOUR LEGS TAIL WHISKERS 30 Dog N N 1 Y Y N N FOUR LEGS TAIL 33 Fish Y N 1 Y Y N N FINS 43 CAR Y N S Y Y N N WHEELS DOORS... (7 Replies)
Discussion started by: jl487
7 Replies

7. Shell Programming and Scripting

How to merge lines based off of text?

Hello Everyone, I have two files, similar to the following: File 1: 8010 ITEM01 CODE1 FLAG1 filler filler 7020 OBJECT CODE2 FLAG2 filler 6010 THING1 CODE4 FLAG4 6011 ITEM20 CODE7 FLAG7 File 2 contains: 6020 ITEM01 CODEA FLAGA filler filler filler 7000 OBJECT CODEB... (2 Replies)
Discussion started by: jl487
2 Replies

8. UNIX for Dummies Questions & Answers

Extracting lines from a text file based on another text file with line numbers

Hi, I am trying to extract lines from a text file given a text file containing line numbers to be extracted from the first file. How do I go about doing this? Thanks! (1 Reply)
Discussion started by: evelibertine
1 Replies

9. Shell Programming and Scripting

Need to merge lines based on pattern

Hi, I have a requirement to merge multiple lines based on search pattern. The search criteria is : it will search for CONSTRAINT and when it found CONSTRAINT, it will merge all lines to 1 line till it founds blank line. For Example: CREATE TABLE "AMS_DISTRIBUTOR_XREF" ( "SOURCE"... (5 Replies)
Discussion started by: satyaatcgi
5 Replies

10. Shell Programming and Scripting

Help with order lines from a file based on a pattern

Hi I need to order these lines from a txt file my file looks like this IMSI ........................ 1234567890 APN ......................... INTERNET.COM APN ......................... MMS.COM APN ......................... WAP.COM APN ......................... BA.COM IMSI... (4 Replies)
Discussion started by: alone77
4 Replies
AWK(1)							      General Commands Manual							    AWK(1)

NAME
awk - pattern scanning and processing language SYNOPSIS
awk [ -Fc ] [ prog ] [ file ] ... DESCRIPTION
Awk scans each input file for lines that match any of a set of patterns specified in prog. With each pattern in prog there can be an asso- ciated action that will be performed when a line of a file matches the pattern. The set of patterns may appear literally as prog, or in a file specified as -f file. Files are read in order; if there are no files, the standard input is read. The file name `-' means the standard input. Each line is matched against the pattern portion of every pattern-action statement; the associated action is performed for each matched pattern. An input line is made up of fields separated by white space. (This default can be changed by using FS, vide infra.) The fields are denoted $1, $2, ... ; $0 refers to the entire line. A pattern-action statement has the form pattern { action } A missing { action } means print the line; a missing pattern always matches. An action is a sequence of statements. A statement can be one of the following: if ( conditional ) statement [ else statement ] while ( conditional ) statement for ( expression ; conditional ; expression ) statement break continue { [ statement ] ... } variable = expression print [ expression-list ] [ >expression ] printf format [ , expression-list ] [ >expression ] next # skip remaining patterns on this input line exit # skip the rest of the input Statements are terminated by semicolons, newlines or right braces. An empty expression-list stands for the whole line. Expressions take on string or numeric values as appropriate, and are built using the operators +, -, *, /, %, and concatenation (indicated by a blank). The C operators ++, --, +=, -=, *=, /=, and %= are also available in expressions. Variables may be scalars, array elements (denoted x[i]) or fields. Variables are initialized to the null string. Array subscripts may be any string, not necessarily numeric; this allows for a form of associative memory. String constants are quoted "...". The print statement prints its arguments on the standard output (or on a file if >file is present), separated by the current output field separator, and terminated by the output record separator. The printf statement formats its expression list according to the format (see printf(3)). The built-in function length returns the length of its argument taken as a string, or of the whole line if no argument. There are also built-in functions exp, log, sqrt, and int. The last truncates its argument to an integer. substr(s, m, n) returns the n-character sub- string of s that begins at position m. The function sprintf(fmt, expr, expr, ...) formats the expressions according to the printf(3) for- mat given by fmt and returns the resulting string. Patterns are arbitrary Boolean combinations (!, ||, &&, and parentheses) of regular expressions and relational expressions. Regular expressions must be surrounded by slashes and are as in egrep. Isolated regular expressions in a pattern apply to the entire line. Regu- lar expressions may also occur in relational expressions. A pattern may consist of two patterns separated by a comma; in this case, the action is performed for all lines between an occurrence of the first pattern and the next occurrence of the second. A relational expression is one of the following: expression matchop regular-expression expression relop expression where a relop is any of the six relational operators in C, and a matchop is either ~ (for contains) or !~ (for does not contain). A condi- tional is an arithmetic expression, a relational expression, or a Boolean combination of these. The special patterns BEGIN and END may be used to capture control before the first input line is read and after the last. BEGIN must be the first pattern, END the last. A single character c may be used to separate the fields by starting the program with BEGIN { FS = "c" } or by using the -Fc option. Other variable names with special meanings include NF, the number of fields in the current record; NR, the ordinal number of the current record; FILENAME, the name of the current input file; OFS, the output field separator (default blank); ORS, the output record separator (default newline); and OFMT, the output format for numbers (default "%.6g"). EXAMPLES
Print lines longer than 72 characters: length > 72 Print first two fields in opposite order: { print $2, $1 } Add up first column, print sum and average: { s += $1 } END { print "sum is", s, " average is", s/NR } Print fields in reverse order: { for (i = NF; i > 0; --i) print $i } Print all lines between start/stop pairs: /start/, /stop/ Print all lines whose first field is different from previous one: $1 != prev { print; prev = $1 } SEE ALSO
lex(1), sed(1) A. V. Aho, B. W. Kernighan, P. J. Weinberger, Awk - a pattern scanning and processing language BUGS
There are no explicit conversions between numbers and strings. To force an expression to be treated as a number add 0 to it; to force it to be treated as a string concatenate "" to it. AWK(1)
All times are GMT -4. The time now is 09:18 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy