Removing \n within a record (awk/gawk)


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Removing \n within a record (awk/gawk)
# 1  
Old 05-12-2009
Removing \n within a record (awk/gawk)

I am using a solution that was provided by a member:

awk '{s=$0;if(length(s) < 700){getline; s=s " " $0}printf("%s\n",s)}'

This scans through a file and removes '\n' within a record but not the record delimiter.

However, there are instances where there are MULTIPLE instances of '\n' within the record. How do a modify this code to account for multiple instance of \n and not remove the record delimiter?

It is a fixed width file with each record of 700 characters in length.

Please help.
Thanks
# 2  
Old 05-12-2009
show a sample of file.
# 3  
Old 05-13-2009
There are actually 2 issues
1. A fixed with record with multiple \n instances with \n as its record delimiter
2. A comma delimited file with \n embedded in the record with \n as its record delimiter.

Expected Result
1234abcd4569My Name is JackSmithJS1231900-01-01

Example (Fixed Width - with multiple \n within record)
1234abcd4569My Name is Jack
SmithJS123
1900-01-01

Example (Comma Delimted - with multiple \n within record)
1234,"abcd",4569,"My Name is Jack
Smith","JS123
",1900-01-01

In both cases, I do not want to remove the \n as its record delimiter

Last edited by CKT_newbie88; 05-13-2009 at 12:54 PM..
# 4  
Old 05-13-2009
Can you provide the expected o/p for the above 2 inputs?
# 5  
Old 05-13-2009
Give an exact format of your file, not only lines you want to combine. Post an example within code tags.

Regards
# 6  
Old 05-13-2009
Due to sensitive data, the following is a sample of 4 lines of data (in reality, the file contains 18 fields with the 11th field being problematic - since it is a Varchar 256 free-from field). All records have a \n as its record delimiter.

RecordID Integer
NameID Char
SubRecID Integer
Desc Char100
UserCD Char
Date Date

Sample Rows (FTP'd from Mainframe) - Comma Delimited
2345,"wxyz",2345,"Her Name is Nancy Drew","ND001","1900-01-01"
1234,"abcd",4569,"My Name is
Jack
Smith","JS123","1900-01-01"
5667,"gghd",9984,"His Name is
Joe Hardy","JH007","1900-01-01"
3333,"aaaa",9999,"Our Group is Excel Point","EP009","1900-01-01"

First Row shows stantdard format (no issue)
Second Row shows 2 embedded \n instances
Third Row shows 1 embedded \n instance
Fourth Row shows standard format (no issue)

Expected Output:
2345,"wxyz",2345,"Her Name is Nancy Drew","ND001","1900-01-01"
1234,"abcd",4569,"My Name is Jack Smith","JS123","1900-01-01"
5667,"gghd",9984,"His Name is Joe Hardy","JH007","1900-01-01"
3333,"aaaa",9999,"Our Group is Excel Point","EP009","1900-01-01"

Thanks
# 7  
Old 05-13-2009
Try this:

Code:
awk -F, '{printf("%s%s", $0,$NF ~ /[0-9]-[0-9]/?RS:"")}' file

Regards
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Removing Header and Trailer record of a EBCDIC file

I have a EBCDIC multi layout file which has a header record which is 21 bytes, The Detail records are 2427 bytes long and the trailer record is 9 bytes long. Is there a command to remove the header as well as trailer record and read only the detail records while at the same time not altering... (1 Reply)
Discussion started by: abhilashnair
1 Replies

2. Shell Programming and Scripting

How to compare current record,with next and previous record in awk without using array?

Hi! all can any one tell me how to compare current record of column with next and previous record in awk without using array my case is like this input.txt 0 32 1 26 2 27 3 34 4 26 5 25 6 24 9 23 0 32 1 28 2 15 3 26 4 24 (7 Replies)
Discussion started by: Dona Clara
7 Replies

3. UNIX for Dummies Questions & Answers

Doubts About awk, and Gawk

well i have some doubts about the use of this commands: my first doubt is to know if there is a way to execute a awk program from a file? (now i do copy paste, i copy the script of a notepad on the terminal and then i press enter, but i want to put this scripts in some folder and execute them)... (3 Replies)
Discussion started by: matius_88
3 Replies

4. UNIX for Dummies Questions & Answers

gawk asort to sort record groups based on one subfield

input ("/" delimited fields): style1/book1 (author_C)/editor1/2000 style1/book2 (author_A)/editor2/2004 style1/book3 (author_B)/editor3/2001 style2/book8 (author_B)/editor4/2010 style2/book5 (author_A)/editor2/1998 Records with same field 1 belong to the same group. Using asort (not sort),... (3 Replies)
Discussion started by: lucasvs
3 Replies

5. Shell Programming and Scripting

Removing spaces from record

HI i have record as shown below 402665,4X75,754X_FERNIE BC,12F2,008708,FERNIE BC,1,UTC ,UTC ,250 402665,4X75,754X_FERNIE BC,F212,008708,FERNIE BC,1,UTC ,UTC ,250 402665,4Y75,754Y_FERNIE BC,22F2,008708,FERNIE BC,1,UTC ,UTC ,250 here i want to remove multiple spaces into no... (3 Replies)
Discussion started by: raghavendra.cse
3 Replies

6. Shell Programming and Scripting

Removing \n within a fixed width record

I am trying to remove a line feed (\n) within a fixed width record. I tried the tr -d ‘\n' command, but it also removes the record delimiter. Is there a way to remove the line feed without removing the record delimiter? (10 Replies)
Discussion started by: CKT_newbie88
10 Replies

7. Shell Programming and Scripting

Removing duplicate field from MARC Record

Hello, I'm new to Perl programming and I have a duplicate 035 tag Voyager application field. The first 035 tag has the information I need but the second 035 tag created the bib id, which I don't need. This incident was performed on several records so I would have to run this script on several... (1 Reply)
Discussion started by: rcnick
1 Replies

8. Shell Programming and Scripting

Substitution using awk/gawk

Hello, I have a file containing lines such as: (1 104 (16) (17) (18) (102))$ (1 105 (16) (17) (19:21) (102))$ I would like to extract the numbers, only by using awk (or gawk). I do not want to use "sed" as it is very slow. For now my solution consists in... (2 Replies)
Discussion started by: jolecanard
2 Replies

9. Shell Programming and Scripting

awk,gawk in bat file

Hi. I'm trying to convert bat file into shell script. Bat file invokes awk file in one section: c:\upg\exe\gawk -f c:\upg\awk\gen_sae.awk -v OP=C:\\upg\\lod\\... ...c:\upg\ref\saaxi.ref c:\upg\log\SAAEPWO.log c:\upg\ref\saaepref.log First of all I issued unix2dos command on that awk file.... (0 Replies)
Discussion started by: andrej
0 Replies

10. Shell Programming and Scripting

Removing Carriage return to create one record

I have a file with multiple records in it and want to create a single record by removing all the carriage returns, is there a sed command or another command that will easily allow this to happen. current layout 813209A 813273C 812272B expected result 813209A813273C812272B previously I... (3 Replies)
Discussion started by: r1500
3 Replies
Login or Register to Ask a Question