Sponsored Content
Top Forums Shell Programming and Scripting Newline between unequal record fields Post 302780557 by DGPickett on Thursday 14th of March 2013 04:37:59 PM
Old 03-14-2013
One approach is to make the fields all lines - homogenous if separated, but my standard sed looper is fine for merging lines:
Code:
tr ' ' '\12' < in_file | sed '
  :loop
  $q
  N
  s/\(....\)\n\1/\1 \1/
  t loop
  P
  s/.*\n//
  t loop
 ' > out_file

But this might mess up for two lines of the same number. In some apps, that might be great; you can put a "| sort" after the "tr" and merge far separated numbers, or a "| sort | uniq -c" and reduce them to a count.

Maybe pure sed is actually better yet:
Code:
sed '
  s/ /\
/g
  s/\(....\)\n\1/\1 \1/g
  s/\(....\)\n\1/\1 \1/g
 ' in_file > out_file

Cheap trick, making all the spaces line feeds and then making them back into spaces where equal. There's a lesson about negative cases there. Mostly, line feed was a certainly not in use substitute character. Once I swapped line feed and form feed so I could sed pages into insert statements (one page per row in one column) and then reversed the fomr feeds back to line feeds. Note that you have to sub twice, for the odd and even spaces. Also, if you know what a string is, you do not have to source the original bytes, any dup quad looks the same!

Last edited by DGPickett; 03-14-2013 at 05:53 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk: record has too many fields

Hi, I'm trying this command - but get this error. Do you guys have any workaround for this? cat tf|sed 's/{//g'|sed 's/,//g'|awk '{for (i=1;i<=NF;i++) {if ($i == "OPTIME") {k = i + 2; print $i,$k}}}' awk: record `2005 Jul 28 17:35:29...' has too many fields record number 15 This is how... (3 Replies)
Discussion started by: chaandana
3 Replies

2. Shell Programming and Scripting

Manipulating fields record wise

Hi all, I have an input file with no delimiter. Let us say the file is abc.txt having values for fields namely, EmpNumEnameDesigSalDept. Ofcourse the file has got several records. Every field has got a fixed start and end position. I need to assign the fields to corresponding varibles say... (1 Reply)
Discussion started by: rinku11
1 Replies

3. Shell Programming and Scripting

Making changes in the fields of a record

:confused: Hi Friends, In the record below i have to make changes in the fields by putting the values stored in the temporary variables, x, y, z, p, q, r: 2) In the TBT record store the values in the various fields as: a) X in a field position 51 to 56 b) Y... (5 Replies)
Discussion started by: kanu_pathak
5 Replies

4. Shell Programming and Scripting

remove newline chars in each record of file

Hi, I have a fixed width file with record length 10. I need to remove multiple newline characters present in each record. EX: af\n72/7\n s\n3\nad\n 2\n\n33r\n In the above file I want to remove new lines in red color(\n) but not (\n) Please provide me a solution. Thanks, Sri (1 Reply)
Discussion started by: srilaxmi
1 Replies

5. Shell Programming and Scripting

awk puts newline between fields

I have a='123, abc, def, ghi' var1=`echo $a | awk -F", " '{print RS $1}'` echo "something: $var1" which outputs something 123 how can I tell awk not to put a newline between fields? I want it to output: something: 123 (4 Replies)
Discussion started by: unclecameron
4 Replies

6. Shell Programming and Scripting

Remove newline character or join the broken record

Hi, I have a very huge file, around 1GB of data. I want to remove the newline characters in the file but not preceded by the original end delimiter {} sample data will look like this 1234567 abcd{} 1234sssss as67 abcd{} 12dsad3dad 4sdad567 abcdsadd{} this should look like this... (6 Replies)
Discussion started by: ratheeshjulk
6 Replies

7. Shell Programming and Scripting

Newline characters in fields of a file

My source file is pipe delimeted file with 53 fields.In 33 rd column i am getting mutlple new line characters,dule to that record is breaking into multiple records. Note : here record delimter also \n sample Source file with 6 fields : 1234|abc| \nabcd \n bvd \n cde \n |678|890|900\n ... (6 Replies)
Discussion started by: lakshmi001
6 Replies

8. Shell Programming and Scripting

Delete last 2 fields from every record in a file

Sample file record : "20130617003","2013-06-18T07:00:03","OUTWARD","01001011","TEST PLC","","HFX834346364364","20130617","10","DUM87534758","","1.28","826","020201","65879278","","","","","","010101","56789","DUMMY... (3 Replies)
Discussion started by: bigbuk
3 Replies

9. Shell Programming and Scripting

awk - compare 1st 15 fields of record with 20 fields

I'm trying to compare 2 files for differences in a selct number of fields. When differnces are found it will write the whole record of the second file including appending '|C' out to a delta file. Each record will have 20 fields, but only want to do comparison of 1st 15 fields. The 1st field of... (7 Replies)
Discussion started by: sljnk
7 Replies

10. Shell Programming and Scripting

Removinf newline characters in first 62 fields

Hi All, I receive a | delimited text file containing 63 columns. There is no delimiter at the end of the 63rd field, instead there would be a newline character at the end of the text in 63rd column. I wanted to retain this newline character at the end of the 63rd column, as it is desired newline... (1 Reply)
Discussion started by: sagarparadkar
1 Replies
COL(1)							    BSD General Commands Manual 						    COL(1)

NAME
col -- filter reverse line feeds from input SYNOPSIS
col [-bfpx] [-l num] DESCRIPTION
col filters out reverse (and half reverse) line feeds so that the output is in the correct order with only forward and half forward line feeds, and replaces white-space characters with tabs where possible. This can be useful in processing the output of nroff(1) and tbl(1). col reads from the standard input and writes to the standard output. The options are as follows: -b Do not output any backspaces, printing only the last character written to each column position. -f Forward half line feeds are permitted (``fine'' mode). Normally characters printed on a half line boundary are printed on the fol- lowing line. -p Force unknown control sequences to be passed through unchanged. Normally, col will filter out any control sequences from the input other than those recognized and interpreted by itself, which are listed below. -x Output multiple spaces instead of tabs. -l num Buffer at least num lines in memory. By default, 128 lines are buffered. The control sequences for carriage motion that col understands and their decimal values are listed in the following table: ESC-7 reverse line feed (escape then 7) ESC-8 half reverse line feed (escape then 8) ESC-9 half forward line feed (escape then 9) backspace moves back one column (8); ignored in the first column carriage return (13) newline forward line feed (10); also does carriage return shift in shift to normal character set (15) shift out shift to alternative character set (14) space moves forward one column (32) tab moves forward to next tab stop (9) vertical tab reverse line feed (11) All unrecognized control characters and escape sequences are discarded. col keeps track of the character set as characters are read and makes sure the character set is correct when they are output. If the input attempts to back up to the last flushed line, col will display a warning message. SEE ALSO
expand(1), nroff(1), tbl(1) STANDARDS
The col utility conforms to X/Open Portability Guide Issue 4, Version 2 (``XPG4.2''). The -l option is an extension to the standard. HISTORY
A col command appeared in Version 6 AT&T UNIX. BSD
February 22, 1999 BSD
All times are GMT -4. The time now is 10:37 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy