Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Newline between unequal record fields
Top Forums Shell Programming and Scripting Newline between unequal record fields Post 302780557 by DGPickett on Thursday 14th of March 2013 04:37:59 PM
Old 03-14-2013
One approach is to make the fields all lines - homogenous if separated, but my standard sed looper is fine for merging lines:
Code:
tr ' ' '\12' < in_file | sed '
  :loop
  $q
  N
  s/\(....\)\n\1/\1 \1/
  t loop
  P
  s/.*\n//
  t loop
 ' > out_file

But this might mess up for two lines of the same number. In some apps, that might be great; you can put a "| sort" after the "tr" and merge far separated numbers, or a "| sort | uniq -c" and reduce them to a count.

Maybe pure sed is actually better yet:
Code:
sed '
  s/ /\
/g
  s/\(....\)\n\1/\1 \1/g
  s/\(....\)\n\1/\1 \1/g
 ' in_file > out_file

Cheap trick, making all the spaces line feeds and then making them back into spaces where equal. There's a lesson about negative cases there. Mostly, line feed was a certainly not in use substitute character. Once I swapped line feed and form feed so I could sed pages into insert statements (one page per row in one column) and then reversed the fomr feeds back to line feeds. Note that you have to sub twice, for the odd and even spaces. Also, if you know what a string is, you do not have to source the original bytes, any dup quad looks the same!
Last edited by DGPickett; 03-14-2013 at 05:53 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk: record has too many fields

Hi, I'm trying this command - but get this error. Do you guys have any workaround for this? cat tf|sed 's/{//g'|sed 's/,//g'|awk '{for (i=1;i<=NF;i++) {if ($i == "OPTIME") {k = i + 2; print $i,$k}}}' awk: record `2005 Jul 28 17:35:29...' has too many fields record number 15 This is how... (3 Replies)
Discussion started by: chaandana
3 Replies

2. Shell Programming and Scripting

Manipulating fields record wise

Hi all, I have an input file with no delimiter. Let us say the file is abc.txt having values for fields namely, EmpNumEnameDesigSalDept. Ofcourse the file has got several records. Every field has got a fixed start and end position. I need to assign the fields to corresponding varibles say... (1 Reply)
Discussion started by: rinku11
1 Replies

3. Shell Programming and Scripting

Making changes in the fields of a record

:confused: Hi Friends, In the record below i have to make changes in the fields by putting the values stored in the temporary variables, x, y, z, p, q, r: 2) In the TBT record store the values in the various fields as: a) X in a field position 51 to 56 b) Y... (5 Replies)
Discussion started by: kanu_pathak
5 Replies

4. Shell Programming and Scripting

remove newline chars in each record of file

Hi, I have a fixed width file with record length 10. I need to remove multiple newline characters present in each record. EX: af\n72/7\n s\n3\nad\n 2\n\n33r\n In the above file I want to remove new lines in red color(\n) but not (\n) Please provide me a solution. Thanks, Sri (1 Reply)
Discussion started by: srilaxmi
1 Replies

5. Shell Programming and Scripting

awk puts newline between fields

I have a='123, abc, def, ghi' var1=`echo $a | awk -F", " '{print RS $1}'` echo "something: $var1" which outputs something 123 how can I tell awk not to put a newline between fields? I want it to output: something: 123 (4 Replies)
Discussion started by: unclecameron
4 Replies

6. Shell Programming and Scripting

Remove newline character or join the broken record

Hi, I have a very huge file, around 1GB of data. I want to remove the newline characters in the file but not preceded by the original end delimiter {} sample data will look like this 1234567 abcd{} 1234sssss as67 abcd{} 12dsad3dad 4sdad567 abcdsadd{} this should look like this... (6 Replies)
Discussion started by: ratheeshjulk
6 Replies

7. Shell Programming and Scripting

Newline characters in fields of a file

My source file is pipe delimeted file with 53 fields.In 33 rd column i am getting mutlple new line characters,dule to that record is breaking into multiple records. Note : here record delimter also \n sample Source file with 6 fields : 1234|abc| \nabcd \n bvd \n cde \n |678|890|900\n ... (6 Replies)
Discussion started by: lakshmi001
6 Replies

8. Shell Programming and Scripting

Delete last 2 fields from every record in a file

Sample file record : "20130617003","2013-06-18T07:00:03","OUTWARD","01001011","TEST PLC","","HFX834346364364","20130617","10","DUM87534758","","1.28","826","020201","65879278","","","","","","010101","56789","DUMMY... (3 Replies)
Discussion started by: bigbuk
3 Replies

9. Shell Programming and Scripting

awk - compare 1st 15 fields of record with 20 fields

I'm trying to compare 2 files for differences in a selct number of fields. When differnces are found it will write the whole record of the second file including appending '|C' out to a delta file. Each record will have 20 fields, but only want to do comparison of 1st 15 fields. The 1st field of... (7 Replies)
Discussion started by: sljnk
7 Replies

10. Shell Programming and Scripting

Removinf newline characters in first 62 fields

Hi All, I receive a | delimited text file containing 63 columns. There is no delimiter at the end of the 63rd field, instead there would be a newline character at the end of the text in 63rd column. I wanted to retain this newline character at the end of the 63rd column, as it is desired newline... (1 Reply)
Discussion started by: sagarparadkar
1 Replies

LEARN ABOUT DEBIAN

shtool-subst

SHTOOL-SUBST.TMP(1)					      GNU Portable Shell Tool					       SHTOOL-SUBST.TMP(1)

NAME

       shtool-subst - GNU shtool sed(1) substitution operations

SYNOPSIS

       shtool subst [-v|--verbose] [-t|--trace] [-n|--nop] [-w|--warning] [-q|--quiet] [-s|--stealth] [-i|--interactive] [-b|--backup ext]
       [-e|--exec cmd] [-f|--file cmd-file] [file] [file ...]

DESCRIPTION

       This command applies one or more sed(1) substitution operations to stdin or any number of files.

OPTIONS

       The following command line options are available.

       -v, --verbose
	   Display some processing information.

       -t, --trace
	   Enable the output of the essential shell commands which are executed.

       -n, --nop
	   No operation mode. Actual execution of the essential shell commands which would be executed is suppressed.

       -w, --warning
	   Show warning on substitution operation resulting in no content change on every file. The default is to show a warning on substitution
	   operations resulted in no content change on all files.

       -q, --quiet
	   Suppress warning on substitution operation resulting in no content change.

       -s, --stealth
	   Stealth operation. Preserve timestamp on file.

       -i, --interactive
	   Enter interactive mode where the user has to approve each operation.

       -b, --backup ext
	   Preserve backup of original file using file name extension ext.  Default is to overwrite the original file.

       -e, --exec cmd
	   Specify sed(1) command directly.

       -f, --file cmd-file
	   Read sed(1) command from file.

EXAMPLE

	#   shell script
	shtool subst -i -e 's;(c) ([0-9]*)-2000;(c) 1-2001;' *.[ch]

	#    RPM spec-file
	%install
	    shtool subst -v -n 
		-e 's;^(prefix=).*;1 $RPM_BUILD_ROOT%{_prefix};g' 
		-e 's;^(sysconfdir=).*;1 $RPM_BUILD_ROOT%{_prefix}/etc;g' 
		`find . -name Makefile -print`
	    make install

HISTORY

       The GNU shtool subst command was originally written by Ralf S.  Engelschall <rse@engelschall.com> in 2001 for GNU shtool.  It was prompted
       by the need to have a uniform and convenient patching frontend to sed(1) operations in the OpenPKG package specifications.

SEE ALSO

       shtool(1), sed(1).

18-Jul-2008							   shtool 2.0.8 					       SHTOOL-SUBST.TMP(1)
All times are GMT -4. The time now is 06:54 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy