Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Newline between unequal record fields

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Newline between unequal record fields
# 1  
Old 03-14-2013
Newline between unequal record fields
Assume the following 5 records (field separator is a space):
Code:
0903 0903 0910 0910 0910 0910 0910 0910 0917 0917 0917 0917 0924
1001 1001 1001 1001 1008 1008 1008 1008 1015 1015 1015 1015 1022
1029 1029 1029 1029 1105 1105 1105 1105 1112 1112 1112 1112 1119
1126 1126 1126 1126 1203 1203 1203 1203 1210 1210 1210 1210 1217
1224 1224 1224 1224 1224 1224 1224 1224 1231 1231 1231 1231

The output result needed:
Code:
0903 0903
0910 0910 0910 0910 0910 0910
0917 0917 0917 0917
0924
1001 1001 1001 1001
1008 1008 1008 1008
1015 1015 1015 1015
1022
1029 1029 1029 1029
1105 1105 1105 1105
1112 1112 1112 1112
1119
1126 1126 1126 1126
1203 1203 1203 1203
1210 1210 1210 1210
1217
1224 1224 1224 1224 1224 1224 1224 1224
1231 1231 1231 1231

Assume additional records will have different values. Without doing this by hand I've been unable solve it. I tried using a combination of sed, awk, and grep scripts with no success. Any help would be appreciated.
Last edited by Scrutinizer; 03-14-2013 at 02:29 PM.. Reason: code tags
# 2  
Old 03-14-2013
Try:
Code:
awk '{for(i=1; i<NF; i++) $i=$i ($(i+1)==$i?FS:RS)}1' OFS=

# 3  
Old 03-14-2013
Totally answered my problem. Been working on this for a week. Finished reading the O'Reilly book on bash scripting but could find the answer. It's good but doesn't go into too much detail on sed or awk. Thanks.
# 4  
Old 03-14-2013
awk is its own programming language, it's hard to go over it in detail without it becoming its own book. Not a difficult language mind you, but quite different.
# 5  
Old 03-14-2013
try also (in case same values wrap on the next line):
Code:
awk '{w=(w)?w:$1;for(i=1; i<=NF; i++) {printf ($i==w)? $i" ":"\n"$i" "; w=$i}} END {print ""}' infile

# 6  
Old 03-14-2013
If you are interested, here is a solution using bash:
Code:
#!/bin/bash

while read line
do
        for c_num in $line
        do
                [[ "$c_num" == "$p_num" ]] && printf "%s " $c_num || printf "\n%s " $c_num
                p_num="$c_num"
        done
done < file
printf "\n"

Last edited by Yoda; 03-14-2013 at 04:44 PM.. Reason: correction
# 7  
Old 03-14-2013
One approach is to make the fields all lines - homogenous if separated, but my standard sed looper is fine for merging lines:
Code:
tr ' ' '\12' < in_file | sed '
  :loop
  $q
  N
  s/\(....\)\n\1/\1 \1/
  t loop
  P
  s/.*\n//
  t loop
 ' > out_file

But this might mess up for two lines of the same number. In some apps, that might be great; you can put a "| sort" after the "tr" and merge far separated numbers, or a "| sort | uniq -c" and reduce them to a count.

Maybe pure sed is actually better yet:
Code:
sed '
  s/ /\
/g
  s/\(....\)\n\1/\1 \1/g
  s/\(....\)\n\1/\1 \1/g
 ' in_file > out_file

Cheap trick, making all the spaces line feeds and then making them back into spaces where equal. There's a lesson about negative cases there. Mostly, line feed was a certainly not in use substitute character. Once I swapped line feed and form feed so I could sed pages into insert statements (one page per row in one column) and then reversed the fomr feeds back to line feeds. Note that you have to sub twice, for the odd and even spaces. Also, if you know what a string is, you do not have to source the original bytes, any dup quad looks the same!
Last edited by DGPickett; 03-14-2013 at 05:53 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removinf newline characters in first 62 fields

Hi All, I receive a | delimited text file containing 63 columns. There is no delimiter at the end of the 63rd field, instead there would be a newline character at the end of the text in 63rd column. I wanted to retain this newline character at the end of the 63rd column, as it is desired newline... (1 Reply)
Discussion started by: sagarparadkar
1 Replies

2. Shell Programming and Scripting

awk - compare 1st 15 fields of record with 20 fields

I'm trying to compare 2 files for differences in a selct number of fields. When differnces are found it will write the whole record of the second file including appending '|C' out to a delta file. Each record will have 20 fields, but only want to do comparison of 1st 15 fields. The 1st field of... (7 Replies)
Discussion started by: sljnk
7 Replies

3. Shell Programming and Scripting

Delete last 2 fields from every record in a file

Sample file record : "20130617003","2013-06-18T07:00:03","OUTWARD","01001011","TEST PLC","","HFX834346364364","20130617","10","DUM87534758","","1.28","826","020201","65879278","","","","","","010101","56789","DUMMY... (3 Replies)
Discussion started by: bigbuk
3 Replies

4. Shell Programming and Scripting

Newline characters in fields of a file

My source file is pipe delimeted file with 53 fields.In 33 rd column i am getting mutlple new line characters,dule to that record is breaking into multiple records. Note : here record delimter also \n sample Source file with 6 fields : 1234|abc| \nabcd \n bvd \n cde \n |678|890|900\n ... (6 Replies)
Discussion started by: lakshmi001
6 Replies

5. Shell Programming and Scripting

Remove newline character or join the broken record

Hi, I have a very huge file, around 1GB of data. I want to remove the newline characters in the file but not preceded by the original end delimiter {} sample data will look like this 1234567 abcd{} 1234sssss as67 abcd{} 12dsad3dad 4sdad567 abcdsadd{} this should look like this... (6 Replies)
Discussion started by: ratheeshjulk
6 Replies

6. Shell Programming and Scripting

awk puts newline between fields

I have a='123, abc, def, ghi' var1=`echo $a | awk -F", " '{print RS $1}'` echo "something: $var1" which outputs something 123 how can I tell awk not to put a newline between fields? I want it to output: something: 123 (4 Replies)
Discussion started by: unclecameron
4 Replies

7. Shell Programming and Scripting

remove newline chars in each record of file

Hi, I have a fixed width file with record length 10. I need to remove multiple newline characters present in each record. EX: af\n72/7\n s\n3\nad\n 2\n\n33r\n In the above file I want to remove new lines in red color(\n) but not (\n) Please provide me a solution. Thanks, Sri (1 Reply)
Discussion started by: srilaxmi
1 Replies

8. Shell Programming and Scripting

Making changes in the fields of a record

:confused: Hi Friends, In the record below i have to make changes in the fields by putting the values stored in the temporary variables, x, y, z, p, q, r: 2) In the TBT record store the values in the various fields as: a) X in a field position 51 to 56 b) Y... (5 Replies)
Discussion started by: kanu_pathak
5 Replies

9. Shell Programming and Scripting

Manipulating fields record wise

Hi all, I have an input file with no delimiter. Let us say the file is abc.txt having values for fields namely, EmpNumEnameDesigSalDept. Ofcourse the file has got several records. Every field has got a fixed start and end position. I need to assign the fields to corresponding varibles say... (1 Reply)
Discussion started by: rinku11
1 Replies

10. Shell Programming and Scripting

awk: record has too many fields

Hi, I'm trying this command - but get this error. Do you guys have any workaround for this? cat tf|sed 's/{//g'|sed 's/,//g'|awk '{for (i=1;i<=NF;i++) {if ($i == "OPTIME") {k = i + 2; print $i,$k}}}' awk: record `2005 Jul 28 17:35:29...' has too many fields record number 15 This is how... (3 Replies)
Discussion started by: chaandana
3 Replies
Login or Register to Ask a Question