Show Password


Answers to Frequently Asked Questions

Answers to Beginners BSD, Linux, and UNIX shell scripting Frequently Asked Questions — This forum contains answers to frequently asked questions about how to use awk, bash, csh, ksh, perl, php, python, sed, sh, shell scripts, and other shell scripting languages. (Only Moderators and Administrators can add a thread to this forum. If you would like to make an addition to this forum, post your suggestion in the FAQ Submission Queue sub-forum.)

Fixed width file issue and resolutions

👤 Login to reply

Thread Tools Search this Thread Display Modes
Old 07-19-2017
santoshkm santoshkm is offline
Registered User
Join Date: Jul 2017
Last Activity: 19 July 2017, 2:14 AM EDT
Posts: 1
Thanks: 0
Thanked 0 Times in 0 Posts
Fixed width file issue and resolutions

I have a fixed width file with no separators , but the file has JUNK characters in it and i know how to remove it. but in few cases these junk characters having created a mess by splitting single row into 2 or 3 rows.
I need to put them back to a single line like rest of the rows. This fixed width file as rows which is of length 1-484.

The only way to identify that the row is spilt into 2 or 3 rows is by its length only and that too is not consistent.

Example :
Row expected :

0401000000030020170628000000710000366400201706280002750035*T000100N▒101892928550383900000009201 00000000000008402017062800000020  0000006435000000000000000840A1E098D09D9279BE4000561510A00003220000000000000000000000FF                            05A17DD6318008000048000        83337602E0F8C80002420300    00000000000000                                                   0000002278BF8AC19546B1EE0012            A0000000042203                  A0000000042203                  R 000000199551

Row with the issue : CASE1 :

T000100N▒101909856755446700000018201 00000000000008402017062800000000  00000067540000000000000008402ED730917E9D1DC040000B0810A04003240000000000000000000000FF                            083426C5739000000001000        83337602E0F8C800021E0300    00000000000000                                                   00000022F85D9594E4FC84E2031A            A0000000041010                  A0000000041010                  R 000000199442

Row with the issue : CASE2 :

0000034655000000000000000840A6258473ADBF88E24000740110A04001220000000000000000000000FF                            06E09E43739000000008000        88131669E0F8C800021E0300    00000000000000                                                   00000022B050CC3E5A75C2E90012            A0000000041010                  A0000000041010                  R 000002274242

Moderator's Comments:
Fixed width file issue and resolutions Please use CODE tags as required by forum rules!

Last edited by RudiC; 07-19-2017 at 03:21 AM.. Reason: Added CODE tags.
Sponsored Links
Old 07-19-2017
RudiC RudiC is offline Forum Staff  
Join Date: Jul 2012
Last Activity: 16 July 2018, 7:55 AM EDT
Location: Aachen, Germany
Posts: 13,059
Thanks: 447
Thanked 4,008 Times in 3,685 Posts
Welcome to the forum.

Why don't you post the method / script / code that you use to remove the junk characters? It might be better to avoid the errors in the first place than to run a second code to deal with them.
Sponsored Links
Old 07-19-2017
rbatte1 rbatte1 is offline Forum Staff  
Root armed
Join Date: Jun 2007
Last Activity: 13 July 2018, 1:03 PM EDT
Location: Lancashire, UK
Posts: 3,582
Thanks: 1,590
Thanked 706 Times in 633 Posts
Welcome santoshkm,

I have a few to questions pose in response first:-
  • What have you tried so far?
  • What output/errors do you get?
  • Can you show us the data piped through od -x so we can see the character codes? Having the original data on the next line would be useful too.
  • What OS and version are you using?
  • What are your preferred tools? (C, shell, perl, awk, etc.)
  • What logical process have you considered? (to help steer us to follow what you are trying to achieve)
Most importantly, What have you tried so far?

There are probably many ways to achieve most tasks, so giving us an idea of your style and thoughts will help us guide you to an answer most suitable to you so you can adjust it to suit your needs in future.

We're all here to learn and getting the relevant information will help us all.

Thanks, in advance,
Sponsored Links
👤 Login to reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
awk issue splitting a fixed-width file containing line feed in data pchang Shell Programming and Scripting 8 02-17-2017 03:34 PM
Alter Fixed Width File vinus Shell Programming and Scripting 6 08-27-2014 11:30 AM
Length of a fixed width file Amrutha24 UNIX for Dummies Questions & Answers 2 06-19-2013 01:54 AM
Help with Fixed width File Parsing sate911 UNIX Desktop Questions & Answers 4 05-19-2008 11:18 AM
Fixed Width file using AWK alok.benjwal UNIX for Dummies Questions & Answers 2 12-05-2005 10:39 AM

All times are GMT -4. The time now is 08:36 AM.

Unix & Linux Forums Content Copyright©1993-2018. All Rights Reserved.