Today (Saturday) We will make some minor tuning adjustments to MySQL.

You may experience 2 up to 10 seconds "glitch time" when we restart MySQL. We expect to make these adjustments around 1AM Eastern Daylight Saving Time (EDT) US.

Fixed width file issue and resolutions

Login or Register to Reply

Thread Tools Search this Thread
# 1  
Fixed width file issue and resolutions

I have a fixed width file with no separators , but the file has JUNK characters in it and i know how to remove it. but in few cases these junk characters having created a mess by splitting single row into 2 or 3 rows.
I need to put them back to a single line like rest of the rows. This fixed width file as rows which is of length 1-484.

The only way to identify that the row is spilt into 2 or 3 rows is by its length only and that too is not consistent.

Example :
Row expected :
0401000000030020170628000000710000366400201706280002750035*T000100N▒101892928550383900000009201 00000000000008402017062800000020  0000006435000000000000000840A1E098D09D9279BE4000561510A00003220000000000000000000000FF                            05A17DD6318008000048000        83337602E0F8C80002420300    00000000000000                                                   0000002278BF8AC19546B1EE0012            A0000000042203                  A0000000042203                  R 000000199551

Row with the issue : CASE1 :
T000100N▒101909856755446700000018201 00000000000008402017062800000000  00000067540000000000000008402ED730917E9D1DC040000B0810A04003240000000000000000000000FF                            083426C5739000000001000        83337602E0F8C800021E0300    00000000000000                                                   00000022F85D9594E4FC84E2031A            A0000000041010                  A0000000041010                  R 000000199442

Row with the issue : CASE2 :

0000034655000000000000000840A6258473ADBF88E24000740110A04001220000000000000000000000FF                            06E09E43739000000008000        88131669E0F8C800021E0300    00000000000000                                                   00000022B050CC3E5A75C2E90012            A0000000041010                  A0000000041010                  R 000002274242

Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!

Last edited by RudiC; 07-19-2017 at 04:21 AM.. Reason: Added CODE tags.
# 2  
Welcome to the forum.

Why don't you post the method / script / code that you use to remove the junk characters? It might be better to avoid the errors in the first place than to run a second code to deal with them.
# 3  
Welcome santoshkm,

I have a few to questions pose in response first:-
  • What have you tried so far?
  • What output/errors do you get?
  • Can you show us the data piped through od -x so we can see the character codes? Having the original data on the next line would be useful too.
  • What OS and version are you using?
  • What are your preferred tools? (C, shell, perl, awk, etc.)
  • What logical process have you considered? (to help steer us to follow what you are trying to achieve)
Most importantly, What have you tried so far?

There are probably many ways to achieve most tasks, so giving us an idea of your style and thoughts will help us guide you to an answer most suitable to you so you can adjust it to suit your needs in future.

We're all here to learn and getting the relevant information will help us all.

Thanks, in advance,
Login or Register to Reply

Thread Tools Search this Thread
Search this Thread:
Advanced Search

More UNIX and Linux Forum Topics You Might Find Helpful
awk issue splitting a fixed-width file containing line feed in data
Hi Forum. I have the following script that splits a large fixed-width file into smaller multiple fixed-width files based on input segment type. The main command in the script is: awk -v search_col_pos=$search_col_pos -v search_str_len=$search_str_len -v segment_type="$segment_type"...... Shell Programming and Scripting
Shell Programming and Scripting
Alter Fixed Width File
Thank u so much .Its working fine as expected. ---------- Post updated at 03:41 PM ---------- Previous update was at 01:46 PM ---------- I need one more help. I have another file(fixed length) that will get negative value (ex:-00000000003000) in postion (98 - 112) then i have to...... Shell Programming and Scripting
Shell Programming and Scripting
Length of a fixed width file
I have a fixed width file of length 53. when is try to get the lengh of the record of that file i get 2 different answers. awk '{print length;exit}' <File_name> The above code gives me length 50. wc -L <File_name> The above code gives me length 53. Please clarify on...... UNIX for Dummies Questions & Answers
UNIX for Dummies Questions & Answers
Comparing two fixed width file
Hi Guys I am checking the treads to get the answer but i am not able to get the answer for my question. I have two files. First file is a pattern file and the second file is the file i want to search in it. Output will be the lines from file2. File1: P2797f12af 44751228...... Shell Programming and Scripting
Shell Programming and Scripting
Fixed Width file using AWK
I am using the following command at the Unix prompt to make my 'infile' into a fixed width file of 100 characters. awk '{printf "%-100s\n",$0}' infile > outfile However, there are some records with a special character "" These records are using 3 characters in place of one and my record...... UNIX for Dummies Questions & Answers
UNIX for Dummies Questions & Answers

Featured Tech Videos