Fixed width file issue and resolutions


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
# 1  
Fixed width file issue and resolutions

I have a fixed width file with no separators , but the file has JUNK characters in it and i know how to remove it. but in few cases these junk characters having created a mess by splitting single row into 2 or 3 rows.
I need to put them back to a single line like rest of the rows. This fixed width file as rows which is of length 1-484.

The only way to identify that the row is spilt into 2 or 3 rows is by its length only and that too is not consistent.

Example :
Row expected :
Code:
0401000000030020170628000000710000366400201706280002750035*T000100N▒101892928550383900000009201 00000000000008402017062800000020  0000006435000000000000000840A1E098D09D9279BE4000561510A00003220000000000000000000000FF                            05A17DD6318008000048000        83337602E0F8C80002420300    00000000000000                                                   0000002278BF8AC19546B1EE0012            A0000000042203                  A0000000042203                  R 000000199551

Row with the issue : CASE1 :
Code:
0401000000030020170628000000710000365400201706280001041125
T000100N▒101909856755446700000018201 00000000000008402017062800000000  00000067540000000000000008402ED730917E9D1DC040000B0810A04003240000000000000000000000FF                            083426C5739000000001000        83337602E0F8C800021E0300    00000000000000                                                   00000022F85D9594E4FC84E2031A            A0000000041010                  A0000000041010                  R 000000199442

Row with the issue : CASE2 :

Code:
0401000000480020170628000000830000656500201706280004755339
T000100N 
0000034655000000000000000840A6258473ADBF88E24000740110A04001220000000000000000000000FF                            06E09E43739000000008000        88131669E0F8C800021E0300    00000000000000                                                   00000022B050CC3E5A75C2E90012            A0000000041010                  A0000000041010                  R 000002274242


Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!

Last edited by RudiC; 07-19-2017 at 04:21 AM.. Reason: Added CODE tags.
# 2  
Welcome to the forum.

Why don't you post the method / script / code that you use to remove the junk characters? It might be better to avoid the errors in the first place than to run a second code to deal with them.
# 3  
Welcome santoshkm,

I have a few to questions pose in response first:-
  • What have you tried so far?
  • What output/errors do you get?
  • Can you show us the data piped through od -x so we can see the character codes? Having the original data on the next line would be useful too.
  • What OS and version are you using?
  • What are your preferred tools? (C, shell, perl, awk, etc.)
  • What logical process have you considered? (to help steer us to follow what you are trying to achieve)
Most importantly, What have you tried so far?

There are probably many ways to achieve most tasks, so giving us an idea of your style and thoughts will help us guide you to an answer most suitable to you so you can adjust it to suit your needs in future.


We're all here to learn and getting the relevant information will help us all.


Thanks, in advance,
Robin
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk issue splitting a fixed-width file containing line feed in data

Hi Forum. I have the following script that splits a large fixed-width file into smaller multiple fixed-width files based on input segment type. The main command in the script is: awk -v search_col_pos=$search_col_pos -v search_str_len=$search_str_len -v segment_type="$segment_type"... (8 Replies)
Discussion started by: pchang
8 Replies

2. Shell Programming and Scripting

Replace using awk on fixed width file.

All, I used to use following command to replace specific location in a fixed width file. Recently looks like my command stopped working as intended. We are on AIX unix. awk 'function repl(s,f,t,v) { return substr(s,1,f-1) sprintf("%-*s", t-f+1, v) substr(s,t+1) } NR<=10 {... (3 Replies)
Discussion started by: pinnacle
3 Replies

3. Shell Programming and Scripting

Alter Fixed Width File

Thank u so much .Its working fine as expected. ---------- Post updated at 03:41 PM ---------- Previous update was at 01:46 PM ---------- I need one more help. I have another file(fixed length) that will get negative value (ex:-00000000003000) in postion (98 - 112) then i have to... (6 Replies)
Discussion started by: vinus
6 Replies

4. Shell Programming and Scripting

To replace the value of the column in a fixed width file

I have a fixed with file with header & trailer length having the same length of the detail record file. The details record length of this file is 24, for Header and Trailer the records will be padded with spaces to match the record length of the file Currently I am adding 3 spaces in header... (14 Replies)
Discussion started by: ginrkf
14 Replies

5. UNIX for Dummies Questions & Answers

Length of a fixed width file

I have a fixed width file of length 53. when is try to get the lengh of the record of that file i get 2 different answers. awk '{print length;exit}' <File_name> The above code gives me length 50. wc -L <File_name> The above code gives me length 53. Please clarify on... (2 Replies)
Discussion started by: Amrutha24
2 Replies

6. Shell Programming and Scripting

Comparing two fixed width file

Hi Guys I am checking the treads to get the answer but i am not able to get the answer for my question. I have two files. First file is a pattern file and the second file is the file i want to search in it. Output will be the lines from file2. File1: P2797f12af 44751228... (10 Replies)
Discussion started by: anshul_er
10 Replies

7. Shell Programming and Scripting

Fixed-Width file from Oracle

Hi All, I have created a script which generates FIXED-WIDTH file by executing Oracle query. SELECT RPAD(NVL(col1,CHR(9)),20)||NVL(col2,CHR(9))||NVL(col3,CHR(9) FROM XYZ It generates the data file with proper alignment. But if same file i transfer to windows server or Mainframe... (5 Replies)
Discussion started by: Amit.Sagpariya
5 Replies

8. UNIX Desktop Questions & Answers

Help with Fixed width File Parsing

I am trying to parse a Fixed width file with data as below. I am trying to assign column values from each record to variables. When I parse the data, the spaces in all coumns are dropped. I would like to retain the spaces as part of the dat stored in the variables. Any help is appreciated. I... (4 Replies)
Discussion started by: sate911
4 Replies

9. Shell Programming and Scripting

Changing particular field in fixed width file

I have a fixed width file and i need to change 36th field to "G" in for about random 20 records? How can I do it? (4 Replies)
Discussion started by: dsravan
4 Replies

10. UNIX for Dummies Questions & Answers

Fixed Width file using AWK

I am using the following command at the Unix prompt to make my 'infile' into a fixed width file of 100 characters. awk '{printf "%-100s\n",$0}' infile > outfile However, there are some records with a special character "" These records are using 3 characters in place of one and my record... (2 Replies)
Discussion started by: alok.benjwal
2 Replies

Featured Tech Videos