Shell Programming and Scripting

Hi Robin,
We are getting a fixed length file got compressed with special character "ú07 & ú1A?, etc.," in between the characters, need to read the special characters along with the next two bytes hexadecimal value. Once we read the hexadecimal value need to convert to decimal and add mentioned byte (symbol or space) after hexadecimal value in a file. We tried using awk, sed but no progress in output. we need some sample script or information to run the script on SunOS and drive further.

Thanks,
Dines

Is this packed-decimal data from a mainframe perhaps? It would probably be easier to generate the file as truly plain text at the source before transferring it. If you are using FTP, make sure the transfer is forced to be an ASCII transfer.

From your example in the first post, I think what you want is to read the 1A? to mean 'please insert 26 (decimal) question marks' and any other time we hit the special character in the same line or any other line.

It makes it all a bit complex, hence why I suggest you generate a fully expanded file at the source. If it won't fit, or the transfer takes too long, then there are commercial compression tools that are available for pretty much any platform combination. We changed one transfer from 23 hours to 4 by using one, but there will be others out there.

What is your source system?



Robin

Hi Robin,

We encounter fixed format special characters like "? or spaces" in the file between characters after hexadecimal values. The file got FTP from windows machine, provide some information how to handle the situtation using UNIX script.

Thanks,
Dines

Is this a Windows compressed file, a Winzip file or something else?

It may be possible to expand this with gunzip or similar utilities if these are available to you, but we'd need to know how it is generated in the first place.


Robin

The substitution regex above matches any character which is not a letter, number or space (as defined by the current locale) followed by 2 characters that could be interpreted as hexadecimal, followed by any character and replaces them with the 3 spaces followed by the character repeated n times.

The fact that Perl allows executable code in the substitution block means we can do things like this not available in sed or awk as a single substitution.

The e flag marks the substitution block for evaluation, the g flag would allow the substitution to be applied globally rather than to just the first match.

man perlre for more details on Perl regex

Hi Robin,

File FTP from windows system without any compression, only data got compressed, we need to expand the data by finding the special character ie. "ú07 & ú1A?.," lies in the file and to pad the next byte ie(space or ?) after the hexadecimal value convert to decimal.

Sample Data:
Input Data:

Code:

ABCD172 2 B10001ú0E Fú08 DineshGú14 KumarNakkaú0E ú3C?INú14?EFGH340

Output Data:

Code:

ABCD172 2 B10001   (14 spaces) DineshG   (14 spaces) KumarNakka   (14 spaces)    ??????(60-?symbol)IN   ??????????????EFGH340

Thanks,
Dines

Moderator's Comments:

Please use CODE tags as in the forum rules. You have two warning already. Using CODE tags means that the output will be fixed-width characters and multiple spaces will be respected

From your input of:-

Code:

ABCD172 2 B10001ú0E Fú08 DineshGú14 KumarNakkaú0E ú3C?INú14?EFGH340

.....I would expect this:-

Code:

ABCD172 2 B10001              F        DineshG                    KumarNakka              ????????????????????????????????????????????????????????????IN????????????????????EFGH340

.... which does not match your description of the expected output.

I'd still like to pursue how this is generated in the first place and see if there is a simpler alternative.



Robin