I am trying to generate a file with the C Cedilla delimiter.
I have a file that uses the below DML (In Dev region, the file that i am trying to generate):
decimal("Ç") acct_id; and so and so for new columns
When I cat this file I get below output:
Code:
bankbtch@jackets:/prod/home10/data/serial/rtl_baseldw_us_card/rbuc_cr_bur/temp=> cat ori_test.dat | head -1
10045087403Ç07Ç-2Ç*2Ç*2Ç*2Ç*2Ç5Ç37630Ç40500Ç22000Ç20515Ç19Ç422Ç0Ç0Ç0Ç0Ç0Ç0Ç0Ç2Ç0Ç5Ç*2Ç38501Ç-2Ç-2Ç-2Ç-2ÇÇ-2Ç0Ç1Ç*2Ç*2Ç*2Ç*2Ç*2Ç*2Ç*2Ç*2Ç0Ç*2Ç-2Ç*2Ç0Ç*2Ç*2Ç-2Ç0Ç-2Ç-2Ç-2Ç-2Ç-2Ç-2Ç*2Ç*2Ç*2Ç*2Ç*2Ç*2Ç3Ç-2
But when i vi the same file I get the below output:
Code:
bankbtch@jackets:/prod/home10/data/serial/rtl_baseldw_us_card/rbuc_cr_bur/temp=> vi ori_test.dat
10045087403▒~G07▒~G-2▒~G*2▒~G*2▒~G*2▒~G*2▒~G5▒~G37630▒~G40500▒~G22000▒~G20515▒~G19▒~G422▒~G0▒~G0▒~G0▒~G0▒~G0▒~G0▒~G0▒~G2▒~G0▒~G5▒~G*2▒~G38501▒~G-2▒~G-2▒~G-2▒~G-2▒~G▒~G-2▒~G0▒~G1▒~G*2▒~G*2▒~G*2▒~G*2▒~G*2▒~G*2▒~G*2▒~G*2▒~G0▒~G*2▒~G-2▒~G*2▒~G0▒~G*2▒~G*2▒~G-2▒~G0▒~G-2▒~G-2▒~G-2▒~G-2▒~G-2▒~G-2▒~G*2▒~G*2▒~G*2
The same file uses the below DML in prod looking like:
decimal("Ç") acct_id; and so and so for new columns
When I cat this file I get below output:
Code:
bankbtch@jackets:/prod/home10/data/serial/rtl_baseldw_us_card/rbuc_cr_bur/temp=> cat ori_test.dat | head -1
bankbtch@jackets:/prod/home01/data/serial/rtl_baseldw_us_card/rbuc_misc/temp=> cat prod_test.dat_new_efx
1147073▒39▒▒▒▒▒▒7▒532▒832▒4500▒323▒21▒277▒0▒0▒0▒4▒1▒3▒0▒0▒0▒3▒Y▒1032▒0▒0▒▒▒▒▒0▒1▒N▒N▒N▒N▒N▒N▒N▒Y▒0▒02A8F588C159E82802▒199403▒T▒4▒002▒N▒0▒0▒0▒0▒0▒0▒0▒0▒▒▒▒▒▒▒0▒0▒▒▒0▒102▒1693▒11122▒10▒4▒5▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒1032▒2▒1▒0▒0▒0▒0▒3▒▒12▒0▒2▒3▒3▒0▒17▒10▒1693▒11122▒7▒5▒0▒0▒0▒1▒0▒0▒0▒1▒9▒9▒1193▒10590▒4500▒532▒80▒500▒0▒0▒0▒0▒0▒0▒532▒80▒500▒0▒0▒0▒0▒0▒0▒6▒1▒7500▒3▒2▒Y▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒4▒1▒3▒0▒0▒0▒0▒0▒0▒7▒8▒0▒0▒0▒4▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒000000▒000000▒1▒0▒0▒0▒0▒0▒0▒0▒0▒▒0▒▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒▒0▒0▒0▒▒▒▒▒▒▒▒▒0▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒171▒200▒105▒338▒4590▒6▒0▒201707▒0▒0▒0▒▒▒▒3▒3▒3▒5▒2▒2▒2▒2▒2▒2▒2▒2▒0▒0▒0▒0▒0▒0▒0▒0▒▒201805▒200408▒▒31994▒0▒0▒0▒0▒0▒4▒1▒▒▒N▒▒MORINSH HWY▒▒STONY BROOK▒NY▒11790▒▒▒▒▒▒▒5▒FOREST CREEK HIGHWAY▒AVE▒STONY BROOK▒NY▒11790▒HT▒1405830▒▒▒▒▒2▒1▒1▒0▒1432▒243▒54▒4599▒3938▒661▒Y▒▒▒0▒0▒▒0▒N▒Y▒Y▒N▒N▒N▒▒▒▒▒▒0▒0▒▒N▒N▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒N▒N▒N▒N▒999999999▒0▒0▒0▒0▒0▒0▒200409▒999999999▒999999999▒9▒1193▒0▒0▒2▒1▒3▒500▒999999998▒201707▒1113▒999999998▒500▒999999998▒201707▒1113▒999999998▒0▒0▒0▒0▒199506▒200201▒200201▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒1▒0▒0▒999▒13▒998▒0▒0▒0▒999▒999▒▒▒999▒999▒999▒999▒1▒207▒999▒999▒999▒999▒99▒99▒2▒0▒0▒0▒999▒999▒999▒999▒0▒0▒0▒0▒588▒39▒18▒13▒10▒99▒182▒3▒2▒▒▒▒▒▒▒▒▒▒▒▒▒▒2018-07-31▒0▒297▒
But when i vi the same file I get the below output:
Code:
bankbtch@jackets:/prod/home10/data/serial/rtl_baseldw_us_card/rbuc_cr_bur/temp=> vi prod_test.dat_new_efx
1147073▒39▒▒▒▒▒▒7▒532▒832▒4500▒323▒21▒277▒0▒0▒0▒4▒1▒3▒0▒0▒0▒3▒Y▒1032▒0▒0▒▒▒▒▒0▒1▒N▒N▒N▒N▒N▒N▒N▒Y▒0▒02A8F588C159E82802▒199403▒T▒4▒002▒N▒0▒0▒0▒0▒0▒0▒0▒0▒▒▒▒▒▒▒0▒0▒▒▒0▒102▒1693▒11122▒10▒4▒5▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒1032▒2▒1▒0▒0▒0▒0▒3▒▒12▒0▒2▒3▒3▒0▒17▒10▒1693▒11122▒7▒5▒0▒0▒0▒1▒0▒0▒0▒1▒9▒9▒1193▒10590▒4500▒532▒80▒500▒0▒0▒0▒0▒0▒0▒532▒80▒500▒0▒0▒0▒0▒0▒0▒6▒1▒7500▒3▒2▒Y▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒4▒1▒3▒0▒0▒0▒0▒0▒0▒7▒8▒0▒0▒0▒4▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒000000▒000000▒1▒0▒0▒0▒0▒0▒0▒0▒0▒▒0▒▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒▒0▒0▒0▒▒▒▒▒▒▒▒▒0▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒171▒200▒105▒338▒4590▒6▒0▒201707▒0▒0▒0▒▒▒▒3▒3▒3▒5▒2▒2▒2▒2▒2▒2▒2▒2▒0▒0▒0▒0▒0▒0▒0▒0▒▒201805▒200408▒▒31994▒0▒0▒0▒0▒0▒4▒1▒▒▒N▒▒MORINSH HWY▒▒STONY BROOK▒NY▒11790▒▒▒▒▒▒▒5▒FOREST CREEK HIGHWAY▒AVE▒STONY BROOK▒NY▒11790▒HT▒1405830▒▒▒▒▒2▒1▒1▒0▒1432▒243▒54▒4599▒3938▒661▒Y▒▒▒0▒0▒▒0▒N▒Y▒Y▒N▒N▒N▒▒▒▒▒▒0▒0▒▒N▒N▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒N▒N▒N▒N▒999999999▒0▒0▒0▒0▒0▒0▒200409▒999999999▒999999999▒9▒1193▒0▒0▒2▒1▒3▒500▒999999998▒201707▒1113▒999999998▒500▒999999998▒201707▒1113▒999999998▒0▒0▒0▒0▒199506▒200201▒200201▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒0▒1▒0▒0▒999▒13▒998▒0▒0▒0▒999▒999▒▒▒999▒999▒999▒999▒1▒207▒999▒999▒999▒999▒99▒99▒2▒0▒0▒0▒999▒999▒999▒999▒0▒0▒0▒0▒588▒39▒18▒13▒10▒99▒182▒3▒2▒▒▒▒▒▒▒▒▒▒▒▒▒▒2018-07-31▒0▒297▒
Need to find a way to make the Dev file consistent with the prod file eliminating ~G from the delimiter content.
Currently i am passing UC007 as the unicode delimiter value.
Please let me know if anyone knows the fix for the same.
Moderator's Comments:
Please use CODE tags as required by forum rules!
Last edited by RudiC; 08-17-2018 at 06:48 AM..
Reason: Added CODE tags.
Please become accustomed to provide decent context info of your problem.
It is always helpful to carefully and detailedly phrase a request, and to support it with system info like OS and shell, related environment (variables, directory structures, options), preferred tools, adequate (representative) sample input and desired output data and the logics connecting the two including your own attempts at a solution, and, if existent, system (error) messages verbatim, to avoid ambiguities and keep people from guessing.
Some additional info is necessary:
Where do those files come from (HW, OS, DB-tool)?
What is those files' character encoding?
You could create a temp file with a "normal" or "standard" delimiter like ; or , , and then transliterate those with a text tool like tr, sed, or awk.
It is Unix OS and korn shell. The source is Hadoop EMR and this file is generated on the S3 lake.
We run a script where we input the delimiter to the script and it generates the output file accordingly. we tried providing both UC007 and uc007, it does not help.
Request is to provide the encoding that we need to pass to my script to generate just ▒ (C cedilla) instead of ▒~G (we don't need the extra ~G)
Greetings Experts,
I do have some basic knowledge of Unix. The task I am trying to do through shell script is to generate the view script for all of the tables which is in YYYYMMDD format (I assume I am on Ksh).
I have certain tables that ends in YYYYMMDD format (eg: tbl_20150630)
For each... (1 Reply)
Hi,
Extremely new to Perl scripting, but need a quick fix without using TEXT::CSV
I need to read in a file, pass any delimiter as an argument, and convert it to bar delimited on the output. In addition, enclose fields within double quotes in case of any embedded delimiters.
Any help would... (2 Replies)
Hi,
I have a No Delimiter variable length text file with following schema -
Column Name Data length
Firstname 5
Lastname 5
age 3
phoneno1 10
phoneno2 10
phoneno3 10
sample data - ... (16 Replies)
hello All,
I am new user to shell scripting, kindly advise on the below?
I have a file where i have gaps & the delimiter falls in next line and new line is also created , plz see the example :employee.txt
Now the issue here is , i wan to write a script , where i can use "|" to get the... (6 Replies)
Hi all,
I have a number of strings like below:
//mnt/autocor/43°13'(33")W/
and i'm trying to get the numbers in this string, for example
431333
please help
thanks ahead (14 Replies)
Hi,
I have a string like ABC.123.XYZ-A1-B2-P1-C4. I want to delimit the string based on "-" and then get result as only two strings. One with string till last hyphen and other with value after last hyphen... For this case, it would be something like first string as "ABC.123.XYZ-A1-B2-P1" and... (6 Replies)
Having just transferred a project from Sun Solaris to Red Hat Linux 5.1 on an HP Blade, we have an issue seeing C-Cedilla (Ç) characters in Putty.
While a 'more' command shows the Ç correctly, cut and paste converts it to ^A and vi converts it to yet another ctrl A character.
The Ç character... (4 Replies)
I have used SAR -b to get some Unix cache / buffer metrics and the results are confusing me a bit.
The pread/s & pwrit/s are showing 0. However the lread/s and lwrit/s are showing figures. I note also that the bread/s and bwrit/s are showing figures. I believe that pread/s and pwrit/s is not... (3 Replies)
I executed the following command in the korn shell:
$ variable1="qwerty" ls | sort
and the shell executed the 'ls | sort' command.
I would have expected an error message from the shell, but instead of that the shell ran the 'ls | sort' command and didn't realize the variable assignement. ... (1 Reply)
08-17-2018
