Sponsored Content
Full Discussion: Data Cleaning in a file
Top Forums Shell Programming and Scripting Data Cleaning in a file Post 302533957 by mora on Sunday 26th of June 2011 04:39:54 AM
Old 06-26-2011
Data Cleaning in a file

Hi ,

I have the below source data, I need to clean the data in 2nd,4th,5th columns.

Code:
Source Data
RECORD,CASH_TRANS,BEJING,AUG2011/CASH_TRANS,Y/N150/CASH_TRANS,N/201108
RECORD,CASH_TRANS,INDIA,AUG2011/CASH_TRANS,Y/NC110/CASH_TRANS,N/201108
RECORD,LOAN_GRANTED_AMOUNT,RUSSIA,AUG2011/LOAN_GRANTED_AMOUNT,Y/NP874/LOAN_GRANTED_AMOUNT,N/201108
RECORD,LOAN_GRANTED_AMOUNT,PARIS,AUG2011/LOAN_GRANTED_AMOUNT,Y/NB6543/LOAN_GRANTED_AMOUNT,N/201108


Code:
Target Data
RECORD,TRANS,BEJING,AUG2011/TRANS,Y/N150/TRANS,N/201108
RECORD,TRANS,INDIA,AUG2011/TRANS,Y/NC110/TRANS,N/201108
RECORD,GRANTED_AMOUNT,RUSSIA,AUG2011/GRANTED_AMOUNT,Y/NP874/GRANTED_AMOUNT,N/201108
RECORD,GRANTED_AMOUNT,PARIS,AUG2011/GRANTED_AMOUNT,Y/NB6543/GRANTED_AMOUNT,N/201108

Please give me some hints on how can i achieve this.

-Mora
 

10 More Discussions You Might Find Interesting

1. AIX

doing some spring cleaning....

USERS="me you jim joe sue" for user in ${USERS}; do rmuser -p $user usrdir=`cat /etc/passwd|grep $user|awk -F":" '{ print $6 }'` rm -fr `cat /etc/passwd|grep $user|awk -F":" '{ print $6 }'` echo Deleting: $user '\t' REMOVING: $usrdir done This is for AIX ONLY!!! but easily ported to... (0 Replies)
Discussion started by: Optimus_P
0 Replies

2. UNIX for Dummies Questions & Answers

Cleaning file system

Hello all, I have a crontab entry to clean up a file system once a week that is used as tmp directory. 00 12 * * 0 find /mytmp -mtime +2 -exec rm {} \; The job starts ok but I always get an error message in mails because it is a file system and there is a directory lost+found that should... (1 Reply)
Discussion started by: qfwfq
1 Replies

3. Shell Programming and Scripting

Help me with the cleaning of a file using shell script

Dear, I have an input file and need to clean and get the following output: ATM1/0/0,19072000,97848000 ATM1/0/1,18015000,83270000 ATM1/0/2,16879000,89491000 ATM1/0/3,21684000,122096000 Input file: show interface ATM1/0/0 | inc rate Interface ATM1/0/0 queueing strategy: PXF... (3 Replies)
Discussion started by: He2
3 Replies

4. UNIX for Dummies Questions & Answers

cleaning up spaces from fixed width file while converting to csv file

Open to a sed/awk/or perl alternative so that i can stick command into my bash script. This is a problem I resolve using a combination of cut commands - but that is getting convoluted. So would really appreciate it if someone could provide a better solution which basically replaces all... (3 Replies)
Discussion started by: svn
3 Replies

5. UNIX for Dummies Questions & Answers

AWK Data Cleaning

Hello, I am trying to analyze data I recently ran, and the only way to efficiently clean up the data is by using an awk file. I am very new to awk and am having great difficulty with it. In $8 and $9, for example, I am trying to delete numbers that contain 1. I cannot find any tutorials that... (20 Replies)
Discussion started by: carmar87
20 Replies

6. Shell Programming and Scripting

File cleaning

HI , I am getting the source data as below. Source Data CDR_Data,,,,, F1,F2,F3,F4,F5,F6 5,5,6,7,8,7 6,6,g,,, 7,7,76,,, 8,8,gt,,, 9,9,df ,d,d,d ,,,,, (4 Replies)
Discussion started by: wangkc
4 Replies

7. Shell Programming and Scripting

cleaning the file

Hi, I have a file with multiple rows. each row has 8 columns. Column 8 has entries separated by commas. I want to exclude all the rows in which column 8 has more than 3 commas. 1234#0/1 - ABC_1234 3 ATGCATGCATGC HHHIIIGIHVF 1 49:T>C,60:T>C,78:C>A,76:G>T,65:T>G Thanks, Diya (3 Replies)
Discussion started by: Diya123
3 Replies

8. UNIX for Dummies Questions & Answers

Mapping a data in a file and delete line in source file if data does not exist.

Hi Guys, Please help me with my problem here: I have a source file: 1212 23232 343434 ASAS1 4 3212 23232 343434 ASAS2 4 3234 23232 343434 QWQW1 4 1134 23232 343434 QWQW2 4 3212 23232 343434 QWQW3 4 and a mapping... (4 Replies)
Discussion started by: kokoro
4 Replies

9. Shell Programming and Scripting

Extract header data from one file and combine it with data from another file

Hi, Great minds, I have some files, in fact header files, of CTD profiler, I tried a lot C programming, could not get output as I was expected, because my programming skills are very poor, finally, joined unix forum with the hope that, I may get what I want, from you people, Here I have attached... (17 Replies)
Discussion started by: nex_asp
17 Replies

10. Shell Programming and Scripting

Replace data of a file with data from another file using shell scripting.

Dears, I'm new to shell scripting and i was wondering if you can help me with following matter. I have a file containing 400,000 records. The file contains two columns like: 00611291,0270404000005453 25262597,1580401000016155 25779812,1700403000001786 00388934,1200408000000880... (1 Reply)
Discussion started by: paniklas
1 Replies
CUNM2R(l)								 )								 CUNM2R(l)

NAME
CUNM2R - overwrite the general complex m-by-n matrix C with Q * C if SIDE = 'L' and TRANS = 'N', or Q'* C if SIDE = 'L' and TRANS = 'C', or C * Q if SIDE = 'R' and TRANS = 'N', or C * Q' if SIDE = 'R' and TRANS = 'C', SYNOPSIS
SUBROUTINE CUNM2R( SIDE, TRANS, M, N, K, A, LDA, TAU, C, LDC, WORK, INFO ) CHARACTER SIDE, TRANS INTEGER INFO, K, LDA, LDC, M, N COMPLEX A( LDA, * ), C( LDC, * ), TAU( * ), WORK( * ) PURPOSE
CUNM2R overwrites the general complex m-by-n matrix C with Q * C if SIDE = 'L' and TRANS = 'N', or Q'* C if SIDE = 'L' and TRANS = 'C', or C * Q if SIDE = 'R' and TRANS = 'N', or C * Q' if SIDE = 'R' and TRANS = 'C', where Q is a complex unitary matrix defined as the product of k elementary reflectors Q = H(1) H(2) . . . H(k) as returned by CGEQRF. Q is of order m if SIDE = 'L' and of order n if SIDE = 'R'. ARGUMENTS
SIDE (input) CHARACTER*1 = 'L': apply Q or Q' from the Left = 'R': apply Q or Q' from the Right TRANS (input) CHARACTER*1 = 'N': apply Q (No transpose) = 'C': apply Q' (Conjugate transpose) M (input) INTEGER The number of rows of the matrix C. M >= 0. N (input) INTEGER The number of columns of the matrix C. N >= 0. K (input) INTEGER The number of elementary reflectors whose product defines the matrix Q. If SIDE = 'L', M >= K >= 0; if SIDE = 'R', N >= K >= 0. A (input) COMPLEX array, dimension (LDA,K) The i-th column must contain the vector which defines the elementary reflector H(i), for i = 1,2,...,k, as returned by CGEQRF in the first k columns of its array argument A. A is modified by the routine but restored on exit. LDA (input) INTEGER The leading dimension of the array A. If SIDE = 'L', LDA >= max(1,M); if SIDE = 'R', LDA >= max(1,N). TAU (input) COMPLEX array, dimension (K) TAU(i) must contain the scalar factor of the elementary reflector H(i), as returned by CGEQRF. C (input/output) COMPLEX array, dimension (LDC,N) On entry, the m-by-n matrix C. On exit, C is overwritten by Q*C or Q'*C or C*Q' or C*Q. LDC (input) INTEGER The leading dimension of the array C. LDC >= max(1,M). WORK (workspace) COMPLEX array, dimension (N) if SIDE = 'L', (M) if SIDE = 'R' INFO (output) INTEGER = 0: successful exit < 0: if INFO = -i, the i-th argument had an illegal value LAPACK version 3.0 15 June 2000 CUNM2R(l)
All times are GMT -4. The time now is 08:27 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy