Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

File cleaning

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting File cleaning
# 1  
Old 09-20-2011
File cleaning
HI ,

I am getting the source data as below.

Code:

Source Data

CDR_Data,,,,,

F1,F2,F3,F4,F5,F6
5,5,6,7,8,7
6,6,g,,,
7,7,76,,,
8,8,gt,,,
9,9,df ,d,d,d


,,,,,
Logic for CDR Records,,,,,
<Indicate the Records Flag>,,,,,
,,,,,
,, Load,,,
,,,,,
,,Data files loaded at hourly basis.,,,
,,,,,
,,Data files loaded at hourly basis.,,,
,,,,,
,,Data files loaded at hourly basis.,,,
,,,,,
,,Data files loaded at hourly basis.,,,
,,,,,
,Data files loaded at hourly basis.,,,
,,,,,
,,Data files loaded at hourly basis.,,,
,,,,,

,,,,,
,, Load Status,,,
,,,,,
,,,,,
,,,,,
,,,,,
,,,,,
,,,,,
,,,,,

How can I clean this to the below format., I tried using awk but I am not getting the desired output., what can i use to get the target data as below from the source files

Code:
Target Data

5,5,6,7,8,7
6,6,g,,,
7,7,76,,,
8,8,gt,,,
9,9,df ,d,d,d

Regards,
Wang
# 2  
Old 09-20-2011
Hi wangkc,

Try:
Code:
$ cat infile
CDR_Data,,,,,

F1,F2,F3,F4,F5,F6
5,5,6,7,8,7
6,6,g,,,
7,7,76,,,
8,8,gt,,,
9,9,df ,d,d,d


,,,,,
Logic for CDR Records,,,,,
<Indicate the Records Flag>,,,,,
,,,,,
,, Load,,,
,,,,,
,,Data files loaded at hourly basis.,,,
,,,,,
,,Data files loaded at hourly basis.,,,
,,,,,
,,Data files loaded at hourly basis.,,,
,,,,,
,,Data files loaded at hourly basis.,,,
,,,,,
,Data files loaded at hourly basis.,,,
,,,,,
,,Data files loaded at hourly basis.,,,
,,,,,

,,,,,
,, Load Status,,,
,,,,,
,,,,,
,,,,,
,,,,,
,,,,,
,,,,,
,,,,,
$ sed -ne '/^[0-9]/p' infile
5,5,6,7,8,7
6,6,g,,,
7,7,76,,,
8,8,gt,,,
9,9,df ,d,d,d

This User Gave Thanks to birei For This Post:
wangkc
# 3  
Old 09-20-2011
Code:
grep ^[0-9] input_file

--ahamed
This User Gave Thanks to ahamed101 For This Post:
wangkc
# 4  
Old 09-21-2011
Thanks both of you , Can you pls explain me what it is doing

Code:
 sed -ne '/^[0-9]/p' infile

# 5  
Old 09-21-2011
it will print the lines which starts with a number ...
one more in awk ..
Code:
$  awk '/^[0-9]/ {print $0}' infile

This User Gave Thanks to jayan_jay For This Post:
birei
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Cleaning output using awk

I have some small problem with my code. data.html <TD class="statuscol2">c</TD> <TD class="statuscol3">18</TD> <TD class="statuscol4"><SPAN TITLE="#04">test4</SPAN></TD> <TD... (4 Replies)
Discussion started by: Jotne
4 Replies

2. Shell Programming and Scripting

cleaning the file

Hi, I have a file with multiple rows. each row has 8 columns. Column 8 has entries separated by commas. I want to exclude all the rows in which column 8 has more than 3 commas. 1234#0/1 - ABC_1234 3 ATGCATGCATGC HHHIIIGIHVF 1 49:T>C,60:T>C,78:C>A,76:G>T,65:T>G Thanks, Diya (3 Replies)
Discussion started by: Diya123
3 Replies

3. Shell Programming and Scripting

Data Cleaning in a file

Hi , I have the below source data, I need to clean the data in 2nd,4th,5th columns. Source Data RECORD,CASH_TRANS,BEJING,AUG2011/CASH_TRANS,Y/N150/CASH_TRANS,N/201108 RECORD,CASH_TRANS,INDIA,AUG2011/CASH_TRANS,Y/NC110/CASH_TRANS,N/201108... (7 Replies)
Discussion started by: mora
7 Replies

4. UNIX for Dummies Questions & Answers

AWK Data Cleaning

Hello, I am trying to analyze data I recently ran, and the only way to efficiently clean up the data is by using an awk file. I am very new to awk and am having great difficulty with it. In $8 and $9, for example, I am trying to delete numbers that contain 1. I cannot find any tutorials that... (20 Replies)
Discussion started by: carmar87
20 Replies

5. UNIX for Dummies Questions & Answers

cleaning up spaces from fixed width file while converting to csv file

Open to a sed/awk/or perl alternative so that i can stick command into my bash script. This is a problem I resolve using a combination of cut commands - but that is getting convoluted. So would really appreciate it if someone could provide a better solution which basically replaces all... (3 Replies)
Discussion started by: svn
3 Replies

6. Shell Programming and Scripting

Help me with the cleaning of a file using shell script

Dear, I have an input file and need to clean and get the following output: ATM1/0/0,19072000,97848000 ATM1/0/1,18015000,83270000 ATM1/0/2,16879000,89491000 ATM1/0/3,21684000,122096000 Input file: show interface ATM1/0/0 | inc rate Interface ATM1/0/0 queueing strategy: PXF... (3 Replies)
Discussion started by: He2
3 Replies

7. UNIX for Dummies Questions & Answers

Cleaning core files

Hello *! Just a short question. Where on the system i can find core files. I have one SUN server (Solaris 8) and from time to time I must clean core files on it. But i am not sure where i can find those files. Thank you in advance. :) (5 Replies)
Discussion started by: ghost01
5 Replies

8. UNIX for Dummies Questions & Answers

Cleaning file system

Hello all, I have a crontab entry to clean up a file system once a week that is used as tmp directory. 00 12 * * 0 find /mytmp -mtime +2 -exec rm {} \; The job starts ok but I always get an error message in mails because it is a file system and there is a directory lost+found that should... (1 Reply)
Discussion started by: qfwfq
1 Replies

9. UNIX for Dummies Questions & Answers

cleaning log files

Say I have a log file called is there a way to clean out certain files in this log by or when I go to VI this file it tells me and I am guessing this means the file has become to large to append with VI so I was wondering if there was command with some argument to clean this out? Thanks ... (6 Replies)
Discussion started by: rocker40
6 Replies

10. AIX

doing some spring cleaning....

USERS="me you jim joe sue" for user in ${USERS}; do rmuser -p $user usrdir=`cat /etc/passwd|grep $user|awk -F":" '{ print $6 }'` rm -fr `cat /etc/passwd|grep $user|awk -F":" '{ print $6 }'` echo Deleting: $user '\t' REMOVING: $usrdir done This is for AIX ONLY!!! but easily ported to... (0 Replies)
Discussion started by: Optimus_P
0 Replies
Login or Register to Ask a Question