Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Data Cleaning in a file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Data Cleaning in a file
# 8  
Old 06-26-2011
@wangkc With GNU sed you can do so:
Code:
sed -r 's/CASH_|LOAN_|ABC_|XYZ_|etc//g' testfile

But it's really almost the same. In the former example a line reads and every command applies to it. Then reads another line and so on. Here sed checks a line for CASH_ and for LOAN_ and etc., makes substitution then reads another line. Maybe a little faster but I don't know. It should do more work in one command.

---------- Post updated at 06:06 PM ---------- Previous update was at 06:03 PM ----------

Quote:
Originally Posted by mora
Hi Yazu,

I want to apply the logic,only if the first column is equal to RECORD,not for all the records in the file,how can I do this in sed.

--Mora
Code:
sed '/^RECORD/ { s/CASH_//g; s/LOAN_//g }'

Last edited by yazu; 06-26-2011 at 08:13 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replace data of a file with data from another file using shell scripting.

Dears, I'm new to shell scripting and i was wondering if you can help me with following matter. I have a file containing 400,000 records. The file contains two columns like: 00611291,0270404000005453 25262597,1580401000016155 25779812,1700403000001786 00388934,1200408000000880... (1 Reply)
Discussion started by: paniklas
1 Replies

2. Shell Programming and Scripting

Extract header data from one file and combine it with data from another file

Hi, Great minds, I have some files, in fact header files, of CTD profiler, I tried a lot C programming, could not get output as I was expected, because my programming skills are very poor, finally, joined unix forum with the hope that, I may get what I want, from you people, Here I have attached... (17 Replies)
Discussion started by: nex_asp
17 Replies

3. UNIX for Dummies Questions & Answers

Mapping a data in a file and delete line in source file if data does not exist.

Hi Guys, Please help me with my problem here: I have a source file: 1212 23232 343434 ASAS1 4 3212 23232 343434 ASAS2 4 3234 23232 343434 QWQW1 4 1134 23232 343434 QWQW2 4 3212 23232 343434 QWQW3 4 and a mapping... (4 Replies)
Discussion started by: kokoro
4 Replies

4. Shell Programming and Scripting

cleaning the file

Hi, I have a file with multiple rows. each row has 8 columns. Column 8 has entries separated by commas. I want to exclude all the rows in which column 8 has more than 3 commas. 1234#0/1 - ABC_1234 3 ATGCATGCATGC HHHIIIGIHVF 1 49:T>C,60:T>C,78:C>A,76:G>T,65:T>G Thanks, Diya (3 Replies)
Discussion started by: Diya123
3 Replies

5. Shell Programming and Scripting

File cleaning

HI , I am getting the source data as below. Source Data CDR_Data,,,,, F1,F2,F3,F4,F5,F6 5,5,6,7,8,7 6,6,g,,, 7,7,76,,, 8,8,gt,,, 9,9,df ,d,d,d ,,,,, (4 Replies)
Discussion started by: wangkc
4 Replies

6. UNIX for Dummies Questions & Answers

AWK Data Cleaning

Hello, I am trying to analyze data I recently ran, and the only way to efficiently clean up the data is by using an awk file. I am very new to awk and am having great difficulty with it. In $8 and $9, for example, I am trying to delete numbers that contain 1. I cannot find any tutorials that... (20 Replies)
Discussion started by: carmar87
20 Replies

7. UNIX for Dummies Questions & Answers

cleaning up spaces from fixed width file while converting to csv file

Open to a sed/awk/or perl alternative so that i can stick command into my bash script. This is a problem I resolve using a combination of cut commands - but that is getting convoluted. So would really appreciate it if someone could provide a better solution which basically replaces all... (3 Replies)
Discussion started by: svn
3 Replies

8. Shell Programming and Scripting

Help me with the cleaning of a file using shell script

Dear, I have an input file and need to clean and get the following output: ATM1/0/0,19072000,97848000 ATM1/0/1,18015000,83270000 ATM1/0/2,16879000,89491000 ATM1/0/3,21684000,122096000 Input file: show interface ATM1/0/0 | inc rate Interface ATM1/0/0 queueing strategy: PXF... (3 Replies)
Discussion started by: He2
3 Replies

9. UNIX for Dummies Questions & Answers

Cleaning file system

Hello all, I have a crontab entry to clean up a file system once a week that is used as tmp directory. 00 12 * * 0 find /mytmp -mtime +2 -exec rm {} \; The job starts ok but I always get an error message in mails because it is a file system and there is a directory lost+found that should... (1 Reply)
Discussion started by: qfwfq
1 Replies

10. AIX

doing some spring cleaning....

USERS="me you jim joe sue" for user in ${USERS}; do rmuser -p $user usrdir=`cat /etc/passwd|grep $user|awk -F":" '{ print $6 }'` rm -fr `cat /etc/passwd|grep $user|awk -F":" '{ print $6 }'` echo Deleting: $user '\t' REMOVING: $usrdir done This is for AIX ONLY!!! but easily ported to... (0 Replies)
Discussion started by: Optimus_P
0 Replies
Login or Register to Ask a Question