want to remove separators from file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting want to remove separators from file
# 1  
Old 12-16-2008
want to remove separators from file

Hi,

I have huge file,

head -1 filneame gives,

I just want to remove "##colsep##" from the file, and also want to count the no. of fileds present, as in

Output shld be in newfile as TRADE_KEY,TRADE_DATE
and total no. of fileds separated by these comma's

TRADE_KEY##colsep##TRADE_DATE_TIME##colsep##TRANS_TYPE_CD##colsep##ACCOUNT_KEY##colsep##LOCAL_CURREN CY_AMOUNT##colsep##LOCAL_CURRENCY_CD##colsep##ORIGINAL_CURRENCY_AMOUNT##colsep##ORIGINAL_CURRENCY_CD ##colsep##TRADE_QUANTITY##colsep##SETTLE_DATE_TIME##colsep##ORDER_ENTRY_DATE_TIME##colsep##PRODUCT_K EY##colsep##TRADE_PRICE##colsep##COMMISSION##colsep##IS_CORRECT##colsep##IS_CANCEL##colsep##CORRECTI ON_TYPE##colsep##EXCHANGE_CD##colsep##IS_SOLICITED##colsep##EXECUTING_REPRESENTATIVE_KEY##colsep##AG G_UNIT##colsep##CAPACITY##colsep##ORDER_ID##colsep##ORIGINAL_ORDER_ID##colsep##LOCAL_CURRENCY_NET_AM OUNT##colsep##ORIGINAL_CURRENCY_NET_AMOUNT##colsep##BUSINESS_DATE##colsep##COUNTERPARTY_ACCOUNT_KEY# #colsep##COUNTERPARTY_CATEGORY_CD##colsep##AVERAGE_PRICE_IND##colsep##SOURCE_SYSTEM_CD##colsep##DESK ##colsep##REGION##colsep##ACCRUED_INTEREST##colsep##CURRENCY_EXCHANGE_RATE##colsep##PRODUCT_TYPE_COD E##colsep##SEDOL_CODE##colsep##COUNTERPARTY_ACCOUNT_LABEL##colsep##TRADE_TYPE##colsep##PROP_ACCOUNT# #colsep##DERIVATIVE_TYPE##colsep##PORTFOLIO

Any script shell,awk,perl would do, please help as soon as possible,
# 2  
Old 12-16-2008
man sed
# 3  
Old 12-16-2008
[quote]
by these comma's
[/quota]

Which commas? I don't see any nor do you speak about them before. So maybe just post an example output you would like to have.
Btw, the forum support code tags to display code, logs etc.
# 4  
Old 12-16-2008
Try...
Code:
awk 'BEGIN{FS="##colsep##";OFS=","}{print $1, $2, NF}' file1

# 5  
Old 12-16-2008
Code:
$str='TRADE_KEY##colsep##TRADE_DATE_TIME##colsep##TRANS_TYPE_CD##colsep##ACCOUNT_KEY##colsep##LOCAL_CURREN CY_AMOUNT##colsep##LOCAL_CURRENCY_CD##colsep##ORIGINAL_CURRENCY_AMOUNT##colsep##ORIGINAL_CURRENCY_CD ##colsep##TRADE_QUANTITY##colsep##SETTLE_DATE_TIME##colsep##ORDER_ENTRY_DATE_TIME##colsep##PRODUCT_K EY##colsep##TRADE_PRICE##colsep##COMMISSION##colsep##IS_CORRECT##colsep##IS_CANCEL##colsep##CORRECTI ON_TYPE##colsep##EXCHANGE_CD##colsep##IS_SOLICITED##colsep##EXECUTING_REPRESENTATIVE_KEY##colsep##AG G_UNIT##colsep##CAPACITY##colsep##ORDER_ID##colsep##ORIGINAL_ORDER_ID##colsep##LOCAL_CURRENCY_NET_AM OUNT##colsep##ORIGINAL_CURRENCY_NET_AMOUNT##colsep##BUSINESS_DATE##colsep##COUNTERPARTY_ACCOUNT_KEY# #colsep##COUNTERPARTY_CATEGORY_CD##colsep##AVERAGE_PRICE_IND##colsep##SOURCE_SYSTEM_CD##colsep##DESK ##colsep##REGION##colsep##ACCRUED_INTEREST##colsep##CURRENCY_EXCHANGE_RATE##colsep##PRODUCT_TYPE_COD E##colsep##SEDOL_CODE##colsep##COUNTERPARTY_ACCOUNT_LABEL##colsep##TRADE_TYPE##colsep##PROP_ACCOUNT# #colsep##DERIVATIVE_TYPE##colsep##PORTFOLIO';
$num=($str=~s/##colsep##/,/g);
print "Replaced $num of seprator,","the string after subsituted is:",$str;

# 6  
Old 12-16-2008
awk 'BEGIN{FS="##colsep##";OFS=","}{print $1, $2, NF}' file1

this command works, but giving only 2 columns, NF not working...

there are such 42 cloumns, ......

thanks....and looking fwd
# 7  
Old 12-16-2008
That's what you asked for...
Quote:
Output shld be in newfile as TRADE_KEY,TRADE_DATE
and total no. of fileds separated by these comma's
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Multiple long field separators

How do I use multiple field separators in awk? I know that if I use awk -F"", both a and b will be field separators. But what if I need two field separators that both are longer than one letter? If I want the field separators to be "ab" and "cd", I will not be able to use awk -F"". The ... (2 Replies)
Discussion started by: locoroco
2 Replies

2. Shell Programming and Scripting

Awk with mutliple filed separators

I try to trim a FTP log using awk, but the escape sequences does not work the way I want it. Input data Wed 3Oct10 21:48:00 - (002117) Sent file d:\ftp\home\tools\htmake.rar successfully (48.2 kB/sec - 40997 Bytes) Wed 3Oct10 22:25:46 - (002118) Sent file d:\ftp\files\main\oct\camera1... (4 Replies)
Discussion started by: Jotne
4 Replies

3. Shell Programming and Scripting

How to ignore quoted separators

Hi, I'm trying to parse a text file which uses commas as field separators. Fields are double quoted, and may themselves contain commas, like this: "1","John Smith","London","123" "2","Mary Robertson","Horsham, Sussex","456" This causes problems for the following command cut -d","... (7 Replies)
Discussion started by: sven44
7 Replies

4. UNIX for Advanced & Expert Users

Exclude dash (-) from word separators in vi

vi uses dash and space as word separators. is there any way to exclude dash from word separators ? This is required to work with the symbols generated by ctags exe. when symbol contain a "-" ,vi tags fails to locate that even though symbol is generated properly. For example Symbol -... (3 Replies)
Discussion started by: cabhi
3 Replies

5. UNIX for Dummies Questions & Answers

Can one use 2 field separators in awk?

I have files such as n02-z30-dsr65-terr0.25-dc0.008-16x12drw-run1.cmd I am wondering if it is possible to define two field separators "-" and "." for these strings so that $7 is run1. (5 Replies)
Discussion started by: kristinu
5 Replies

6. UNIX Desktop Questions & Answers

awk Varing Field Separators

Hi Guys, I have small dilemma which I could do with a little help solving . I currently have text HDD S.M.A.R.T report which I have pasted below: smartctl 5.39 2008-10-24 22:33 (openSUSE RPM) Copyright (C) 2002-8 by Bruce Allen, http://smartmontools.sourceforge.net Device: COMPAQ... (2 Replies)
Discussion started by: bikerben
2 Replies

7. Shell Programming and Scripting

how to use two separators in awk.

Hi, Gurus, I have a file like 1 234, 345, 456 2 345, 456, 345 I want to use awk with multipe separator ( one is comma, another is space)print out $1, $3 which should be: 1, 345 2, 456 but I don't know how to put space as separator with another separator. Thanks in advance (7 Replies)
Discussion started by: ken002
7 Replies

8. Shell Programming and Scripting

Problem with changing field separators in a file

I have a file with content as shown below. cat t2 : 100,100,"X",1234,"12A",,,"ab,c" Comma is the field seperator, however string fields will be within double quotes and comma within double quotes should not be treated as field seperator. I am trying to replace this field seperator to a... (7 Replies)
Discussion started by: mk1216
7 Replies

9. Shell Programming and Scripting

AWK multiple fields separators

I need to print the second field of a file, taking spaces, tab and = as field separators. ; for 16-bit app support MAPI=1 CMC=1 CMCDLLNAME32=mapi32.dll CMCDLLNAME=mapi.dll MAPIX=1 MAPIXVER=1.0.0.1 OLEMessaging=1 asf=MPEGVideo asx=MPEGVideo ivf=MPEGVideo m3u=MPEGVideo (2 Replies)
Discussion started by: PamPam
2 Replies

10. Shell Programming and Scripting

Fixed width file with newline field separators

I have some huge files that are produced daily from a production system written in basic (really). The files are fixed width records, 512 bytes, with newline field separators, newlines if the field is null, and trailing newlines for null fields. The data in the fields can be any ascii... (0 Replies)
Discussion started by: vtischuk@yahoo.
0 Replies
Login or Register to Ask a Question