Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk and sed problem

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk and sed problem
# 1  
Old 09-13-2010
awk and sed problem
Hi,
I am able to generate a csv file which has 3 columns. But the header is not coming out exactly i.e it doesnot have a comma in between it but the data is getting produced with the comma.
The first line i.e the header is coming out as:-
Counter Number Counterparty1 Counterparty2

where Counter Number is the first column, Counterparty1 is the 2nd and Counterparty2 is the 3rd column

The process that i followed is:-
HEADER1="Counter Number"
HEADER2="Counterparty1"
HEADER3="Counterparty2
HEADER="$HEADER1,$HEADER2,$HEADER3"

Then I am deleting the first line from the csv file and then
echo $HEADER > output1.csv
cat input.csv >> output1.csv
mv output1.csv output.csv

It works , but I know that this is not the correct way. I think it can be done with awk and sed. So can somebody help me to put commas inbetween the headers in the first line of the csv...

Thanks
# 2  
Old 09-13-2010
Quote:
Originally Posted by scripter12
...
I am able to generate a csv file which has 3 columns. But the header is not coming out exactly i.e it doesnot have a comma in between it but the data is getting produced with the comma.
The first line i.e the header is coming out as:-
Counter Number Counterparty1 Counterparty2

where Counter Number is the first column, Counterparty1 is the 2nd and Counterparty2 is the 3rd column

The process that i followed is:-
HEADER1="Counter Number"
HEADER2="Counterparty1"
HEADER3="Counterparty2
HEADER="$HEADER1,$HEADER2,$HEADER3"

Then I am deleting the first line from the csv file and then
echo $HEADER > output1.csv
cat input.csv >> output1.csv
mv output1.csv output.csv

It works , but I know that this is not the correct way. I think it can be done with awk and sed. So can somebody help me to put commas inbetween the headers in the first line of the csv...
...
There's a missing double-quote at the end of HEADER3 value. It should throw an error in Bash -

Code:
$
$
$ cat input.csv
a,b,c,d
e,f,g,h
i,j,k,l
$
$ cat cp_with_header.sh
HEADER1="Counter Number"
HEADER2="Counterparty1"
HEADER3="Counterparty2
HEADER="$HEADER1,$HEADER2,$HEADER3"
echo $HEADER > output1.csv
cat input.csv >> output1.csv
mv output1.csv output.csv
$
$
$ . cp_with_header.sh
bash: cp_with_header.sh: line 4: unexpected EOF while looking for matching `"'
bash: cp_with_header.sh: line 10: syntax error: unexpected end of file
$
$

I am able to see the expected output after fixing it:

Code:
$
$
$ cat cp_with_header.sh
HEADER1="Counter Number"
HEADER2="Counterparty1"
HEADER3="Counterparty2"
HEADER="$HEADER1,$HEADER2,$HEADER3"
echo $HEADER > output1.csv
cat input.csv >> output1.csv
mv output1.csv output.csv
$
$ . cp_with_header.sh
$
$
$ cat output.csv
Counter Number,Counterparty1,Counterparty2
a,b,c,d
e,f,g,h
i,j,k,l
$
$

A shorter way in the shell could've been thus -

Code:
$
$
$ echo $HEADER
Counter Number,Counterparty1,Counterparty2
$
$
$ echo $HEADER | cat - input.csv
Counter Number,Counterparty1,Counterparty2
a,b,c,d
e,f,g,h
i,j,k,l
$
$
$ # just redirect the pipeline of commands to output.csv
$
$

Or if you want awk -

Code:
$
$
$ cat input.csv
a,b,c,d
e,f,g,h
i,j,k,l
$
$
$ awk -v HDR="$HEADER" 'BEGIN {print HDR} {print}' input.csv
Counter Number,Counterparty1,Counterparty2
a,b,c,d
e,f,g,h
i,j,k,l
$
$
$ # just redirect the output of the awk one-liner to output.csv
$
$

tyler_durden
# 3  
Old 09-13-2010
@Tyler, I think he want to replace the first line with a new line.

Code:
awk 'NR==1{$1="Counter Number, Counterparty1, Counterparty2"}1' output.csv > tempfile && mv tempfile output.csv

If your sed version supports the -i option:
Code:
sed -i '1 s/.*/Counter Number, Counterparty1, Counterparty2/' output.csv

otherwise:
Code:
sed '1 s/.*/Counter Number, Counterparty1, Counterparty2/' output.csv  > tempfile && mv tempfile output.csv

# 4  
Old 09-13-2010
@Franklin & Tyler,

Thanks for the reply.
Still it is hard coding in the script giving the header names directly. If that the case then my lines were working fine.

But i want a generic one which puts a comma between the header names i.e the first line of any csv file.
I just want to put a comma in column1, column2 and column3.
I had to generate 10 csv files and may be there can be more than three columns. So hard coding is not the option for me...

Thanks

Thanks
# 5  
Old 09-13-2010
Difficult if the header has more names with spaces.
# 6  
Old 09-13-2010
Atleast please tell me for the above three columns that I have posted, so that i can get some insight and idea....

Another thing that i can do is, I can delete the spaces between any header names as it comes from a stored procedure, so now if the header becomes..
CounterNumber Counterparty1 Counterparty2

then what will be the solution...

Thanks
# 7  
Old 09-13-2010
Code:
awk -vh="$HEADER" 'NR==1{$0=h"\n"$0}1' input.csv > output.csv
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed or awk script problem

Hi All I am having a file as shown below File1# modyle 1 { test jsj hhsjh 13e3 jsjjs } memP(dbg_trace) { ajjs jsjs jsjs Test(claer) { jsjs (7 Replies)
Discussion started by: kshitij
7 Replies

2. Shell Programming and Scripting

Problem with output awk and sed

I have file, i am extracting email address from file. but problem is that output is very ugly. I am using this command REMOVED "CSS OFFENDING CODE"... While original filename have no such character. Please suggest. (20 Replies)
Discussion started by: learnbash
20 Replies

3. Shell Programming and Scripting

Some problem with awk or sed in if conditions

#set -x path=`pwd` while read line do dd=`echo $line|awk -F"," '{print $1}'|awk -F"/" '{print $2}'` if then mm=`echo $line|awk -F"," '{print $1}'|awk -F"/" '{print $1}'` yy=`echo $line|awk -F"," '{print $1}'|awk -F"/" '{print $3}'` var=`echo $line|awk -F"," '{print $2}'` var1=`echo... (2 Replies)
Discussion started by: nikhil jain
2 Replies

4. Shell Programming and Scripting

awk/sed problem

I have a bunch of files I need to change a letter to another letter only in the first column of a space delimited file. The problem is that after the first column, there's comments and such that cause problems with awk. Here's what my data looks like: 16a28.0a27.0a26.0a25.0a24.67.255.200... (3 Replies)
Discussion started by: supradave
3 Replies

5. Shell Programming and Scripting

Another parsing line awk or sed problem

Hi, After looking on different forums, I'm still in trouble to parse a parameters line received in KSH. $* is equal to "/AAA:111 /BBB:222 /CCC:333 /DDD:444" I would like to parse it and be able to access anyone from his name in my KSH after. like echo myArray => display 111 ... (1 Reply)
Discussion started by: RickTrader
1 Replies

6. Shell Programming and Scripting

Problem facing with sed and awk

Hi All, I have a got a problem .. I have t files as below: 1.txt contains ----- ----- ----- column 1, "cat", column 24, "dog", column 100, "rat", ----- ----- ----- 2.sh should contain ----- ----- ----- awk 'BEGIN { printf ("%1s","cat")}' (19 Replies)
Discussion started by: jisha
19 Replies

7. Shell Programming and Scripting

awk...sed problem

Hi guys, I am tryingto delete or ignore columns in a large dataset using 'sed'. For example, I have the following dataset: - 20060714,X.XX,1,043004,Q,T,24.0000,1,25.5000,4, 20060714,X.XX,1,081209,Q,T,24.0000,1,25.5000,5, As you can see, there are 10 columns here and the table that I am... (1 Reply)
Discussion started by: aarif
1 Replies

8. Shell Programming and Scripting

awk,sed or cut problem

Good afternoon, Sir's, I would like to seek your assistance regarding on this matter. $cat file1 111 aaaa bbb aass aaa files file1 temp temp1 pix 222 11 22 1 33 44 desired output: aaaa bbb aass files file1 temp1 222 11 22 1 33 44 thanks (7 Replies)
Discussion started by: invinzin21
7 Replies

9. Shell Programming and Scripting

awk & sed problem

Hello, I am new to shell scripting. I want to optimize my one of the script. I have one file and i want to remove selected zones for domains from that file.In this file i have almost 3500 zones for domains.Sample data for the file.... named.backup... (0 Replies)
Discussion started by: nrbhole
0 Replies

10. UNIX for Dummies Questions & Answers

sed/awk String problem

I would appreciate it if any one can guide me in using awk perhaps sed in extracting some values from a long string. here is an example. .......some lines here........ ........ aaaa bbbb cccc ddddd eeeee fffff gggg (time:hhhh)........ ......some lines here also.......... How can I extract... (2 Replies)
Discussion started by: odogbolu98
2 Replies
Login or Register to Ask a Question