Output to csv contains quotes


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Output to csv contains quotes
# 1  
Old 07-22-2013
Output to csv contains quotes

I borrowed scripting information from thread
Quote:
111391-script-monitor-disk-space-usage
and modified it for our environment to uuencode it and mail a csv to me. The issue I have is the output contains quotes that hinder the csv use appropriately.

My script looks like this

Code:
 
cd /tmp
df -Pg | awk 'BEGIN{
print "\t\t\t SPACE OF IMPORTANT FILE SYSTEMS\t\n-------------------------------------------------------------------------------\n FILE-SYSTEM\t TOTAL-SPACE\t USED-SPACE\t FREE-SPACE\t %USED\n"}
function disp(v1,v2,v3,v4,v5){
printf "%-21s\t %10s\t %13s\t %10s\t %5s \n",v1,v2,v3,v4,v5
}
NR>2&&(($3>($2*.80)&&disp($6,$2,$3,$4,$5))||($3>($2*.40)&&$3<($2*.80)&&disp($6,$2,$3,$4,$5))||($3<($2*.40)&&disp($6,$2,$3,$4,$5)))
END{print "-------------------------------------------------------------------------------\n"}' > email.csv
uuencode email.csv email.csv | /bin/mailx -s "$HOST diskusage usage" email@email.com
rm -rf email.csv


The output looks like this when pulled out of the csv

Code:
" SPACE OF IMPORTANT FILE SYSTEMS "
------------------------------------------------------------------------
" FILE-SYSTEM TOTAL-SPACE USED-SPACE FREE-SPACE %USED"
"/usr 5.00 3.37 1.63 68% "
"/var 3.00 2.35 0.65 79% "
"/tmp 3.75 0.05 3.70 2% "
"/home 2.00 1.30 0.70 65% "
"/admin 0.06 0.00 0.06 1% "
"/proc - - - - "
"/opt 3.00 1.52 1.48 51% "

If I can remove the quotes from the output, the spreadsheet works perfectly (since this is now the practice I have to do to get it to populate the rows and columns correctly.)

What can I do in the script side to remove the quotes from the end product?

I appreciate any input.

Stryker

Last edited by Scrutinizer; 07-22-2013 at 06:40 PM.. Reason: extra code tags
# 2  
Old 07-22-2013
Filter your quote-laden file through
Code:
sed 's;*;;g'

# 3  
Old 07-22-2013
I can't see where all those quotes are added (unless uuencode does so, which would surprise me). Please post the contents of email.csv before being uuencoded and mailed.

BTW - as this is a sheer ASCII file, no need to uuencode - that's intended to be used on binary files.
# 4  
Old 07-22-2013
I suppose the best reason I am using uuencode is I need an actual csv file and not just text output....and most likely am missing something very simple in my mail or mailx command that will send me a file not just text. Why did I do this? Simply because the output, no matter what I tried would not paste back into the spreadsheet "correctly, column and row specific, until after I remove the quotes. I saw a very similar issue 7 years ago on a unix2dos type conversion that remained hidden until I did the same thing.

RudiC - the very odd thing here is the only place the quotes show up is when I pull the data from the csv file and put it into notepad++. Even Notepad, wordpad and vi/vim do not show the characters. It just looks like regular ol' ascii txt.

So the output from all of them looks identical. The forum editor doesnt do it justice on formatting.

Quote:
SPACE OF IMPORTANT FILE SYSTEMS
-------------------------------------------------------------------------------
FILE-SYSTEM TOTAL-SPACE USED-SPACE FREE-SPACE %USED

/usr 5.00 3.37 1.63 68%
/var 3.00 2.35 0.65 79%
/tmp 3.75 0.05 3.70 2%
/home 2.00 1.30 0.70 65%
/admin 0.06 0.00 0.06 1%
/proc - - - -
/opt 3.00 1.52 1.48 51%
/var/adm/ras/livedump 0.25 0.01 0.24 4%
/usr/local 1.00 0.01 0.99 1%
/opt/Tivoli 3.00 1.50 1.50 50%
/root 1.00 0.01 0.99 2%
/mksysb_dir 20.00 8.83 11.17 45%
/opt/esm 2.00 0.48 1.52 25%
/opt/openv 4.00 1.56 2.44 40%
/db 100.00 66.75 33.25 67%
/ai 5.00 0.02 4.98 1%
/bi 30.00 1.12 28.88 4%
However the text pulled from the csv looks like this, still no quotes are visable

Quote:
SPACE OF IMPORTANT FILE SYSTEMS ------------------------------------------------------------------------ FILE-SYSTEM TOTAL-SPACE USED-SPACE FREE-SPACE %USED/ai 5.00 0.02 4.98 1% /batchrpt 1.00 0.02 0.98 2% /bi 30.00 1.12 28.88 4% / 2.00 0.54 1.46 28% /db 100.00 66.75 33.25 67% /db_bkup 120.00 67.52 52.48 57% /dbashell 5.00 0.06 4.94 2% /dbtmp 10.50 3.65 6.85 35% /DUshare 2.00 0.01 1.99 1% /home 2.00 1.30 0.70 65%
Ultimately, this is where the quotes show up.
Quote:
" SPACE OF IMPORTANT FILE SYSTEMS "
------------------------------------------------------------------------
" FILE-SYSTEM TOTAL-SPACE USED-SPACE FREE-SPACE %USED"
"/ai 5.00 0.02 4.98 1% "
"/batchrpt 1.00 0.02 0.98 2% "
"/bi 30.00 1.12 28.88 4% "
"/ 2.00 0.54 1.46 28% "
"/db 100.00 66.75 33.25 67% "
"/db_bkup 120.00 67.52 52.48 57% "
"/dbashell 5.00 0.06 4.94 2% "
"/dbtmp 10.50 3.65 6.85 35% "
"/DUshare 2.00 0.01 1.99 1% "
"/home 2.00 1.30 0.70 65% "
# 5  
Old 07-22-2013
Jpradley, I used the sed with no change in the results.
This User Gave Thanks to strykergli250hp For This Post:
# 6  
Old 07-23-2013
Hold it, hold it - you're talking windows now. Why don't you make it a real csv- file, using comma or semicolon as the output field separator in your awk script, run unix2dos across it to insert the \r char needed by almost all windows editors, and there you go. (You might as well try to make \r\n the output record separator)
# 7  
Old 07-25-2013
Re to RudiC

No. No, RudiC. I would never disrespect this forum with a Doze problem. The only reason I have to deal with it in the first place is the final results come via email and open in Excel. I will have to verify if unix2dos is on any of the servers, predominantly they are AIX. The rest are RedHat. But as you probably are aware, in todays corporate gardens, not everything is how we used to like it to be.

But I think that as intriguing as it sounds, If I can locate one server in 450 +- that has it, you may have a good suggestion. I can run the script from there.

Have you tried the steps, like I suggested, and saw the output in Notepad++? If not, please do. It would help immensely if I can get an outside view, so that this does not look like it is environmentally the issue.

Let me know if you are up to that challenge. If not I need to locate someone who can. It also may be a new bug, that until now, no one has cared that the format is funky.

Any ways, I truly appreciate your input and advice (yes I will re-attempt to use \r\n).
Stryker
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replace Double quotes within double quotes in a column with space while loading a CSV file

Hi All, I'm unable to load the data using sql loader where there are double quotes within the double quotes As these are optionally enclosed by double quotes. Sample Data : "221100",138.00,"D","0019/1477","44012075","49938","49938/15043000","Television - 22" Refurbished - Airwave","Supply... (6 Replies)
Discussion started by: mlavanya
6 Replies

2. Shell Programming and Scripting

Shell script that should remove unnecessary commas between double quotes in CSV file

i have data as below 123,"paul phiri",paul@yahoo.com,"po.box 23, BT","Eco Bank,Blantyre,Malawi" i need an output to be 123,"paul phiri",paul@yahoo.com,"po.box 23 BT","Eco Bank Blantyre Malawi" (5 Replies)
Discussion started by: mathias23
5 Replies

3. Shell Programming and Scripting

How to delete the commas in a .CSV file that are enclosed in a string with double quotes?

Okay, I would like to delete all the commas in a .CSV file (TEST.CSV) or at least substitute them with empty space, that are enclosed in double quote. Please see the sample file as below: column 1,column 2,column 3,column 4,column 5,column 6,column 7,column 8,column 9,column 10... (8 Replies)
Discussion started by: dhruuv369
8 Replies

4. Shell Programming and Scripting

Convert csv to pipe delimited except the ones in double quotes

I have a csv data file : A,B,C,D,"A,B",E,"GG,H" E,F,G,H,I,J,"S,P" I need to replace all "," with "|" except the ones between double quotes i.e A|B|C|D|"A,B"|E|"GG,H" E|F|G|H|I|J|"S,P" CAn someone assist? (8 Replies)
Discussion started by: Shivdatta
8 Replies

5. Shell Programming and Scripting

HELP with AWK or SED. Need to replace the commas between double quotes in CSV file

Hello experts, I need to validate a csv file which contains data like this: Sample.csv "ABCD","I",23,0,9,,"23/12/2012","OK","Street,State, 91135",0 "ABCD","I",23,0,9,,"23/12/2012","OK","Street,State, 91135",0 I just need to check if all the records contain exactly the number of... (5 Replies)
Discussion started by: shell_boy23
5 Replies

6. Shell Programming and Scripting

Excel CSV with Double Quotes and Carriage Returns

Hi List, I am sure this issue must have been encountered before. I have saved an excel CSV and sent it to my linux box where I require to parse it as a CSV line by line. The file displays fine in Excel with csv headers and consistent records and fields. However when I look at it in linux I see... (2 Replies)
Discussion started by: landossa
2 Replies

7. Shell Programming and Scripting

adding quotes around each column in a csv file

I saved the csv file in a comma delimited format. Sample input input.csv 1 abc 2 2 def 4 3 ghi 6 4 jkl 8 5 mno 10 output.csv should look like this with single quotes around each field '1' 'abc' '2' '2' 'def' '4' '3' 'ghi' '6' '4' 'jkl' '8' '5' 'mno' '10' Please help me :confused:... (3 Replies)
Discussion started by: melannie
3 Replies

8. Shell Programming and Scripting

Replacing comma with in double quotes in a csv file

Hello, I need to read a csv file and I am trying to replace a comma with a text DSEE?DSEE. Example Input "Chapter","NewTrains, "oldTrains","Delayed",10,"London" "Chapter","Newbuses,oldbuses","On Time",20,"London" Output "Chapter","NewTrainsDSEE?DSEE... (5 Replies)
Discussion started by: venkatvani
5 Replies

9. Shell Programming and Scripting

Fix CSV file with column missing quotes

I have a CSV file that is missing quotes around a column that contains text with commas. Example: Column1, Column2, Column3, Column4, Column5, Column6 Data1, Data2, Data3, Data, 4, Data5, Data6 Data1, Data3, Data3, Data, text, 4, Data5, Data6 I think the easiest way for me to fix this is to... (2 Replies)
Discussion started by: EmptyH
2 Replies

10. Shell Programming and Scripting

replace value with double quotes of specific coulmn value in csv file

Hi, I am trying to replace a specific column values in a csv file with double quotes. Example: SNO,NAME,ZIPCODE,RANK 1,Robert,74538,12 2,Sam,07564,13 3,Kim, Ed,12345,14 Desired Output: SNO,NAME,ZIPCODE,RANK 1,Robert Ken,74538,12 2,Sam Mik,"07564",13 3,"Kim, Ed",12345,14 I... (3 Replies)
Discussion started by: techmoris
3 Replies
Login or Register to Ask a Question