CSV Delimiter Settings


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting CSV Delimiter Settings
# 1  
Old 11-05-2012
Question CSV Delimiter Settings

Hi All,

I am really new to Shell Scripting and would appreciate any help here.

I have a script that runs throguh a cron job on a daily basis, problem is when the file is attached and e-mailed, it uses a comma delimiter to seperate fields.

How can I prevent the script from using a comma delimiter, maybe even changing the delimiter to something like a pipe delimiter?

Here is my exisiting script:

Code:
#!/bin/bash
IFS= "|"
# Backup directory location e.g /backups
BACKUPDIR="/mnt/data6/reports"
MAILCONTENT="files"
# Set the maximum allowed email size in k. (4000 = approx 5MB email [see docs])
MAXATTSIZE="4000"
# Email Address to send mail to? (user@domain.com)
MAILADDR=user1@mydomain.com
#MAILADDR=user2@mydomain.com
 
# Choose Compression type. (gzip or bzip2)
COMP=gzip
 
# The maximum size of the buffer for client/server communication. e.g
#=====================================================================
#=====================================================================
PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/mysql/bin 
DATE=`date +%Y-%m-%d_%Hh%Mm` # Datestamp e.g 2002-09-21
DOW=`date +%A` # Day of the week e.g. Monday
DNOW=`date +%u` # Day number of the week 1 to 7 where 1 represents Monday
DOM=`date +%d` # Date of the Month e.g. 27
M=`date +%B` # Month e.g January
W=`date +%V` # Week Number e.g 37
LOGFILE=$BACKUPDIR/SMSCampaignsReport_`date +$DATE`.csv # Logfile Name
LOGERR=$BACKUPDIR/ERRORS_SMSCampaignsReport_`date +$DATE`.log # Error Logfile Name
SUBJECTFILE=$BACKUPDIR/CSOPSMessage.txt
DELIMITER= |
 
# IO redirection for logging.
touch $LOGFILE
exec 6>&1 # Link file descriptor #6 with stdout.
# Saves stdout.
exec > $LOGFILE # stdout replaced with file $LOGFILE.
touch $LOGERR
exec 7>&2 # Link file descriptor #7 with stderr.
# Saves stderr.
exec 2> $LOGERR # stderr replaced with file $LOGERR.
 
# Functions
# Database dump function
dbdump () {
mysql -t -u user -ppassword -h"10.249.34.27" -Bse"SELECT submitDate AS 'Date Received'
, message
, sender
FROM MESSAGES
WHERE submitDate >= CURDATE()-8
AND sender not like 27741000123
AND message LIKE '%Smart%'" > $LOGFILE
 
sed -i 's/\+//g' $LOGFILE
sed -i 's/\-//g' $LOGFILE
replace -s '|' ',' -- $LOGFILE
return 0
}
 

echo ======================================================================
echo Verimark Report File
echo Total disk space used for backup storage..
#Clean up IO redirection
exec 1>&6 6>&- # Restore stdout and close file descriptor #6.
exec 1>&7 7>&- # Restore stdout and close file descriptor #7.
#does actual extract
dbdump
if [ "$MAILCONTENT" = "files" ]
then
if [ -s "$LOGERR" ]
then
# Include error log if is larger than zero.
BACKUPFILES="$BACKUPFILES $LOGERR"
ERRORNOTE="WARNING: Error Reported - "
fi

BACKUPFILES=`echo "$BACKUPFILES" | sed -e "s# # -a #g"` #enable multiple attachments
mutt -s "SMS Campaigns Report $DATE" -a $LOGFILE -- $MAILADDR < $SUBJECTFILE

fi
# Clean up Logfile
eval rm -f "$LOGFILE"
eval rm -f "$LOGERR"
exit 0

# 2  
Old 11-05-2012
try commenting below line..
Here it replaces | with ,
Code:
#replace -s '|' ',' -- $LOGFILE

# 3  
Old 11-05-2012
Thanks I didn't see that. Took over the script from someone else. So the problem doesn't seem to be the ',' as I had thought. When my file exports, some of the data is placed on a new line, rather than being kept on the line it was originally on.

I originally thoguht this was because of the comma delimiter.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Match columns from two csv files and update field in one of the csv file

Hi, I have a file of csv data, which looks like this: file1: 1AA,LGV_PONCEY_LES_ATHEE,1,\N,1,00020460E1,0,\N,\N,\N,\N,2,00.22335321,0.00466628 2BB,LES_POUGES_ASF,\N,200,200,00006298G1,0,\N,\N,\N,\N,1,00.30887539,0.00050312... (10 Replies)
Discussion started by: djoseph
10 Replies

2. Shell Programming and Scripting

Perl Code to change file delimiter (passed as argument) to bar delimiter

Hi, Extremely new to Perl scripting, but need a quick fix without using TEXT::CSV I need to read in a file, pass any delimiter as an argument, and convert it to bar delimited on the output. In addition, enclose fields within double quotes in case of any embedded delimiters. Any help would... (2 Replies)
Discussion started by: JPB1977
2 Replies

3. Shell Programming and Scripting

Compare 2 csv files in ksh and o/p the difference in a new csv file

(say) I have 2 csv files - file1.csv & file2.csv as mentioned below: file1.csv ID,version,cost 1000,1,30 2000,2,40 3000,3,50 4000,4,60 file2.csv ID,version,cost 1000,1,30 2000,2,45 3000,4,55 6000,5,70 The... (7 Replies)
Discussion started by: Naresh101
7 Replies

4. Shell Programming and Scripting

Shell script to put delimiter for a no delimiter variable length text file

Hi, I have a No Delimiter variable length text file with following schema - Column Name Data length Firstname 5 Lastname 5 age 3 phoneno1 10 phoneno2 10 phoneno3 10 sample data - ... (16 Replies)
Discussion started by: Gaurav Martha
16 Replies

5. Shell Programming and Scripting

How to avoid Delimiter occuring in column values in .csv file

Hello Gurus, I need to create a file from a .csv file extracting specific columns only. File structure is Column1,Column2,Column3,Column4 abcd,1234,"asdf, tew,123",123456 efgh,234,asdf,654321 My output file should have abcd,123456 efgh,654321 Can you pls help me with the code. ... (10 Replies)
Discussion started by: ritesh.bhawsar
10 Replies

6. Shell Programming and Scripting

Perl search csv fileA where two strings exist on another csv fileB

Hi I have two csv files, with the following formats: FileA.log: Application, This occured blah Application, That occured blah Application, Also this AnotherLog, Bob did this AnotherLog, Dave did that FileB.log: Uk, London, Application, datetime, LaterDateTime, Today it had'nt... (8 Replies)
Discussion started by: PerlNewbRP
8 Replies

7. Shell Programming and Scripting

How to cut by delimiter, and delimiter can be anything except numbers?

Hi all, I have a number of strings like below: //mnt/autocor/43°13'(33")W/ and i'm trying to get the numbers in this string, for example 431333 please help thanks ahead (14 Replies)
Discussion started by: sunnydanniel
14 Replies

8. Shell Programming and Scripting

CSV to SQL insert: Awk for strings with multiple lines in csv

Hi Fellows, I have been struggling to fix an issue in csv records to compose sql statements and have been really losing sleep over it. Here is the problem: I have csv files in the following pipe-delimited format: Column1|Column2|Column3|Column4|NEWLINE Address Type|some descriptive... (4 Replies)
Discussion started by: khayal
4 Replies

9. Shell Programming and Scripting

2 problems: Mailing CSV file / parsing CSV for display

I have been trying to find a good solution for this seemingly simple task for 2 days, and I'm giving up and posting a thread. I hope someone can help me out! I'm on HPUX, using sqlplus, mailx, awk, have some other tools available, but can't install stuff that isn't already in place (without a... (6 Replies)
Discussion started by: soldstatic
6 Replies

10. Shell Programming and Scripting

Substring based on delimiter, finding last delimiter

Hi, I have a string like ABC.123.XYZ-A1-B2-P1-C4. I want to delimit the string based on "-" and then get result as only two strings. One with string till last hyphen and other with value after last hyphen... For this case, it would be something like first string as "ABC.123.XYZ-A1-B2-P1" and... (6 Replies)
Discussion started by: gupt_ash
6 Replies
Login or Register to Ask a Question