Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: CSV Delimiter Settings
Top Forums Shell Programming and Scripting CSV Delimiter Settings Post 302726623 by SalientAnimal on Monday 5th of November 2012 02:27:27 AM
Old 11-05-2012
Question CSV Delimiter Settings
Hi All,

I am really new to Shell Scripting and would appreciate any help here.

I have a script that runs throguh a cron job on a daily basis, problem is when the file is attached and e-mailed, it uses a comma delimiter to seperate fields.

How can I prevent the script from using a comma delimiter, maybe even changing the delimiter to something like a pipe delimiter?

Here is my exisiting script:

Code:
#!/bin/bash
IFS= "|"
# Backup directory location e.g /backups
BACKUPDIR="/mnt/data6/reports"
MAILCONTENT="files"
# Set the maximum allowed email size in k. (4000 = approx 5MB email [see docs])
MAXATTSIZE="4000"
# Email Address to send mail to? (user@domain.com)
MAILADDR=user1@mydomain.com
#MAILADDR=user2@mydomain.com
　
# Choose Compression type. (gzip or bzip2)
COMP=gzip
　
# The maximum size of the buffer for client/server communication. e.g
#=====================================================================
#=====================================================================
PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/mysql/bin 
DATE=`date +%Y-%m-%d_%Hh%Mm` # Datestamp e.g 2002-09-21
DOW=`date +%A` # Day of the week e.g. Monday
DNOW=`date +%u` # Day number of the week 1 to 7 where 1 represents Monday
DOM=`date +%d` # Date of the Month e.g. 27
M=`date +%B` # Month e.g January
W=`date +%V` # Week Number e.g 37
LOGFILE=$BACKUPDIR/SMSCampaignsReport_`date +$DATE`.csv # Logfile Name
LOGERR=$BACKUPDIR/ERRORS_SMSCampaignsReport_`date +$DATE`.log # Error Logfile Name
SUBJECTFILE=$BACKUPDIR/CSOPSMessage.txt
DELIMITER= |
　
# IO redirection for logging.
touch $LOGFILE
exec 6>&1 # Link file descriptor #6 with stdout.
# Saves stdout.
exec > $LOGFILE # stdout replaced with file $LOGFILE.
touch $LOGERR
exec 7>&2 # Link file descriptor #7 with stderr.
# Saves stderr.
exec 2> $LOGERR # stderr replaced with file $LOGERR.
　
# Functions
# Database dump function
dbdump () {
mysql -t -u user -ppassword -h"10.249.34.27" -Bse"SELECT submitDate AS 'Date Received'
, message
, sender
FROM MESSAGES
WHERE submitDate >= CURDATE()-8
AND sender not like 27741000123
AND message LIKE '%Smart%'" > $LOGFILE
　
sed -i 's/\+//g' $LOGFILE
sed -i 's/\-//g' $LOGFILE
replace -s '|' ',' -- $LOGFILE
return 0
}
　

echo ======================================================================
echo Verimark Report File
echo Total disk space used for backup storage..
#Clean up IO redirection
exec 1>&6 6>&- # Restore stdout and close file descriptor #6.
exec 1>&7 7>&- # Restore stdout and close file descriptor #7.
#does actual extract
dbdump
if [ "$MAILCONTENT" = "files" ]
then
if [ -s "$LOGERR" ]
then
# Include error log if is larger than zero.
BACKUPFILES="$BACKUPFILES $LOGERR"
ERRORNOTE="WARNING: Error Reported - "
fi

BACKUPFILES=`echo "$BACKUPFILES" | sed -e "s# # -a #g"` #enable multiple attachments
mutt -s "SMS Campaigns Report $DATE" -a $LOGFILE -- $MAILADDR < $SUBJECTFILE

fi
# Clean up Logfile
eval rm -f "$LOGFILE"
eval rm -f "$LOGERR"
exit 0

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Substring based on delimiter, finding last delimiter

Hi, I have a string like ABC.123.XYZ-A1-B2-P1-C4. I want to delimit the string based on "-" and then get result as only two strings. One with string till last hyphen and other with value after last hyphen... For this case, it would be something like first string as "ABC.123.XYZ-A1-B2-P1" and... (6 Replies)
Discussion started by: gupt_ash
6 Replies

2. Shell Programming and Scripting

2 problems: Mailing CSV file / parsing CSV for display

I have been trying to find a good solution for this seemingly simple task for 2 days, and I'm giving up and posting a thread. I hope someone can help me out! I'm on HPUX, using sqlplus, mailx, awk, have some other tools available, but can't install stuff that isn't already in place (without a... (6 Replies)
Discussion started by: soldstatic
6 Replies

3. Shell Programming and Scripting

CSV to SQL insert: Awk for strings with multiple lines in csv

Hi Fellows, I have been struggling to fix an issue in csv records to compose sql statements and have been really losing sleep over it. Here is the problem: I have csv files in the following pipe-delimited format: Column1|Column2|Column3|Column4|NEWLINE Address Type|some descriptive... (4 Replies)
Discussion started by: khayal
4 Replies

4. Shell Programming and Scripting

How to cut by delimiter, and delimiter can be anything except numbers?

Hi all, I have a number of strings like below: //mnt/autocor/43°13'(33")W/ and i'm trying to get the numbers in this string, for example 431333 please help thanks ahead (14 Replies)
Discussion started by: sunnydanniel
14 Replies

5. Shell Programming and Scripting

Perl search csv fileA where two strings exist on another csv fileB

Hi I have two csv files, with the following formats: FileA.log: Application, This occured blah Application, That occured blah Application, Also this AnotherLog, Bob did this AnotherLog, Dave did that FileB.log: Uk, London, Application, datetime, LaterDateTime, Today it had'nt... (8 Replies)
Discussion started by: PerlNewbRP
8 Replies

6. Shell Programming and Scripting

How to avoid Delimiter occuring in column values in .csv file

Hello Gurus, I need to create a file from a .csv file extracting specific columns only. File structure is Column1,Column2,Column3,Column4 abcd,1234,"asdf, tew,123",123456 efgh,234,asdf,654321 My output file should have abcd,123456 efgh,654321 Can you pls help me with the code. ... (10 Replies)
Discussion started by: ritesh.bhawsar
10 Replies

7. Shell Programming and Scripting

Shell script to put delimiter for a no delimiter variable length text file

Hi, I have a No Delimiter variable length text file with following schema - Column Name Data length Firstname 5 Lastname 5 age 3 phoneno1 10 phoneno2 10 phoneno3 10 sample data - ... (16 Replies)
Discussion started by: Gaurav Martha
16 Replies

8. Shell Programming and Scripting

Compare 2 csv files in ksh and o/p the difference in a new csv file

(say) I have 2 csv files - file1.csv & file2.csv as mentioned below: file1.csv ID,version,cost 1000,1,30 2000,2,40 3000,3,50 4000,4,60 file2.csv ID,version,cost 1000,1,30 2000,2,45 3000,4,55 6000,5,70 The... (7 Replies)
Discussion started by: Naresh101
7 Replies

9. Shell Programming and Scripting

Perl Code to change file delimiter (passed as argument) to bar delimiter

Hi, Extremely new to Perl scripting, but need a quick fix without using TEXT::CSV I need to read in a file, pass any delimiter as an argument, and convert it to bar delimited on the output. In addition, enclose fields within double quotes in case of any embedded delimiters. Any help would... (2 Replies)
Discussion started by: JPB1977
2 Replies

10. Shell Programming and Scripting

Match columns from two csv files and update field in one of the csv file

Hi, I have a file of csv data, which looks like this: file1: 1AA,LGV_PONCEY_LES_ATHEE,1,\N,1,00020460E1,0,\N,\N,\N,\N,2,00.22335321,0.00466628 2BB,LES_POUGES_ASF,\N,200,200,00006298G1,0,\N,\N,\N,\N,1,00.30887539,0.00050312... (10 Replies)
Discussion started by: djoseph
10 Replies

LEARN ABOUT REDHAT

logfile

LOGFILE(1)							       mrtg								LOGFILE(1)

NAME

       logfile - description of the mrtg-2 logfile format

SYNOPSIS

       This document provides a description of the contents of the mrtg-2 logfile.

OVERVIEW

       The logfile consists of two main sections. A very short one at the beginning:

       The first Line
	   It stores the traffic counters from the most recent run of mrtg

       The rest of the File
	   Stores past traffic rate averates and maxima at increassing intervals

       The first number on each line is a unix time stamp. It represents the number of seconds since 1970.

DETAILS

       The first Line

       The first line has 3 numbers which are:

       A (1st column)
	   A timestamp of when MRTG last ran for this interface.  The timestamp is the number of non-skip seconds passed since the standard UNIX
	   "epoch" of midnight on 1st of January 1970 GMT.

       B (2nd column)
	   The "incoming bytes counter" value.

       C (3rd column)
	   The "outgoing bytes counter" value.

       The rest of the File

       The second and remaining lines of the file 5 numbers which are:

       A (1st column)
	   The Unix timestamp for the point in time the data on this line is relevant.	Note that the interval between timestamps increases as you
	   prograss through the file. At first it is 5 minutes and at the end it is one day between two lines.

	   This timestamp may be converted in EXCEL by using the following formula:

	    =(x+y)/86400+DATE(1970,1,1)

	   you can also ask perl to help by typing

	    perl -e 'print scalar localtime(x),"
"'

	   x is the unix timestamp and y is the offset in seconds from UTC. (Perl knows y).

       B (2nd column)
	   The average incoming transfer rate in bytes per second. This is valid for the time between the A value of the current line and the A
	   value of the previous line.

       C (3rd column)
	   The average outgoing transfer rate in bytes per second since the previous measurement.

       D (4th column)
	   The maximum incoming transfer rate in bytes per second for the current interval. This is calculated from all the updates which have
	   occured in the current interval. If the current interval is 1 hour, and updates have occured every 5 minutes, it will be the biggest 5
	   minute transferrate seen during the hour.

       E (5th column)
	   The maximum outgoing transfer rate in bytes per second for the current interval.

AUTHOR

       Butch Kemper <kemper@bihs.net> and Tobias Oetiker <oetiker@ee.ethz.ch>

3rd Berkeley Distribution					      2.9.17								LOGFILE(1)
All times are GMT -4. The time now is 01:15 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy