Sponsored Content
Top Forums UNIX for Advanced & Expert Users shell script to format .CSV data Post 302363078 by tmcmurtr on Monday 19th of October 2009 10:35:36 AM
Old 10-19-2009
shell script to format .CSV data

Hi all,

I have written a shell script to search a specified directory (e.g. /home/user) for a list of specific words (shown as ${TMPDIR}/wordlist below). The script works well enough, but I was wondering if there was a way to display the line number that the word is found on?

Thanks!

Code:
cat << EOF > ${TMPDIR}/scanit
rm -f ${TMPDIR}/strings
strings "\$1" | egrep -i -f ${TMPDIR}/wordlist >> ${TMPDIR}/strings
if [ -s ${TMPDIR}/strings ]
then
	echo >> ${TMPDIR}/${HOSTNAME}.o
	echo "File:  \$1" >> ${TMPDIR}/${HOSTNAME}.o
	file "\$1"  >> ${TMPDIR}/${HOSTNAME}.o
	cat ${TMPDIR}/strings >> ${TMPDIR}/${HOSTNAME}.o
fi
rm -f ${TMPDIR}/strings
EOF

HOSTNAME=`hostname`
export HOSTNHAME

if [ $# -eq 0 ]
then
	echo "You must specify the start of the directory tree to search"
	exit
fi

find $1 -type f 2> ${TMPDIR}/${HOSTNAME}_find_errors | tee ${TMPDIR}/${HOSTNAME}_filelist | \
head -100 |\
sed -e "s+^+sh -x ${TMPDIR}/scanit \"+" -e 's/$/"/' > ${TMPDIR}/scanitnow

sh -x ${TMPDIR}/scanitnow 1> ${TMPDIR}/${HOSTNAME}_scan_run 2>&1

cd ${TMPDIR}
if [ -s ${HOSTNAME}.o ]
then
	date "+%Y%M%d_%H:%m:%S: indicators found on ${HOSTNAME}" > ${HOSTNAME}_scan_results.csv
	cat ${HOSTNAME}.o >> ${HOSTNAME}_scan_results.csv
else
	date "+%Y%M%d_%H:%m:%S:  No indicators found on ${HOSTNAME}" > ${HOSTNAME}_scan_results.csv
fi

 

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

format csv file using shell script

i have a report.csv file from oracle datavase In that file data is like this with report heading and date SALES DAILY REPORT DATE: 06-26-2007 REPORT NAME: SALES DATA AA.BB.CCCC.DDDD,BBBBB,06-26-2007,0,BEGIN,END ZZ.VV.DDDD.XXXXXXX,MMMMMM,06-25-2007,18,BEGIN,END... (3 Replies)
Discussion started by: raosurya
3 Replies

2. Shell Programming and Scripting

Shell script to format a .CSV data

Hi There I needed to write a Unix shell script which will pick up the data from a .CSV file and reformat it as per the requirement and write it to another .CSV file. Currently I am in the proess of Data Import to "Remedy System" (A one kind of incident mangement Application) and this... (8 Replies)
Discussion started by: Uday1982
8 Replies

3. Shell Programming and Scripting

Exporting data as a CSV file from Unix shell script

Friends...This is the first time i am trying the report generation using shell script... any suggestions are welcome. Is there a way to set the font size & color when i am exporting the data from unix shell script as a CSV file ? The following sample data is saved as a .csv file in the... (2 Replies)
Discussion started by: appu2176
2 Replies

4. Shell Programming and Scripting

Reading the data from CSV and performing search through shell script

Hello, I am working on building a script that does the below actions together in my Linux server. 1) First, have to read the list of strings mentioned in CSV and store it in the shell script 2) Second, pick one by one from the string list, and search a particular folder for files that... (2 Replies)
Discussion started by: vikrams
2 Replies

5. Shell Programming and Scripting

Need perl or shell script to sort vertical lines to horizontal line in csv format

Need perl or shell script to sort vertical lines to horizontal line in csv format My file like below ------------------------- ================================================================================ PATH PINKY1000#I1-1-ZENTA1000-2#I7-1-ASON-SBR-UP-943113845 ... (4 Replies)
Discussion started by: sreedhargouda.h
4 Replies

6. Shell Programming and Scripting

Format CSV file from a shell script

I have a shell script which creates a CSV files. there are 3 fields, field1, field2 and comp. I will generates the values for field1 and field2 and Want to compare both. If field1>filed2 then comp should be success written in green in CSV file, else it should fail in red color. How can I change the... (5 Replies)
Discussion started by: sauravrout
5 Replies

7. UNIX for Dummies Questions & Answers

Shell script to extract data from csv file

Hi Guys, I am new to shell script.I need your help to write a shell script. I need to write a shell script to extract data from a .csv file where columns are ',' separated. The file has 7 columns having values say column 1,column 2.....column 7 as below along with their values. Name, Address,... (7 Replies)
Discussion started by: Vivekit82
7 Replies

8. Shell Programming and Scripting

Shell script to extract data from csv file

Hi everyone, I have a csv file which has data with different heading and column names as below. Static Data Ingested ,,,,,,,,,,,,Known Explained Rejections Column_1,column_2,Column_3,Column_4,,Column_6,Column_7,,% Column_8,,Column_9 ,Column_10 ,... (14 Replies)
Discussion started by: Vivekit82
14 Replies

9. Programming

Python or Shell script to Grep strings from input file and output in csv format

Hi Experts, I am writing a python script to grep string from file and display output in csv file as in attached screenshot https://drive.google.com/file/d/1gfUUdfmQma33tz65NskThYDhkZUGQO0H/view Input file(result_EPFT_config_device) Below is the python script i have prepared as of... (1 Reply)
Discussion started by: as7951
1 Replies
mktemp(1)							   User Commands							 mktemp(1)

NAME
mktemp - make temporary filename SYNOPSIS
mktemp [-dtqu] [-p directory] [template] DESCRIPTION
The mktemp utility makes a temporary filename. To do this, mktemp takes the specified filename template and overwrites a portion of it to create a unique filename. See OPERANDS. The template is passed to mkdtemp(3C) for directories or mkstemp(3C) for ordinary files. If mktemp can successfully generate a unique filename, the file (or directory) is created with file permissions such that it is only read- able and writable by its owner (unless the -u flag is given) and the filename is printed to standard output. mktemp allows shell scripts to safely use temporary files. Traditionally, many shell scripts take the name of the program with the PID as a suffix and used that as a temporary filename. This kind of naming scheme is predictable and the race condition it creates is easy for an attacker to win. A safer, though still inferior approach is to make a temporary directory using the same naming scheme. While this guaran- tees that a temporary file is not subverted, it still allows a simple denial of service attack. Use mktemp instead. OPTIONS
The following options are supported: -d Make a directory instead of a file. -p directory Use the specified directory as a prefix when generating the temporary filename. The directory is overridden by the user's TMPDIR environment variable if it is set. This option implies the -t flag. -q Fail silently if an error occurs. This is useful if a script does not want error output to go to standard error. -t Generate a path rooted in a temporary directory. This directory is chosen as follows: If the user's TMPDIR environment variable is set, the directory contained therein is used. Otherwise, if the -p flag was given the specified directory is used. If none of the above apply, /tmp is used. In this mode, the template (if specified) should be a directory component (as opposed to a full path) and thus should not contain any forward slashes. -u Operate in unsafe mode. The temp file is unlinked before mktemp exits. This is slightly better than mktemp(3C), but still introduces a race condition. Use of this option is discouraged. OPERANDS
The following operands are supported: template template can be any filename with one or more Xs appended to it, for example /tmp/tfile.XXXXXX. If template is not specified, a default of tmp.XXXXXX is used and the -t flag is implied. EXAMPLES
Example 1 Using mktemp The following example illustrates a simple use of mktemp in a sh(1) script. In this example, the script quits if it cannot get a safe tem- porary file. TMPFILE=`mktemp /tmp/example.XXXXXX` if [ -z "$TMPFILE" ]; then exit 1; fi echo "program output" >> $TMPFILE Example 2 Using mktemp to Support TMPDIR The following example uses mktemp to support for a user's TMPDIR environment variable: TMPFILE=`mktemp -t example.XXXXXX` if [ -z "$TMPFILE" ]; then exit 1; fi echo "program output" >> $TMPFILE Example 3 Using mktemp Without Specifying the Name of the Temporary File The following example uses mktemp without specifying the name of the temporary file. In this case the -t flag is implied. TMPFILE=`mktemp` if [ -z "$TMPFILE" ]; then exit 1; fi echo "program output" >> $TMPFILE Example 4 Using mktemp with a Default Temporary Directory Other than /tmp The following example creates the temporary file in /extra/tmp unless the user's TMPDIR environment variable specifies otherwise: TMPFILE=`mktemp -p /extra/tmp example.XXXXX` if [ -z "$TMPFILE" ]; then exit 1; fi echo "program output" >> $TMPFILE Example 5 Using mktemp to Remove a File The following example attempts to create two temporary files. If creation of the second temporary file fails, mktemp removes the first file before exiting: TMP1=`mktemp -t example.1.XXXXXX` if [ -z "$TMP1" ]; then exit 1; fi TMP2=`mktemp -t example.2.XXXXXX` if [ -z "$TMP2" ]; then rm -f $TMP1 exit 1 fi Example 6 Using mktemp The following example does not exit if mktemp is unable to create the file. That part of the script has been protected. TMPFILE=`mktemp -q -t example.XXXXXX` if [ ! -z "$TMPFILE" ] then # Safe to use $TMPFILE in this block echo data > $TMPFILE ... rm -f $TMPFILE fi ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of mktemp with the -t option: TMPDIR. EXIT STATUS
The following exit values are returned: 0 Successful completion. 1 An error occurred. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ |Interface Stability |Committed | +-----------------------------+-----------------------------+ SEE ALSO
sh(1), mkdtemp(3C), mkstemp(3C), attributes(5), environ(5) NOTES
The mktemp utility appeared in OpenBSD 2.1. The Solaris implementation uses only as many `Xs' as are significant for mktemp(3C) and mkstemp(3C). SunOS 5.11 10 Jan 2008 mktemp(1)
All times are GMT -4. The time now is 08:27 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy