CSV File with Multiple Search Parameter


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting CSV File with Multiple Search Parameter
# 1  
Old 03-08-2017
CSV File with Multiple Search Parameter

Dear Team Members,

I have a unique problem. Below is the dataset which I have. I am writing a script which will read through the file and pull the invoice no. (Field 2 of C1 row).

Code:
"C1",990001,"L1","HERO","MOTORCYCLE","ASIA-PACIFIC","BEIJING"
"C2","CLUTCH","HYUNDAI",03032017
"C2","BRAKES","EUROPARTS",03032017
"C2","TYRES","MICHELIN",03032017

"C1",990002,"L2","HERO","CAR","EU1","BUDAPEST"
"C2","CLUTCH","FORD",03032017
"C2","BRAKES","EUROPARTS",03032017
"C2","TYRES","PIRELLI",03032017

For example. My script needs to accept parameters like TYRES MICHELIN.
Script will search the file for the C2 rows and whereever it will find the above two parameters - it will bring the Invoice number along wih the MICHELIN from respective C1 rows. It will also remove the doubtquotes from the texts.

Code:
./findInvoice.sh MICHELIN

Output of the script will give me
Code:
990001 MICHELIN

Another example is

Code:
./findInvoice.sh  PIRELLI

and output of the script will be
Code:
990002 PIRELLI

Any help would be useful.

Thanks.

Last edited by chetanojha; 03-08-2017 at 06:18 PM..
# 2  
Old 03-08-2017
You are calling your shell script differently.
./findInvoice.sh TYRE MICHELIN
AND
./findInvoice.sh PIRELLI
Which one is correct, as the _positional_ parameter gives '$2' in the first and '$1' in the second.
Please clarify.
I am assuming 'TYRE' should not be there and that '$1' is needed for both.
# 3  
Old 03-08-2017
I have now corrected the calling of script.

Code:
./findInvoice.sh MICHELIN
or
./findInvoice.sh PIRELLI

Above is the correct one
# 4  
Old 03-08-2017
Hi,

Is this the sort of thing you're after ? Note that 'example.txt' is just populated with the sample input that you supplied in your post.

Code:
$ cat script.sh
#!/bin/bash
file=example.txt

if [ "$1" == "" ]
then
        echo You must supply a search term
        exit 1
fi

while read -r line
do
        case "$line" in
                \"C1\",*)
                        invoice=`echo $line | /usr/bin/awk -F, '{gsub(/"/,""); print $2}'`
                        ;;
                \"C2\",*)
                        if echo $line | /bin/grep ,\"$1\" >/dev/null 2>/dev/null
                        then
                                echo $invoice $1
                                unset invoice
                        fi
                        ;;
        esac
done < "$file"

$ ./script.sh PIRELLI
990002 PIRELLI
$ ./script.sh MICHELIN
990001 MICHELIN
$ ./script.sh EUROPARTS
990001 EUROPARTS
990002 EUROPARTS
$

Hope this helps. If not, then if you could advise how the behaviour or output differs from your expectations I can have another crack at solving this one.
# 5  
Old 03-09-2017
If you want to search for multiple strings on one invocation of your script or if you want to print the 2nd field from C1 lines even if a given search string is also found on a C1 (as well as on a C2) line, you could try this simplified script:
Code:
#!/bin/ksh
printf '%s\n\n' "$@" |
awk -F, -v RS= '
NR == FNR {
	keys[$0]
	next
}
{	for(key in keys)
		if(index($0, key))
			print $2, key
}' - dataset

which, if invoked with the command:
Code:
./script.sh  BEIJING FORD HYUNDAI TYRE

with your sample input in a file named dataset would produce the output:
Code:
990001 HYUNDAI
990001 BEIJING
990001 TYRE
990002 TYRE
990002 FORD

This depends on the C1 lines being the 1st line in a record and there being a blank line in your file between records which seems to fit your sample input file.

This was written and tested with a Korn shell, but will also work with any other shell that uses Bourne shell syntax (such as bash). With minor changes, this could be made to print only records that matched all operands given to your script instead of matching any of the operands. But, to do that, you'd have to specify which of the operands is to be printed when they all match.

If you want to try this on a Solaris/SunOS system, change awk in the script to /usr/xpg4/bin/awk or nawk.
This User Gave Thanks to Don Cragun For This Post:
# 6  
Old 03-09-2017
Hello Don,

I am not able to run this script. It says cannot open the -dataset.

Other Few questions:
1. How do I pass multiple files? So that this script can look into multiple files (over 100K+).
2. I am running this script in bash shell. Script will look into over 100m records to find out the selected few.
3. This will output the result in a file?
# 7  
Old 03-09-2017
Please consider the space between "-" and "dataset" in Don Cragun's proposal. "-" designates the stdin file descriptor.

1. specify several file names in lieu of just "dataset". Please be aware that the source of the data (file name) will not be printed with the data.
2. record count shouldn't be a problem as long as system limits are not exceeded.
3. As given the code prints to stdout. Use redirection to capture it in a file.
This User Gave Thanks to RudiC For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Combine and complete multiple CSV files based on 1 parameter

I have to create a new CSV file based on the value listed on the 3rd column from different CSV files. This is what I need: 1. I should substitute the first column from each file, excluding the headers, with the file name InputXX. 2. Then, I need to look for rows with 0 on the third column in... (7 Replies)
Discussion started by: Xterra
7 Replies

2. Shell Programming and Scripting

Replace multiple file by passing parameter value

Hello All, I want to change date part in file name to yesterday date in the file name. example file name file-12122017-06-30-41.dat want file-12112017-06-30-41.dat I am doing like below. Below it is not changing the filename. Actually it is not parsing the $today and $yesterday value in... (1 Reply)
Discussion started by: looney
1 Replies

3. Shell Programming and Scripting

Multiple Parameter search script - OLD Post

Dear Forum Member, Hope you are well. I am sending this email to your reply to one of my post in the forum. I raised an help request from all of you regarding an issue I was facing. The original thread is here ... (3 Replies)
Discussion started by: chetanojha
3 Replies

4. Shell Programming and Scripting

Multiple file and single parameter file

Hi Team, In our project we have written below 2 scripts like Script1: Shell script start & END Begin Audit process - uses teradata bteq END Audit Process Script 2: Environemtal variable file different Now Client ask to change this requirement and need below files: Script1:... (1 Reply)
Discussion started by: tusharzaware1
1 Replies

5. Shell Programming and Scripting

Read in search strings from text file, search for string in second text file and output to CSV

Hi guys, I have a text file named file1.txt that is formatted like this: 001 , ID , 20000 002 , Name , Brandon 003 , Phone_Number , 616-234-1999 004 , SSNumber , 234-23-234 005 , Model , Toyota 007 , Engine ,V8 008 , GPS , OFF and I have file2.txt formatted like this: ... (2 Replies)
Discussion started by: An0mander
2 Replies

6. UNIX for Dummies Questions & Answers

Search file and print everything except multiple search terms

I'm trying to find a way to search a range of similar words in a file. I tried using sed but can't get it right:sed 's/\(ca01\)*//'It only removes "ca01" but leaves the rest of the word. I still want the rest of the information on the lines just not these specific words listed below. Any... (3 Replies)
Discussion started by: seekryts15
3 Replies

7. Shell Programming and Scripting

Perl search csv fileA where two strings exist on another csv fileB

Hi I have two csv files, with the following formats: FileA.log: Application, This occured blah Application, That occured blah Application, Also this AnotherLog, Bob did this AnotherLog, Dave did that FileB.log: Uk, London, Application, datetime, LaterDateTime, Today it had'nt... (8 Replies)
Discussion started by: PerlNewbRP
8 Replies

8. Shell Programming and Scripting

search and replace in csv file

I have csv file where I want the second column has to be replaced with value 1. Source file 919568760477,1,2011-07-11T22:34:27.000+05:30, 919557735692,2,2011-07-11T22:36:16.000+05:30, 917417384969,2,2011-07-11T22:33:26.000+05:30, Final file ... (30 Replies)
Discussion started by: dondilip
30 Replies

9. Shell Programming and Scripting

Appending a parameter value to a .csv file???

Hi I have a date which I get as parameter. I want it to be added as first column in all the rows in a csv file. I have tried the below code but no success. date=$1 awk -F"," '{print $date","$0}' z3.csv > z4.csv Could you tell the correct code for the above req.? How to use code... (1 Reply)
Discussion started by: msp2244
1 Replies

10. Shell Programming and Scripting

Handle Configuration File with same name of Parameter in multiple Sections

Hi I have a config file with multiple section and a parameter with the same name in each section. I need to read each parameter for distinct section. Parameter = 1 .... Parameter = 2 .... Parameter = 4 .... Tried this: grep -m1 '^*ProcessorsNumber' ServiceBrokerFramework.cfg |... (7 Replies)
Discussion started by: potro
7 Replies
Login or Register to Ask a Question