Replace query by reading the file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Replace query by reading the file
# 1  
Old 08-17-2017
Replace query by reading the file

Hi Guys,

I am having below file which holds data like this
file.txt
Code:
name,id,flag
apple,1,Y
apple,2,N
mango,1,Y
mango,2,Y

I need to read the above file and frame a query like this
Code:
hive -s -e "create apple_view as select 1 from main_table;"
hive -s -e "create mango_view as select 1,2 from main_table;"

i need take the column id where the flag is Y so here its '1' for apple and 1,2 for mango
# 2  
Old 08-17-2017
Code:
 
[skrynesaver@busybox]$cat tmp.dat
name,id,flag
apple,1,Y
apple,2,N
mango,1,Y
mango,2,Y
[skrynesaver@busybox]$ perl -ne 'chomp;@r=split/,/; push @{$fields{$r[0]}},$r[1] if $r[2] eq "Y";}{for $fruit (keys %fields){print "hive -s -e \"create ${fruit}_view as select ", join",",@{$fields{$fruit}}," from main table\"\n"}' tmp.dat
hive -s -e "create mango_view as select 1,2, from main table"
hive -s -e "create apple_view as select 1, from main table"
[skrynesaver@busybox]$

This User Gave Thanks to Skrynesaver For This Post:
# 3  
Old 08-17-2017
Thanks a lot,

Can this be achived in Shell scripting instead of perl
# 4  
Old 08-17-2017
That's a shell command Smilie I can break it into digestible chunks if you wish
Code:
perl -ne '               #Call the Perl interpreter and iterate over every line in the files listed as arguments with the following script
chomp;                 # Strip newlines
@r=split/,/;           #Create an array r of each field (separated by ",") in the record
push @{$fields{$r[0]}},$r[1] if $r[2] eq "Y";  # if the falg is "Y" then add the id to an array stored in a hash and keyed on name
}{                                                                # Eskino's nose execute the remainder when you've finished processing the file(s)
for $fruit (keys %fields){                               # for each name field we stored an array for
print "hive -s -e \"create ${fruit}_view as select ", join",",@{$fields{$fruit}}," from main table\"\n"} # Print the required string for the named joining the id's with a ","
' tmp.dat            # for this file

This User Gave Thanks to Skrynesaver For This Post:
# 5  
Old 08-17-2017
Thanks a lot, what i was trying to say is instead of using perl interpretor can i use any other unix interpreter and i would like to pass the fruit as parameter
# 6  
Old 08-17-2017
Pure shell:
Code:
{ read
   while IFS=, read NAME ID FLAG
     do if [ ! "$LAST" = "$NAME" ]
          then  [ "$LAST" ] &&  printf " from main_table;\"\n"
                printf "hive -s -e \"create ${NAME}_view as select "
                SEP=" "
        fi
        [ "$FLAG" = "Y" ] && printf "%s%s" $SEP $ID
        SEP=","
        LAST=$NAME
     done
   printf " from main_table;\"\n"
} < file
hive -s -e "create apple_view as select 1 from main_table;"
hive -s -e "create mango_view as select 1,2 from main_table;"

# 7  
Old 08-17-2017
Thanks Rudic,

In your code where you will be passing fruit as parameter , because i wanted to pass that as parameter so that it can extract that from file
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Emergency UNIX and Linux Support

sed replace file contents by reading from another file

Hello, My input file1 is like this by tab-delimited chr1 mm10_knownGene stop_codon 3216022 3216024 0.000000 - . gene_id "uc007aeu.1"; transcript_id "uc007aeu.1"; chr1 mm10_knownGene CDS 3216025 3216968 0.000000 - 2 gene_id "uc007aeu.1"; transcript_id "uc007aeu.1"; ... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

2. Shell Programming and Scripting

Search & Replace in Multiple Files by reading a input file

I have a environment property file which contains: Input file: value1 = url1 value2 = url2 value3 = url3 and so on. I need to search all *.xml files under directory for value1 and replace it with url1. Same thing I have to do for all values mentioned in input file. I need script in unix bash... (7 Replies)
Discussion started by: Shamkamde
7 Replies

3. Shell Programming and Scripting

Reading values from sql query

I have sql query in shell script. select distinct account_no from adj order by account_no; This query returns account number daily.Sometimes it may return 90 rows sometime it may return 1 row only and someday it may return 0 rows I am storing the output of this query in sql_output.txt. ... (5 Replies)
Discussion started by: rafa_fed2
5 Replies

4. Programming

Conditional replace after reading in a file

I need to read the contents of a file. Then I need to grep for a keyword and replace part of the grepped line based on the condition of previous and present line. Example input file: V { port1 = P; port2 = 0; shift_port = P0; /* if next shift_port is P0 I need... (7 Replies)
Discussion started by: naveen@
7 Replies

5. Shell Programming and Scripting

Replace a file reading from another file

Hi Guys, I have a file which has string which needs to be replaced by what. Change.txt Former Replace with ASD AAD ABP NAID I like to read this file and search another file and replace the content. For an example, read Change.txt and replace ASD with AAD in the... (2 Replies)
Discussion started by: mac4rfree
2 Replies

6. Shell Programming and Scripting

Shell Script to execute Oracle query taking input from a file to form query

Hi, I need to query Oracle database for 100 users. I have these 100 users in a file. I need a shell script which would read this User file (one user at a time) & query database. For instance: USER CITY --------- ---------- A CITY_A B CITY_B C ... (2 Replies)
Discussion started by: DevendraG
2 Replies

7. Shell Programming and Scripting

Search & Replace in Multiple Files by reading a input file

Hi, I have a folder which contains multiple config.xml files and one input file, Please see the below format. Config Files format looks like :- Code: <application name="SAMPLE-ARCHIVE"> <NVPairs name="Global Variables"> <NameValuePair> ... (0 Replies)
Discussion started by: haiksuresh
0 Replies

8. Solaris

Mysql query reading a text file.

Hi, Is there any way that mysql query reads the content from a text file which has data in the below format: 1,2,3,4,5 and selects matched data from another table. ie I have a table named xyz which has same ids as in that file. I want this query to get count of the ids from xyz file by... (6 Replies)
Discussion started by: jyothi_wipro
6 Replies

9. Shell Programming and Scripting

need help in reading a output of a sql query in unix script

i'm used a sql query in a unix script to get the information from table. but unable to extract the output which i need. Any help with logic will be greatly appreciated. my sql query provide output some thing like this - col1 col2 count ---- ---- ------ A B 10 c D 6 e... (8 Replies)
Discussion started by: pharos467
8 Replies

10. UNIX for Dummies Questions & Answers

Reading from a file and passing the value to a select query

Hi all, Here is my problem. I want to read data from a file and pass the variable to a select query. I tried but it doesn't seem to work. Please advise. Example below. FileName='filekey.txt' while read LINE do var=$LINE print "For File key $var" ${ORACLE_HOME}/bin/sqlplus -s... (1 Reply)
Discussion started by: er_ashu
1 Replies
Login or Register to Ask a Question