Sponsored Content
Top Forums Shell Programming and Scripting Writing hive scripts in bash script file Post 302899176 by shree11 on Monday 28th of April 2014 01:19:50 AM
Old 04-28-2014
Writing hive scripts in bash script file

Hi,

I wanted to load data from HDFS to HIVE by writing bash script.
Description:
I have written a bash script to validate the data and loaded validated data from local file system to HDFS. Now in the same bash script i wanted to load the data from HDFS to HIVE. How can i do it ? Also how tyhe hive shell is called when i excecute the bash script (.sh file)?
Below is the code i have written to load the data form local to hdfs.
Code:
#!/bin/bash
<code to validate the file on local>
hadoop fs -put /home/hduser/validate/badrec /user/hduser/Dataparse/
hadoop fs -put /home/hduser/validate/goodrec /user/hduser/Dataparse/

No i wanted to add the below hive create table statement and load statement in the same bash script file
Code:
create table GoodRecords(
id int,
name string,
state string,
phone_no int,
gender string) row format delimited fields terminated by ',' stored as textfile;

LOAD DATA INPATH '/user/hduser/Dataparse/goodrec' INTO TABLE GoodRecords;

Please suggest me how this can be done ?

Thanks,
Shree
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Problems writing bash script to unzip files

I'm getting the following errors when I try to write a script to unzip some zip files. When I use the free trial copy of the commerical winzip program, however, they work fine. When I use -l or -t on unzip it indicates no errors. When I use the -o switch interactively from the bash command line it... (1 Reply)
Discussion started by: siegfried
1 Replies

2. Shell Programming and Scripting

Writing Bash script

Could anyone help me to Write a script in BASH Shell to determine the percentage of system disk space you are using. (1 Reply)
Discussion started by: boris
1 Replies

3. Shell Programming and Scripting

Writing a file to RAM within Bash and using it

I've got a few scripts I use for various things, but there is one I have taking the output from a specific command, writing it to disk, and repeating for another command, then reading both (after some formatting) and writing the output to another file. Part of the reasoning for writing to the... (7 Replies)
Discussion started by: forgottenwizard
7 Replies

4. Shell Programming and Scripting

Help with writing simple bash script

I want to write a bash script to: 1. Send an email from localhost to an external gmail account. (gmail then automatically forwards the message back to a pop account on the same server. 2. Script waits 3 minutes then checks to see if the email arrived, and if not, it sends an email to... (9 Replies)
Discussion started by: sallyanne
9 Replies

5. Shell Programming and Scripting

Writing Bash shell scripts corresponding to windows bat files

Experts, I am newbie in shell scripting. I want to write Bash shell scripts corresponding to windows bat files. I have installed cygwin at c:\cygwin and i am trying to crate the sh file using vi editor. i am not able to understand how to use linux/unix convention for the code. following is my... (15 Replies)
Discussion started by: rajuchacha007
15 Replies

6. Homework & Coursework Questions

brand new user!.. Lost on BASH script writing

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: I have just gotten into writing bash scripts for a class, part of the assignment is to read and be able to tell... (4 Replies)
Discussion started by: Byrang
4 Replies

7. Shell Programming and Scripting

Writing a bash script using host

Im trying to write a script using the host command but its not working properly. I cant understand what Im doing wrong. When I use it at the command prompt, it works fine. But its being used actually in the script, it says its not found: 2 SERVFAIL. Can anyone help me? Here's what I have so far: no... (6 Replies)
Discussion started by: relsha
6 Replies

8. Shell Programming and Scripting

Mkbootfs writing to stdout in bash script

Hi, I need to automate some repacking tasks of a boot image for Android When in command line, I can use this command: mkbootfs /path/to/root > /path/to/ramdisk-recovery.cpio;However, if I try to run the command from a shell script under Ubuntu, it fails and outputs to stdout instead of the... (27 Replies)
Discussion started by: Phil3759
27 Replies

9. Shell Programming and Scripting

Question about writing a bash script

Hello, I want to write a bash script to delete the content after '#'. However, if '#' appears in a string with "", ignore this. For example, input file: test #delete "test #not delete" Output file: test "test #not delete" Does anyone know how to write this script? Thanks (1 Reply)
Discussion started by: jeffwang66
1 Replies

10. Shell Programming and Scripting

Writing Hbase and pig scripts in the bash script file

Hi, I have a script file where i'm validatig the input file and storing the validated records on HDFS. I wanted to load data from HDFS to HBASE using pig script. So for that i have created a HBASE table and written pig script to load data from HDFS to HBASE which is working fine. Now i wanted... (0 Replies)
Discussion started by: shree11
0 Replies
SYSPROFILE(8)						      System Manager's Manual						     SYSPROFILE(8)

NAME
sysprofile - modular centralized shell configuration DESCRIPTION
sysprofile is a generic approach to configure shell settings in a modular and centralized way mostly aimed at avoiding work for lazy sysad- mins. It has only been tested to work with the bash shell. It basically consists of the small /etc/sysprofile shell script which invokes other small shell scripts having a .bash suffix which are contained in the /etc/sysprofile.d/ directory. The system administrator can drop in any script he wants without any naming convention other than that the scripts need to have a .bash suffix to enable automagic sourcing by /etc/sysprofile. This mechanism is set up by inserting a small shell routine into /etc/profile for login shells and optionally into /etc/bashrc and/or /etc/bash.bashrc for non-login shells from where the actual /etc/sysprofile script is invoked: if [ -f /etc/sysprofile ]; then . /etc/sysprofile fi For using "sysprofile" under X11, one can source it in a similar way from /etc/X11/Xsession or your X display manager's Xsession file to provide the same shell environment as under the console in X11. See the example files in /usr/share/doc/sysprofile/ for illustration. For usage of terminal emulators with a non-login bash shell under X11, take care to enable sysprofile via /etc/bash.bashrc. If not set this way, your terminal emulators won't come up with the environment defined by the scripts in /etc/sysprofile.d/. Users not wanting /etc/sysprofile to be sourced for their environment can easily disable it's automatic mechanism. It can be disabled by simply creating an empty file called $HOME/.nosysprofile in the user's home directory using e.g. the touch(1) command. Any single configuration file in /etc/sysprofile.d/ can be overridden by any user by creating a private $HOME/.sysprofile.d/ directory which may contain a user's own version of any configuration file to be sourced instead of the system default. It's names have just to match exactly the system's default /etc/sysprofile.d/ configuration files. Empty versions of these files contained in the $HOME/.syspro- file.d/ directory automatically disable sourcing of the system wide version. Naturally, users can add and include their own private script inventions to be automagically executed by /etc/sysprofile at login time. OPTIONS
There are no options other than those dictated by shell conventions. Anything is defined within the configuration scripts themselves. SEE ALSO
The README files and configuration examples contained in /etc/sysprofile.d/ and the manual pages bash(1), xdm(1x), xdm.options(5), and wdm(1x). Recommended further reading is everything related with shell programming. If you need a similar mechanism for executing code at logout time check out the related package syslogout(8) which is a very close compan- ion to sysprofile. BUGS
sysprofile in its current form is mainly restricted to bash(1) syntax. In fact it is actually a rather embarrassing quick and dirty hack than anything else - but it works. It serves the practical need to enable a centralized bash configuration until something better becomes available. Your constructive criticism in making this into something better" is very welcome. Before i forget to mention it: we take patches... ;-) AUTHOR
sysprofile was developed by Paul Seelig <pseelig@debian.org> specifically for the Debian GNU/Linux system. Feel free to port it to and use it anywhere else under the conditions of either the GNU public license or the BSD license or both. Better yet, please help to make it into something more worthwhile than it currently is. SYSPROFILE(8)
All times are GMT -4. The time now is 03:26 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy