Downloading hdfs file to local UNIX through UNIX script
Hi All ,
I am very new to unix script.I am aware of unix commands but never put together in unix script level.If any one can suggest me technical guidance in the below scenario that will highly beneficial.
Data have been already migrated from mainframe to Hadoop file system(HDFS).HDFS server is hosted on a unix box.HDFS file is just .txt file.We are currently downloading HDFS to local unix file system by HDFS command and then adding sequence number to the local unix file.We just want to automate this process for any file.
Currently we are using below steps :
Downlading HDFS file to local unix system by below command :
Adding sequnce generated No :
We need to automate the above process through unix script for any user provided input file .If anyone can provide me technical approach/block of code to automate the above scenario that will be real help.Thanks in advance !
---------- Post updated at 08:24 AM ---------- Previous update was at 03:52 AM ----------
Hi All ,
I have written the below script which is working for a single file.I have hardcoded the hdfs path in the script itself.And the script is creating pipe dilimited sequence generated o/p file in the local directory.We just need to modify this script for any user provided input file.Can anyone suggest me how to automate this script for any user provided input file which should not be hardcoded in the script.Any help in this regard will be highly appreciated.Thanks.
Last edited by rbatte1; 04-30-2015 at 01:33 PM..
Reason: Added CODE and LIST=1 tags
I am not sure, yet i want to learn.. therefor ..
I want to put some Unix on my machine.. Unix 03, nut i have heard its not free and not open source..
please advise me on what to do.. (4 Replies)
Hi,
Which command I can use to download a file from website?
I tried with wget and lwp-dowload but no gain.
Can any one suggest me the good way to dowload? (2 Replies)
Hi All,
please help me to write a shell that ftp a file which is in the local (C:\) drive to a Unix server.
Where as i know the IP for the Unix server.
i could do this process by using ftp command.
pls help me to write as Shell script.
Thanks in advance for all of your answers.:b::b: (3 Replies)
Dear Friends,
How to transfer files from my local PC to Unix directory without using FTP.
Scenario: Transfer/Upload a file from PC to unix using web browser without using FTP technologies. I heard something like sendunix and sendpc used to transfer files from unix to Desktop and... (1 Reply)
Dear All,
i am trying to get the file from windows location to unix location without using FTP and neither thru entering the user id and password. I have one unix form which is running on web application and user is entering the location and file name there now i know the file name and path. So i... (8 Replies)
Hi All ,
I need a help regarding file ftp ing from mainframe to unix.Our source file is mainframe file.I need to download the mainframe file to local unix server through unix script.If anyone can help me how we can do it through unix script ,it will be really helpful.Thanks. (7 Replies)
Hi All ,
I am facing one issue here...I have a huge file in hadoop file system.Some disk space issues are thr ,thatswhy I want to copy 1st 100 records from hdfs to local unix.I tried below command but it is no working .Its giving error like cat: Unable to write to output stream.if any one can... (2 Replies)
I am learning Hadoop. As a part of that, Hdfs - Hadoop distributed file system has commands similar to unix where we can create,copy,move files from unix/linux file system to HDFS.
My question is
1) how two file systems (unix and hdfs) can coexist on thr same partition..
2)What if block used... (0 Replies)
I am learning Hadoop. As a part of that, Hdfs - Hadoop distributed file system has commands similar to unix where we can create,copy,move files from unix/linux file system to HDFS.
My question is
1) how two file systems (unix and hdfs) can coexist on thr same partition..
2)What if block... (1 Reply)
Discussion started by: Narendra Eliset
1 Replies
LEARN ABOUT SUSE
sieveshell
SIEVESHELL(1) User Contributed Perl Documentation SIEVESHELL(1)NAME
sieveshell - remotely manipulate sieve scripts
SYNOPSIS
sieveshell [--user=user] [--authname=authname] [--realm=realm] [--exec=script] server[:port]
sieveshell --help
DESCRIPTION
sieveshell allows users to manipulate their scripts on a remote server. It works via MANAGESIEVE, a work in progress.
The following commands are recognized:
list list scripts on server.
put <filename> upload script to server.
get <name> [<filename>] get script. if no filename display to stdout
delete <name> delete script.
activate <name> activate script.
deactivate deactivate all scripts.
OPTIONS -u user, --user=user
The authorization name to request; by default, derived from the authentication credentials.
-a authname, --authname=authname
The user to use for authentication (defaults to current user).
-r realm, --realm=realm
The realm to attempt authentication in.
-e script, --exec=script
Instead of working interactively, run commands from script, and exit when done.
REFERENCES
[MANAGESIEVE] Martin, T.; "A Protocol for Remotely Managing Sieve Scripts", draft-ietf-managesieve-03.txt, Mirapoint, Inc.; May 2001, work
in progress.
AUTHOR
Tim Martin <tmartin@mirapoint.com>, and the rest of the Cyrus team <cyrus-bugs@andrew.cmu.edu>.
perl v5.10.0 2008-04-04 SIEVESHELL(1)