Sponsored Content
Top Forums Shell Programming and Scripting Downloading hdfs file to local UNIX through UNIX script Post 302943804 by STCET22 on Wednesday 13th of May 2015 03:10:43 AM
Old 05-13-2015
Hi Don/Corona688 ,
Thanks a lot for your reply.As suggested by you ,after removing the carriage return from the script ,the script is working fine for me.As I am new to unix script level ,the script is not totally understandable to me.If you kindly explain the below block of code ,that will be really helpful for me.
Code:
#!/bin/bash
#Downloading HDFS file to Local Unix & Reformatting

N=0

DIR="$1" ; shift

while [ "$#" -gt 0 ]
do
    hdfs dfs -copyToLocal "$DIR"/"$1" .

    FILE="$1"

    awk '{printf "%06d,",NR} 1 ; END { printf "\n" }' "$FILE" >output${N}.txt
    let N=N+1
    shift
done

I am not sure why shift is being used in the above code .And if DIR variable holds the value of "$1" ,then why the FILE variable is again holding the value of "$1".Kindy bear with me and if you please explain the above code block ,that will be really beneficial for my understanding.Thanks !
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Downloading Unix..

I am not sure, yet i want to learn.. therefor .. I want to put some Unix on my machine.. Unix 03, nut i have heard its not free and not open source.. please advise me on what to do.. (4 Replies)
Discussion started by: binary_w0lf
4 Replies

2. Shell Programming and Scripting

How i ftp a unix file to my local window

Hello Sir/ Madam, i m new user in unix shell scripting.Please guide me to crack this problem. Thanking you. (1 Reply)
Discussion started by: Nirmal
1 Replies

3. UNIX for Dummies Questions & Answers

Unix command used for downloading file from net

Hi, Which command I can use to download a file from website? I tried with wget and lwp-dowload but no gain. Can any one suggest me the good way to dowload? (2 Replies)
Discussion started by: siba.s.nayak
2 Replies

4. Shell Programming and Scripting

how to FTP a file from the local folder to unix server

Hi All, please help me to write a shell that ftp a file which is in the local (C:\) drive to a Unix server. Where as i know the IP for the Unix server. i could do this process by using ftp command. pls help me to write as Shell script. Thanks in advance for all of your answers.:b::b: (3 Replies)
Discussion started by: little_wonder
3 Replies

5. UNIX for Dummies Questions & Answers

How to transfer file from Local PC to Unix Directory without FTP!!!

Dear Friends, How to transfer files from my local PC to Unix directory without using FTP. Scenario: Transfer/Upload a file from PC to unix using web browser without using FTP technologies. I heard something like sendunix and sendpc used to transfer files from unix to Desktop and... (1 Reply)
Discussion started by: kk_c2il2
1 Replies

6. Shell Programming and Scripting

How to transfer file from Local PC to UNIX Directory without FTP?

Dear All, i am trying to get the file from windows location to unix location without using FTP and neither thru entering the user id and password. I have one unix form which is running on web application and user is entering the location and file name there now i know the file name and path. So i... (8 Replies)
Discussion started by: ripudaman.singh
8 Replies

7. Shell Programming and Scripting

Downloading file from mainframe to UNIX

Hi All , I need a help regarding file ftp ing from mainframe to unix.Our source file is mainframe file.I need to download the mainframe file to local unix server through unix script.If anyone can help me how we can do it through unix script ,it will be really helpful.Thanks. (7 Replies)
Discussion started by: STCET22
7 Replies

8. Shell Programming and Scripting

Need help how to copy few records from hdfs to UNIX

Hi All , I am facing one issue here...I have a huge file in hadoop file system.Some disk space issues are thr ,thatswhy I want to copy 1st 100 records from hdfs to local unix.I tried below command but it is no working .Its giving error like cat: Unable to write to output stream.if any one can... (2 Replies)
Discussion started by: STCET22
2 Replies

9. UNIX for Advanced & Expert Users

UNIX and HDFS - file systems on same partition.

I am learning Hadoop. As a part of that, Hdfs - Hadoop distributed file system has commands similar to unix where we can create,copy,move files from unix/linux file system to HDFS. My question is 1) how two file systems (unix and hdfs) can coexist on thr same partition.. 2)What if block used... (0 Replies)
Discussion started by: Narendra Eliset
0 Replies

10. UNIX for Beginners Questions & Answers

UNIX and HDFS - file systems on same partition.

I am learning Hadoop. As a part of that, Hdfs - Hadoop distributed file system has commands similar to unix where we can create,copy,move files from unix/linux file system to HDFS. My question is 1) how two file systems (unix and hdfs) can coexist on thr same partition.. 2)What if block... (1 Reply)
Discussion started by: Narendra Eliset
1 Replies
script(1)						      General Commands Manual							 script(1)

NAME
script - Makes a transcript of terminal session SYNOPSIS
script [-a] [file] The script command makes a transcript of everything printed on your terminal. OPTIONS
Appends the transcript to file rather than writing it to file. OPERANDS
The name of an output file that will contain the transcript of the session. If this parameter is omitted, the file typescript is written. DESCRIPTION
The transcript is written to file, or appended to file if the -a option is given. If no file name is given, the transcript is saved in the file typescript. The script ends when the forked shell exits. This program is useful when you are using a CRT and want a hard-copy record of the dialog (for example, a technical writer might create an example of a working session this way). If you specify the -a option and the file does not exist, it is created. If you do not specify the -a option and the file exists, it is replaced. RESTRICTIONS
The script command requires a streams based terminal. In single user mode, streams may not be enabled. Under these circumstances, script will exit with no action. If you are the superuser and need to run this command in single user mode, use the following special instruc- tions. Enabling Streams If it is necessary to enable a streams environment in the single user mode, enter the command /sbin/init.d/streams. This command is avail- able to the superuser only. SEE ALSO
Commands: autopush(8), cat(1), echo(1), strsetup(8), tee(1) System Administration script(1)
All times are GMT -4. The time now is 11:54 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy