Sponsored Content
Top Forums Shell Programming and Scripting Downloading hdfs file to local UNIX through UNIX script Post 302942006 by Corona688 on Thursday 23rd of April 2015 12:38:32 PM
Old 04-23-2015
Code:
case "$#" in
2)
        hdfs dfs -copyToLocal "$1"/"$2" .
        FILE="$2"
        ;;
1)
        hdfs dfs -copyToLocal "$1" .
        OLDIFS="$IFS"
        # Split $1="a/b/filename" into $1="a", $2="b", $3="filename"
        IFS="/"
                set -- $1
        IFS="$OLDIFS"

        # Get rid of "a", "b"
        shift "$(( $# - 1 ))

        FILE="$1"
        ;;
*)
        echo "Usage:  $0 path file"
        echo "Alternate usage:  $0 path/file"
        exit 1
        ;;
esac

awk '{printf "%06d|",NR} 1' "$FILE" >output.txt

This User Gave Thanks to Corona688 For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Downloading Unix..

I am not sure, yet i want to learn.. therefor .. I want to put some Unix on my machine.. Unix 03, nut i have heard its not free and not open source.. please advise me on what to do.. (4 Replies)
Discussion started by: binary_w0lf
4 Replies

2. Shell Programming and Scripting

How i ftp a unix file to my local window

Hello Sir/ Madam, i m new user in unix shell scripting.Please guide me to crack this problem. Thanking you. (1 Reply)
Discussion started by: Nirmal
1 Replies

3. UNIX for Dummies Questions & Answers

Unix command used for downloading file from net

Hi, Which command I can use to download a file from website? I tried with wget and lwp-dowload but no gain. Can any one suggest me the good way to dowload? (2 Replies)
Discussion started by: siba.s.nayak
2 Replies

4. Shell Programming and Scripting

how to FTP a file from the local folder to unix server

Hi All, please help me to write a shell that ftp a file which is in the local (C:\) drive to a Unix server. Where as i know the IP for the Unix server. i could do this process by using ftp command. pls help me to write as Shell script. Thanks in advance for all of your answers.:b::b: (3 Replies)
Discussion started by: little_wonder
3 Replies

5. UNIX for Dummies Questions & Answers

How to transfer file from Local PC to Unix Directory without FTP!!!

Dear Friends, How to transfer files from my local PC to Unix directory without using FTP. Scenario: Transfer/Upload a file from PC to unix using web browser without using FTP technologies. I heard something like sendunix and sendpc used to transfer files from unix to Desktop and... (1 Reply)
Discussion started by: kk_c2il2
1 Replies

6. Shell Programming and Scripting

How to transfer file from Local PC to UNIX Directory without FTP?

Dear All, i am trying to get the file from windows location to unix location without using FTP and neither thru entering the user id and password. I have one unix form which is running on web application and user is entering the location and file name there now i know the file name and path. So i... (8 Replies)
Discussion started by: ripudaman.singh
8 Replies

7. Shell Programming and Scripting

Downloading file from mainframe to UNIX

Hi All , I need a help regarding file ftp ing from mainframe to unix.Our source file is mainframe file.I need to download the mainframe file to local unix server through unix script.If anyone can help me how we can do it through unix script ,it will be really helpful.Thanks. (7 Replies)
Discussion started by: STCET22
7 Replies

8. Shell Programming and Scripting

Need help how to copy few records from hdfs to UNIX

Hi All , I am facing one issue here...I have a huge file in hadoop file system.Some disk space issues are thr ,thatswhy I want to copy 1st 100 records from hdfs to local unix.I tried below command but it is no working .Its giving error like cat: Unable to write to output stream.if any one can... (2 Replies)
Discussion started by: STCET22
2 Replies

9. UNIX for Advanced & Expert Users

UNIX and HDFS - file systems on same partition.

I am learning Hadoop. As a part of that, Hdfs - Hadoop distributed file system has commands similar to unix where we can create,copy,move files from unix/linux file system to HDFS. My question is 1) how two file systems (unix and hdfs) can coexist on thr same partition.. 2)What if block used... (0 Replies)
Discussion started by: Narendra Eliset
0 Replies

10. UNIX for Beginners Questions & Answers

UNIX and HDFS - file systems on same partition.

I am learning Hadoop. As a part of that, Hdfs - Hadoop distributed file system has commands similar to unix where we can create,copy,move files from unix/linux file system to HDFS. My question is 1) how two file systems (unix and hdfs) can coexist on thr same partition.. 2)What if block... (1 Reply)
Discussion started by: Narendra Eliset
1 Replies
FCGIWRAP(8)						      System Manager's Manual						       FCGIWRAP(8)

NAME
fcgiwrap - serve CGI applications over FastCGI SYNOPSIS
fcgiwrap [OPTIONS] DESCRIPTION
fcgiwrap is a simple server for running CGI applications over FastCGI. It hopes to provide clean CGI support to Nginx (and other web servers that may need it). OPTIONS
-c number Number of fcgiwrap processes to prefork. -s socket_url A URL for the listen socket to bind to. By default fcgiwrap expects a listen socket to be passed on file descriptor 0, matching the FastCGI convention. The recommended way to deploy fcgiwrap is to run it under a process manager that takes care of opening the socket. However, for simple configurations and one-off tests this option may be used. Valid socket_urls include unix:/path/to/unix/socket, tcp:dot.ted.qu.ad:port and tcp6:[ipv6_addr]:port. -h Show a help message and exit. ENVIRONMENT
When running, fcgiwrap evaluates these environment variables set by the web server calling an fcgi-script. The variables DOCUMENT_ROOT and SCRIPT_NAME will be concatenated and the resulting executable run as CGI script wrapped as FastCGI, with the remainder after the script name available as PATH_INFO. To disable PATH_INFO mangling, set up your web server to pass SCRIPT_FILENAME, which should contain the com- plete path to the script. Then PATH_INFO will not be modified. DOCUMENT_ROOT directory which the script resides in SCRIPT_NAME actual executable SCRIPT_FILENAME complete path to CGI script. When set, overrides DOCUMENT_ROOT and SCRIPT_NAME EXAMPLE
The fastest way to see fcgiwrap do something is to launch it at the command line like this: fcgiwrap -s unix:/var/run/fcgiwrap.sock Apart from potential permission problems etc., it should be ready to accept FastCGI requests and run CGI scripts. Most probably you will want to launch fcgiwrap by spawn-fcgi using a configuration like this: FCGI_SOCKET=/var/run/fcgiwrap.sock FCGI_PROGRAM=/usr/sbin/fcgiwrap FCGI_USER=nginx FCGI_GROUP=www FCGI_EXTRA_OPTIONS="-M 0700" ALLOWED_ENV="PATH" Nginx can be configured to have the arbitrary CGI cgit run as FastCGI as follows: location / { fastcgi_param DOCUMENT_ROOT /var/www/localhost/htdocs/cgit/; fastcgi_param SCRIPT_NAME cgit; fastcgi_pass unix:/var/run/fastcgi.sock; } AUTHOR
fcgiwrap was written by Grzegorz Nosek <root@localdomain.pl> with contributions by W-Mark Kubacki <wmark@hurrikane.de>. This manual page was written by Jordi Mallach <jordi@debian.org> (with contributions by Grzegorz Nosek) for the Debian project (and may be used by others). Jun 3, 2010 FCGIWRAP(8)
All times are GMT -4. The time now is 07:18 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy