Sponsored Content
Top Forums Shell Programming and Scripting I am trying to merge all csv files from source path into 1 file Post 302974974 by cplusplus1 on Monday 6th of June 2016 08:55:53 PM
Old 06-06-2016
I am trying to merge all csv files from source path into 1 file

I am trying to merge all csv files from source path into one single csv file in target. but getting error message:
Code:
hadoop fs -cat /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/TCH-charge_*.csv > /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/final/TCH_pb_charge.csv

getting error message:
Code:
-bash no such file exists at this path: /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/final/TCH_pb_charge.csv

I have 11 csv files in the first path, now thinking the above command will automatically merge all 11 csv files into 1 csv file.

I tried to use this command also is this a proper merge command line with csv files into 1 single csv file.
Code:
hadoop fs -text /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/TCH-charge_*.csv | hadoop fs -put - /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/final/TCH_pb_charge.csv

Thanks a lot for the helpful info.
Moderator's Comments:
Mod Comment As has been said many times before:
Use CODE tags to display full-line and multi-line sample input, sample output, and code segments.
Do Not use ICODE tags to display full-line and multi-line sample input, sample output, and code segments.
Use ICODE tags to display partial-line sample input, sample output, and code segments as part of in-line text.

Last edited by Don Cragun; 06-07-2016 at 01:24 AM.. Reason: Change ICODE tags to CODE tags, add missing CODE tags.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Unzip and merge into .csv file

Hi all, I am having some 6 .zip files. inside this zip am having my .csv. now i wants to merge this 6 zip file into a single .csv. can anyone help me on this? Thanks in advance, warm regards, Raji. (1 Reply)
Discussion started by: rajinavaneethan
1 Replies

2. Shell Programming and Scripting

Merge 2 csv files with awk

I have 2 files pipe delimted and want to merge them based on a key e.g file 1 123$aaa$yyy$zzz 345$xab$yzy$zyz 456$sss$ttt$foo 799$aaa$ggg$dee file 2 123$hhh 345$ddd 456$xxx 888$zzz so if the key is the first field, and the result should be the common key between file 1 and 2 (6 Replies)
Discussion started by: loloAix
6 Replies

3. Shell Programming and Scripting

Merge 2 CSV files using sed

Help in writing a script using sed which updates fileOne with the contents from fileTwo Example: Contents of fileOne 1,111111 2,897823 3,235473 4,222222 Contents of fileTwo 1,111111,A,1,2 4,222222,A,2,2 5,374632,A,3,2 6,374654,A,4,2 Final File should be: 1,111111,A,1,2... (9 Replies)
Discussion started by: NewToSed
9 Replies

4. Shell Programming and Scripting

Merge CSV files and create a column with the filename from the original file

Hello everyone!! I am not completely new to shell script but I havent been able to find the answer to my problem and I'm sure there are some smart brains here up for the challenge :D. I have several CSV files that I need to combine into one, but I also need to know where each row came from.... (7 Replies)
Discussion started by: fransanchezoria
7 Replies

5. Linux

rpmbuild, how to specify a different source and destination path for files

I'd like to specify a different build and deployment path for files, by default the same path is used for both build and install, I wasn't able to find a way to make these different. With Solaris pkgadd, one can specify different paths in prototype, so I would assume something like that is possible... (0 Replies)
Discussion started by: tiburblium
0 Replies

6. Shell Programming and Scripting

Merge *.csv files, each in separate sheets

Does anyone know how to Merge *.csv files, each in seperate sheets? (7 Replies)
Discussion started by: frhling
7 Replies

7. Shell Programming and Scripting

Merge CSV files

I have lot of csv file collected from script like below : Name of files (some examples) there are thousands of it: 192.168.0.123_251_18796_1433144473.csv 192.168.0.123_251_18796_1433144772.csv 192.168.0.123_251_18796_1433145073.csv 192.168.0.123_251_18796_1433145372.csvContent of each... (5 Replies)
Discussion started by: rk4k
5 Replies

8. UNIX for Dummies Questions & Answers

Merge two csv files using column name

Hi all, I have two separate csv files(comma delimited) file 1 and file 2. File 1 contains PAN,NAME,Salary AAAAA5467D,Raj,50000 AAFAC5467D,Ram,60000 BDCFA5677D,Kumar,90000 File 2 contains PAN,NAME,Dept,Salary ASDFG6756T,Karthik,ABC,450000 QWERT8765Y,JAX,CDR,780000... (5 Replies)
Discussion started by: Nivas
5 Replies

9. Shell Programming and Scripting

Compare and merge two big CSV files

Hi all, i need help. I have two csv files with a huge amount of data. I need the first column of the first file, to be compared with the data of the second, to have at the end a file with the data not present in the second file. Example File1: (only one column) profile_id 57036226... (11 Replies)
Discussion started by: SirMannu
11 Replies

10. UNIX for Beginners Questions & Answers

Merge the three csv files as one according to first coloumn.

I have three files with similar pattern i need to merge all the coloumns side by side from all three files according to the first coloumn example as shown below I mentioned 5 coloumns only in example but i have around 15 coloumns in each file. file1: Name,Samples,Error,95RT,90RT... (4 Replies)
Discussion started by: Raghuram717
4 Replies
OWX(1)							      General Commands Manual							    OWX(1)

NAME
owx - utility to program Wouxun dual-band handheld radios SYNOPSIS
owx-check [ -v | -h ] [ -f ] [ -p <port> ] [ -t <timeout> ] owx-get [ -v | -h ] [ -f ] [ -p <port> ] [ -t <timeout> ] -o <path> owx-put [ -v | -h ] [ -f ] [ -p <port> ] [ -t <timeout> ] -i <path> -r <path> owx-export [ -v | -h ] -i <bin path> -o <csv path> owx-import [ -v | -h ] -i <csv path> -o <bin path> DESCRIPTION
This manual page documents briefly the owx commands. owx is a set of commands that will allow you to fetch and update configuration on your Wouxun handheld radio. It is made of multiple com- mands. COMMON OPTIONS
These options are common to all commands. -h Show summary of options. -v Show version of program. -c <command> Invoke owx-<command>. Makes sense only if called directly as owx. OPTIONS FOR check, get AND put -f Force operation even if your radio identifies different from a KG669V. Use this option with extreme caution - it is very possible that your radio will be rendered unusable after you use this. It was NEVER tested with any radio different from mentoined above. -p <port> Use port <port>, defaults to /dev/ttyUSB0. Of course you must have appropriate read and write permissions for this device. -t <timeout> Specify the receive timeout for communication with radio. If you disable it (by setting to 0) and the communication fails, the pro- gram will hang forever. You probably don't need to change the default value (5 seconds). USAGE
owx-check This program just checks for the connection and identification string. It can be used to check that your cable and port works. owx-get This program downloads memory map from radio to binary file. Options: -o <path>: binary file to write to owx-put This program uploads memory map from binary file to radio. Options: -i <path>: binary file to read from -r <path>: reference file Option -r is not mandatory, but recommended. You can specify original, unchanged file (exactly as downloaded using owx-get) and this will speed up memory uploading, as owx will compare input file to this reference file and upload only changed memory pages. When using this option, be sure that nothing has changed in the radio (even the currently selected memory channel) between downloading reference file and using it for upload. This is important as some variables that cross the page boundaries (if there are any in the memory map) could be corrupted by this. Example: owx-get -o file.bin cp file.bin backup.bin owx-export -i file.bin -o wouxun.csv oocalc wouxun.csv owx-import -i wouxun.csv -o file.bin owx-put -i file.bin -r backup.bin Please do yourself a favour and double-check that you upload the correct file. If you try to upload incorrect or corrupted file, your radio will power down and fail to power up. owx will refuse to upload any file with incorrect size, but this is the only safety check. owx-export This program exports channel data from binary file to CSV file. This file can be later edited using your favourite spreadsheet edi- tor or even text editor. Options: -i <path>: binary file to read from -o <path>: csv file to write to owx-import This program reads the specified, possibly edited by you CSV file, and patches existing binary file with this updated data. The file is now prepared to be uploaded with owx-put. Options: -i <path>: csv file to read from -o <path>: binary file to write to (must already exist) SEE ALSO
chirp(1), the README file. AUTHOR
owx was written by SP5GOF (Adam Wysocki). This manual page was written by Antoine Beaupre <anarcat@debian.org>, for the Debian project (and may be used by others). October 26, 2011 OWX(1)
All times are GMT -4. The time now is 04:08 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy