Sponsored Content
Top Forums Shell Programming and Scripting I am trying to merge all csv files from source path into 1 file Post 302974974 by cplusplus1 on Monday 6th of June 2016 08:55:53 PM
Old 06-06-2016
I am trying to merge all csv files from source path into 1 file

I am trying to merge all csv files from source path into one single csv file in target. but getting error message:
Code:
hadoop fs -cat /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/TCH-charge_*.csv > /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/final/TCH_pb_charge.csv

getting error message:
Code:
-bash no such file exists at this path: /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/final/TCH_pb_charge.csv

I have 11 csv files in the first path, now thinking the above command will automatically merge all 11 csv files into 1 csv file.

I tried to use this command also is this a proper merge command line with csv files into 1 single csv file.
Code:
hadoop fs -text /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/TCH-charge_*.csv | hadoop fs -put - /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/final/TCH_pb_charge.csv

Thanks a lot for the helpful info.
Moderator's Comments:
Mod Comment As has been said many times before:
Use CODE tags to display full-line and multi-line sample input, sample output, and code segments.
Do Not use ICODE tags to display full-line and multi-line sample input, sample output, and code segments.
Use ICODE tags to display partial-line sample input, sample output, and code segments as part of in-line text.

Last edited by Don Cragun; 06-07-2016 at 01:24 AM.. Reason: Change ICODE tags to CODE tags, add missing CODE tags.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Unzip and merge into .csv file

Hi all, I am having some 6 .zip files. inside this zip am having my .csv. now i wants to merge this 6 zip file into a single .csv. can anyone help me on this? Thanks in advance, warm regards, Raji. (1 Reply)
Discussion started by: rajinavaneethan
1 Replies

2. Shell Programming and Scripting

Merge 2 csv files with awk

I have 2 files pipe delimted and want to merge them based on a key e.g file 1 123$aaa$yyy$zzz 345$xab$yzy$zyz 456$sss$ttt$foo 799$aaa$ggg$dee file 2 123$hhh 345$ddd 456$xxx 888$zzz so if the key is the first field, and the result should be the common key between file 1 and 2 (6 Replies)
Discussion started by: loloAix
6 Replies

3. Shell Programming and Scripting

Merge 2 CSV files using sed

Help in writing a script using sed which updates fileOne with the contents from fileTwo Example: Contents of fileOne 1,111111 2,897823 3,235473 4,222222 Contents of fileTwo 1,111111,A,1,2 4,222222,A,2,2 5,374632,A,3,2 6,374654,A,4,2 Final File should be: 1,111111,A,1,2... (9 Replies)
Discussion started by: NewToSed
9 Replies

4. Shell Programming and Scripting

Merge CSV files and create a column with the filename from the original file

Hello everyone!! I am not completely new to shell script but I havent been able to find the answer to my problem and I'm sure there are some smart brains here up for the challenge :D. I have several CSV files that I need to combine into one, but I also need to know where each row came from.... (7 Replies)
Discussion started by: fransanchezoria
7 Replies

5. Linux

rpmbuild, how to specify a different source and destination path for files

I'd like to specify a different build and deployment path for files, by default the same path is used for both build and install, I wasn't able to find a way to make these different. With Solaris pkgadd, one can specify different paths in prototype, so I would assume something like that is possible... (0 Replies)
Discussion started by: tiburblium
0 Replies

6. Shell Programming and Scripting

Merge *.csv files, each in separate sheets

Does anyone know how to Merge *.csv files, each in seperate sheets? (7 Replies)
Discussion started by: frhling
7 Replies

7. Shell Programming and Scripting

Merge CSV files

I have lot of csv file collected from script like below : Name of files (some examples) there are thousands of it: 192.168.0.123_251_18796_1433144473.csv 192.168.0.123_251_18796_1433144772.csv 192.168.0.123_251_18796_1433145073.csv 192.168.0.123_251_18796_1433145372.csvContent of each... (5 Replies)
Discussion started by: rk4k
5 Replies

8. UNIX for Dummies Questions & Answers

Merge two csv files using column name

Hi all, I have two separate csv files(comma delimited) file 1 and file 2. File 1 contains PAN,NAME,Salary AAAAA5467D,Raj,50000 AAFAC5467D,Ram,60000 BDCFA5677D,Kumar,90000 File 2 contains PAN,NAME,Dept,Salary ASDFG6756T,Karthik,ABC,450000 QWERT8765Y,JAX,CDR,780000... (5 Replies)
Discussion started by: Nivas
5 Replies

9. Shell Programming and Scripting

Compare and merge two big CSV files

Hi all, i need help. I have two csv files with a huge amount of data. I need the first column of the first file, to be compared with the data of the second, to have at the end a file with the data not present in the second file. Example File1: (only one column) profile_id 57036226... (11 Replies)
Discussion started by: SirMannu
11 Replies

10. UNIX for Beginners Questions & Answers

Merge the three csv files as one according to first coloumn.

I have three files with similar pattern i need to merge all the coloumns side by side from all three files according to the first coloumn example as shown below I mentioned 5 coloumns only in example but i have around 15 coloumns in each file. file1: Name,Samples,Error,95RT,90RT... (4 Replies)
Discussion started by: Raghuram717
4 Replies
CSV2REC(1)							   User Commands							CSV2REC(1)

NAME
csv2rec - csv to rec converter SYNOPSIS
csv2rec [OPTIONS]... [CSV_FILE] DESCRIPTION
Convert csv data into rec data. -t, --type=TYPE type name for the converted records; if this parameter is ommited then no type is used. -s, --strict be strict parsing the csv file. -e, --omit-empty omit empty fields. --help print a help message and exit. --version show version and exit. AUTHOR
Written by Jose E. Marchesi. REPORTING BUGS
Report bugs to: bug-recutils@gnu.org GNU recutils home page: <http://www.gnu.org/software/recutils/> General help using GNU software: <http://www.gnu.org/gethelp/> COPYRIGHT
Copyright (C) 2010, 2011, 2012 Jose E. Marchesi. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. SEE ALSO
The full documentation for csv2rec is maintained as a Texinfo manual. If the info and csv2rec programs are properly installed at your site, the command info csv2rec should give you access to the complete manual. csv2rec 1.4.93 January 2012 CSV2REC(1)
All times are GMT -4. The time now is 05:57 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy