Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

I am trying to merge all csv files from source path into 1 file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting I am trying to merge all csv files from source path into 1 file
# 1  
Old 06-06-2016
I am trying to merge all csv files from source path into 1 file
I am trying to merge all csv files from source path into one single csv file in target. but getting error message:
Code:
hadoop fs -cat /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/TCH-charge_*.csv > /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/final/TCH_pb_charge.csv

getting error message:
Code:
-bash no such file exists at this path: /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/final/TCH_pb_charge.csv

I have 11 csv files in the first path, now thinking the above command will automatically merge all 11 csv files into 1 csv file.

I tried to use this command also is this a proper merge command line with csv files into 1 single csv file.
Code:
hadoop fs -text /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/TCH-charge_*.csv | hadoop fs -put - /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/final/TCH_pb_charge.csv

Thanks a lot for the helpful info.
Moderator's Comments:
Mod Comment As has been said many times before:
Use CODE tags to display full-line and multi-line sample input, sample output, and code segments.
Do Not use ICODE tags to display full-line and multi-line sample input, sample output, and code segments.
Use ICODE tags to display partial-line sample input, sample output, and code segments as part of in-line text.
Last edited by Don Cragun; 06-07-2016 at 01:24 AM.. Reason: Change ICODE tags to CODE tags, add missing CODE tags.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Merge the three csv files as one according to first coloumn.

I have three files with similar pattern i need to merge all the coloumns side by side from all three files according to the first coloumn example as shown below I mentioned 5 coloumns only in example but i have around 15 coloumns in each file. file1: Name,Samples,Error,95RT,90RT... (4 Replies)
Discussion started by: Raghuram717
4 Replies

2. Shell Programming and Scripting

Compare and merge two big CSV files

Hi all, i need help. I have two csv files with a huge amount of data. I need the first column of the first file, to be compared with the data of the second, to have at the end a file with the data not present in the second file. Example File1: (only one column) profile_id 57036226... (11 Replies)
Discussion started by: SirMannu
11 Replies

3. UNIX for Dummies Questions & Answers

Merge two csv files using column name

Hi all, I have two separate csv files(comma delimited) file 1 and file 2. File 1 contains PAN,NAME,Salary AAAAA5467D,Raj,50000 AAFAC5467D,Ram,60000 BDCFA5677D,Kumar,90000 File 2 contains PAN,NAME,Dept,Salary ASDFG6756T,Karthik,ABC,450000 QWERT8765Y,JAX,CDR,780000... (5 Replies)
Discussion started by: Nivas
5 Replies

4. Shell Programming and Scripting

Merge CSV files

I have lot of csv file collected from script like below : Name of files (some examples) there are thousands of it: 192.168.0.123_251_18796_1433144473.csv 192.168.0.123_251_18796_1433144772.csv 192.168.0.123_251_18796_1433145073.csv 192.168.0.123_251_18796_1433145372.csvContent of each... (5 Replies)
Discussion started by: rk4k
5 Replies

5. Shell Programming and Scripting

Merge *.csv files, each in separate sheets

Does anyone know how to Merge *.csv files, each in seperate sheets? (7 Replies)
Discussion started by: frhling
7 Replies

6. Linux

rpmbuild, how to specify a different source and destination path for files

I'd like to specify a different build and deployment path for files, by default the same path is used for both build and install, I wasn't able to find a way to make these different. With Solaris pkgadd, one can specify different paths in prototype, so I would assume something like that is possible... (0 Replies)
Discussion started by: tiburblium
0 Replies

7. Shell Programming and Scripting

Merge CSV files and create a column with the filename from the original file

Hello everyone!! I am not completely new to shell script but I havent been able to find the answer to my problem and I'm sure there are some smart brains here up for the challenge :D. I have several CSV files that I need to combine into one, but I also need to know where each row came from.... (7 Replies)
Discussion started by: fransanchezoria
7 Replies

8. Shell Programming and Scripting

Merge 2 CSV files using sed

Help in writing a script using sed which updates fileOne with the contents from fileTwo Example: Contents of fileOne 1,111111 2,897823 3,235473 4,222222 Contents of fileTwo 1,111111,A,1,2 4,222222,A,2,2 5,374632,A,3,2 6,374654,A,4,2 Final File should be: 1,111111,A,1,2... (9 Replies)
Discussion started by: NewToSed
9 Replies

9. Shell Programming and Scripting

Merge 2 csv files with awk

I have 2 files pipe delimted and want to merge them based on a key e.g file 1 123$aaa$yyy$zzz 345$xab$yzy$zyz 456$sss$ttt$foo 799$aaa$ggg$dee file 2 123$hhh 345$ddd 456$xxx 888$zzz so if the key is the first field, and the result should be the common key between file 1 and 2 (6 Replies)
Discussion started by: loloAix
6 Replies

10. Shell Programming and Scripting

Unzip and merge into .csv file

Hi all, I am having some 6 .zip files. inside this zip am having my .csv. now i wants to merge this 6 zip file into a single .csv. can anyone help me on this? Thanks in advance, warm regards, Raji. (1 Reply)
Discussion started by: rajinavaneethan
1 Replies
Login or Register to Ask a Question