Sponsored Content
Top Forums Shell Programming and Scripting I am trying to merge all csv files from source path into 1 file Post 302974974 by cplusplus1 on Monday 6th of June 2016 08:55:53 PM
Old 06-06-2016
I am trying to merge all csv files from source path into 1 file

I am trying to merge all csv files from source path into one single csv file in target. but getting error message:
Code:
hadoop fs -cat /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/TCH-charge_*.csv > /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/final/TCH_pb_charge.csv

getting error message:
Code:
-bash no such file exists at this path: /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/final/TCH_pb_charge.csv

I have 11 csv files in the first path, now thinking the above command will automatically merge all 11 csv files into 1 csv file.

I tried to use this command also is this a proper merge command line with csv files into 1 single csv file.
Code:
hadoop fs -text /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/TCH-charge_*.csv | hadoop fs -put - /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/final/TCH_pb_charge.csv

Thanks a lot for the helpful info.
Moderator's Comments:
Mod Comment As has been said many times before:
Use CODE tags to display full-line and multi-line sample input, sample output, and code segments.
Do Not use ICODE tags to display full-line and multi-line sample input, sample output, and code segments.
Use ICODE tags to display partial-line sample input, sample output, and code segments as part of in-line text.

Last edited by Don Cragun; 06-07-2016 at 01:24 AM.. Reason: Change ICODE tags to CODE tags, add missing CODE tags.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Unzip and merge into .csv file

Hi all, I am having some 6 .zip files. inside this zip am having my .csv. now i wants to merge this 6 zip file into a single .csv. can anyone help me on this? Thanks in advance, warm regards, Raji. (1 Reply)
Discussion started by: rajinavaneethan
1 Replies

2. Shell Programming and Scripting

Merge 2 csv files with awk

I have 2 files pipe delimted and want to merge them based on a key e.g file 1 123$aaa$yyy$zzz 345$xab$yzy$zyz 456$sss$ttt$foo 799$aaa$ggg$dee file 2 123$hhh 345$ddd 456$xxx 888$zzz so if the key is the first field, and the result should be the common key between file 1 and 2 (6 Replies)
Discussion started by: loloAix
6 Replies

3. Shell Programming and Scripting

Merge 2 CSV files using sed

Help in writing a script using sed which updates fileOne with the contents from fileTwo Example: Contents of fileOne 1,111111 2,897823 3,235473 4,222222 Contents of fileTwo 1,111111,A,1,2 4,222222,A,2,2 5,374632,A,3,2 6,374654,A,4,2 Final File should be: 1,111111,A,1,2... (9 Replies)
Discussion started by: NewToSed
9 Replies

4. Shell Programming and Scripting

Merge CSV files and create a column with the filename from the original file

Hello everyone!! I am not completely new to shell script but I havent been able to find the answer to my problem and I'm sure there are some smart brains here up for the challenge :D. I have several CSV files that I need to combine into one, but I also need to know where each row came from.... (7 Replies)
Discussion started by: fransanchezoria
7 Replies

5. Linux

rpmbuild, how to specify a different source and destination path for files

I'd like to specify a different build and deployment path for files, by default the same path is used for both build and install, I wasn't able to find a way to make these different. With Solaris pkgadd, one can specify different paths in prototype, so I would assume something like that is possible... (0 Replies)
Discussion started by: tiburblium
0 Replies

6. Shell Programming and Scripting

Merge *.csv files, each in separate sheets

Does anyone know how to Merge *.csv files, each in seperate sheets? (7 Replies)
Discussion started by: frhling
7 Replies

7. Shell Programming and Scripting

Merge CSV files

I have lot of csv file collected from script like below : Name of files (some examples) there are thousands of it: 192.168.0.123_251_18796_1433144473.csv 192.168.0.123_251_18796_1433144772.csv 192.168.0.123_251_18796_1433145073.csv 192.168.0.123_251_18796_1433145372.csvContent of each... (5 Replies)
Discussion started by: rk4k
5 Replies

8. UNIX for Dummies Questions & Answers

Merge two csv files using column name

Hi all, I have two separate csv files(comma delimited) file 1 and file 2. File 1 contains PAN,NAME,Salary AAAAA5467D,Raj,50000 AAFAC5467D,Ram,60000 BDCFA5677D,Kumar,90000 File 2 contains PAN,NAME,Dept,Salary ASDFG6756T,Karthik,ABC,450000 QWERT8765Y,JAX,CDR,780000... (5 Replies)
Discussion started by: Nivas
5 Replies

9. Shell Programming and Scripting

Compare and merge two big CSV files

Hi all, i need help. I have two csv files with a huge amount of data. I need the first column of the first file, to be compared with the data of the second, to have at the end a file with the data not present in the second file. Example File1: (only one column) profile_id 57036226... (11 Replies)
Discussion started by: SirMannu
11 Replies

10. UNIX for Beginners Questions & Answers

Merge the three csv files as one according to first coloumn.

I have three files with similar pattern i need to merge all the coloumns side by side from all three files according to the first coloumn example as shown below I mentioned 5 coloumns only in example but i have around 15 coloumns in each file. file1: Name,Samples,Error,95RT,90RT... (4 Replies)
Discussion started by: Raghuram717
4 Replies
obd2csv(1)						      General Commands Manual							obd2csv(1)

NAME
obd2csv - Convert obdgpslogger(1) logs to csv files SYNOPSIS
obd2csv [ options ] DESCRIPTION
Convert obdgpslogger(1) logs to csv files OPTIONS
-o|--out <output filename> Output to this .csv file -d|--db <database> Work from logs stored in this database file -s|--start <time> Only dump rows more recent than this -e|--end <time> Only dump rows older than this -z|--gzip gzip compress output using zlib [if available] -v|--version Print out version number and exit. -h|--help Print out help and exit. NOT OPTIONS
These options aren't intended for end-users, they're for the GUI. -p|--progress Print out progress. It will occasionally print a number in the range [0..100], indicating progress percentage. SEE ALSO
obdgpslogger(1), obd2kml(1), obd2gpx(1), obdsim(1), obdgui(1), obdlogrepair(1) AUTHORS
Gary "Chunky Ks" Briggs <chunky@icculus.org> obd2csv(1)
All times are GMT -4. The time now is 04:26 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy