Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Merge files in a directory
Top Forums Shell Programming and Scripting Merge files in a directory Post 302980312 by rbatte1 on Friday 26th of August 2016 04:30:38 AM
Old 08-26-2016
The loop suggested seems odd to me. The sed is difficult to read and is a large process to start, so for multiple files will be slow. It also assumes that the target output files do not exist.

Would you be better with:-
Code:
filenames="$(for i in access*
do
  echo ${i%%.*}                     # Variable substitution removes everything after the first dot
done | sort -u)"                    # Building a list of required output files

for filename in $filenames
do
   cat ${filename}.* > $filename   # Write all matching files in one step
done

I know it is a two step process, but I've kept IO to a minimum and used internal functions rather than calling external processes.

Because of the trailing .* on the cat, this will not match the output file as an input file if it already exists. If you run this and then tidy away the input files, a re-run will match the output files as potential input, but then the cat will complain about missing input.




I hope that this helps,
Robin
This User Gave Thanks to rbatte1 For This Post:
MadeInGermany
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

merge files

Hi, i have the files f1 and f2 like: files f1: c1 a1 c2 a2 c3 a3 file f2: c1 b1 c2 b2 c3 b3 i want merge the f1 and f2 file to f3 file like: c1 a1 b1 c2 a2 b3 c3 a3 b3........ .... . . please help me onthis..... (5 Replies)
Discussion started by: koti_rama
5 Replies

2. Shell Programming and Scripting

how to merge these two files?

I have two files, each of them has 12 lines, fileA has 3 columns, fileB has 1 column, like the following FileA a 1 b 2 c 3 ..blabla FileB A B C ..blabla Now I am trying to put the content of fileB as column 3 of fileA, e.g. a 1 A b 2 B c 3 C (3 Replies)
Discussion started by: fedora
3 Replies

3. Shell Programming and Scripting

Merge files of differrent size with one field common in both files using awk

hi, i am facing a problem in merging two files using awk, the problem is as stated below, file1: A|B|C|D|E|F|G|H|I|1 M|N|O|P|Q|R|S|T|U|2 AA|BB|CC|DD|EE|FF|GG|HH|II|1 .... .... .... file2 : 1|Mn|op|qr (2 Replies)
Discussion started by: shashi1982
2 Replies

4. Shell Programming and Scripting

Merge 2 files

Hello, i'd like a bash script to merge 2 files without duplicate lines. Example : file1 : toto titi file2 : toto tata Expected result, file3 : toto (5 Replies)
Discussion started by: Celmar
5 Replies

5. Shell Programming and Scripting

Merge files

Hello, I have a application software plink. It can merge files with some kinds of way. The command likes: plink --file 1 --merge 2 --recode --out merged That means merge file 1 and 2 then output file "merged". However I have 23 files (1,2,3,...22,23)to be merged together. How can I use... (2 Replies)
Discussion started by: zhshqzyc
2 Replies

6. UNIX for Dummies Questions & Answers

Merge files

Hi, I would like to know how can I merge files based on their coordinates, but mantaining the score of each file in the output file like: Note: 1st column is for chromosome, 2nd for start, 3rd for end of segment, 4th for score file1: 1 200 300 20 1 400 500 30 file2: 1 200 350 30 1... (1 Reply)
Discussion started by: fadista
1 Replies

7. Shell Programming and Scripting

Checking in a directory how many files are present and basing on that merge all the files

Hi, My requirement is,there is a directory location like: :camp/current/ In this location there can be different flat files that are generated in a single day with same header and the data will be different, differentiated by timestamp, so i need to verify how many files are generated... (10 Replies)
Discussion started by: srikanth_sagi
10 Replies

8. Shell Programming and Scripting

Merge files and generate a resume in two files

Dear Gents, Please I need your help... I need small script :) to do the following. I have a thousand of files in a folder produced daily. I need first to merge all files called. txt (0009.txt, 0010.txt, 0011.txt) and and to output a resume of all information on 2 separate files in csv... (14 Replies)
Discussion started by: jiam912
14 Replies

9. Shell Programming and Scripting

How to merge two files?

Dear Frens, I have two files and need to merge into one file. Like File_1: Field1 Field2 1 4 File_2: Field1 Field2 3 5 I need one single output as File_1: Field1 Field2 1 4 3 5 This means taking header from either file. (8 Replies)
Discussion started by: manisha_singh
8 Replies

10. Shell Programming and Scripting

Directory containing files,Print names of the files in the directory that are exactly same content.

Given a directory containing say a few thousand files, please output a list of all the names of the files in the directory that are exactly the same, i.e. have the same contents. func(a_directory_name) output -> {“matches”: , ... ]} e.g. func(“/home/my/files”) where the directory... (7 Replies)
Discussion started by: anuragpgtgerman
7 Replies

LEARN ABOUT OSX

opendiff

opendiff(1)						    BSD General Commands Manual 					       opendiff(1)

NAME

     opendiff -- Use FileMerge to graphically compare or merge file or directories

SYNOPSIS

     opendiff file1 file2 [-ancestor ancestorFile] [-merge mergeFile]
     opendiff dir1 dir2 [-ancestor ancestorDirectory] [-merge mergeDirectory]

DESCRIPTION

     opendiff is a command line utility that provides a convenient way to launch the FileMerge application from Terminal to graphically compare
     files or directories.  If FileMerge is already running, opendiff will connect to that running instance for the new comparison.  opendiff
     exits immediately after the comparison request has been sent to FileMerge.

     opendiff and FileMerge can be used to compare two files file1 and file2 or to compare two directories dir1 and dir2.

     If the -ancestor flag is given, FileMerge will compare the two files or directories to a common ancestor.	This is useful if two people inde-
     pendently modify copies of a single original file or directory.

     FileMerge lets you merge two files or directories together to create a third file or directory.  To see the contents of a merged file, drag
     the splitter bar at the bottom of FileMerge's file comparison window.  The contents of the merged file can be directly edited within File-
     Merge.  After editing, the merged file can be saved to the file (or into the directory) specified with the Fl merge flag.	If a destination
     is not specified with the -merge flag, FileMerge will ask for a destination file or directory when you try to save a merged file.

     For further information, please consult the Help information available from the FileMerge application.

FILES

     /Developer/Applications/Utilities/FileMerge.app

     opendiff and FileMerge are installed as part of the Mac OS X Developer Tools.

SEE ALSO

     diff(1), diff3(1), cmp(1)

Mac OS X							  August 3, 2004							  Mac OS X
All times are GMT -4. The time now is 06:12 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy