Loop for file merging in a folder


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Loop for file merging in a folder
# 15  
Old 01-26-2014
those are complicated things for me to answer ...
Code:
Combine ----- test.sh
....... ----- result ---- april
.................... ---- may
.................... ---- june
.................... ---- july

hope this is a help
I dont have any more details but I execute it with
Code:
bash test.sh


Last edited by Scrutinizer; 01-26-2014 at 03:07 PM.. Reason: quote tags to code tags
# 16  
Old 01-26-2014
Code:
#!/bin/bash
#
#	Script to be placed in "Combine" folder
#	Retrieve files from "Combine/result"
#	Output to "Combine/output"
#
#
#	Vars
#
	FZ=(result/*) 		# all entries of ./result
	ZC=0 			# Z Counter
#
#	Action
#
	[[ -d output ]] || mkdir output

	for Z in "${FZ[@]}"
	do
		THESE="$Z"	# Set '$THESE' to a new single value
				# As we have a new variable to start with
		for I in 1 2 3
		do
			THESE+=" ${FZ[$[ $ZC + $I ]]}"
			str=$(echo "$THESE"|sed s,' ','-',g)
			cat "$THESE"|sort -u > output/${str}.txt
		done

		((ZC++))
	done
#
#	Verify
#
	ls output

hth & gn8

EDIT:
If you have additional sub folders in Combine/result, add another '/*' to the FZ= line...
Finaly got that part

Last edited by sea; 01-26-2014 at 04:18 PM..
This User Gave Thanks to sea For This Post:
# 17  
Old 01-27-2014
unfortunately, it is not working and its too complicated for me to figure what the problem is... i was thinking it would be easier to create a loop with a program for it but it seems not Smilie
I think its better if I give you the files that I want and the output...
april
Code:
Anna|Smith|2013|Testing
Andrew|Smith|2012|data
Mark|Green|2010|system
Tony|Green|2009|Testing

may
Code:
George|White|2010|system
Hanna|Brown|2009|data

june
Code:
Anna|Smith|2013|Testing
Andrew|Smith|2012|data
Hanna|Brown|2009|data
Tony|Green|2009|Testing

july
Code:
Anna|Smith|2013|Testing
Tony|Green|2009|Testing
Hanna|Brown|2009|data

and the output files should be 12 followings
Code:
april (or april-april) extension not important
april-may (or visa versa for all)
april-june
april-july
may-june
may-july
june-july
april-may-june
april-may-july
april-june-july
may-june-july
april-may-june-july

Moderator's Comments:
Mod Comment Please: Use CODE tage, not QUOTE tags for data samples

Last edited by Scrutinizer; 01-27-2014 at 11:19 AM.. Reason: quote tags ===>> code tags
# 18  
Old 01-27-2014
Try something like:
Code:
#!/bin/bash
#
# cd to Combine folder and call script there. Otherwise put "cd /path/to/Combine" in your script
#
combine_files() {
  local p i
  to=../output/$*
  to=${to// /-}
  [ -f "$to" ] || cat "$@" > "$to"
  if [ $# -gt 2 ]; then
    for((i=0; i<$#; i++))
    do
      p=("$@")
      unset p[i]
      combine_files "${p[@]}"
    done
  fi
}

# cd /path/to/Combine
[ -d output ] || mkdir output
cd result
combine_files *


Last edited by Scrutinizer; 01-27-2014 at 04:14 PM..
This User Gave Thanks to Scrutinizer For This Post:
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merging two columns from two files with similar names into a loop

I have two files like this: fileA.net A B C fileA.dat 1 2 3 and I want the output output_expected A 1 B 2 C 3 I know that the easier way is to do a paste fileA.net fileA.dat, but the problem is that I have 10,000 couple of files (fileB.net with fileB.dat; fileC.net with... (3 Replies)
Discussion started by: valente
3 Replies

2. Shell Programming and Scripting

For loop for number of files in a folder

Hi All, Need a for loop which should run for number of files in a folder and should pass the file name as parameter to another shell script for each loop. Please help me. Thanks. (2 Replies)
Discussion started by: chillblue
2 Replies

3. Shell Programming and Scripting

For loop to read folder which are not under processing

Hi, I want to read folders which do not have a file "processing" in a for loop ordered by timestamp. Currently im doing like this. Like cd /home/working for i in `ls -c1` do some command... done I want to exclude folders which have that "processing" file. The directory... (2 Replies)
Discussion started by: chetan.c
2 Replies

4. Shell Programming and Scripting

File merging based on different counter loop

hello, File 1 main Group sub group MIT VAR_1D_DATA_TYPE 23-03-2012 MIT VAR_1D_DATA_TYPE 22-03-2012 MIT VAR_10D_DATA_TYPE 23-03-2012 MIT VAR_10D_DATA_TYPE 22-03-2012 MIT ... (0 Replies)
Discussion started by: manas_ranjan
0 Replies

5. Shell Programming and Scripting

Loop through text file > Copy Folder > Edit XML files in bulk?

I have a text file which contains lines in this format - it contains 105 lines in total, but I'm just putting 4 here to keep it short: 58571,east_ppl_ppla_por 58788,east_pcy_hd_por 58704,east_pcy_ga_por 58697,east_pcy_pcybs_por It's called id_key.txt I have a sample folder called... (9 Replies)
Discussion started by: biscuitcreek
9 Replies

6. Shell Programming and Scripting

Merging folder of files according to name and contents.

I have absolutely no idea how to do this and everything I have done doesn't even come close. Here's the scenario: There are a number of files in a folder named in this manner: agd.txt stv.txt frk.txt dqp.txt There is also a series of other files in the same folder with like file... (5 Replies)
Discussion started by: Trapper
5 Replies

7. Shell Programming and Scripting

File in a folder and loop until time

Hi All, I am stuck with this requirement. My requirement is that a job will run daily at anytime before and would look for a file named filename.txt in a folder and mv to a different folder and if not found until 9 A.M it will exit with a success code. Now I did the follwing ..but there... (8 Replies)
Discussion started by: RubinPat
8 Replies

8. Shell Programming and Scripting

Shell script for merging lines in a loop

Dear All, I need a script to merge lines of an input file in a loop, please guide me for the script or one liner(awk, sed, tr, shell, perl). I/P File------------------------- APaul,,,,SDH,,23,,,PPH,,2 ,,,,KKH,,19,,,MMH,,12, ,,,,CCH,,22,,,MNH,,19, ,,,,TCH,,55,,,NNH,,67,... (3 Replies)
Discussion started by: ashis.tewari
3 Replies

9. Shell Programming and Scripting

File Management: How do I move all JPGS in a folder structure to a single folder?

This is the file structure: DESKTOP/Root of Photo Folders/Folder1qweqwasdfsd/*jpg DESKTOP/Root of Photo Folders/Folder2asdasdasd/*jpg DESKTOP/Root of Photo Folders/Folder3asdadfhgasdf/*jpg DESKTOP/Root of Photo Folders/Folder4qwetwdfsdfg/*jpg DESKTOP/Root of Photo... (4 Replies)
Discussion started by: guptaxpn
4 Replies

10. Shell Programming and Scripting

Parse the .txt file for folder name and FTP to the corrsponding folder.

Oracle procedure create files on UNIX folder on a regular basis. I need to FTP files onto windows server and place the files, based on their name, in the corresponding folders. File name is as follows: ccyymmddfoldernamefile.txt; Folder Name length could be of any size; however, the prefix and... (3 Replies)
Discussion started by: MeganP
3 Replies
Login or Register to Ask a Question