Assistance with combining, sorting and saving multi files into one new file


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Assistance with combining, sorting and saving multi files into one new file
# 1  
Old 01-27-2011
Assistance with combining, sorting and saving multi files into one new file

Good morning. I have a piece of code that is currently taking multiple files and using the CAT.exe command to combine into one file that is then sorted in reverse order based on the 3rd field of the file, then displayed on screen. I am trying to change this so that the files are being combined into one file, that one file sorted on the 3rd field in reverse order, and then saved. So I want to take it from 1 step/line of code to 3 steps. Combine, sort and then save.

This is what I had originally:
Code:
#print system("cat.exe tmp\\*.log | c:\\wbin\\sort.exe -k 3 -r ");

This is what I have been trying to change to, with no luck - No error messages apprear, but no new file appears either.
Code:
#print system("cat.exe tmp\\*.log | tmp\\TTLog.txt");
#print system("c:\\wbin\\sort.exe -k 3 -r  tmp\\TTLog.txt");

Any assistance, suggestions, and help would be greatly appreciated! I have tried with the .txt file created with no data in it and also with the .txt file not created in the directory. Neither way gave error message, and went through debugger with no problems. However, neither way populated the txt file as I wanted, let along sorted it out.

I have also tried to use a '>' rather than the '|' character in the line of code with Cat.exe in it. This did not help either.

The text file will then be moved from this location to a history location and the next new file will be created the following month. Thank you in advance!!
# 2  
Old 01-27-2011
How are your fields delimited?

Is this Cygwin, gnu stuff under dos or ??? with the exe files?

This is how UNIX does it:
Code:
sort -r +2 -3 files*.txt


Last edited by DGPickett; 01-27-2011 at 03:24 PM..
# 3  
Old 01-28-2011
The fields in the Log files are all delimited by tabs. I have a total of 14 Log files in this folder. I need to combine them into one large file first. That file will have to be sorted and saved.

I thought if I could do a Cat command in Unix and then a Sort command in Unix this would solve the issue. I can find MANY examples of both commands, but when I try to replicate the examples, my ONE file is NOT being created/updated/saved ever.

I would think the Cat command would be set up similar to (logically) Cat.exe all files with the extension of .log (*.log) into ('|' or '>') one large text file (file.txt). Is there a problem with combining files of one extension into one file with a different extension?

Then of course logically I would think I could then use sort to sort the now populated large file (.txt file) in reverse order based on the 3rd field in this large file.

Finally I want to save this file, then move it to another location so it can be utilized in the future for aduits.
# 4  
Old 01-28-2011
I think your syntax is wrong. You don't pipe to/from files, only between processes. To files, you redirect. <file reads, >file writes.

If you're using the windows CMD shell and not a real UNIX shell, "cat *.txt" won't work, * won't expand. * only works for a few special command like DEL and COPY in windows CMD.

You could try find, which will run cat many times and put them all together, but I don't think it will sort the names that way.

Code:
c:\wbin\find.exe .\tmp -name '*.log' -exec cat '{}' ';' | c:\wbin\sort.exe -k 3 -r > outputfile

If you have a c:\wbin\sh.exe, you should be able to run it to get a real shell, and 'cat tmp/*.log' in that and get what you expect. "c:\wbin\sh.exe scriptfile.sh" to run scriptfile.sh from disk so you don't have to type a painful number of arguments into your system() command.

Last edited by Corona688; 01-28-2011 at 02:08 PM..
This User Gave Thanks to Corona688 For This Post:
# 5  
Old 01-31-2011
Thank you for your assistance! This worked perfectly for what I want to have accomplished!!!
This User Gave Thanks to jaacmmason For This Post:
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Saving files with file name as output

Hi, i need help with a file creation of an output program. I've got a program that with #find creates an output for each files in a directory. If i give this command : -o spec$(date -u +%Y%m%dt%H%M) it creates just one file, overwriting all the others since it is the creation date .... (2 Replies)
Discussion started by: Board27
2 Replies

2. UNIX for Dummies Questions & Answers

Sorting and saving values based on unique entries

Hi all, I wanted to save the values of a file that contains unique entries based on a specific column (column 4). my sample file looks like the following: input file: 200006-07file.txt 145 35 10 3 147 35 12 4 146 36 11 3 145 34 12 5 143 31 15 4 146 30 14 5 desired output files:... (5 Replies)
Discussion started by: ida1215
5 Replies

3. Shell Programming and Scripting

Multi line sorting in Linux

I have log files with following format - YYYY/MM/DD HH:mm:ss.msec|field2|filed3| log message Now the message itself can be multi line message containing new line character. for e.g. 2013/02/05 15:33:12.234|abc|xyz| This is first single line message. 2013/02/05 15:33:12.786|abc|xyz| This... (6 Replies)
Discussion started by: gini32
6 Replies

4. Shell Programming and Scripting

Assistance for sorting files

Hi there, I have tried using the "find" command to do this but to no avail as the "find -mtime" command I used descend to the directories from my current working directory. Say in "directoryA" has multiple files and those files are created on a daily basis. Under "directoryB", there are... (4 Replies)
Discussion started by: chewku
4 Replies

5. Shell Programming and Scripting

Multi level sorting script

I want to sort like below Suppose few lines in a file is like this systemid:ABC messagedestination:batchxpr replytoqname: myca systemid:BCD messagedestination:realtime replytoqname: myca systemid:ABC messagedestination:realtime replytoqname: eac systemid: BCD messagedestination:mqonline... (1 Reply)
Discussion started by: srkmish
1 Replies

6. Shell Programming and Scripting

parsing txt file, saving graphics files

hi everyone, i am a newbie in shell programming. and i want to simply go through a text file that contains 3 "columns", split by ';' customerID ; link-to-contract ; save-as-filename so an example would simply look like this now i want to loop through every line, and save the file from... (3 Replies)
Discussion started by: Confidence
3 Replies

7. Shell Programming and Scripting

Sorting multi-column values from a specific file

Hi, all. I need a shell script which gathers data from a remote XML file and then displays it according to my needs.. I need this for my job due to the fact that I need to keep track price changes of euro, usd, gold, etc. The XML file I am talking about is located at this page: cnnturk dot... (4 Replies)
Discussion started by: canimsin
4 Replies

8. Shell Programming and Scripting

sorting multi dimensional array

Hi there, Can someone let me know how to sort the 2 dimensional array below by column 1 then by column 2? 22 55 2222 2230 33 66 44 58 222 240 11 25 22 60 33 45 output: 11 25 22 55 22 60 33 45 33 66 44 58 (6 Replies)
Discussion started by: phoeberunner
6 Replies

9. Shell Programming and Scripting

In need of multi threaded perl assistance

I need to write a perl script to execute external programs and grab the output and return code. Each program should be killed if it has not completed within X seconds. Imagine that the script goes something like this : @commands = &get_commands(); foreach $cmd (@commands) { $pid =... (4 Replies)
Discussion started by: SandmanCL
4 Replies

10. UNIX for Advanced & Expert Users

need help sorting and combining files

I have 2 files with many lines. File one example: CN=VXX,CN=ug88888,OU=America,OU=State,OU=VXX, File two example: ul22222 Jon Smith ug8888 Mildred Apple I need a routine that will take the ug88888 portion of the first file and pull out the corresponding full line in file two. Then add the... (2 Replies)
Discussion started by: scriptarg
2 Replies
Login or Register to Ask a Question