single output of awk script processing multiple files
Helllo UNIX Forum
Since I am posting on this board, yes, I am new to UNIX!
I read a copy of "UNIX made easy" from 1990, which felt like a making a "computer-science time jump" backwards
So, basically I have some sort of understanding what the basic concept is.
Problem Description:
What I am currently trying to do is writing an awk script.
This awk script should be able to repeat the same task on multiple input textfiles (extracting information (numerical values) from specified columns) and write the output to one single output file.
The output should be formatted in such a way, that it appears in 2 columns:
1st: index, 2nd: extracted value
I got to the point of extracting the information from multiple files and writing it into 1 output file. But my problem is, that the index starts all over again every time a new input file is read in, I would like it to increase every time, regardless whether it is a new file or not.
Ansatz:
My code looks the following way:
I guess that each time the loop completes one cycle the awk script exits, effectively resetting the index-value i.
Own thoughts:
1a) Is there some sort of "save-attribute" so the awk-script doesn't "forget" the index-value?
1b) Alternatively could the index i of the awk-script get saved as a "global" variable in the shell-script and locally in the awk-script?
2) The other solution I considered was to use wc -l or some awk-command, to see how many lines the output_file would already consists of, but I think that would create a problem, when the file to be analyzed does not exist at that point (which would happen in the vey first run I presume). That could probably also be fixed by creating an empty output_file before any output is written (appended) to it. Then again, if there was no output written (if the value does not match the specified one), I would need to include a control structure checking for content in output_file. In case that there is none, output_file gets removed.
3) The other idea is to create one temporary file containing all the input files and give it to awk. After output is written the temporary file gets deleted again.
I doubt that solutions 1a) or 1b) are possible (are they?) and I don't really like solutions 2) (too complicated) and 3) (use of a temporary file).
My actual goal was to use the awk script, to write a code as smooth and easy as possible...
Question
What solution would you try (if at all any of the before mentioned), or do you have any hints at solving the problem?
Thanks in advance,
Kasimir
Last edited by DukeNuke2; 08-08-2009 at 01:02 PM..
Reason: added code tags
Thank you very much for your help. I think this solved my problem.
What do the quotation marks around output_file actually mean, though - why were they necessary? What happened when not using them?
It changes output_file from an awk variable into a string.
If you don't use the quotes, depending on your awk version either no output file will be written, or you'll get an error. If you declare an awk variable called output_file, then it would be fine:
I tested it with a couple test files and it worked just fine (under latest cygwin), to finally conclude the case, I will be looking forward to test the code at work tomorrow.
Every input file is not to be read completely - only up to the position of the appearance of a certain string. Once this string is read, awk should stop processing the current file and start on the next one, until all input-files are processed.
My idea was something like this:
if ($1=="exit_string)
"go to the next input_file"
If all input files are read, exit.
My problem is, that I don't know how I could realize the "go to the next input_file" command. If I would just use the exit command, instead of the "go to next file", awk would quit after reading the first input file.
Hi,
I'm using awk command in bash script. I'm able to pass multiple files to awk for processing.The code i can use is as below(sample code)
#!/bin/bash
awk -F "," 'BEGIN {
...
...
...
}' file1 file2 file3
In the above code i'm passing the file names manually and it is fine till my... (7 Replies)
Hi All,
I am new here but I have a scripting question that I can't seem to figure out with the "find" cmd.
What I am trying to do is to only have to run a single find cmd parsing the directories and output the different file types to induvidual files and I have been running into problems.... (3 Replies)
hai i need my single awk script to act on 4 trace files of ns2 and to calculate througput and it should print result from each trace file in a single trace file. i tried with the following code but it doesnt work awk -f awkscript inputfile1 inputfile2 inputfile3 inputfile4>outputfile ... (4 Replies)
Hi!
I'm new in awk and I need some help.
I have a folder with a lot of files and I need that awk do something in each file and print a new file with the output. The input file name should be modified when I print the outpu files.
Thanks in advance for help!
:-)
ciao (5 Replies)
Hi,
I'd like to process multiple files. For example:
file1.txt
file2.txt
file3.txt
Each file contains several lines of data. I want to extract a piece of data and output it to a new file.
file1.txt ----> newfile1.txt
file2.txt ----> newfile2.txt
file3.txt ----> newfile3.txt
Here is... (3 Replies)
Hi!
I have 2 files containing data that I need to process at the same time, I have problems in reading a different number of lines from the different files.
Here is an explanation of what I need to do (possibly with an awk script).
File "samples.txt" contains data in the format:
time_instant... (6 Replies)
Hi,
I am trying to do the following using AWK program.
1. Read the input data file
2. Parse the record and see if it contains errors
3. If the record contains errors, then write it into Reject file, else, write into usual output file or display it on the screen
Here is what I have done -... (6 Replies)
I need a script to process a huge single line text file:
The sample of the text is:
"forward_inline_item": "Inline", "options_region_Australia": "Australia", "server_event_err_msg": "There was an error attempting to save", "Token": "Yes", "family": "Family","pwd_login_tab": "Enter Your... (1 Reply)
Is it possible to process multiple variables in a single sed command?
I have the following ksh with three variables and I want to search for all variables which start with "var" inside input.txt.
I tired "$var$" but it just prints out everyting in input.txt and does not work.
$ more test.ksh... (5 Replies)