run awk on one file for each line in a second file
I have a file with a list of 'samples' all in one column and a second file with a list of 'results' for these samples. I am trying to use a FOR loop to go through each sample and gawk the second file to return (the first field of) all the results for that sample.
All I am getting is errors so far. I have deduced (via commenting out gawk) that it generates an error for each line in the first file and then there is an error on my gawk statement. I can't make sense out of any of them:
Hello all,
I realized that I was actually executing the wrong script. I had two versions and was editing one but executing the other. No wonder I kept getting the same error no matter what I changed
Hmmm. I take it from your response that fgrep would have provided a much easier way to perform the task, but after looking it up I don't see how. (I found this definition: "use fgrep to find all the lines of a file that contain a particular word.")
Seems that I would have just been doing the same 'for' loop except to fgrep instead of gawk. In fact, there would have been an extra step because that would have just given me the lines, but I'd still need to parse out the first field from the rest of the line.
Am I missing something?
The -f option to grep provides a facility for finding all lines in a file that match any of the strings in the -f file. Historically this was only available with fgrep, but with GNU grep, you can always do it. So
Granted, it doesn't ensure that its the 5th field. But you can use -w option to make sure that it IS a field.
In the awk below I am trying to cp and paste each matching line in f2 to $3 in f1 if $2 of f1 is in the line in f2 somewhere. There will always be a match (usually more then 1) and my actual data is much larger (several hundreds of lines) in both f1 and f2. When the line in f2 is pasted to $3 in... (4 Replies)
My file (the output of an experiment) starts off looking like this,
_____________________________________________________________
Subjects incorporated to date: 001
Data file started on machine PKSHS260-05CP
**********************************************************************
Subject 1,... (9 Replies)
I cannot seem to get what should be a simple awk one-liner to work correctly and cannot figure out why. I would like to use patterns from a specific field in one file as regex to search for matching strings in the entire line ($0) of another file.
I would like to output the lines of File2 which... (1 Reply)
hi,
i'm trying to write a tcsh script that reads in a text file (one column) and the runs a different command for each line of text.
i've found lots of example commands for bash, but not for tcsh.
can anyone give me a hint?
thanks,
jill (8 Replies)
I have a script that builds a database ~30 million lines, ~3.7 GB .cvs file. After multiple optimzations It takes about 62 min to bring in and parse all the files and used to take 10 min to remove duplicates until I was requested to add another column. I am using the highly optimized awk code:
awk... (34 Replies)
Say I have a text file, with several lines. Each line may contain spaces or the # symbol.
For each line, I want to pass that line as the path of a file, in order to add it to a tar file.
I've tried this but doesn't work:
cat contents.txt | xargs -0 `tar -uvf contents.tar $1`Any ideas?
... (3 Replies)
Hey, not too good at this, so I only managed a clumsy and SLOW solution to my problem that needs a drastic speed up. Any ideas how I write the following in awk only?
Code is supposed to do...
For every line read column values $6, $7, $8 and do a calculation with the same column values of every... (6 Replies)
I have several hundreds of tiny files which need to be concatenated into one single line and all those in a single file. Some files have several blank lines. Tried to use this script but failed on it.
awk 'END { print r } r && !/^/ { print FILENAME, r; r = "" }{ r = r ? r $0 : $0 }' *.txt... (8 Replies)
Hi Sorry to multipost. I am opening the new thread because the earlier threads head was misleading to my current doubt.
and i am stuck.
list=`cat /u/Test/programs`;
psg "ServTest" | awk -v listawk=$list '{
cmd_name=($5 ~ /^/)? $9:$8
for(pgmname in listawk)
... (6 Replies)
Hi,
My awk program is failing. I figured out using command
od -c filename
that the last line of the file doesnt end with a new line character.
Mine is an automated process because of this data is missing.
How do i handle this?
I want to append new line character at the end of last... (2 Replies)