awk: too many output files created from while loop
I am using awk to read lines from a CSV file then put data into other files. These other files are named using the value of a certain column. Column 7 is a name such as "att" or "charter" . I want to end up with file names with the value of column 7 appended to them, like this:
I am getting these files alright, but I am ending up with a couple extra files as if column 7 was not populated.
stockton-migrate-1-.cfg
stockton-migrate-2-.cfg
The last part of the file name after the 3rd "-" comes from the $7 variable which I have as part of the output file name. I don't understand why my script is creating these files that end in "-.cfg" since $7 is always populated with data. Also, I instruct the script not to run through the while loop when $4 equals "d" and $7 equals "none".
Here is a snippet of my CSV file, notice that line 17 has "d" in $4 and "none" in $7.
Here is my code. Is there something I am doing wrong? I am getting the data I need but I am ending up with extra data that I don't want which makes me think there is a better way of doing this.
Last edited by dodgerfan78; 10-08-2012 at 09:59 PM..
Reason: modify title
I would have to say that somewhere in your data you have at least one record where field 4 is not a 'd' and field 7 is empty. What is produced when you run this across your data?
If it generates even one line, that is the cause of your problem.
Thanks. I ran that command as you suggested and it gave back no input. I am using a spreadsheet editor to make the CSV file so I am positive that there is data in column 7 for every row. It seems to me that my loop somewhere is clearing the value of column 7 then re-reading the dataset against the template.
This User Gave Thanks to dodgerfan78 For This Post:
Thanks for running the test -- too painful, if not impossible, to ask to see all of the data
I didn't see this before...
You've already processed the record, and using getline you'll read to the end of the file and when you drop into the next block you'll always match because $7 will be nil.
Adding a 'next' will cause awk to read the next record and start over.
Last edited by agama; 10-08-2012 at 11:14 PM..
Reason: correction -- clarification
You can also simplify the if statement in the next section:
can become
because you already processed a 'd' record above and skipped the rest of the processing. It's minor, but makes it more efficient and that could help if your input is huge.
while IFS= read -r line
do
# sV for version detection
nmap -T4 -Pn -v -sS "$line" > "text/$line"
done < <(grep '' $file)
Hi,
where line represents the IP.
I am using NMAP to do scanning.
How can I set to execute that command in the loop several concurrently at a time instead of one... (5 Replies)
Hi Expert,
I have a .sh script in my home/new_dir/script.sh
This script creates number of output files at home/new_dir/email, home/new_dir/logs dir.
I am running this script using crontab (owner root).
Now this output files are getting created with
rw-r----- 1 root root
So if i... (2 Replies)
Have built this script, the output is what I needed, but NR 6 is omitted. Why? Is it an error? I am using Gawk.
'{nr=$2;f = $1} END{for (i=1;i<=f;i++) if (nr != i) print i, nr }' input1.csv >output1.csvinput1.csv
1 9
3 5
4 1
7 6
8 5
10 6
output1.csv > with the missing line number 6. 6 is... (5 Replies)
Hi all,
after hours of playing around with this and scouring the web I decided to ask my fellow UNIX operators as I can't wrap my head around this.
First off,
I want to parse an input file with tabs (I could pull this off easily with different delimiters) but I was trying to make nicer... (2 Replies)
I need to get total number of hdisk not assigned to any VGs.
PDC # lspv |grep None |awk '{print $1}' |wc
131 131 1099
So, it shows 131 hdisks.
I need to look at the individual hdisk fget_config info like below:
PDC # fget_config -Av |grep hdisk230
hdisk230 dac1 229... (4 Replies)
When I run the following command in the shell it works fine. It prints a city name and then a path for a file.
~$ for i in `awk -F':' '{print $0}' /home/knoppix/Desktop/data/subs | grep -m 1 $ city | sed "s/:/ /"`
>do
>echo $i
>done
Now, when I place it in this shell script (sh) it prints... (6 Replies)
I'm trying to understand if it's possible to create a set of variables that are numbered based on another variable (using eval) in a loop, and then call on it before the loop ends.
As an example I've written a script called question (The fist command is to show what is the contents of the... (2 Replies)
I have a while loop which looks for files and then sets a variable to give me the record count of each file:
current_members=`wc -l ${DATA_DIR}/$MEMBERS_FILENAME | nawk '{ printf "%d\n", $0}'`
I am out putting the totals into a file:
echo $current_members >> ../data/out_total_members.dat
... (7 Replies)