Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Use awk to pipe output from one file into multiple files

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Use awk to pipe output from one file into multiple files
# 8  
Old 05-10-2012
Quote:
Originally Posted by codestar1
cat ${landing_dir}/${lfile}|awk -F "${file_delimiter}" -v bad=${bad_file} -v good=${rootDir}/landing/Validation_EDITED/new_${lfile} -v fc=${file_field_count} '{if(NF!=fc) {print NR,$0 > bad;} else {{print > good}}}' ${lfile}
Why the useless...cat | awk...when all of this can be done with an awk 1 liner.
Code:
awk -F "${file_delimiter}" -v bad=${bad_file} -v good=${rootDir}/landing/Validation_EDITED/new_${lfile} -v fc=${file_field_count} '{if(NF!=fc) {print NR,$0 > bad;} else {{print > good}}}' ${landing_dir}/${lfile}

# 9  
Old 05-10-2012
I tried that exact code. The issue is that something is wrong with the use of "${file_delimiter}". when I echo the statement and exit the script, the value comes as an empty string. I'm not sure why. But when I cat the file it comes. The echo statement with the above code shows:

Code:
awk -F   -v bad

The varaible is defined as

Code:
file_delimiter=`echo $f|cut -d ',' -f6`
        file_delimiter=`echo ${file_delimiter}|sed -e 's/tab/\t/g'`

 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Pipe or combine output of three awk commands

What is the correct syntax to pipe or run three awk commands? Basically, using the output of the first awk as input in the second. Then using the output of the second awk in the third. Thank you :). awk 'FNR==NR {E; next }$3 in E {print $3, $5}' panel_genes.txt RefSeqGene.txt > update.txt |... (3 Replies)
Discussion started by: cmccabe
3 Replies

2. Shell Programming and Scripting

Awk: Print Error While Redirecting output in multiple Files

Hi, I have a following code in which I am unable to redirect to multiple files. Can anybody please help with some corrections awk -F, '{ if ( substr($1,26,2)=="02" && substr($1,184,14)=="MTSCC_VALFIRST") { array1++ array2++ array3++ } else if (substr($1,26,2)=="03" &&... (4 Replies)
Discussion started by: siramitsharma
4 Replies

3. Shell Programming and Scripting

Pipe awk's output to sed for deletion

Hi Friends, I am using a command that prints certain lines from a file. For ex: cat input abc chr1 456 def chr1 789 ghi chr1 999 jjj chr1 777 jhk chr7 914 My command awk '{if($2=="chr1" && $3>=456 && $3<=999) {print $0}}' OFS="\t" input Output being printed is abc chr1 456 (7 Replies)
Discussion started by: jacobs.smith
7 Replies

4. UNIX for Dummies Questions & Answers

Using multiple pipe output

I have a script that finds all sffs and extracts them into .fastq file types. What I need to do is change the .fastq to .fasta using the below script. How can I change the input.fastq and output.fasta to mirror the file's name? Would I use an array and use the default iterator? #!/bin/bash ... (3 Replies)
Discussion started by: jrymer
3 Replies

5. Shell Programming and Scripting

awk, multiple files input and multiple files output

Hi! I'm new in awk and I need some help. I have a folder with a lot of files and I need that awk do something in each file and print a new file with the output. The input file name should be modified when I print the outpu files. Thanks in advance for help! :-) ciao (5 Replies)
Discussion started by: gabrysfe
5 Replies

6. UNIX for Dummies Questions & Answers

awk to match multiple regex and create separate output files

Howdy Folks, I have a list that looks like this: (file2.txt) AAA BBB CCC DDD and there are 24 of these short words. I am matching these patterns to another file with 755795 lines (file1.txt). I have this code for matching: awk -v f2=file2.txt ' BEGIN { while(... (2 Replies)
Discussion started by: heecha
2 Replies

7. UNIX for Dummies Questions & Answers

Using AWK: Extract data from multiple files and output to multiple new files

Hi, I'd like to process multiple files. For example: file1.txt file2.txt file3.txt Each file contains several lines of data. I want to extract a piece of data and output it to a new file. file1.txt ----> newfile1.txt file2.txt ----> newfile2.txt file3.txt ----> newfile3.txt Here is... (3 Replies)
Discussion started by: Liverpaul09
3 Replies

8. Shell Programming and Scripting

pipe'ing grep output to awk

This script is supposed to find out if tomcat is running or not. #!/bin/sh if netstat -a | grep `grep ${1}: /tomcat/bases | awk -F: '{print $3}'` > /dev/null then echo Tomcat for $1 running else echo Tomcat for $1 NOT running fi the /tomcat/bases is a file that... (2 Replies)
Discussion started by: ziggy25
2 Replies

9. UNIX for Dummies Questions & Answers

single output of awk script processing multiple files

Helllo UNIX Forum :) Since I am posting on this board, yes, I am new to UNIX! I read a copy of "UNIX made easy" from 1990, which felt like a making a "computer-science time jump" backwards ;) So, basically I have some sort of understanding what the basic concept is. Problem Description:... (6 Replies)
Discussion started by: Kasimir
6 Replies

10. UNIX for Dummies Questions & Answers

pipe output to two files

I am using grep and I want the output to go into two files without going to the screen. I used tee to get the output into two files, but it is also putting the output on the screen which i do not want. Can this be fixed. (2 Replies)
Discussion started by: NobluesFDT
2 Replies
Login or Register to Ask a Question