awk to match multiple regex and create separate output files Post: 302544585

Sponsored Content

Top Forums UNIX for Dummies Questions & Answers awk to match multiple regex and create separate output files Post 302544585 by heecha on Thursday 4th of August 2011 09:22:34 AM

08-04-2011

Registered User

awk to match multiple regex and create separate output files

Howdy Folks,

I have a list that looks like this:
(file2.txt)

AAA
BBB
CCC
DDD

and there are 24 of these short words.

I am matching these patterns to another file with 755795 lines (file1.txt).

I have this code for matching:

Code:

awk -v f2=file2.txt '
    BEGIN {
        while( (getline<f2) > 0 )   # read and collect records from f2
        {
            key = $1;
            ki = kidx[key]++;        # track number of duplicate keys (0 based)
            k2rec[key,ki] = $0;      # save unique record by key and dup count
        }
        close( f2 );
    }

    {
        key = $1;
        for( i = 0; i < kidx[key]; i++ )          # for each duplicate of key
            printf( "%s\t%s\n", k2rec[key,i], $0 );   # print f2 record, followed by current f1 record
    }
' <file1.txt > output

In this form every line in file1 that is matched goes into the same output file. What I would like to do is change the code such that each of the patterns in file2.txt gets its own file. For example:

all of the lines in file1.txt that match AAA would go in AAA.txt while all of the lines matching BBB would go in BBB.txt.

I'm not exactly sure how to handle controlling the output.

Thanks guys, I appreciate your help.
Robert

heecha

View Public Profile for heecha

Find all posts by heecha

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Using AWK to separate data from a large XML file into multiple files

I have a 500 MB XML file from a FileMaker database export, it's formatted horribly (no line breaks at all). The node structure is basically <FMPXMLRESULT> <METADATA> <FIELD att="............." id="..."/> </METADATA> <RESULTSET FOUND="1763457"> <ROW att="....." etc="...."> ...

2. Shell Programming and Scripting

handling multiple files using awk command and wants to get separate out file for each

hai all I am new to the world of shell scripting I wanted to extract two columns from multiple files say around 25 files and i wanted to get the separate outfile for each input file tired using the following command to extract two columns from 25 files awk...

3. UNIX for Dummies Questions & Answers

Using AWK: Extract data from multiple files and output to multiple new files

Hi, I'd like to process multiple files. For example: file1.txt file2.txt file3.txt Each file contains several lines of data. I want to extract a piece of data and output it to a new file. file1.txt ----> newfile1.txt file2.txt ----> newfile2.txt file3.txt ----> newfile3.txt Here is...

4. Shell Programming and Scripting

extract DDL - output every match to separate file

Hi, i want to extract the 'CREATE INDEX' or 'CREATE UNIQUE INDEX' statements from a ddl file and output each match to a separate file. i was looking around the net but couldn�t find anything. a possible sed-script could be: sed -n '/CREATE*INDEX*/,/COMMIT/p' filename.ddlbut i couldn�t find out...

5. Shell Programming and Scripting

create separate files from one excel file with multiple sheets

Hi, I have one requirement, create separate files (".csv") from one excel file(xlsx) with multiple sheets. These ".csv" files are my source files. So anybody please suggest me the process. Thanks in Advance. Regards, Harris

6. Shell Programming and Scripting

Create Multiple UNIX Files for Multiple SQL Rows output

Dear All, I am trying to write a Unix Script which fires a sql query. The output of the sql query gives multiple rows. Each row should be saved in a separate Unix File. The number of rows of sql output can be variable. I am able save all the rows in one file but in separate files. Any...

7. Shell Programming and Scripting

Join two files combining multiple columns and produce mix and match output

I would like to join two files when two columns in each file matches with each other and then produce an output when taking multiple columns. Like I have file A 1234,ABCD,23,JOHN,NJ,USA 2345,ABCD,24,SAM,NY,USA 5678,GHIJ,24,TOM,NY,USA 5678,WXYZ,27,MAT,NJ,USA and file B ...

8. Shell Programming and Scripting

awk to create separate files but not include specific field in output

I am trying to use awk to create (in this example) 3 seperate text file from the unique id in $1 in file, if it starts with the pattern aa. The contents of each row is used to populate each text file except for $1 which is not needed. It seems I am close but not quite get there. Thank you :). ...

9. UNIX for Beginners Questions & Answers

Awk: output lines with common field to separate files

Hi, A beginner one. my input.tab (tab-separated): h1 h2 h3 h4 h5 item1 grpA 2 3 customer1 item2 grpB 4 6 customer1 item3 grpA 5 9 customer1 item4 grpA 0 0 customer2 item5 grpA 9 1 customer2 objective: output a file for each customer ($5) with the item number ($1) only if $2 matches...

10. Shell Programming and Scripting

awk to create subdirectory based on match between two files

In the below awk I am trying to mkdir based of an exact match between file2 line starting with R_2019.... and file1 line starting with R_2019. When a match is found there is a folder located at /home/cmccabe/run with the same name as the match where each $2 in file1 is a new subdirectory in that...

LEARN ABOUT DEBIAN

sc_warts2text

SC_WARTS2TEXT(1)					    BSD General Commands Manual 					  SC_WARTS2TEXT(1)

NAME

     sc_warts2text -- simple dump of information contained in a warts file.

SYNOPSIS

     sc_warts2text [-d ip2descr-file] [file ...]

DESCRIPTION

     The sc_warts2text utility provides a simple dump of information contained in a sequence of warts files.  The output is the same as that which
     would have been provided by scamper if the text output option had been chosen instead of the warts output option when the data was collected.
     The options are as follows:

     -d ip2descr-file
	     specifies the name of a file with IP-address, description mappings, one mapping per line.	See the examples section for further
	     information.

     While the output of sc_warts2text is structured and suitable for initial analyses of results, the format of the output is not suitable for
     automated parsing and analysis as the output of sc_warts2text will change overtime with no regard to backwards compatibility.  Analyses of
     the contents of a warts file should be made using specialised programs which link against the scamper file API.

EXAMPLES

     The command:

	sc_warts2text file1.warts file2.warts

     will decode and print the contents of file1.warts, followed by the contents of file2.warts.

     The command:

	gzcat file1.warts.gz | sc_warts2text

     will print the contents of the uncompressed file supplied on stdin.

     Given a set of IP-address, description pairs in a file name mappings.txt:

	192.0.2.1 "foo"
	192.0.2.2 "bar"

     then the command gzcat file1.warts.gz | sc_warts2text -d mappings.txt will print the description associated with a given destination address
     before each result is presented.

SEE ALSO

     scamper(1), sc_wartsdump(1)

AUTHORS

     sc_warts2text is written by Matthew Luckie <mjl@luckie.org.nz>.

BSD
								 October 15, 2010							       BSD