Sponsored Content
Top Forums Shell Programming and Scripting Awk/sed/cut to filter out records from a file based on criteria Post 302999707 by vgersh99 on Monday 26th of June 2017 04:51:13 PM
Old 06-26-2017
a bit of a longhand - making some assumptions based on the file samples.....
Code:
awk -f mia.awk file1.txt file2.txt

where mia.awk is:
Code:
FNR==NR {
  if (split($1,a, "_") ==3)
     f1[$1]
  next
}
NF{
  if (split($1, a, "_") ==3 && $1 in f1)
     p=1
  rec=(!rec)?$0:rec ORS $0
  next
}
p{
    print rec ORS
    p=0; rec=""
}


Last edited by vgersh99; 06-26-2017 at 06:02 PM..
This User Gave Thanks to vgersh99 For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Selecting records from file on criteria.

Can I have 2 files as in input to the awk command? Situation is somewhat below, File A contains number & value delimited by a space. File B contains number as a part of a line. I am not supposed to retrieve more than 1 number from a line. If number from file B matches with number from... (7 Replies)
Discussion started by: videsh77
7 Replies

2. UNIX for Dummies Questions & Answers

Select records based on search criteria on first column

Hi All, I need to select only those records having a non zero record in the first column of a comma delimited file. Suppose my input file is having data like: "0","01/08/2005 07:11:15",1,1,"Created",,"01/08/2005" "0","01/08/2005 07:12:40",1,1,"Created",,"01/08/2005"... (2 Replies)
Discussion started by: shashi_kiran_v
2 Replies

3. Shell Programming and Scripting

Filter records in a file using AWK

I want to filter records in one of my file using AWK command (or anyother command). I am using the below code awk -F@ '$1=="0003"&&"$2==20100402" print {$0}' $INPUT > $OUTPUT I want to pass the 0003 and 20100402 values through a variable. How can I do this? Any help is much... (1 Reply)
Discussion started by: gpaulose
1 Replies

4. Shell Programming and Scripting

awk - splitting 1 large file into multiple based on same key records

Hello gurus, I am new to "awk" and trying to break a large file having 4 million records into several output files each having half million but at the same time I want to keep the similar key records in the same output file, not to exist accross the files. e.g. my data is like: Row_Num,... (6 Replies)
Discussion started by: kam66
6 Replies

5. Shell Programming and Scripting

Filter/remove duplicate .dat file with certain criteria

I am a beginner in Unix. Though have been asked to write a script to filter(remove duplicates) data from a .dat file. File is very huge containig billions of records. contents of file looks like 30002157,40342424,OTC,mart_rec,100, ,0 30002157,40343369,OTC,mart_rec,95, ,0... (6 Replies)
Discussion started by: mukeshguliao
6 Replies

6. Shell Programming and Scripting

Extract error records based on specific criteria from Unix file

Hi, I look for a awk one liner for below issue. input file ABC 1234 abc 12345 ABC 4567 678 XYZ xyz ght 678 ABC 787 yyuu ABC 789 7890 777 zxr hyip hyu mno uii 678 776 ABC ty7 888 All lines should be started with ABC as first field. If a record has another value for 1st... (7 Replies)
Discussion started by: ratheesh2011
7 Replies

7. Shell Programming and Scripting

Filter records based on 2nd file

Hello, I want to filter records of a file if they fall in range associated with a second file. First the chr number (2nd col of 1st file and 1st col of 2nd file) needs to be matched. Then if the 3rd col of the first file falls within any of the ranges specified by the 2nd and 3rd cols , then... (4 Replies)
Discussion started by: ritakadm
4 Replies

8. Shell Programming and Scripting

awk to print specific line in file based on criteria

In the file below I am trying to extract a specific instance of path, if the adjacent plugin": "/rundb/api/v1/plugin/49/. Thank you :). file "path": "/results/analysis/output/Home/Auto_user_S5-00580-4-Medexome_65_028/plugin_out/FileExporter_out.52", "plugin": "/rundb/api/v1/plugin/49/",... (8 Replies)
Discussion started by: cmccabe
8 Replies

9. Shell Programming and Scripting

awk to filter file based on seperate conditions

The below awk will filter a list of 30,000 lines in the tab-delimited file. What I am having trouble with is adding a condition to SVTYPE=CNV that will only print that line if CI= must be >.05 . The other condition to add is if SVTYPE=Fusion, then in order to print that line READ_COUNT must... (3 Replies)
Discussion started by: cmccabe
3 Replies

10. UNIX for Beginners Questions & Answers

Filter records from a log file based on timestamp

Dear Experts, I have a log file that contains a timestamp, I would like to filter record from that file based on timestamp. For example refer below file - cat sample.txt Jan 19 20:51:48 mukul-Vostro-14-3468 systemd: pam_unix(systemd-user:session): session opened for user root by (uid=0)... (6 Replies)
Discussion started by: mukulverma2408
6 Replies
cut(1)							      General Commands Manual							    cut(1)

Name
       cut - cut out selected fields of each line of a file

Syntax
       cut -clist [file1 file2...]
       cut -flist [-dchar] [-s] [file1 file2...]

Description
       Use  the  command to cut out columns from a table or fields from each line of a file.  The fields as specified by list can be fixed length,
       that is, character positions as on a punched card (-c option), or the length can vary from line to line and be marked with a  field  delim-
       iter character like tab (-f option).  The command can be used as a filter.  If no files are given, the standard input is used.

       Use to make horizontal ``cuts'' (by context) through a file, or to put files together in columns.  To reorder columns in a table, use and

Options
       list	   Specifies  ranges  that must be a comma-separated list of integer field numbers in increasing order.  With optional - indicates
		   ranges as in the -o option of nroff/troff for page ranges; for example, 1,4,7; 1-3,8; -5,10 (short for 1-5,10);  or	3-  (short
		   for third through last field).

       -clist	   Specifies character positions to be cut out.  For example, -c1-72 would pass the first 72 characters of each line.

       -flist	   Specifies  the  fields  to be cut out.  For example, -f1,7 copies the first and seventh field only.	Lines with no field delim-
		   iters are passed through intact (useful for table subheadings), unless -s is specified.

       -dchar	   Uses the specified character as the field delimiter.  Default is tab.  Space or other characters with special  meaning  to  the
		   shell must be quoted.  The -d option is used only in combination with the -f option, according to XPG3 and SVID2/SVID3.

       -s	   Suppresses  lines  with  no	delimiter  characters.	 Unless  specified, lines with no delimiters are passed through untouched.
		   Either the -c or -f option must be specified.

Examples
       Mapping of user IDs to names:
       cut -d: -f1,5 /etc/passwd
       To set name to the current login name for the csh shell:
       set name=`who am i | cut -f1 -d" "`
       To set name to the current login name for the sh, sh5, and ksh shells:
       name=`who am i | cut -f1 -d" "`

Diagnostics
       "line too long"	   A line can have no more than 511 characters or fields.

       "bad list for c/f option"
			   Missing -c or -f option or incorrectly specified list.  No error occurs if a line has fewer fields than the list  calls
			   for.

       "no fields"	   The list is empty.

See Also
       grep(1), paste(1)

																	    cut(1)
All times are GMT -4. The time now is 10:23 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy