Visit Our UNIX and Linux User Community


Extract error records based on specific criteria from Unix file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract error records based on specific criteria from Unix file
# 1  
Old 07-12-2012
Extract error records based on specific criteria from Unix file

Hi,

I look for a awk one liner for below issue.

input file

Code:
ABC 1234 abc 12345
ABC 4567 678 XYZ
xyz ght 678
ABC 787 yyuu
ABC 789 7890 777
zxr hyip hyu
mno uii 678 776
ABC ty7 888

All lines should be started with ABC as first field. If a record has another value for 1st field, the previuos record with ABC also an error record. I need to get the output as below

Code:
ABC 4567 678 XYZ
xyz ght 678
ABC 789 7890 777
zxr hyip hyu
mno uii 678 776

i have written a code by using while ... do ... done. but for bigger files this takes lot of time as this code reads file record by record.
Please help.

Thanks
RAR.
# 2  
Old 07-12-2012
I think your O/P and the explanation doesn't match Smilie
# 3  
Old 07-12-2012
i have written a code by using while ... do ... done

post your code.......
# 4  
Old 07-12-2012
you can use this perl code

Code:
 
$ cat pp.pl
#!/usr/bin/perl -w
use strict;
chomp(my @arr=<>);
my $i=0;
for (@arr)
{
        if (!/^ABC/)
        {
                my $x=$i-1;
                print "Line ".$i.":".$arr[$x]."\n";
                print "Line ".($i+1).":".$_."\n";
        }
$i++;
}

while runing this code, club it with sort -u

Code:
 
$ ./pp.pl a.txt| sort -u

output
Code:
 
Line 2:ABC 4567 678 XYZ
Line 3:xyz ght 678
Line 5:ABC 789 7890 777
Line 6:zxr hyip hyu
Line 7:mno uii 678 776

# 5  
Old 07-12-2012
My code for this

My requirement is extract error records. Output shows error records.

My code is

Code:
cnt=1
cat input_file | while read line
do
typ=`echo "$line" | cut -c 1-3`
if [ $cnt -gt 1 ]
then
if [ typ != "ABC" ] || [ pre_typ != "ABC" ]
then
echo $pre_line >> output_file
fi
fi
cnt = 2
pre_typ=$typ
pre_line=$line
done
if [ typ != "ABC" ] || [ pre_typ != "ABC" ]
then
echo $pre_line >> output_file
fi

Please give me a better logic.
# 6  
Old 07-12-2012
I don't understand what you mean by extract error records,
my code is giving the exact output you wanted (as mentioned in your post)
if you want the output in a file you can just direct the output towards one

Code:
$ ./pp.pl a.txt| sort -u > ErrorFile.txt

# 7  
Old 07-13-2012
Code:
$awk '$1 ~ /ABC/{if(x ~/ABC/){print $0};x=$1}($1 !~ /ABC/){x=$1;print}'  inputfile


Last edited by raj_saini20; 07-13-2012 at 11:19 AM..

Previous Thread | Next Thread
Test Your Knowledge in Computers #699
Difficulty: Medium
The noosphere represents the highest stage of biospheric development, its defining factor being the development of humankind's rational activities, according to biogeochemist Vladimir Vernadsky.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Awk/sed/cut to filter out records from a file based on criteria

I have two files and would need to filter out records based on certain criteria, these column are of variable lengths, but the lengths are uniform throughout all the records of the file. I have shown a sample of three records below. Line 1-9 is the item number "0227546_1" in the case of the first... (15 Replies)
Discussion started by: MIA651
15 Replies

2. Shell Programming and Scripting

Need a Linux command for find/replace column based on specific criteria.

I'm new to shell programming, I have a huge text file in the following format, where columns are separated by single space: ACA MEX 4O_ $98.00 $127.40 $166.60 0:00 0:00 0 ; ACA YUL TS_ $300.00 $390.00 $510.00 0:00 0:00 0 ; ACA YYZ TS_ $300.00 $390.00 $510.00 0:00 0:00 0 ; ADZ YUL TS_ $300.00... (3 Replies)
Discussion started by: transat
3 Replies

3. Shell Programming and Scripting

awk to print specific line in file based on criteria

In the file below I am trying to extract a specific instance of path, if the adjacent plugin": "/rundb/api/v1/plugin/49/. Thank you :). file "path": "/results/analysis/output/Home/Auto_user_S5-00580-4-Medexome_65_028/plugin_out/FileExporter_out.52", "plugin": "/rundb/api/v1/plugin/49/",... (8 Replies)
Discussion started by: cmccabe
8 Replies

4. Shell Programming and Scripting

Extract specific lines based on another file

I have a folder containing text files. I need to extract specific lines from the files of this folder based on another file input.txt. How can I do this with awk/sed? file1 ARG 81.9 8 81.9 0 LEU 27.1 9 27.1 0 PHE .0 10 .0 0 ASP 59.8 11 59.8 0 ASN 27.6 12 27.6 0 ALA .0 13 .0 0... (5 Replies)
Discussion started by: alanmathew84
5 Replies

5. Shell Programming and Scripting

Extract records from Oracle to UNIX file with headers

Hi, I have a shell script which extracts records form oracle to unix file. sqlplus -s ${WMD_DM_CONNECT} <<EOF >$tmpfile set heading off set pagesize 0 set feedback off select CD_DESC||'|'||CD_ID||'|'||'Arun'||'|'||'Montu' from WMD_SYS_CD_LKUP where CD_TYP =... (5 Replies)
Discussion started by: Arun Mishra
5 Replies

6. Shell Programming and Scripting

Extract file records based on some field conditions

Hello Friends, I have a file(InputFile.csv) with the following columns(the columns are pipe-delimited): ColA|ColB|ColC|ColD|ColE|ColF Now for this file, I have to get those records which fulfil the following condition: If "ColB" is NOT NULL and "ColD" has values one of the following... (9 Replies)
Discussion started by: mehimadri
9 Replies

7. Shell Programming and Scripting

Extract data based on specific search criteria

I have a huge file (about 2 millions records) contains data separated by “,” (comma). As part of the requirement, I can't change the format. The objective is to remove some of the records with the following condition. If the 23rd field on each line start with 302 , I need to remove that from the... (4 Replies)
Discussion started by: jaygamini
4 Replies

8. UNIX for Dummies Questions & Answers

Extract a specific number from an XML file based on the start and end tags

Hello People, I have the following contents in an XML file ........... ........... .......... ........... <Details = "Sample Details"> <Name>Bob</Name> <Age>34</Age> <Address>CA</Address> <ContactNumber>1234</ContactNumber> </Details> ........... ............. .............. (4 Replies)
Discussion started by: sushant172
4 Replies

9. Shell Programming and Scripting

Append specific lines to a previous line based on sequential search criteria

I'll try explain this as best I can. Let me know if it is not clear. I have large text files that contain data as such: 143593502 09-08-20 09:02:13 xxxxxxxxxxx xxxxxxxxxxx 09-08-20 09:02:11 N line 1 test line 2 test line 3 test 143593503 09-08-20 09:02:13... (3 Replies)
Discussion started by: jesse
3 Replies

10. UNIX for Dummies Questions & Answers

Select records based on search criteria on first column

Hi All, I need to select only those records having a non zero record in the first column of a comma delimited file. Suppose my input file is having data like: "0","01/08/2005 07:11:15",1,1,"Created",,"01/08/2005" "0","01/08/2005 07:12:40",1,1,"Created",,"01/08/2005"... (2 Replies)
Discussion started by: shashi_kiran_v
2 Replies

Featured Tech Videos