Sponsored Content
Top Forums UNIX for Advanced & Expert Users Script to parse and compare information in two fields of file Post 302953430 by GERMANOS on Friday 28th of August 2015 04:49:42 AM
Old 08-28-2015
Script to parse and compare information in two fields of file

Hello,
I am working parsing a large input file1(field CFA)
I have to compare the the file1 field(CFA byte 88-96) with the content of the file2(It contains only one field) and and insert rows equal
in another file.
Here is my code and sample input file:


Code:
#########################################
# F.ne: CheckNBS
#########################################

function CheckNBS
{
    
writeInfo "************************************************************************************************"
writeInfo "----------------- CHECK FILE NBS FILE2 Start: $d ------------------"    
FILE2="$DIR_OUT"/"FILE2"_"${NamingDate}.data"
FILE_OUT="$DIR_OUT"/"OUT_CAMPIONE_NBS"_"${NamingDate}.ctrl"


ListFILE=`cat "$NBSPATH"/"*"${DATA_RIFERIMENTO}"*"`
for FILE1 in ${ListFILE}
do

writeInfo "Elaborazione FILE1 : ${FILE1}"

ListCFA=`cat ${FILE2}`
for CFA in ${ListCFA}
do

zcat "$NBSPATH"/"$FILE1" | grep $CFA | awk '$1 == "201" { print $0 }' >> ${FILE_OUT}

done
done
}

Execution is very slow. I can use awk also on compressed files ?

Can you help me?

Last edited by Don Cragun; 08-28-2015 at 03:52 PM.. Reason: Add CODE tags.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need awk script to compare 2 fields in fixed length file.

Need a script that manipulates a fixed length file that will compare 2 fields in that file and if they are equal write that line to a new file. i.e. If fields 87-93 = fields 119-125, then write the entire line to a new file. Do this for every line in the file. After we get only the fields... (1 Reply)
Discussion started by: Muga801
1 Replies

2. Shell Programming and Scripting

Trying to Parse Version Information from Text File

I have a file name version.properties with the following data: major.version=14 minor.version=234 I'm trying to write a grep expression to only put "14" to stdout. The following is not working. grep "major.version=(+)" version.properties What am I doing wrong? (6 Replies)
Discussion started by: obfunkhouser
6 Replies

3. Shell Programming and Scripting

How to read and parse the content of csv file containing # as delimeter into fields using Bash?

#!/bin/bash i=0 cat 1.csv | while read fileline do echo "$fileline" IFS="#" flds=( $fileline ) nrofflds=${#flds} echo "noof fields$nrofflds" fld=0 while do echo "noof counter$fld" echo "$nrofflds" #fld1="${flds}" trying to store the content of line to fields but i... (4 Replies)
Discussion started by: barani75
4 Replies

4. Shell Programming and Scripting

Perl: Parse Hex file into fields

Hi, I want to split/parse certain bits of the hex data into another field. Example: Input data is Word1: 4f72abfd Output: Parse bits (5 to 0) into field word1data1=0x00cd=205 decimal Parse bits (7 to 6) into field word1data2=0x000c=12 decimal etc. Word2: efff3d02 Parse bits (13 to... (1 Reply)
Discussion started by: morrbie
1 Replies

5. Shell Programming and Scripting

Compare fields within a file

Hi Folks, I have a file with 22 columns. In which, I need to remove nulls if found at $2, $4, $14 & $16. Then, needs to compare the fields such as $2 == $14 && $4 == $16. The final output will print all the fields where the above conditions are satisfied. Could you please anyone help me... (6 Replies)
Discussion started by: Jerald
6 Replies

6. Shell Programming and Scripting

Parse files in directory and compare with another file

I have two files File 1 in reading directory is of following format Read 1 A T Read 3 T C Read 5 G T Read 7 A G Read 10 A G Read 12 C G File 2 in directory contains Read 5 A G Read 6 T C Read 7 G A Read 8 G A Read 20 A T File2 contains (1 Reply)
Discussion started by: empyrean
1 Replies

7. UNIX for Advanced & Expert Users

Nawk coding trying to compare two fields in a file

I have a file. We can call file1. It has these contents. STOPLOSS Control_file 0000000010.000 STOPLOSS Audit_file_Signoff +0000000010.00 nawk '{ fmt="%3s %15s %15s %15s %21s\n"; if ($3==$6) { tpy="Successful Match"; jnme=$1;... (4 Replies)
Discussion started by: wawa
4 Replies

8. Shell Programming and Scripting

awk script to parse case with information in two fields of file

The below awk parser works for most data inputs, but I am having trouble with the last one. The problem is in the below rules steps 1 and 2 come from $2 (NC_000013.10:g.20763686_20763687delinsA) and steps 3 and 4 come from $1 (NM_004004.5:c.34_35delGGinsT). Parse Rules: The header is... (0 Replies)
Discussion started by: cmccabe
0 Replies

9. Shell Programming and Scripting

Parse file for fields and specific text

I have a file of ~500,000 entries in the following: file.txt chr1 11868 12227 ENSG00000223972.5 . + HAVANA exon . gene_id "ENSG00000223972.5"; transcript_id "ENST00000456328.2"; gene_type "transcribed_unprocessed_pseudogene"; gene_status "KNOWN"; gene_name "DDX11L1"; transcript_type... (17 Replies)
Discussion started by: cmccabe
17 Replies

10. UNIX for Beginners Questions & Answers

Is there a UNIX command that can compare fields of files with differing number of fields?

Hi, Below are the sample files. x.txt is from an Excel file that is a list of users from Windows and y.txt is a list of database account. $ head -500 x.txt y.txt ==> x.txt <== TEST01 APP_USER_PROFILE USER03 APP_USER_PROFILE TEST02 APP_USER_EXP_PROFILE TEST04 APP_USER_PROFILE USER01 ... (3 Replies)
Discussion started by: newbie_01
3 Replies
TEST(1) 								FSF								   TEST(1)

NAME
test - check file types and compare values SYNOPSIS
test EXPRESSION [ EXPRESSION ] test OPTION DESCRIPTION
Exit with the status determined by EXPRESSION. --help display this help and exit --version output version information and exit EXPRESSION is true or false and sets exit status. It is one of: ( EXPRESSION ) EXPRESSION is true ! EXPRESSION EXPRESSION is false EXPRESSION1 -a EXPRESSION2 both EXPRESSION1 and EXPRESSION2 are true EXPRESSION1 -o EXPRESSION2 either EXPRESSION1 or EXPRESSION2 is true [-n] STRING the length of STRING is nonzero -z STRING the length of STRING is zero STRING1 = STRING2 the strings are equal STRING1 != STRING2 the strings are not equal INTEGER1 -eq INTEGER2 INTEGER1 is equal to INTEGER2 INTEGER1 -ge INTEGER2 INTEGER1 is greater than or equal to INTEGER2 INTEGER1 -gt INTEGER2 INTEGER1 is greater than INTEGER2 INTEGER1 -le INTEGER2 INTEGER1 is less than or equal to INTEGER2 INTEGER1 -lt INTEGER2 INTEGER1 is less than INTEGER2 INTEGER1 -ne INTEGER2 INTEGER1 is not equal to INTEGER2 FILE1 -ef FILE2 FILE1 and FILE2 have the same device and inode numbers FILE1 -nt FILE2 FILE1 is newer (modification date) than FILE2 FILE1 -ot FILE2 FILE1 is older than FILE2 -b FILE FILE exists and is block special -c FILE FILE exists and is character special -d FILE FILE exists and is a directory -e FILE FILE exists -f FILE FILE exists and is a regular file -g FILE FILE exists and is set-group-ID -h FILE FILE exists and is a symbolic link (same as -L) -G FILE FILE exists and is owned by the effective group ID -k FILE FILE exists and has its sticky bit set -L FILE FILE exists and is a symbolic link (same as -h) -O FILE FILE exists and is owned by the effective user ID -p FILE FILE exists and is a named pipe -r FILE FILE exists and is readable -s FILE FILE exists and has a size greater than zero -S FILE FILE exists and is a socket -t [FD] file descriptor FD (stdout by default) is opened on a terminal -u FILE FILE exists and its set-user-ID bit is set -w FILE FILE exists and is writable -x FILE FILE exists and is executable Beware that parentheses need to be escaped (e.g., by backslashes) for shells. INTEGER may also be -l STRING, which evaluates to the length of STRING. AUTHOR
Written by FIXME: ksb and mjb. REPORTING BUGS
Report bugs to <bug-coreutils@gnu.org>. COPYRIGHT
Copyright (C) 2002 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICU- LAR PURPOSE. SEE ALSO
The full documentation for test is maintained as a Texinfo manual. If the info and test programs are properly installed at your site, the command info test should give you access to the complete manual. GNU coreutils 4.5.3 February 2003 TEST(1)
All times are GMT -4. The time now is 04:29 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy