08-28-2015
You're zcatting "$NBSPATH"/"$FILE1" and running grep | awk once for every CFA in $FILE2. That consumes a lot of resources. Why don't you uncompress once into a temp file and use e.g. grep -f $FILE2 on the temp file? Does your system offer the zgrep command?
This User Gave Thanks to RudiC For This Post:
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Need a script that manipulates a fixed length file that will compare 2 fields in that file and if they are equal write that line to a new file.
i.e. If fields 87-93 = fields 119-125, then write the entire line to a new file. Do this for every line in the file. After we get only the fields... (1 Reply)
Discussion started by: Muga801
1 Replies
2. Shell Programming and Scripting
I have a file name version.properties with the following data:
major.version=14
minor.version=234
I'm trying to write a grep expression to only put "14" to stdout. The following is not working.
grep "major.version=(+)" version.properties
What am I doing wrong? (6 Replies)
Discussion started by: obfunkhouser
6 Replies
3. Shell Programming and Scripting
#!/bin/bash
i=0
cat 1.csv | while read fileline
do
echo "$fileline"
IFS="#" flds=( $fileline )
nrofflds=${#flds}
echo "noof fields$nrofflds"
fld=0
while
do
echo "noof counter$fld"
echo "$nrofflds"
#fld1="${flds}" trying to store the content of line to fields but i... (4 Replies)
Discussion started by: barani75
4 Replies
4. Shell Programming and Scripting
Hi,
I want to split/parse certain bits of the hex data into another field.
Example:
Input data is
Word1: 4f72abfd
Output:
Parse bits (5 to 0) into field word1data1=0x00cd=205 decimal
Parse bits (7 to 6) into field word1data2=0x000c=12 decimal
etc.
Word2: efff3d02
Parse bits (13 to... (1 Reply)
Discussion started by: morrbie
1 Replies
5. Shell Programming and Scripting
Hi Folks,
I have a file with 22 columns. In which, I need to remove nulls if found at $2, $4, $14 & $16. Then, needs to compare the fields such as $2 == $14 && $4 == $16. The final output will print all the fields where the above conditions are satisfied.
Could you please anyone help me... (6 Replies)
Discussion started by: Jerald
6 Replies
6. Shell Programming and Scripting
I have two files
File 1 in reading directory is of following format
Read 1 A T
Read 3 T C
Read 5 G T
Read 7 A G
Read 10 A G
Read 12 C G
File 2 in directory contains
Read 5 A G
Read 6 T C
Read 7 G A
Read 8 G A
Read 20 A T
File2 contains (1 Reply)
Discussion started by: empyrean
1 Replies
7. UNIX for Advanced & Expert Users
I have a file. We can call file1. It has these contents.
STOPLOSS Control_file 0000000010.000 STOPLOSS Audit_file_Signoff +0000000010.00
nawk '{ fmt="%3s %15s %15s %15s %21s\n";
if ($3==$6) {
tpy="Successful Match"; jnme=$1;... (4 Replies)
Discussion started by: wawa
4 Replies
8. Shell Programming and Scripting
The below awk parser works for most data inputs, but I am having trouble with the last one. The problem is in the below rules steps 1 and 2 come from $2 (NC_000013.10:g.20763686_20763687delinsA) and steps 3 and 4 come from $1 (NM_004004.5:c.34_35delGGinsT).
Parse Rules:
The header is... (0 Replies)
Discussion started by: cmccabe
0 Replies
9. Shell Programming and Scripting
I have a file of ~500,000 entries in the following:
file.txt
chr1 11868 12227 ENSG00000223972.5 . + HAVANA exon . gene_id "ENSG00000223972.5"; transcript_id "ENST00000456328.2"; gene_type "transcribed_unprocessed_pseudogene"; gene_status "KNOWN"; gene_name "DDX11L1"; transcript_type... (17 Replies)
Discussion started by: cmccabe
17 Replies
10. UNIX for Beginners Questions & Answers
Hi,
Below are the sample files. x.txt is from an Excel file that is a list of users from Windows and y.txt is a list of database account.
$ head -500 x.txt y.txt
==> x.txt <==
TEST01 APP_USER_PROFILE
USER03 APP_USER_PROFILE
TEST02 APP_USER_EXP_PROFILE
TEST04 APP_USER_PROFILE
USER01 ... (3 Replies)
Discussion started by: newbie_01
3 Replies
JOIN(1) FSF JOIN(1)
NAME
join - join lines of two files on a common field
SYNOPSIS
join [OPTION]... FILE1 FILE2
DESCRIPTION
For each pair of input lines with identical join fields, write a line to standard output. The default join field is the first, delimited
by whitespace. When FILE1 or FILE2 (not both) is -, read standard input.
-a SIDE
print unpairable lines coming from file SIDE
-e EMPTY
replace missing input fields with EMPTY
-i, --ignore-case ignore differences in case when comparing fields
-j FIELD
(obsolescent) equivalent to `-1 FIELD -2 FIELD'
-j1 FIELD
(obsolescent) equivalent to `-1 FIELD'
-j2 FIELD
(obsolescent) equivalent to `-2 FIELD'
-o FORMAT
obey FORMAT while constructing output line
-t CHAR
use CHAR as input and output field separator
-v SIDE
like -a SIDE, but suppress joined output lines
-1 FIELD
join on this FIELD of file 1
-2 FIELD
join on this FIELD of file 2
--help display this help and exit
--version
output version information and exit
Unless -t CHAR is given, leading blanks separate fields and are ignored, else fields are separated by CHAR. Any FIELD is a field number
counted from 1. FORMAT is one or more comma or blank separated specifications, each being `SIDE.FIELD' or `0'. Default FORMAT outputs the
join field, the remaining fields from FILE1, the remaining fields from FILE2, all separated by CHAR.
AUTHOR
Written by Mike Haertel.
REPORTING BUGS
Report bugs to <bug-coreutils@gnu.org>.
COPYRIGHT
Copyright (C) 2002 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICU-
LAR PURPOSE.
SEE ALSO
The full documentation for join is maintained as a Texinfo manual. If the info and join programs are properly installed at your site, the
command
info join
should give you access to the complete manual.
join (coreutils) 4.5.3 February 2003 JOIN(1)