Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

To extract selected rows

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting To extract selected rows
# 1  
Old 02-06-2012
To extract selected rows
Hi,

I have two files Input1.txt and Input2.txt and i need to have a output with selected lines starting with the values present in Input2.txt.

For example:

Input1.txt:
Code:
9653752570,405932410770019,89919321111147700191,1234,4321,19682011,74021309,82CFEEB44A7E7BCF96FB0BDA6F5FCCF2,01,01,02,8991932111114770019,1128149
9653757361,405932410770020,89919321111147700209,1234,4321,92086703,79121574,235E047A708A6BEF0C8970365C8A9500,01,01,02,8991932111114770020,1128149
9653153160,405932410770021,89919321111147700217,1234,4321,64443192,64800609,E5E91988852DBAB559EF8FD3D045DDFF,01,01,02,8991932111114770021,1128149
9653485678,405932410770022,89919321111147700225,1234,4321,54737476,65617118,A5CFF9E6500E1A1884BA1E3CE3EB8E2F,01,01,02,8991932111114770022,1128149
9653497712,405932410770023,89919321111147700233,1234,4321,79940033,96840212,9CAAFF9987B06054D5F89381CEE10F11,01,01,02,8991932111114770023,1128149
9653001302,405932410770024,89919321111147700241,1234,4321,77360544,12637345,841CD432DBB19014229F067134598413,01,01,02,8991932111114770024,1128149
9653001643,405932410770025,89919321111147700258,1234,4321,34179830,94439735,BF644C4545C9FFB701FB90BE62DEDC09,01,01,02,8991932111114770025,1128149
9653001977,405932410770026,89919321111147700266,1234,4321,92159534,98564636,49E5B2D1A284183CCDAECBA45D443F0A,01,01,02,8991932111114770026,1128149
9653002491,405932410770027,89919321111147700274,1234,4321,20542427,54009311,E17CC8ADBFBF1A000DD2DF28AE9863F0,01,01,02,8991932111114770027,1128149
9653050558,405932410770028,89919321111147700282,1234,4321,75310059,31104923,4700F5103BDE38C31CFE4B56BF9D1202,01,01,02,8991932111114770028,1128149
9653051031,405932410770029,89919321111147700290,1234,4321,43433588,14563286,BAE6B6F4BC09DDA969C59C17DDE5E772,01,01,02,8991932111114770029,1128149

Input2.txt:
Code:
9653752570
9653001302
9653050558

Expected output:
Code:
9653752570,405932410770019,89919321111147700191,1234,4321,19682011,74021309,82CFEEB44A7E7BCF96FB0BDA6F5FCCF2,01,01,02,8991932111114770019,1128149
9653001302,405932410770024,89919321111147700241,1234,4321,77360544,12637345,841CD432DBB19014229F067134598413,01,01,02,8991932111114770024,1128149
9653050558,405932410770028,89919321111147700282,1234,4321,75310059,31104923,4700F5103BDE38C31CFE4B56BF9D1202,01,01,02,8991932111114770028,1128149

Thanks..
Last edited by Franklin52; 02-06-2012 at 06:49 AM.. Reason: Please use code tags for data and code samples, thank you
# 2  
Old 02-06-2012
Code:
grep -f Input2.txt Input1.txt

# 3  
Old 02-06-2012
To secure a little the matching :

Code:
sed 's/^/^/' input2.txt | grep -f - Input1.txt

For example if you have a line like :

9653001643,405932410770025,89919321119653001302,1234,4321,34179830,94439735,BF644C4545C9FFB701FB90BE 62DEDC09,01,01,02,8991932111114770025,1128149

This line is not expected since its first fields is not in the list of the wanted fields pointed by input2.txt
but this the sequence of figures 9653001302 appear in the line, the line will still be displayed with the
grep -f Input2.txt Input1.txt command (even if unexpected).


Short demo : we only want line that start with 'to' :
Code:
$ cat filter
to

Code:
$ cat data
toto
tito
tuto
toto

Code:
$ grep -f filter data
toto
tito
tuto
toto

Code:
$ sed 's/^/^/' filter | grep -f - data
toto
toto

Last edited by ctsgnb; 02-06-2012 at 06:34 AM..
# 4  
Old 02-06-2012
Thanks much cstgnb.

But i am getting an below error
grep: can't open -

my files are huge more that 1 lakh rows

grep -f is working fine for small files.

And also i want the expected output in a seperate Output.txt file.

Please suggest...
# 5  
Old 02-06-2012
This should help

Code:
 
#!/bin/bash

while read F_FIELD; do
        awk -F','  '/^'"$F_FIELD"'/ {print $0}' Input1.txt >> output

done < Input2.txt

# 6  
Old 02-06-2012
Code:
awk -F, 'NR==FNR{a[$1];next}$1 in a' input2.txt input1.txt

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Reading and copying a selected rows

Dear All, I have a data file input.res like below. (Only six column shown here for example.) Sequence of first column starting from 1 to 148. Input file 1 Q0 9_August_2014_Entertainment2 0 20.14967806339729 BM25b1.0 1 Q0 13_October_2012_Page323 1 20.134224346765738 BM25b1.0 1 Q0... (2 Replies)
Discussion started by: imranrasheedamu
2 Replies

2. Shell Programming and Scripting

Combine multiple rows based on selected column keys

Hello I want to collapse a file with multiple rows into consolidated lines of entries based on selected columns as the 'key'. Example: 1 2 3 Abc def ghi 1 2 3 jkl mno p qrts 6 9 0 mno def Abc 7 8 4 Abc mno mno abc 7 8 9 mno mno abc 7 8 9 mno j k So if columns 1, 2 and 3 are... (6 Replies)
Discussion started by: linuxlearner123
6 Replies

3. Shell Programming and Scripting

awk command to print only selected rows in a particular column specified by column name

Dear All, I have a data file input.csv like below. (Only five column shown here for example.) Data1,StepNo,Data2,Data3,Data4 2,1,3,4,5 3,1,5,6,7 3,2,4,5,6 5,3,5,5,6 From this I want the below output Data1,StepNo,Data2,Data3,Data4 2,1,3,4,5 3,1,5,6,7 where the second column... (4 Replies)
Discussion started by: ks_reddy
4 Replies

4. Shell Programming and Scripting

Shell to remove a newline char from selected rows in a file.

Greetings! Can we automate the process of removing a newline char from selected rows in a fixed width file using a shell? Input is like abcd1234 xyzd1234 abcd a1b2c3d4 abcd1234 xyzd1234 xx abcd1234 Expected output - abcd1234xyzd1234 abcda1b2c3d4abcd1234xyzd1234 xxabcd1234 ... (3 Replies)
Discussion started by: mailme0205
3 Replies

5. UNIX for Dummies Questions & Answers

Remove a newline char from selected rows.

Greetings! Can we automate the process of removing a newline char from selected rows in a fixed width file using a shell? Input is like abcd1234 xyzd1234 abcd a1b2c3d4 abcd1234 xyzd1234 xx abcd1234 Expected output - abcd1234xyzd1234 abcda1b2c3d4abcd1234xyzd1234 xxabcd1234 ... (2 Replies)
Discussion started by: mailme0205
2 Replies

6. Shell Programming and Scripting

How to extract duplicate rows

I have searched the internet for duplicate row extracting. All I have seen is extracting good rows or eliminating duplicate rows. How do I extract duplicate rows from a flat file in unix. I'm using Korn shell on HP Unix. For.eg. FlatFile.txt ======== 123:456:678 123:456:678 123:456:876... (5 Replies)
Discussion started by: bobbygsk
5 Replies

7. Shell Programming and Scripting

print selected rows with awk

Hi everybody: Could anybody tell me how I can print from a file a selected rows with awk. In my case I only want print in another file all the rows from NR=8 to NR=2459 and the increment each 8 times. I tried to this: awk '{for (i=8; i=2459; i+=8); NR==i}' file1 > file2 But doesn't... (6 Replies)
Discussion started by: tonet
6 Replies

8. Shell Programming and Scripting

return number of rows selected

Hi all, i am doing a perl script to read from a db. I am able to retrieve the rows, but i am unable to return the number of rows selected. i tried $selectedrows = $sth->numrows; i got the error msg: Can't locate object method "numrows" via package "DBI::st" i changed it to $selectedrows =... (7 Replies)
Discussion started by: new2ss
7 Replies

9. Shell Programming and Scripting

flags to suppress column output, # of rows selected in db2 sql in UNIX

Hello, I am new to db2 SQL in unix so bear with me while I try to explain the situation. I have a text file that has the contents of the where condition that I am using for a db2 SQL in UNIX ksh. Here is the snippet. if ; then echo "Begin processing VALUEs" ... (1 Reply)
Discussion started by: jerardfjay
1 Replies

10. Shell Programming and Scripting

Cutting rows after a selected point

I have a log file from which I want to cut out the ten lines assoictiated to my search requirment (cust_ref #). The cust_ref number will be on te first line and the update information will be on the following ten lines (which dosen't linking data to grep on). Using a script I would like to... (4 Replies)
Discussion started by: nhatch
4 Replies
Login or Register to Ask a Question