Sponsored Content
Top Forums Shell Programming and Scripting Find all lines in file such that each word on that line appears in at least n lines of the file Post 302999175 by uncleMonty on Wednesday 14th of June 2017 09:49:34 PM
Old 06-14-2017
Find all lines in file such that each word on that line appears in at least n lines of the file

I have a file where every line includes four expressions with a caret in the middle (plus some other "words" or fields, always separated by spaces). I would like to extract from this file, all those lines such that each of the four expressions containing a caret appears in at least four different lines of the whole file. Could anyone help me?

Here is a section of my file:
Code:
5^4 + 32^1 = 6^3 + 21^2    (625, 32, 216, 441)
5^4 + 34^2 = 12^3 + 53^1    (625, 1156, 1728, 53)
5^4 + 40^2 = 13^3 + 28^1    (625, 1600, 2197, 28)
5^4 + 42^1 = 7^3 + 18^2    (625, 42, 343, 324)
5^4 + 53^2 = 15^3 + 59^1    (625, 2809, 3375, 59)
5^4 + 56^1 = 8^3 + 13^2    (625, 56, 512, 169)
5^4 + 66^2 = 17^3 + 68^1    (625, 4356, 4913, 68)
5^4 + 75^1 = 6^3 + 22^2    (625, 75, 216, 484)
5^5 + 6^4 = 65^1 + 66^2    (3125, 1296, 65, 4356)
5^5 + 7^1 = 6^3 + 54^2    (3125, 7, 216, 2916)
5^5 + 7^4 = 50^1 + 74^2    (3125, 2401, 50, 5476)
5^5 + 8^3 = 37^1 + 60^2    (3125, 512, 37, 3600)
5^5 + 9^3 = 10^1 + 62^2    (3125, 729, 10, 3844)
5^5 + 10^3 = 8^4 + 29^1    (3125, 1000, 4096, 29)
5^5 + 16^2 = 6^1 + 15^3    (3125, 256, 6, 3375)
5^5 + 17^2 = 15^3 + 39^1    (3125, 289, 3375, 39)
5^5 + 18^2 = 15^3 + 74^1    (3125, 324, 3375, 74)
5^5 + 19^1 = 14^3 + 20^2    (3125, 19, 2744, 400)
5^5 + 20^1 = 6^4 + 43^2    (3125, 20, 1296, 1849)
5^5 + 27^1 = 7^3 + 53^2    (3125, 27, 343, 2809)
5^5 + 32^2 = 8^4 + 53^1    (3125, 1024, 4096, 53)
5^5 + 32^2 = 16^3 + 53^1    (3125, 1024, 4096, 53)
5^5 + 33^1 = 13^3 + 31^2    (3125, 33, 2197, 961)
5^5 + 43^2 = 17^3 + 61^1    (3125, 1849, 4913, 61)
5^5 + 47^1 = 12^3 + 38^2    (3125, 47, 1728, 1444)
5^5 + 55^1 = 11^3 + 43^2    (3125, 55, 1331, 1849)
5^5 + 59^2 = 9^4 + 45^1    (3125, 3481, 6561, 45)
5^5 + 60^1 = 7^4 + 28^2    (3125, 60, 2401, 784)
5^5 + 60^1 = 14^3 + 21^2    (3125, 60, 2744, 441)
5^6 + 8^4 = 27^3 + 38^1    (15625, 4096, 19683, 38)
5^6 + 16^1 = 10^3 + 11^4    (15625, 16, 1000, 14641)
5^6 + 20^4 = 9^1 + 56^3    (15625, 160000, 9, 175616)
5^6 + 35^2 = 7^5 + 43^1    (15625, 1225, 16807, 43)
5^6 + 45^2 = 26^3 + 74^1    (15625, 2025, 17576, 74)

So in what I would like to extract from the file, the last line would only be included if each of "5^6", "45^2", "26^3" and "74^1" appears on at least four different lines of the entire file. Thanks for any help!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

shellscript to find a line in between a particular set of lines of a text file

i have a file a.txt and following is only one portion. I want to search <branch value="/dev36/AREA/" include="yes"></branch> present in between <template_file name="Approve External" path="core/approve/bin" and </template_file> where the no of lines containing "<branch value= " is increasing ... (2 Replies)
Discussion started by: millan
2 Replies

2. Shell Programming and Scripting

Find 5 lines and replace with 18 line in sql file where it contains multiple blocks.

My sql file xyz_abc.sql in this file there are multiple sql block in this block I need to find the following block rem Subset Rows (&&tempName.*) CREATE VIEW &&tempName.* AS SELECT * FROM &&tempName.* WHERE f is not null and replace with following code rem Subset Rows... (9 Replies)
Discussion started by: Zaheer.mic
9 Replies

3. Shell Programming and Scripting

Unix help to find blank lines in a file and print numbers on that line

Hi, I would like to know how to solve one of my problems using expert unix commands. I have a file with occasional blank lines; for example; dertu frthu fghtu frtty frtgy frgtui frgtu ghrye frhutp frjuf I need to edit the file so that the file looks like this; (10 Replies)
Discussion started by: Lucky Ali
10 Replies

4. UNIX for Dummies Questions & Answers

find uniq lines in file, using the first field of line

Hello all, new to unix and have just found the forum. I think I will be here quite often, and hope that in time i will be able to provide soem help, role on not being a newbie anymore :) I have a question which iI am hoping someone could help me with. If i have a file with lines in in thus... (8 Replies)
Discussion started by: grom
8 Replies

5. Shell Programming and Scripting

print lines from a file containing key word

i have a file containing over 1 million records,and i want to print about 300,000 line containing a some specific words. file has content. eg 1,rrt,234 3,fgt,678 4,crf,456 5,cde,drt 6,cfg,123 and i want to print the line with the word fgt,crf this is just an example,my file is so... (2 Replies)
Discussion started by: tomjones
2 Replies

6. UNIX for Dummies Questions & Answers

how to find a word in a file that appears next to a given keyword

Hi Experts, I have a file which contains some text. i need to print the word next to a given keyword. Please help. Ex: test.txt ===================== NEXT HOST ===================== AEADBAS001 access-list 1 permit xxxxxxxxxxxxxx ip access-list extended BLA_Outgoing_Filter... (6 Replies)
Discussion started by: mwrg
6 Replies

7. Shell Programming and Scripting

Get last lines of file after last line with word TEST

i need to get least lines of file after last word TEST in file, and send that lines to mail example of file structure: TEST 10.10.2010 jdfjdnjfndjfndnfkdk djfjdnfjkdjkfnjkdfk jdfjdjfnjdjnfjkdnfjk TEST 11.10.2010 jdjfnjdnfdkdfjdfjdnk jdnfjdnjkfndnfjdnfjk fjdnfjkndnfdfnjdnfjk TEST... (6 Replies)
Discussion started by: waso
6 Replies

8. Shell Programming and Scripting

Read all lines after a string appears in the file.

Hi All, I want to read all lines after a perticular string {SET UP VALUES}apprears in the file. SET UP values contains direcory, number of days and file type. Step1: Read all lines below SET UP VALUES string. Step2: If set up values are not present in each record then read from default... (4 Replies)
Discussion started by: Nagaraja Akkiva
4 Replies

9. UNIX for Advanced & Expert Users

How to find a string in a line in UNIX file and delete that line and previous 3 lines ?

Hi , i have a file with data as below.This is same file. But actual file contains to many rows. i want to search for a string "Field 039 00" and delete that line and previous 3 lines in that file.. Can some body suggested me how can i do using either sed or awk command ? Field 004... (7 Replies)
Discussion started by: vadlamudy
7 Replies

10. UNIX for Beginners Questions & Answers

Search for word in huge logfile and need to continue to print few lines from that line til find date

Guys i need an idea for one logic..in shell scripting am struggling with a logic...So the thing is... i need to search for a word in a huge log file and i need to continue to print few more lines from that line and the consecutive line has to end when it finds the line with date..because i know... (1 Reply)
Discussion started by: Prathi
1 Replies
funtbl(1)							SAORD Documentation							 funtbl(1)

NAME
funtbl - extract a table from Funtools ASCII output SYNOPSIS
funtable [-c cols] [-h] [-n table] [-p prog] [-s sep] <iname> DESCRIPTION
[NB: This program has been deprecated in favor of the ASCII text processing support in funtools. You can now perform fundisp on funtools ASCII output files (specifying the table using bracket notation) to extract tables and columns.] The funtbl script extracts a specified table (without the header and comments) from a funtools ASCII output file and writes the result to the standard output. The first non-switch argument is the ASCII input file name (i.e. the saved output from funcnts, fundisp, funhist, etc.). If no filename is specified, stdin is read. The -n switch specifies which table (starting from 1) to extract. The default is to extract the first table. The -c switch is a space-delimited list of column numbers to output, e.g. -c "1 3 5" will extract the first three odd-numbered columns. The default is to extract all columns. The -s switch specifies the separator string to put between columns. The default is a single space. The -h switch specifies that column names should be added in a header line before the data is output. With- out the switch, no header is prepended. The -p program switch allows you to specify an awk-like program to run instead of the default (which is host-specific and is determined at build time). The -T switch will output the data in rdb format (i.e., with a 2-row header of column names and dashes, and with data columns separated by tabs). The -help switch will print out a message describing program usage. For example, consider the output from the following funcnts command: [sh] funcnts -sr snr.ev "ann 512 512 0 9 n=3" # source # data file: /proj/rd/data/snr.ev # arcsec/pixel: 8 # background # constant value: 0.000000 # column units # area: arcsec**2 # surf_bri: cnts/arcsec**2 # surf_err: cnts/arcsec**2 # summed background-subtracted results upto net_counts error background berror area surf_bri surf_err ---- ------------ --------- ------------ --------- --------- --------- --------- 1 147.000 12.124 0.000 0.000 1600.00 0.092 0.008 2 625.000 25.000 0.000 0.000 6976.00 0.090 0.004 3 1442.000 37.974 0.000 0.000 15936.00 0.090 0.002 # background-subtracted results reg net_counts error background berror area surf_bri surf_err ---- ------------ --------- ------------ --------- --------- --------- --------- 1 147.000 12.124 0.000 0.000 1600.00 0.092 0.008 2 478.000 21.863 0.000 0.000 5376.00 0.089 0.004 3 817.000 28.583 0.000 0.000 8960.00 0.091 0.003 # the following source and background components were used: source_region(s) ---------------- ann 512 512 0 9 n=3 reg counts pixels sumcnts sumpix ---- ------------ --------- ------------ --------- 1 147.000 25 147.000 25 2 478.000 84 625.000 109 3 817.000 140 1442.000 249 There are four tables in this output. To extract the last one, you can execute: [sh] funcnts -s snr.ev "ann 512 512 0 9 n=3" | funtbl -n 4 1 147.000 25 147.000 25 2 478.000 84 625.000 109 3 817.000 140 1442.000 249 Note that the output has been re-formatted so that only a single space separates each column, with no extraneous header or comment informa- tion. To extract only columns 1,2, and 4 from the last example (but with a header prepended and tabs between columns), you can execute: [sh] funcnts -s snr.ev "ann 512 512 0 9 n=3" | funtbl -c "1 2 4" -h -n 4 -s " " #reg counts sumcnts 1 147.000 147.000 2 478.000 625.000 3 817.000 1442.000 Of course, if the output has previously been saved in a file named foo.out, the same result can be obtained by executing: [sh] funtbl -c "1 2 4" -h -n 4 -s " " foo.out #reg counts sumcnts 1 147.000 147.000 2 478.000 625.000 3 817.000 1442.000 SEE ALSO
See funtools(7) for a list of Funtools help pages version 1.4.2 January 2, 2008 funtbl(1)
All times are GMT -4. The time now is 05:28 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy