Sponsored Content
Top Forums Shell Programming and Scripting Find all lines in file such that each word on that line appears in at least n lines of the file Post 302999219 by RudiC on Thursday 15th of June 2017 10:53:02 AM
Old 06-15-2017
I'm certain Don Cragun will accept the apologies. The forum maintainers' attitude is less to not to become useless - people in here REALLY like to help with also minor problems - but to keep up the quality of IT education. If a student fills in the homework form including institution, course and professor, s/he will be helped to develop in the right direction and find a solution of his/her own; c.f. https://www.unix.com/homework-and-coursework-questions/. By the way, vague comments on a person's company like "chemical" or "administration" would have sufficed, or even you telling us you're a hobbyist.

Back to your problem. Outputting the entire line that satisfies a condition means either keep ALL lines in memory (demanding for BIG files) or run through the input file twice - once for counting, once for printing. This is the approach in here:
Code:
awk 'NR == FNR {CNT[$1]++; CNT[$3]++;CNT[$5]++; CNT[$7]++; next} CNT[$1] > 1 && CNT[$3] > 1 && CNT[$5] > 1 && CNT[$7] > 1 ' file file
5^5 + 18^2 = 15^3 + 74^1    (3125, 324, 3375, 74)
5^5 + 32^2 = 8^4 + 53^1    (3125, 1024, 4096, 53)
5^5 + 60^1 = 14^3 + 21^2    (3125, 60, 2744, 441)

For increasing the count limit, set all the 1 s to 3 for the four comparisons in the second part.
And, yes, you're right: awk is a very powerful tool for text file analyses...
This User Gave Thanks to RudiC For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

shellscript to find a line in between a particular set of lines of a text file

i have a file a.txt and following is only one portion. I want to search <branch value="/dev36/AREA/" include="yes"></branch> present in between <template_file name="Approve External" path="core/approve/bin" and </template_file> where the no of lines containing "<branch value= " is increasing ... (2 Replies)
Discussion started by: millan
2 Replies

2. Shell Programming and Scripting

Find 5 lines and replace with 18 line in sql file where it contains multiple blocks.

My sql file xyz_abc.sql in this file there are multiple sql block in this block I need to find the following block rem Subset Rows (&&tempName.*) CREATE VIEW &&tempName.* AS SELECT * FROM &&tempName.* WHERE f is not null and replace with following code rem Subset Rows... (9 Replies)
Discussion started by: Zaheer.mic
9 Replies

3. Shell Programming and Scripting

Unix help to find blank lines in a file and print numbers on that line

Hi, I would like to know how to solve one of my problems using expert unix commands. I have a file with occasional blank lines; for example; dertu frthu fghtu frtty frtgy frgtui frgtu ghrye frhutp frjuf I need to edit the file so that the file looks like this; (10 Replies)
Discussion started by: Lucky Ali
10 Replies

4. UNIX for Dummies Questions & Answers

find uniq lines in file, using the first field of line

Hello all, new to unix and have just found the forum. I think I will be here quite often, and hope that in time i will be able to provide soem help, role on not being a newbie anymore :) I have a question which iI am hoping someone could help me with. If i have a file with lines in in thus... (8 Replies)
Discussion started by: grom
8 Replies

5. Shell Programming and Scripting

print lines from a file containing key word

i have a file containing over 1 million records,and i want to print about 300,000 line containing a some specific words. file has content. eg 1,rrt,234 3,fgt,678 4,crf,456 5,cde,drt 6,cfg,123 and i want to print the line with the word fgt,crf this is just an example,my file is so... (2 Replies)
Discussion started by: tomjones
2 Replies

6. UNIX for Dummies Questions & Answers

how to find a word in a file that appears next to a given keyword

Hi Experts, I have a file which contains some text. i need to print the word next to a given keyword. Please help. Ex: test.txt ===================== NEXT HOST ===================== AEADBAS001 access-list 1 permit xxxxxxxxxxxxxx ip access-list extended BLA_Outgoing_Filter... (6 Replies)
Discussion started by: mwrg
6 Replies

7. Shell Programming and Scripting

Get last lines of file after last line with word TEST

i need to get least lines of file after last word TEST in file, and send that lines to mail example of file structure: TEST 10.10.2010 jdfjdnjfndjfndnfkdk djfjdnfjkdjkfnjkdfk jdfjdjfnjdjnfjkdnfjk TEST 11.10.2010 jdjfnjdnfdkdfjdfjdnk jdnfjdnjkfndnfjdnfjk fjdnfjkndnfdfnjdnfjk TEST... (6 Replies)
Discussion started by: waso
6 Replies

8. Shell Programming and Scripting

Read all lines after a string appears in the file.

Hi All, I want to read all lines after a perticular string {SET UP VALUES}apprears in the file. SET UP values contains direcory, number of days and file type. Step1: Read all lines below SET UP VALUES string. Step2: If set up values are not present in each record then read from default... (4 Replies)
Discussion started by: Nagaraja Akkiva
4 Replies

9. UNIX for Advanced & Expert Users

How to find a string in a line in UNIX file and delete that line and previous 3 lines ?

Hi , i have a file with data as below.This is same file. But actual file contains to many rows. i want to search for a string "Field 039 00" and delete that line and previous 3 lines in that file.. Can some body suggested me how can i do using either sed or awk command ? Field 004... (7 Replies)
Discussion started by: vadlamudy
7 Replies

10. UNIX for Beginners Questions & Answers

Search for word in huge logfile and need to continue to print few lines from that line til find date

Guys i need an idea for one logic..in shell scripting am struggling with a logic...So the thing is... i need to search for a word in a huge log file and i need to continue to print few more lines from that line and the consecutive line has to end when it finds the line with date..because i know... (1 Reply)
Discussion started by: Prathi
1 Replies
PSTAT(1M)																 PSTAT(1M)

NAME
pstat - print system facts SYNOPSIS
pstat [ -aixptuf ] [ suboptions ] [ file ] DESCRIPTION
Pstat interprets the contents of certain system tables. If file is given, the tables are sought there, otherwise in /dev/mem. The required namelist is taken from /unix. Options are -a Under -p, describe all process slots rather than just active ones. -i Print the inode table with the these headings: LOC The core location of this table entry. FLAGS Miscellaneous state variables encoded thus: L locked U update time filsys(5)) must be corrected A access time must be corrected M file system is mounted here W wanted by another process (L flag is on) T contains a text file C changed time must be corrected CNT Number of open file table entries for this inode. DEV Major and minor device number of file system in which this inode resides. INO I-number within the device. MODE Mode bits, see chmod(2). NLK Number of links to this inode. UID User ID of owner. SIZ/DEV Number of bytes in an ordinary file, or major and minor device of special file. -x Print the text table with these headings: LOC The core location of this table entry. FLAGS Miscellaneous state variables encoded thus: T ptrace(2) in effect W text not yet written on swap device L loading in progress K locked w wanted (L flag is on) DADDR Disk address in swap, measured in multiples of 512 bytes. CADDR Core address, measured in multiples of 64 bytes. SIZE Size of text segment, measured in multiples of 64 bytes. IPTR Core location of corresponding inode. CNT Number of processes using this text segment. CCNT Number of processes in core using this text segment. -p Print process table for active processes with these headings: LOC The core location of this table entry. S Run state encoded thus: 0 no process 1 waiting for some event 3 runnable 4 being created 5 being terminated 6 stopped under trace F Miscellaneous state variables, or-ed together: 01 loaded 02 the scheduler process 04 locked 010 swapped out 020 traced 040 used in tracing 0100 locked in by lock(2). PRI Scheduling priority, see nice(2). SIGNAL Signals received (signals 1-16 coded in bits 0-15), UID Real user ID. TIM Time resident in seconds; times over 127 coded as 127. CPU Weighted integral of CPU time, for scheduler. NI Nice level, see nice(2). PGRP Process number of root of process group (the opener of the controlling terminal). PID The process ID number. PPID The process ID of parent process. ADDR If in core, the physical address of the `u-area' of the process measured in multiples of 64 bytes. If swapped out, the position in the swap area measured in multiples of 512 bytes. SIZE Size of process image in multiples of 64 bytes. WCHAN Wait channel number of a waiting process. LINK Link pointer in list of runnable processes. TEXTP If text is pure, pointer to location of text table entry. CLKT Countdown for alarm(2) measured in seconds. -t Print table for terminals (only DH11 and DL11 handled) with these headings: RAW Number of characters in raw input queue. CAN Number of characters in canonicalized input queue. OUT Number of characters in putput queue. MODE See tty(4). ADDR Physical device address. DEL Number of delimiters (newlines) in canonicalized input queue. COL Calculated column position of terminal. STATE Miscellaneous state variables encoded thus: W waiting for open to complete O open S has special (output) start routine C carrier is on B busy doing output A process is awaiting output X open for exclusive use H hangup on close PGRP Process group for which this is controlling terminal. -u print information about a user process; the next argument is its address as given by ps(1). The process must be in main memory, or the file used can be a core image and the address 0. -f Print the open file table with these headings: LOC The core location of this table entry. FLG Miscellaneous state variables encoded thus: R open for reading W open for writing P pipe CNT Number of processes that know this open file. INO The location of the inode table entry for this file. OFFS The file offset, see lseek(2). FILES
/unix namelist /dev/mem default source of tables SEE ALSO
ps(1), stat(2), filsys(5) K. Thompson, UNIX Implementation PSTAT(1M)
All times are GMT -4. The time now is 04:27 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy