Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Bash : Checking Large file for specific lines
Top Forums Shell Programming and Scripting Bash : Checking Large file for specific lines Post 303019077 by RudiC on Friday 22nd of June 2018 02:12:44 PM
Old 06-22-2018
I'd propose to drop the "command substitution" $( ... ) as it's not needed in above construct.


If you have all target lines in a file, and you are sure they occur not more than once in the large file, you could try
Code:
if [ $(grep -cwf targetlines 1000linesfile) = $(wc -l < targetlines) ]
  then    echo "Found all"
   else    echo "Some missing"
   fi

To allow for duplicates of target lines, try $(grep -owf targetlines 1000linesfile | sort -u | wc -l) in the if construct above.
Last edited by RudiC; 06-22-2018 at 03:18 PM..
This User Gave Thanks to RudiC For This Post:
sumguy
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Help with selecting specific lines in a large file

Hello, I need to select the 3 lines above as well as below a search string, including the search string. I have been trying various combinations using sed command without any success. Can anuone help please. Thanking (2 Replies)
Discussion started by: tansha
2 Replies

2. Shell Programming and Scripting

Count specific character(s) very large file

I'm trying to count the number of 2 specific characters in a very large file. I'd like to avoid using gsub because its taking too long. I was thinking something like: awk '-F' { t += NF - 1 } END {print t}' infile > outfile which isn't working Any ideas would be great. (3 Replies)
Discussion started by: dcfargo
3 Replies

3. Shell Programming and Scripting

Searching a specific line in a large file

Hey All Can any one please suggest the procedure to search a part of line in a very large file in which log entries are entered with very high speed. i have trued with grep and egrep grep 'text text text' <file-name> egrep 'text text text' <file-name> here 'text text text' is... (4 Replies)
Discussion started by: NIMISH AGARWAL
4 Replies

4. UNIX for Dummies Questions & Answers

how to display specific lines of a specific file

are there any basic commands that can display lines 99 - 101 of the /etc/passwd file? I'm thinking use of head and tail, but I forget what numbers to use and where to put /etc/passwd in the command. (2 Replies)
Discussion started by: raidkridley
2 Replies

5. Shell Programming and Scripting

remove a specific line in a LARGE file

Hi guys, i have a really big file, and i want to remove a specific line. sed -i '5d' fileThis doesn't really work, it takes a lot of time... The whole script is supposed to remove every word containing less than 5 characters and currently looks like this: #!/bin/bash line="1"... (2 Replies)
Discussion started by: blubbiblubbkekz
2 Replies

6. Shell Programming and Scripting

[bash help]Adding multiple lines of text into a specific spot into a text file

I am attempting to insert multiple lines of text into a specific place in a text file based on the lines above or below it. For example, Here is a portion of a zone file. IN NS ns1.domain.tld. IN NS ns2.domain.tld. IN ... (2 Replies)
Discussion started by: cdn_humbucker
2 Replies

7. Shell Programming and Scripting

Cutting specific line of a file by checking condition

testfile.csv 0","1125209",,"689202CBx18888",,"49",,,"NONMC",,,,,"01112010",,,,,,,"MTM- "1","",,"689202ABx19005",,"49",,,"NONMC",,,,,"01072010",,,,,,,"MTM- testfile.csv looks like above format if the second column is null then get 23rd column and store in a different varible .. add all the... (1 Reply)
Discussion started by: mgant
1 Replies

8. UNIX for Dummies Questions & Answers

Quick UNIX command to display specific lines in the middle of a file from/to specific word

This could be a really dummy question. I have a log text file. What unix command to extract line from specific string to another specific string. Is it something similar to?: more +/"string" file_name Thanks (4 Replies)
Discussion started by: aku
4 Replies

9. Shell Programming and Scripting

Adding lines to a large file

Hello, I have a relatively large text file (25,000K) consisting of records of data. For each record, I need to create a new line based on what is already there. Every record has a block that looks like, M END > <ID> 1 > <SOURCE> KEGG > <SOURCE_ID> C00002 > <NAME> ATP;... (4 Replies)
Discussion started by: LMHmedchem
4 Replies

10. Shell Programming and Scripting

Bash: copying lines with specific character to files with same name as copied line.

I am trying to make my script as simple as a possible but, I am not sure if the way I am approaching is necessarily the most efficient or effective it can be. What I am mainly trying to fix is a for loop to remove a string from the specified files and within this loop I am trying to copy the lines... (2 Replies)
Discussion started by: Allie_gastrator
2 Replies

LEARN ABOUT OPENSOLARIS

dfstab

dfstab(4)							   File Formats 							 dfstab(4)

NAME

       dfstab - file containing commands for sharing resources across a network

DESCRIPTION

       dfstab  resides	in  directory /etc/dfs and contains commands for sharing resources across a network. dfstab gives a system administrator a
       uniform method of controlling the automatic sharing of local resources.

       Each line of the dfstab file consists of a share(1M) command. The dfstab file can be read by the  shell	to  share  all	resources.  System
       administrators can also prepare their own shell scripts to execute particular lines from dfstab.

       The contents of dfstab put into effect when the command shown below is run. See svcadm(1M).

	 /usr/sbin/svcadm enable network/nfs/server

SEE ALSO

       share(1M), shareall(1M), sharemgr(1M), svcadm(1M)

NOTES

       Do  not modify this file directly. This file is reconstructed and only maintained for backwards compatibility. Configuration lines could be
       lost.

       Use the sharemgr(1M) command for all share management.

SunOS 5.11							    15 Aug 2008 							 dfstab(4)
All times are GMT -4. The time now is 11:54 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy