Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Bash : Checking Large file for specific lines

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Bash : Checking Large file for specific lines
# 1  
Old 06-22-2018
I'd propose to drop the "command substitution" $( ... ) as it's not needed in above construct.


If you have all target lines in a file, and you are sure they occur not more than once in the large file, you could try
Code:
if [ $(grep -cwf targetlines 1000linesfile) = $(wc -l < targetlines) ]
  then    echo "Found all"
   else    echo "Some missing"
   fi

To allow for duplicates of target lines, try $(grep -owf targetlines 1000linesfile | sort -u | wc -l) in the if construct above.
Last edited by RudiC; 06-22-2018 at 03:18 PM..
This User Gave Thanks to RudiC For This Post:
sumguy
# 2  
Old 06-22-2018
Thanks guys !!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash: copying lines with specific character to files with same name as copied line.

I am trying to make my script as simple as a possible but, I am not sure if the way I am approaching is necessarily the most efficient or effective it can be. What I am mainly trying to fix is a for loop to remove a string from the specified files and within this loop I am trying to copy the lines... (2 Replies)
Discussion started by: Allie_gastrator
2 Replies

2. Shell Programming and Scripting

Adding lines to a large file

Hello, I have a relatively large text file (25,000K) consisting of records of data. For each record, I need to create a new line based on what is already there. Every record has a block that looks like, M END > <ID> 1 > <SOURCE> KEGG > <SOURCE_ID> C00002 > <NAME> ATP;... (4 Replies)
Discussion started by: LMHmedchem
4 Replies

3. UNIX for Dummies Questions & Answers

Quick UNIX command to display specific lines in the middle of a file from/to specific word

This could be a really dummy question. I have a log text file. What unix command to extract line from specific string to another specific string. Is it something similar to?: more +/"string" file_name Thanks (4 Replies)
Discussion started by: aku
4 Replies

4. Shell Programming and Scripting

Cutting specific line of a file by checking condition

testfile.csv 0","1125209",,"689202CBx18888",,"49",,,"NONMC",,,,,"01112010",,,,,,,"MTM- "1","",,"689202ABx19005",,"49",,,"NONMC",,,,,"01072010",,,,,,,"MTM- testfile.csv looks like above format if the second column is null then get 23rd column and store in a different varible .. add all the... (1 Reply)
Discussion started by: mgant
1 Replies

5. Shell Programming and Scripting

[bash help]Adding multiple lines of text into a specific spot into a text file

I am attempting to insert multiple lines of text into a specific place in a text file based on the lines above or below it. For example, Here is a portion of a zone file. IN NS ns1.domain.tld. IN NS ns2.domain.tld. IN ... (2 Replies)
Discussion started by: cdn_humbucker
2 Replies

6. Shell Programming and Scripting

remove a specific line in a LARGE file

Hi guys, i have a really big file, and i want to remove a specific line. sed -i '5d' fileThis doesn't really work, it takes a lot of time... The whole script is supposed to remove every word containing less than 5 characters and currently looks like this: #!/bin/bash line="1"... (2 Replies)
Discussion started by: blubbiblubbkekz
2 Replies

7. UNIX for Dummies Questions & Answers

how to display specific lines of a specific file

are there any basic commands that can display lines 99 - 101 of the /etc/passwd file? I'm thinking use of head and tail, but I forget what numbers to use and where to put /etc/passwd in the command. (2 Replies)
Discussion started by: raidkridley
2 Replies

8. Shell Programming and Scripting

Searching a specific line in a large file

Hey All Can any one please suggest the procedure to search a part of line in a very large file in which log entries are entered with very high speed. i have trued with grep and egrep grep 'text text text' <file-name> egrep 'text text text' <file-name> here 'text text text' is... (4 Replies)
Discussion started by: NIMISH AGARWAL
4 Replies

9. Shell Programming and Scripting

Count specific character(s) very large file

I'm trying to count the number of 2 specific characters in a very large file. I'd like to avoid using gsub because its taking too long. I was thinking something like: awk '-F' { t += NF - 1 } END {print t}' infile > outfile which isn't working Any ideas would be great. (3 Replies)
Discussion started by: dcfargo
3 Replies

10. UNIX for Dummies Questions & Answers

Help with selecting specific lines in a large file

Hello, I need to select the 3 lines above as well as below a search string, including the search string. I have been trying various combinations using sed command without any success. Can anuone help please. Thanking (2 Replies)
Discussion started by: tansha
2 Replies
Login or Register to Ask a Question