Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Bash : Checking Large file for specific lines
Top Forums Shell Programming and Scripting Bash : Checking Large file for specific lines Post 303019077 by RudiC on Friday 22nd of June 2018 02:12:44 PM
Old 06-22-2018
I'd propose to drop the "command substitution" $( ... ) as it's not needed in above construct.


If you have all target lines in a file, and you are sure they occur not more than once in the large file, you could try
Code:
if [ $(grep -cwf targetlines 1000linesfile) = $(wc -l < targetlines) ]
  then    echo "Found all"
   else    echo "Some missing"
   fi

To allow for duplicates of target lines, try $(grep -owf targetlines 1000linesfile | sort -u | wc -l) in the if construct above.
Last edited by RudiC; 06-22-2018 at 03:18 PM..
This User Gave Thanks to RudiC For This Post:
sumguy
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Help with selecting specific lines in a large file

Hello, I need to select the 3 lines above as well as below a search string, including the search string. I have been trying various combinations using sed command without any success. Can anuone help please. Thanking (2 Replies)
Discussion started by: tansha
2 Replies

2. Shell Programming and Scripting

Count specific character(s) very large file

I'm trying to count the number of 2 specific characters in a very large file. I'd like to avoid using gsub because its taking too long. I was thinking something like: awk '-F' { t += NF - 1 } END {print t}' infile > outfile which isn't working Any ideas would be great. (3 Replies)
Discussion started by: dcfargo
3 Replies

3. Shell Programming and Scripting

Searching a specific line in a large file

Hey All Can any one please suggest the procedure to search a part of line in a very large file in which log entries are entered with very high speed. i have trued with grep and egrep grep 'text text text' <file-name> egrep 'text text text' <file-name> here 'text text text' is... (4 Replies)
Discussion started by: NIMISH AGARWAL
4 Replies

4. UNIX for Dummies Questions & Answers

how to display specific lines of a specific file

are there any basic commands that can display lines 99 - 101 of the /etc/passwd file? I'm thinking use of head and tail, but I forget what numbers to use and where to put /etc/passwd in the command. (2 Replies)
Discussion started by: raidkridley
2 Replies

5. Shell Programming and Scripting

remove a specific line in a LARGE file

Hi guys, i have a really big file, and i want to remove a specific line. sed -i '5d' fileThis doesn't really work, it takes a lot of time... The whole script is supposed to remove every word containing less than 5 characters and currently looks like this: #!/bin/bash line="1"... (2 Replies)
Discussion started by: blubbiblubbkekz
2 Replies

6. Shell Programming and Scripting

[bash help]Adding multiple lines of text into a specific spot into a text file

I am attempting to insert multiple lines of text into a specific place in a text file based on the lines above or below it. For example, Here is a portion of a zone file. IN NS ns1.domain.tld. IN NS ns2.domain.tld. IN ... (2 Replies)
Discussion started by: cdn_humbucker
2 Replies

7. Shell Programming and Scripting

Cutting specific line of a file by checking condition

testfile.csv 0","1125209",,"689202CBx18888",,"49",,,"NONMC",,,,,"01112010",,,,,,,"MTM- "1","",,"689202ABx19005",,"49",,,"NONMC",,,,,"01072010",,,,,,,"MTM- testfile.csv looks like above format if the second column is null then get 23rd column and store in a different varible .. add all the... (1 Reply)
Discussion started by: mgant
1 Replies

8. UNIX for Dummies Questions & Answers

Quick UNIX command to display specific lines in the middle of a file from/to specific word

This could be a really dummy question. I have a log text file. What unix command to extract line from specific string to another specific string. Is it something similar to?: more +/"string" file_name Thanks (4 Replies)
Discussion started by: aku
4 Replies

9. Shell Programming and Scripting

Adding lines to a large file

Hello, I have a relatively large text file (25,000K) consisting of records of data. For each record, I need to create a new line based on what is already there. Every record has a block that looks like, M END > <ID> 1 > <SOURCE> KEGG > <SOURCE_ID> C00002 > <NAME> ATP;... (4 Replies)
Discussion started by: LMHmedchem
4 Replies

10. Shell Programming and Scripting

Bash: copying lines with specific character to files with same name as copied line.

I am trying to make my script as simple as a possible but, I am not sure if the way I am approaching is necessarily the most efficient or effective it can be. What I am mainly trying to fix is a for loop to remove a string from the specified files and within this loop I am trying to copy the lines... (2 Replies)
Discussion started by: Allie_gastrator
2 Replies

LEARN ABOUT SUSE

split

SPLIT(1)							   User Commands							  SPLIT(1)

NAME

       split - split a file into pieces

SYNOPSIS

       split [OPTION]... [INPUT [PREFIX]]

DESCRIPTION

       Output  fixed-size  pieces  of  INPUT to PREFIXaa, PREFIXab, ...; default size is 1000 lines, and default PREFIX is `x'.  With no INPUT, or
       when INPUT is -, read standard input.

       Mandatory arguments to long options are mandatory for short options too.

       -a, --suffix-length=N
	      use suffixes of length N (default 2)

       -b, --bytes=SIZE
	      put SIZE bytes per output file

       -C, --line-bytes=SIZE
	      put at most SIZE bytes of lines per output file

       -d, --numeric-suffixes
	      use numeric suffixes instead of alphabetic

       -l, --lines=NUMBER
	      put NUMBER lines per output file

       --verbose
	      print a diagnostic just before each output file is opened

       --help display this help and exit

       --version
	      output version information and exit

       SIZE may have a multiplier suffix: b 512, kB 1000, K 1024, MB 1000*1000, M 1024*1024, GB 1000*1000*1000, G 1024*1024*1024, and so on for T,
       P, E, Z, Y.

AUTHOR

       Written by Torbjorn Granlund and Richard M. Stallman.

REPORTING BUGS

       Report split bugs to bug-coreutils@gnu.org
       GNU coreutils home page: <http://www.gnu.org/software/coreutils/>
       General help using GNU software: <http://www.gnu.org/gethelp/>

COPYRIGHT

       Copyright (C) 2009 Free Software Foundation, Inc.  License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
       This is free software: you are free to change and redistribute it.  There is NO WARRANTY, to the extent permitted by law.

SEE ALSO

       The  full  documentation  for split is maintained as a Texinfo manual.  If the info and split programs are properly installed at your site,
       the command

	      info coreutils 'split invocation'

       should give you access to the complete manual.

GNU coreutils 7.1						     July 2010								  SPLIT(1)
All times are GMT -4. The time now is 01:03 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy