Verify if line number exist


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Verify if line number exist
# 1  
Old 12-15-2012
Verify if line number exist

Code:
os: linux/sunos

i'm running the following:

Code:
sed -n "2065696{p;q}" /tmp/file.txt

/tmp/file.txt is a very big file. it ranges from 400MB to 4GB in size. i want to know if line 2065696 exist. hence the reason for the above. but the problem is, this command is very slow. i have tried awk and grep and both take a while to come back to the prompt.

does any one have ideas on how to fasten this?
# 2  
Old 12-15-2012
Are all of the file records the same length? (the file has a fixed record length).
if so:
1. get the file size from ls -l (solaris) or stat command (Linux)
2. divide the file size by the fixed record size.
# if the result is >= 2065696 then then record exists.

If not you have to push the file pointer past 2065695
Code:
\n

, not have reached EOF. i.e., read that far into the file.

What are you trying to do? Look for something on line 2065696?
# 3  
Old 12-15-2012
Quote:
Originally Posted by jim mcnamara
Are all of the file records the same length? (the file has a fixed record length).
if so:
1. get the file size from ls -l (solaris) or stat command (Linux)
2. divide the file size by the fixed record size.
# if the result is >= 2065696 then then record exists.

If not you have to push the file pointer past 2065695
Code:
\n

, not have reached EOF. i.e., read that far into the file.

What are you trying to do? Look for something on line 2065696?
yes i'm looking for something on line 2065696.

and i dont understand the second part of what you said. can you please elaborate

also, each line in the file is not the same length.
# 4  
Old 12-15-2012
Because each line ends with \n, the newline character, you have to read from the start of the file, reading line by line, until you have read 2065696 lines. Or found that many newlines minus to be on line 2065696.

The fastest way to do that is to use something that is compiled to do just exactly that:
Code:
// findln.c
// usage:
//  ./findln  file_to_check
//  ./findln  < file_to_check
//  command some_file | ./findln
// compile [g]cc findln.c -o findln

#include <stdlib.h>
#include <stdio.h>
int main(int argc, char **argv)
{
    FILE *in=NULL;
    int i=0;
    if (argc==1)
    	  in=stdin; 
    else 
    	  in=fopen(argv[1], "r");
    
    if(in==NULL) {perror(""); exit(1);}
    char tmp[4096]={0x0};
    while(fgets(tmp, sizeof(tmp), in)!=NULL )
    {
         i++;
         if(i==2065696)
         {
              printf("%s", tmp);
              exit(0);  // no error because we found it
         }   
    }
    return 1;  // error because we did not get the line
}

One of the things you do in UNIX and windows is to write quick and dirty code for things like this. Does one thing: It just reads a file about as fast as possible, line by line.

If you want a faster solution this is how you do it. C, C++, or some other compiled language you know. awk is interpreted, sed is meant to do a lot of stuff, so they all are going to be somewhat slower than stupid code like the above.
# 5  
Old 01-29-2013
how about this

Code:
tail +2065696 /tmp/file.txt | head -1

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash to verify each line in input for specific pattern

In the bash below the out put of a process is written to input. What I am trying to do is read each line in the input and verify/check it for specific text (there are always 6 lines for each file and the specific text for each line is in the description). There will always be 6 lines in each... (5 Replies)
Discussion started by: cmccabe
5 Replies

2. OS X (Apple)

OSX verify username and password in one line

I'm writing a script that has the need to verify the current user's username and password. I'm not entirely sure how to do this. I've read some things on "dscl" but am not sure that's the correct route for me to go. The one condition i have is that i really need to have the verification happen... (4 Replies)
Discussion started by: TheDrizzle
4 Replies

3. Shell Programming and Scripting

Write $line number into textfile and read from line number

Hello everyone, I don't really know anything about scripting, but I have to manage to make this script, out of necessity. #!/bin/bash while read -r line; do #I'm reading from a big wordlist instructions using $line done Is there a way to automatically write the $line number the script... (4 Replies)
Discussion started by: bobylapointe
4 Replies

4. UNIX for Dummies Questions & Answers

how to count number of times each word exist in a file

I'm trying to count the number of times each word in the file exist for example if the file has: today I have a lot to write, but I will not go for it. The main thing is that today I am looking for a way to get each word in this file with a word count after it specifying that this word has... (4 Replies)
Discussion started by: shnkool
4 Replies

5. AIX

command to verify number of files

Guy's I have file system called /appspft has millions of files and I want to know exactly how many file under it Pls advice with that command to verify number of files .. (3 Replies)
Discussion started by: Mr.AIX
3 Replies

6. UNIX for Dummies Questions & Answers

How to read contents of a file from a given line number upto line number again specified by user

Hello Everyone. I am trying to display contains of a file from a specific line to a specific line(let say, from line number 3 to line number 5). For this I got the shell script as shown below: if ; then if ; then tail +$1 $3 | head -n $2 else ... (5 Replies)
Discussion started by: grc
5 Replies

7. Shell Programming and Scripting

get line number where only ) exist

hi guys, i have close bracket ")" accross the file but i want to take the line number where only once close bracket exist in the line. Please help me in SED command i should loop through the file. (4 Replies)
Discussion started by: Gopal_Engg
4 Replies

8. Shell Programming and Scripting

how to get the data from line number 1 to line number 100 of a file

Hi Everybody, I am trying to write a script that will get some perticuler data from a file and redirect to a file. My Question is, I have a Very huge file,In that file I have my required data is started from 25th line and it will ends in 100th line. I know the line numbers, I need to get all... (9 Replies)
Discussion started by: Anji
9 Replies

9. Shell Programming and Scripting

Adding a columnfrom a specifit line number to a specific line number

Hi, I have a huge file & I want to add a specific text in column. But I want to add this text from a specific line number to a specific line number & another text in to another range of line numbers. To be more specific: lets say my file has 1000 lines & 4 Columns. I want to add text "Hello"... (2 Replies)
Discussion started by: Ezy
2 Replies

10. Shell Programming and Scripting

Appending line number to each line and getting total number of lines

Hello, I need help in appending the line number of each line to the file and also to get the total number of lines. Can somebody please help me. I have a file say: abc def ccc ddd ffff The output should be: Instance1=abc Instance2=def Instance3=ccc Instance4=ddd Instance5=ffff ... (2 Replies)
Discussion started by: chiru_h
2 Replies
Login or Register to Ask a Question

Featured Tech Videos