Would like to know if there's a way if we can read a large file(like 1-2GB) using a shell command or a script and that can show the output as well of the contents of the file in the CLI?
Will appreciate your response. Thank you
There are hundreds of ways to read a text file in a shell. The definition of a text file varies from operating system to operating system. The ways to read a text file vary from shell to shell.
What operating system are you using?
What shell are you using?
Is the file you want to read a text file? If it contains binary data or one or more lines containing more than LINE_MAX bytes, your options for reading the file and displaying its contents in a readable form will be much more limited.
Of course, you could write your own CLI in C or C++ and make it read any type of file and print it in whatever format you desire.
The file is text file containing texts separated by new lines but only thing is file size is around more than a GIG, the OS is Red hat linux and shell is Bash, please do let us know how to approach on this?
If you want to display it on your screen with a new line from the file being displayed every second, try something like:
If you're a very fast reader with nimble fingers and you want to print the file to your terminal as quickly as possible stopping once-in-a-while when you see something interesting going by, use:
and while it is printing use ctl-s (i.e. hit the control key and the s key at the same time) to temporarily stop the output and ctl-q to restart the output.
If you want to display the output a page at a time, use:
or:
If you want to skip around in the file searching for certain patterns and you're used to vi editing commands, use:
(note that view is a link to vi that opens its file operands read-only instead of read-write).
If you're looking for certain patterns in your file and only want to see lines that contain that pattern, use grep.
If you're trying to do something else, use or write a utility, that does what you want... ... ...
This User Gave Thanks to Don Cragun For This Post:
Dear Users,
Appreciate your help if you could help me with splitting a large file > 1 million lines with sed or awk. below is the text in the file
input file.txt
scaffold1 928 929 C/T +
scaffold1 942 943 G/C +
scaffold1 959 960 C/T +... (6 Replies)
Hello All,
i am a newbie and need some help when reading a csv file in a bourne shell script. I want to read 10 lines, then wait for a minute and then do a reading of another 10 lines and so on in the same way. I want to do this till the end of file.
Any inputs are appreciated
... (3 Replies)
Hi-
I am trying to search a large file with a number of different search terms that are listed one per line in 3 different files. Most importantly I need to be able to do a case insensitive search.
I have tried just using egrep -f but it doesn't seam to be able to handle the -i option when... (3 Replies)
Hi, I hope the title does not scare people to look into this thread but it describes roughly what I'm trying to do. I need a solution in PHP.
I'm a programming beginner, so it might be that the approach to solve this, might be easier to solve with an other approach of someone else, so if you... (0 Replies)
# include <stdio.h>
# include <fcntl.h>
# include <stdlib.h>
# include <sys/stat.h>
int main(int argc, char *argv)
{
int fRead, fPadded, padVal;
int btRead;
int BUFFSIZE = 512;
char buff;
if (argc != 4)
{
printf ("Please provide all of the... (3 Replies)
Hello Gurus,
We are facing some performance issue in UNIX. If someone had faced such kind of issue in past please provide your suggestions on this .
Problem Definition:
/Few of load processes of our Finance Application are facing issue in UNIX when they uses a shell script having below... (19 Replies)
Hi,
I'm trying to figure out the best solution to the following problem, and I'm not
yet that much experienced like you. :-)
Basically I have to read a fairly large file, composed of "messages" , in order
to display all of them through an user interface (made with QT).
The messages that... (3 Replies)
Hi All,
I have shell script running on AIX 5.3 box. It has 7 to 8 "sed" commands piped(|) together. It has a an Xml file as its input which has many records internally. There are certain record which which have more than hundered tags.The script is taking a huge amount of time more than 1.5 hrs... (10 Replies)
I have one large file, after every 200 line i have to split the file and the add header and footer to each small file?
It is possible to add different header and footer to each file? (1 Reply)
Hi, Im trying to take a database backup. one of the files is 26 GB. I am using cp -pr to create a backup copy of the database. after the copying is complete, if i do du -hrs on the folders i saw a difference of 2GB.
The weird fact is that the BACKUP folder was 2 GB more than the original one!
... (1 Reply)