Sponsored Content
Top Forums Shell Programming and Scripting Very big text file - Too slow! Post 302515277 by Corona688 on Tuesday 19th of April 2011 04:38:31 PM
Old 04-19-2011
Quote:
In var $RAND a random integer 1 to total_lines_of_file is stored using the $RANDOM variable of bash. So with sed $RAND i traverse to that line.
I figured that much out, but I have no idea why you're using sed to snip out random lines. You could even get the same line more than once that way!
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to view a big file(143M big)

1 . Thanks everyone who read the post first. 2 . I have a log file which size is 143M , I can not use vi open it .I can not use xedit open it too. How to view it ? If I want to view 200-300 ,how can I implement it 3 . Thanks (3 Replies)
Discussion started by: chenhao_no1
3 Replies

2. AIX

How to send big files over slow network?

Hi, I am trying to send oracle archives over WAN and it is taking hell a lot of time. To reduce the time, I tried to gzip the files and send over to the other side. That seems to reduce the time. Does anybody have experienced this kind of problem and any possible ways to reduce the time. ... (1 Reply)
Discussion started by: giribt
1 Replies

3. Shell Programming and Scripting

Cut big text file into 2

I have a big text file. I want to cut it into 2 pieces at known point or I know the pattern of the contents from where it can separate the files. Is there any quick command/solution? (4 Replies)
Discussion started by: sandy221
4 Replies

4. UNIX for Dummies Questions & Answers

How to slow down text output?

I found some ascii art that is animated (vt100) and would like to view it properly. However, when I try using 'cat', the file is done almost the instant I press enter. How can I view the file in a slower fashion (similar to the days of 2400baud, for example)? (2 Replies)
Discussion started by: Fangs McWolf
2 Replies

5. UNIX for Dummies Questions & Answers

How big is too big a config.log file?

I have a 5000 line config.log file with several "maybe" errors. Any reccomendations on finding solvable problems? (2 Replies)
Discussion started by: NeedLotsofHelp
2 Replies

6. Shell Programming and Scripting

Helping in parsing subset of text from a big results file

Hi All, I need some help to effectively parse out a subset of results from a big results file. Below is an example of the text file. Each block that I need to parse starts with "reading sequence file 10.codon" (next block starts with another number) and ends with **p-Value(s)**. I have given... (1 Reply)
Discussion started by: Lucky Ali
1 Replies

7. UNIX for Advanced & Expert Users

sed working slow on big files

HI Experts , I'm using the following code to remove spaces appearing at the end of the file. sed "s/*$//g" <filename> > <new_filename> mv <new_filename> <filename> this is working fine for volumes upto 20-25 GB. for the bigger files it is taking more time that it is required... (5 Replies)
Discussion started by: sumoka
5 Replies

8. UNIX and Linux Applications

tabbed text editor without big libraries

I am looking for a tabbed text editor without a big library like gnome, kde, and gtk, I know about gedit, kate with extensions, geany, and bluefish. I would prefer it to be like gedit and be really light weight. So if anyone knows of a text editor that doesn't require those big libraries please let... (3 Replies)
Discussion started by: cokedude
3 Replies

9. Shell Programming and Scripting

Improve script - slow process with big files

Gents, Please can u help me to improve this script to be more faster, it works perfectly but for big files take a lot time to end the job.. I see the problem is in the step (while) and in this part the script takes a lot time.. Please if you can find a best way to do will be great. ... (13 Replies)
Discussion started by: jiam912
13 Replies
FLOCK(1)							  H. Peter Anvin							  FLOCK(1)

NAME
flock - Manage locks from shell scripts SYNOPSIS
flock [-sxon] [-w timeout] lockfile [-c] command... flock [-sxon] [-w timeout] lockdir [-c] command... flock [-sxun] [-w timeout] fd DESCRIPTION
This utility manages flock(2) locks from within shell scripts or the command line. The first and second forms wraps the lock around the executing a command, in a manner similar to su(1) or newgrp(1). It locks a specified file or directory, which is created (assuming appropriate permissions), if it does not already exist. The third form is convenient inside shell scripts, and is usually used the following manner: ( flock -n 9 # ... commands executed under lock ... ) 9>/var/lock/mylockfile The mode used to open the file doesn't matter to flock; using > or >> allows the lockfile to be created if it does not already exist, how- ever, write permission is required; using < requires that the file already exists but only read permission is required. By default, if the lock cannot be immediately acquired, flock waits until the lock is available. OPTIONS
-s, --shared Obtain a shared lock, sometimes called a read lock. -x, -e, --exclusive Obtain an exclusive lock, sometimes called a write lock. This is the default. -u, --unlock Drop a lock. This is usually not required, since a lock is automatically dropped when the file is closed. However, it may be required in special cases, for example if the enclosed command group may have forked a background process which should not be hold- ing the lock. -n, --nb, --nonblock Fail (with an exit code of 1) rather than wait if the lock cannot be immediately acquired. -w, --wait, --timeout seconds Fail (with an exit code of 1) if the lock cannot be acquired within seconds seconds. Decimal fractional values are allowed. -o, --close Close the file descriptor on which the lock is held before executing command. This is useful if command spawns a child process which should not be holding the lock. -c, --command command Pass a single command to the shell with -c. -h, --help Print a help message. AUTHOR
Written by H. Peter Anvin <hpa@zytor.com>. COPYRIGHT
Copyright (C) 2003-2006 H. Peter Anvin. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICU- LAR PURPOSE. SEE ALSO
flock(2) AVAILABILITY
The flock command is part of the util-linux package and is available from ftp://ftp.kernel.org/pub/linux/utils/util-linux/. flock utility 4 Feb 2006 FLOCK(1)
All times are GMT -4. The time now is 02:13 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy