Sponsored Content
Top Forums Shell Programming and Scripting Finiding Files with Perl or awk? Post 302944476 by Corona688 on Tuesday 19th of May 2015 03:04:53 PM
Old 05-19-2015
Quote:
Originally Posted by Michael Stora
Even a 2-3x improvement would help tramendously.
Computers do not work that way.

A slow system call is slow in any language, and the slower it is, the less there is to be gained by 'optimizing' it.

Suppose your program is spending 98% of its time waiting for NFS and 2% of its time actually running. If you find a 200% faster proram, it will be spending 1% of its time actually running and 99% of its time waiting on NFS with theoretical a speed gain of 1% and a realistic speed gain of absolutely zip.

This is also why you can't turbo charge a slow disk with a fast program. No matter how fast your program is, the underlying I/O can't actually move faster.

You might be able to parallelize it, but only to a point.

Perhaps your network connection or NFS can be fine-tuned? That's beyond my expertise, though.

P.S. The find command is not 'known to be slow', certainly not slower than any other file tree walker I know. If you don't understand why it's 'slow' when used on huge file trees, you don't actually know what it's doing.

Last edited by Corona688; 05-19-2015 at 04:12 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Perl or awk/egrep from big files??

Hi experts. In one thread i have asked you how to grep the string from the below sample file- Unfortunately the script did not gave proper output (it missed many strings). It happened may be i did gave you the proper contents of the file That was the script- "$ perl -00nle'print join... (13 Replies)
Discussion started by: thepurple
13 Replies

2. Shell Programming and Scripting

Merge files of differrent size with one field common in both files using awk

hi, i am facing a problem in merging two files using awk, the problem is as stated below, file1: A|B|C|D|E|F|G|H|I|1 M|N|O|P|Q|R|S|T|U|2 AA|BB|CC|DD|EE|FF|GG|HH|II|1 .... .... .... file2 : 1|Mn|op|qr (2 Replies)
Discussion started by: shashi1982
2 Replies

3. Shell Programming and Scripting

perl script for listing files and mailing the all files

Hi, I am new to perl: I need to write perl script to list all the files present in directory and mail should be come to my inbox with all the files present in that directory. advanced thanks for valuable inputs. Thanks Prakash GR (1 Reply)
Discussion started by: prakash.gr
1 Replies

4. Shell Programming and Scripting

Finiding filenames with specific index string

Hi All, I have a file (Names.txt) and the contents of the file is give below. $ cat Names.txt FF313207008.txt FF223207007.txt FF143207006.txt FF372150600.txt FF063407005.txt FF063307005.txt $ From these given file names I want to find the files which has the 6th index value as 2. So... (5 Replies)
Discussion started by: krish_indus
5 Replies

5. Shell Programming and Scripting

Comparison and editing of files using awk.(And also a possible bug in awk for loop?)

I have two files which I would like to compare and then manipulate in a way. File1: pictures.txt 1.1 1.3 dance.txt 1.2 1.4 treehouse.txt 1.3 1.5 File2: pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244 dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2... (1 Reply)
Discussion started by: linuxkid
1 Replies

6. Shell Programming and Scripting

Compare two files and set a third one using awk or perl

Folks I need your help cuz I've a file with 100,000 records that need to be compared against a passwd file (300) and then create a third one with the data in the first one and the passwd from the second one set in it. The format of the first file is: host xxxxxx "" 0,0 Closed control00/... (4 Replies)
Discussion started by: ranrodrig
4 Replies

7. Shell Programming and Scripting

Apply 'awk' to all files in a directory or individual files from a command line

Hi All, I am using the awk command to replace ',' by '\t' (tabs) in a csv file. I would like to apply this to all .csv files in a directory and create .txt files with the tabs. How would I do this in a script? I have the following script called "csvtabs": awk 'BEGIN { FS... (4 Replies)
Discussion started by: ScKaSx
4 Replies

8. Shell Programming and Scripting

Compare intervals (columns) from two files (awk, grep, Perl?)

Hi dear users, I need to compare numeric columns in two files. These files have the following structure. K.txt (4 columns) A001 chr21 9805831 9846011 A002 chr21 9806202 9846263 A003 chr21 9887188 9988593 A003 chr21 9887188 ... (2 Replies)
Discussion started by: jcvivar
2 Replies

9. Shell Programming and Scripting

awk command to compare a file with set of files in a directory using 'awk'

Hi, I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files. To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the... (10 Replies)
Discussion started by: anandek
10 Replies

10. Shell Programming and Scripting

Awk or Perl - to selectively merge two files.

I have two files, these have to be selectively merged into two other files. In addition there will require to be a edit to the last field, where the date format is changed. The first file is a csv file with around 300k lines the data is now nearly 20 years old and I have been asked to move this... (7 Replies)
Discussion started by: gull04
7 Replies
firmdl3(1)							  brickOS Utility							firmdl3(1)

NAME
firmdl3 - A firmware downloader for the RCX. Version 3.0. SYNOPSIS
firmdl3 [options] srecfile DESCRIPTION
This manual page documents the firmdl3 kernel download command. firmdl3 is a program which is used to download the brickOS kernel (firmware). firmdl3 Supports single and quad rate downloading (quad-rate is default). Quad-rate downloading is a feature that significantly reduces firmware download times. It is implemented on top of a fast serial protocol that has a 2x baud rate with no complements or parity bits. The fast protocol is enabled by a short firmware stub that reconfigures some control registers and tweaks some ROM program state. Once the firmware stub is downloaded and executed, the ROM responds to the fast serial protocol, which the downloader then uses to transfer the actual firmware file. Because the initial firmware stub is small, all but the most trivial programs see a download time improvement when using quad-speed downloading. The caveat to using quad-speed downloading is its sensitivity to lighting conditions. Getting quad-speed downloading to work right might require some trial and error. In addition to avoiding harsh lighting conditions, it is important to place the RCX around 4-12 inches from the IR tower. OPTIONS
This program follows the usual GNU command line syntax, with long options starting with two dashes (`-'). The options supported by firmdl3 are: -h, --help Show built-in help (summery of options) -f, --fast Use 'fast' (4x) download algorithm (default) -s, --slow Use 'slow' (1x) download algorithm (Use if experiencing download problems) --tty={ttydevice} Specify serial serial tty where IR tower is connected. NOTE: The environment variable RCXTTY may be used in place of -tty --debug Enable debugging. (Mostly displays raw bytes). ENVIRONMENT VARIABLES
firmdl3 will respond to the environment variable RCXTTY. This should be set to the serial device to which the IR tower is connected. Use of RCXTTY is equivalent the using the -tty command-line option. EXAMPLES
firmdl3 is a simple command to use. The following examples show most of the uses. Ex1: download the brickOS kernal to ttyS0. $ firmdl3 -tty=/dev/ttyS0 /usr/lib/brickos/brickOS.srec Transferring "Fast Download Image" to RCX... 100% Transferring "/usr/lib/brickos/brickOS.srec" to RCX... 100% $ Ex2: download the brickOS kernal to ttyS1 (set env var 1st). $ export RCXTTY=/dev/ttyS1 $ firmdl3 /usr/lib/brickos/brickOS.srec Transferring "Fast Download Image" to RCX... 100% Transferring "/usr/lib/brickos/brickOS.srec" to RCX... 100% $ Ex3: download the brickOS kernal using slow protocol (assumes RCXTTY is already set). $ firmdl3 --slow /usr/lib/brickos/brickOS.srec Transferring "/usr/lib/brickos/brickOS.srec" to RCX... 100% $ NOTE: the missing indication of the "Fast Download Image" being sent. We'll leave the experiment of running firmdl3 with --debug set as an exercise for you our reader... (You really interested in seeing all the bytes fly by? ;-) ERROR MESSAGES
(excerpt from brickOS HOWTO) delete firmware failed Because of a limitation in the Lego firmware, firmdl3 cannot delete LegOS. Instead, before downloading the new firmware, you must first stop the old LegOS program, and then hold down the Prgm button while hitting the On/Off switch. This will delete the old LegOS firmware, allowing a new download. Alternately, if you see no green light on the IR box and get this error message, it may indicate you have a dead battery in the IR tower. /dev/ttyS0: Permission denied, com1: Permission denied, or a similar message. The first part of the error message is the output port that firmdl3 is trying to use. This can mean two things. Either you have specified the wrong device or you have specified the correct device but do not have permission to access it. If it is an access problem, speak to your sysadmin. If not, you can use the --tty argument to specify another device. For example, to specify the standard device under linux, try 'firmdl3 --tty /dev/ttyS0'. Under cygwin, you probably probably want to use 'firmdl3 --tty com1'. If you discover that this is the case, you may wish to recompile firmdl3 to make your setting the default. /dev/ttyS0: Input/output error or a similar message. You forgot to plug the tower in, most likely. unlock firmware failed This indicates that there was a slight corruption in the download. Clear brickOS and download it again. no response from RCX This error should be called "you forgot to turn on your RCX, moron." If I hadn't done it about 10 million times, I'd submit the patch to change the error myself. If firmdl3 gives you repeated problems, you may want to try using the --slow argument, which will make your download much slower but can correct for certain errors caused by ambient light. SEE ALSO
dll(1) AUTHOR
Laurent Demailly, Allen Martin, Markus Noga, Gavin Smyth, and Luis Villa all contributed something to some version of firmdl3. firmdl3 was written by: Kekoa Proudfoot <kekoa@graphics.stanford.edu> - 10/13/99 This manual page was written by Stephen M Moraco <stephen@debian.org>, for the Debian GNU/Linux system (but may be used by others). brickOS January 6, 2001 firmdl3(1)
All times are GMT -4. The time now is 01:10 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy