Sponsored Content
Full Discussion: Help speeding up script
Top Forums Shell Programming and Scripting Help speeding up script Post 302942331 by sea on Monday 27th of April 2015 06:13:24 PM
Old 04-27-2015
Hi

You could use sort directly instead of cat.
Cant say anything else without the content of the Tax_Provision_sample.dat.

hth
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

speeding up the compilation on SUN Solaris environment

Dear friends, Please let me know how do I increase the speed of my compilation in SUN Solaris environment. actually I have many subfolders which contains .cc files. when I compile makefile at the root it will take much time to compile all the subfolders and generates object(.o) files. Can... (2 Replies)
Discussion started by: swamymns
2 Replies

2. Shell Programming and Scripting

Speeding up processing a file

Hi guys, I'm hoping you can help me here. I've knocked up a script that looks at a (huge) log file, and pulls from each line the hour of each transaction and how long each transaction took. The data is stored sequentially as: 07:01 blah blah blah 12456 blah 07:03 blah blah blah 234 blah... (4 Replies)
Discussion started by: dlam
4 Replies

3. UNIX for Dummies Questions & Answers

Speeding up a Shell Script (find, grep and a for loop)

Hi all, I'm having some trouble with a shell script that I have put together to search our web pages for links to PDFs. The first thing I did was: ls -R | grep .pdf > /tmp/dave_pdfs.outWhich generates a list of all of the PDFs on the server. For the sake of arguement, say it looks like... (8 Replies)
Discussion started by: Dave Stockdale
8 Replies

4. UNIX for Dummies Questions & Answers

Speeding/Optimizing GREP search on CSV files

Hi all, I have problem with searching hundreds of CSV files, the problem is that search is lasting too long (over 5min). Csv files are "," delimited, and have 30 fields each line, but I always grep same 4 fields - so is there a way to grep just those 4 fields to speed-up search. Example:... (11 Replies)
Discussion started by: Whit3H0rse
11 Replies

5. Shell Programming and Scripting

speeding up bash script with "while read line"

Hello everybody, I'm still slowly treading my way into bash scripting (without any prior programming experience) and hence my code is mostly what some might call "creative" if they meant well :D I have created a script that serves its purpose but it does so very slowly, since it needs to work... (4 Replies)
Discussion started by: origamisven
4 Replies

6. Shell Programming and Scripting

Speeding up search and replace in a for loop

Hello, I am using sed in a for loop to replace text in a 100MB file. I have about 55,000 entries to convert in a csv file with two entries per line. The following script works to search file.txt for the first field from conversion.csv and then replace it with the second field. While it works fine,... (15 Replies)
Discussion started by: pbluescript
15 Replies

7. Shell Programming and Scripting

Speeding up substitutions

Hi all, I have a lookup table from which I am looking up values (from col1) and replacing them by corresponding values (from col2) in another file. lookup file a,b c,d So just replace a by b, and replace c by d. mainfile a,fvvgeggsegg,dvs a,fgeggefddddddddddg... (7 Replies)
Discussion started by: senhia83
7 Replies

8. Shell Programming and Scripting

Speeding up shell script with grep

HI Guys hoping some one can help I have two files on both containing uk phone numbers master is a file which has been collated over a few years ad currently contains around 4 million numbers new is a file which also contains 4 million number i need to split new nto two separate files... (4 Replies)
Discussion started by: dunryc
4 Replies

9. Shell Programming and Scripting

Help 'speeding' up this 'parsing' script - taking 24+ hours to run

Hi, I've written a ksh script that read a file and parse/filter/format each line. The script runs as expected but it runs for 24+ hours for a file that has 2million lines. And sometimes, the input file has 10million lines which means it can be running for more than 2 days and still not finish.... (9 Replies)
Discussion started by: newbie_01
9 Replies

10. Shell Programming and Scripting

Help with speeding up my working script to take less time - how to use more CPU usage for a script

Hello experts, we have input files with 700K lines each (one generated for every hour). and we need to convert them as below and move them to another directory once. Sample INPUT:- # cat test1 1559205600000,8474,NormalizedPortInfo,PctDiscards,0.0,Interface,BG-CTA-AX1.test.com,Vl111... (7 Replies)
Discussion started by: prvnrk
7 Replies
dbm(3UCB)					     SunOS/BSD Compatibility Library Functions						 dbm(3UCB)

NAME
dbm, dbminit, dbmclose, fetch, store, delete, firstkey, nextkey - data base subroutines SYNOPSIS
/usr/ucb/cc [ flag ... ] file ... -ldbm #include <dbm.h> typedef struct { char *dptr; int dsize; }datum; int dbminit(file); char *file; int dbmclose(); datum fetch( key); datum key; int store( key, dat); datum key, dat; int delete(key); datum key; datum firstkey() datum nextkey(key); datum key; DESCRIPTION
The dbm() library has been superseded by ndbm (see ndbm(3C)). These functions maintain key/content pairs in a data base. The functions will handle very large (a billion blocks) databases and will access a keyed item in one or two file system accesses. key/dat and their content are described by the datum typedef. A datum specifies a string of dsize bytes pointed to by dptr. Arbitrary binary data, as well as normal ASCII strings, are allowed. The data base is stored in two files. One file is a directory containing a bit map and has .dir as its suffix. The second file contains all data and has .pag as its suffix. Before a database can be accessed, it must be opened by dbminit(). At the time of this call, the files file.dir and file.pag must exist. An empty database is created by creating zero-length .dir and .pag files. A database may be closed by calling dbmclose(). You must close a database before opening a new one. Once open, the data stored under a key is accessed by fetch() and data is placed under a key by store. A key (and its associated contents) is deleted by delete(). A linear pass through all keys in a database may be made, in an (apparently) random order, by use of firstkey() and nextkey(). firstkey() will return the first key in the database. With any key nextkey() will return the next key in the database. This code will traverse the data base: for (key = firstkey; key.dptr != NULL; key = nextkey(key)) RETURN VALUES
All functions that return an int indicate errors with negative values. A zero return indicates no error. Routines that return a datum indi- cate errors with a NULL(0) dptr. SEE ALSO
ar(1), cat(1), cp(1), tar(1), ndbm(3C) NOTES
Use of these interfaces should be restricted to only applications written on BSD platforms. Use of these interfaces with any of the system libraries or in multi-thread applications is unsupported. The .pag file will contain holes so that its apparent size may be larger than its actual content. Older versions of the UNIX operating sys- tem may create real file blocks for these holes when touched. These files cannot be copied by normal means ( cp(1), cat(1), tar(1), ar(1)) without filling in the holes. dptr pointers returned by these subroutines point into static storage that is changed by subsequent calls. The sum of the sizes of a key/content pair must not exceed the internal block size (currently 1024 bytes). Moreover all key/content pairs that hash together must fit on a single block. store will return an error in the event that a disk block fills with inseparable data. delete() does not physically reclaim file space, although it does make it available for reuse. The order of keys presented by firstkey() and nextkey() depends on a hashing function, not on anything interesting. There are no interlocks and no reliable cache flushing; thus concurrent updating and reading is risky. The database files (file.dir and file.pag) are binary and are architecture-specific (for example, they depend on the architecture's byte order.) These files are not guaranteed to be portable across architectures. SunOS 5.10 20 Feb 1997 dbm(3UCB)
All times are GMT -4. The time now is 06:15 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy