Sponsored Content
Top Forums Shell Programming and Scripting awk help to make my work faster Post 302271357 by matrixmadhan on Thursday 25th of December 2008 02:48:04 AM
Old 12-25-2008
0.6 M don't seem to be a big number ( I guess )
load everything to memory with a map as

line_no => line_no_contents

then do a lookup with line_no, its done then Smilie
 

10 More Discussions You Might Find Interesting

1. Solaris

looking for different debugger for Solaris or to make sunstudio faster

im using the sunstudio but it is very slow , is there ant other GUI debugger for sun Solaris or at list some ways to make it faster ? im using to debug throw telnet connection connected to remote server thanks (0 Replies)
Discussion started by: umen
0 Replies

2. Shell Programming and Scripting

Can anyone make this script run faster?

One of our servers runs Solaris 8 and does not have "ls -lh" as a valid command. I wrote the following script to make the ls output easier to read and emulate "ls -lh" functionality. The script works, but it is slow when executed on a directory that contains a large number of files. Can anyone make... (10 Replies)
Discussion started by: shew01
10 Replies

3. Red Hat

Re:How to make the linux pc faster

Hi, Can any one help me out in solving the problem i have a linux database server it is tooo slow that i am unable to open even the terminial is there any solution to get rid of this problem.How to make this server faster. Thanks & Regards Venky (0 Replies)
Discussion started by: venky_vemuri
0 Replies

4. Shell Programming and Scripting

How to make copy work faster

I am trying to copy a folder which contains a list of C executables. It takes 2 mins for completion,where as the entire script takes only 3 more minutes for other process. Is there a way to copy the folder faster so that the performance of the script will improve? (2 Replies)
Discussion started by: prasperl
2 Replies

5. Shell Programming and Scripting

Running rename command on large files and make it faster

Hi All, I have some 80,000 files in a directory which I need to rename. Below is the command which I am currently running and it seems, it is taking fore ever to run this command. This command seems too slow. Is there any way to speed up the command. I have have GNU Parallel installed on my... (6 Replies)
Discussion started by: shoaibjameel123
6 Replies

6. Shell Programming and Scripting

Make script faster

Hi all, In bash scripting, I use to read files: cat $file | while read line; do ... doneHowever, it's a very slow way to read file line by line. E.g. In a file that has 3 columns, and less than 400 rows, like this: I run next script: cat $line | while read line; do ## Reads each... (10 Replies)
Discussion started by: AlbertGM
10 Replies

7. Shell Programming and Scripting

awk changes to make it faster

I have script like below, who is picking number from one file and and searching in another file, and printing output. Bu is is very slow to be run on huge file.can we modify it with awk #! /bin/ksh while read line1 do echo "$line1" a=`echo $line1` if then echo "$num" cat file1|nawk... (6 Replies)
Discussion started by: mirwasim
6 Replies

8. Shell Programming and Scripting

How to make awk command faster?

I have the below command which is referring a large file and it is taking 3 hours to run. Can something be done to make this command faster. awk -F ',' '{OFS=","}{ if ($13 == "9999") print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12 }' ${NLAP_TEMP}/hist1.out|sort -T ${NLAP_TEMP} |uniq>... (13 Replies)
Discussion started by: Peu Mukherjee
13 Replies

9. Shell Programming and Scripting

How to make awk command faster for large amount of data?

I have nginx web server logs with all requests that were made and I'm filtering them by date and time. Each line has the following structure: 127.0.0.1 - xyz.com GET 123.ts HTTP/1.1 (200) 0.000 s 3182 CoreMedia/1.0.0.15F79 (iPhone; U; CPU OS 11_4 like Mac OS X; pt_br) These text files are... (21 Replies)
Discussion started by: brenoasrm
21 Replies

10. Shell Programming and Scripting

How to make faster loop in multiple directories?

Hello, I am under Ubuntu 18.04 Bionic. I have one shell script run.sh (which is out of my topic) to run files under multiple directories and one file to control all processes running under those directories (control.sh). I set a cronjob task to check each of them with two minutes of intervals.... (3 Replies)
Discussion started by: baris35
3 Replies
CSPLIT(1P)						     POSIX Programmer's Manual							CSPLIT(1P)

PROLOG
This manual page is part of the POSIX Programmer's Manual. The Linux implementation of this interface may differ (consult the correspond- ing Linux manual page for details of Linux behavior), or the interface may not be implemented on Linux. NAME
csplit - split files based on context SYNOPSIS
csplit [-ks][-f prefix][-n number] file arg1 ...argn DESCRIPTION
The csplit utility shall read the file named by the file operand, write all or part of that file into other files as directed by the arg operands, and write the sizes of the files. OPTIONS
The csplit utility shall conform to the Base Definitions volume of IEEE Std 1003.1-2001, Section 12.2, Utility Syntax Guidelines. The following options shall be supported: -f prefix Name the created files prefix 00, prefix 01, ..., prefixn. The default is xx00 ... xx n. If the prefix argument would create a filename exceeding {NAME_MAX} bytes, an error shall result, csplit shall exit with a diagnostic message, and no files shall be cre- ated. -k Leave previously created files intact. By default, csplit shall remove created files if an error occurs. -n number Use number decimal digits to form filenames for the file pieces. The default shall be 2. -s Suppress the output of file size messages. OPERANDS
The following operands shall be supported: file The pathname of a text file to be split. If file is '-', the standard input shall be used. The operands arg1 ... argn can be a combination of the following: /rexp/[offset] A file shall be created using the content of the lines from the current line up to, but not including, the line that results from the evaluation of the regular expression with offset, if any, applied. The regular expression rexp shall follow the rules for basic regular expressions described in the Base Definitions volume of IEEE Std 1003.1-2001, Section 9.3, Basic Regular Expressions. The application shall use the sequence "/" to specify a slash character within the rexp. The optional offset shall be a positive or negative integer value representing a number of lines. A positive integer value can be preceded by '+' . If the selection of lines from an offset expression of this type would create a file with zero lines, or one with greater than the number of lines left in the input file, the results are unspecified. After the section is created, the current line shall be set to the line that results from the evaluation of the regular expression with any offset applied. If the current line is the first line in the file and a regular expression operation has not yet been performed, the pattern match of rexp shall be applied from the current line to the end of the file. Otherwise, the pattern match of rexp shall be applied from the line following the current line to the end of the file. %rexp%[offset] Equivalent to /rexp/[offset], except that no file shall be created for the selected section of the input file. The application shall use the sequence "\%" to specify a percent-sign character within the rexp. line_no Create a file from the current line up to (but not including) the line number line_no. Lines in the file shall be numbered starting at one. The current line becomes line_no. {num} Repeat operand. This operand can follow any of the operands described previously. If it follows a rexp type operand, that operand shall be applied num more times. If it follows a line_no operand, the file shall be split every line_no lines, num times, from that point. An error shall be reported if an operand does not reference a line between the current position and the end of the file. STDIN
See the INPUT FILES section. INPUT FILES
The input file shall be a text file. ENVIRONMENT VARIABLES
The following environment variables shall affect the execution of csplit: LANG Provide a default value for the internationalization variables that are unset or null. (See the Base Definitions volume of IEEE Std 1003.1-2001, Section 8.2, Internationalization Variables for the precedence of internationalization variables used to determine the values of locale categories.) LC_ALL If set to a non-empty string value, override the values of all the other internationalization variables. LC_COLLATE Determine the locale for the behavior of ranges, equivalence classes, and multi-character collating elements within regular expres- sions. LC_CTYPE Determine the locale for the interpretation of sequences of bytes of text data as characters (for example, single-byte as opposed to multi-byte characters in arguments and input files) and the behavior of character classes within regular expressions. LC_MESSAGES Determine the locale that should be used to affect the format and contents of diagnostic messages written to standard error. NLSPATH Determine the location of message catalogs for the processing of LC_MESSAGES . ASYNCHRONOUS EVENTS
If the -k option is specified, created files shall be retained. Otherwise, the default action occurs. STDOUT
Unless the -s option is used, the standard output shall consist of one line per file created, with a format as follows: "%d ", <file size in bytes> STDERR
The standard error shall be used only for diagnostic messages. OUTPUT FILES
The output files shall contain portions of the original input file; otherwise, unchanged. EXTENDED DESCRIPTION
None. EXIT STATUS
The following exit values shall be returned: 0 Successful completion. >0 An error occurred. CONSEQUENCES OF ERRORS
By default, created files shall be removed if an error occurs. When the -k option is specified, created files shall not be removed if an error occurs. The following sections are informative. APPLICATION USAGE
None. EXAMPLES
1. This example creates four files, cobol00 ... cobol03: csplit -f cobol file '/procedure division/' /par5./ /par16./ After editing the split files, they can be recombined as follows: cat cobol0[0-3] > file Note that this example overwrites the original file. 2. This example would split the file after the first 99 lines, and every 100 lines thereafter, up to 9999 lines; this is because lines in the file are numbered from 1 rather than zero, for historical reasons: csplit -k file 100 {99} 3. Assuming that prog.c follows the C-language coding convention of ending routines with a '}' at the beginning of the line, this example creates a file containing each separate C routine (up to 21) in prog.c: csplit -k prog.c '%main(%' '/^}/+1' {20} RATIONALE
The -n option was added to extend the range of filenames that could be handled. Consideration was given to adding a -a flag to use the alphabetic filename generation used by the historical split utility, but the func- tionality added by the -n option was deemed to make alphabetic naming unnecessary. FUTURE DIRECTIONS
None. SEE ALSO
sed, split COPYRIGHT
Portions of this text are reprinted and reproduced in electronic form from IEEE Std 1003.1, 2003 Edition, Standard for Information Technol- ogy -- Portable Operating System Interface (POSIX), The Open Group Base Specifications Issue 6, Copyright (C) 2001-2003 by the Institute of Electrical and Electronics Engineers, Inc and The Open Group. In the event of any discrepancy between this version and the original IEEE and The Open Group Standard, the original IEEE and The Open Group Standard is the referee document. The original Standard can be obtained online at http://www.opengroup.org/unix/online.html . IEEE
/The Open Group 2003 CSPLIT(1P)
All times are GMT -4. The time now is 02:37 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy