Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: AWK - Pattern Matching & Replacing - Performance
Top Forums Shell Programming and Scripting AWK - Pattern Matching & Replacing - Performance Post 302450612 by srivijay81 on Friday 3rd of September 2010 08:00:20 AM
Old 09-03-2010
AWK - Pattern Matching & Replacing - Performance
Experts,

I am a beginner to Unix Shell Scripting
We have source as a flat file which contains CTRL+F character as the delimiter. We need to count the number of records in the file (CTRL+F) to perform file validation
Following command being used:
Code:
awk '{cnt+=gsub(//,"&")}END {print cnt}' Sri.dat


This takes only fewer seconds to show the count if the file contains less than 25K records. We have files with count exceeding 1000K and this command takes more than one hour to complete

Could you please suggest any other command similar to the above requirement and provide the count within fewer seconds / minutes. The commands can be either in Unix or Perl

Please help

Thanks & regards
Sri
Last edited by radoulov; 09-03-2010 at 09:49 AM.. Reason: Code tags, please!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

help with finding & replacing pattern in a file

Hi everyone. Could u be so kind and help me with on "simple" shell script? 1. i need to search a file line by line for a pattern. example of a lines in that file 2947 domain = feD,id = 00 0A 02 48 17 1E 1D 39 DE 00 0E 00,Name Values:snNo = f10 Add AttFlag = 0 2. i need to find... (0 Replies)
Discussion started by: dusoo
0 Replies

2. Shell Programming and Scripting

Replacing all except for the matching pattern

Hi all I need to replace all characters in a file except for the matching pattern eg. I need to replace all character with '*' except for the pattern "abc" Input "sdfhgsdf abc ##%$#abcsdfh sdfjkfff" Output "******abc******abc*************" Request for single liner solution (1 Reply)
Discussion started by: raghav288
1 Replies

3. UNIX for Dummies Questions & Answers

Pattern matching New to Sed & Awk

Hello, Despite reading the Pattern Matching chapter in the O'Reilly Sed & Awk book several times and looking at numerous examples, I cannot seem to get any kind of conditional script to work in my awk scripts! I am able to do the basic awk and grep script to capture the data but when I do with... (0 Replies)
Discussion started by: pg55
0 Replies

4. Shell Programming and Scripting

matching a pattern in a file & prefixing a word

Hi, In my shell script i have to match a patten in a file , if found i have to prefix the entair line by a "word" eg. pattern = "aaa" prefix= #123 file: bbbb xxx zzzz aaaa qqqq kkkk outPut file: bbbb xxx ... (5 Replies)
Discussion started by: shivarajM
5 Replies

5. Shell Programming and Scripting

Pattern matching & storing result in variable

Hi! i'm trying to parse textfiles against a pattern and storing the result in a variable. The strings i want to get are embraced by and can occur several times in one line, so e.g. some text anything else endwhat i have so far: #!/bin/bash for f in $* do exec 3<&0 exec 0<$f ... (2 Replies)
Discussion started by: thoni
2 Replies

6. Shell Programming and Scripting

Pattern Matching & replacing of content in file1 with file2

I have file 1 & file 2 with content mentioned below. I want to get the output as shown in file3. Requirement: check the content of column 1 & column 2, if value of column 1 in file1 matches with first column of file2 then remaining columns(2&3) of file2 should get replaced, also if value of... (4 Replies)
Discussion started by: siramitsharma
4 Replies

7. Shell Programming and Scripting

Replacing lines matching a multi-line pattern (sed/perl/awk)

Dear Unix Forums, I am hoping you can help me with a pattern matching problem. What am I trying to do? I want to replace multiple lines of a text file (that match a multi-line pattern) with a single line of text. These patterns can span several lines and do not always have the same number of... (10 Replies)
Discussion started by: thefang
10 Replies

8. Shell Programming and Scripting

awk Array matching and replacing from master file.

I have an awk related question that I was hoping you all could help with. I am given 2 input files named OLDFILE and NEWFILE, and a Master file named MASTERFILE. They can be seen below. OLDFILE: a a a a a f g 4 5 7 8 1 2 3 (1 Reply)
Discussion started by: tiktak292
1 Replies

9. Shell Programming and Scripting

How to print & and \n while replacing with sed/awk?

string="din&esh\nisgood" File.txt: the name is sed "s#\#${string}#g" File.txt Output am getting: the name is dinesh is good Expected output: the name is din&esh\nisgood The input string is dynamic it will be keep on changing am able to handle & by placing \& in the string.. (5 Replies)
Discussion started by: dineshaila
5 Replies

10. Shell Programming and Scripting

Performance assessment of using single or combined pattern matching

Hi, I want to know which pattern matching technique will be giving better performance and quick result. I will be having the patterns in a file and want to read that patterns and search through a whole file of say 70 MB size. whether if i initially create a pattern matching string while... (7 Replies)
Discussion started by: ananan
7 Replies

LEARN ABOUT DEBIAN

vconsole.conf

VCONSOLE.CONF(5)						   vconsole.conf						  VCONSOLE.CONF(5)

NAME

       vconsole.conf - configuration file for the virtual console

SYNOPSIS

       /etc/vconsole.conf

DESCRIPTION

       The /etc/vconsole.conf file configures the virtual console, i.e. keyboard mapping and console font.

       The basic file format of the vconsole.conf is a newline-separated list environment-like shell-compatible variable assignments. It is
       possible to source the configuration from shell scripts, however, beyond mere variable assignments no shell features are supported,
       allowing applications to read the file without implementing a shell compatible execution engine.

       Note that the kernel command line options vconsole.keymap=, vconsole.keymap.toggle=, vconsole.font=, vconsole.font.map=,
       vconsole.font.unimap= may be used to override the console settings at boot.

       Depending on the operating system other configuration files might be checked for configuration of the virtual console as well, however only
       as fallback.

OPTIONS

       The following options are understood:

       KEYMAP=, KEYMAP_TOGGLE=
	   Configures the key mapping table of for they keyboard.  KEYMAP= defaults to us if not set. The KEYMAP_TOGGLE= can be used to configured
	   a second toggle keymap and is by default unset.

       FONT=, FONT_MAP=, FONT_UNIMAP=
	   Configures the console font, the console map and the unicode font map.  FONT= defaults to latarcyrheb-sun16.

EXAMPLE

       Example 1. German keyboard and console

       /etc/vconsole.conf:

	   KEYMAP=de-latin1
	   FONT=latarcyrheb-sun16

SEE ALSO

       systemd(1), loadkeys(1), setfont(8), locale.conf(5)

AUTHOR

       Lennart Poettering <lennart@poettering.net>
	   Developer

systemd 							    10/07/2013							  VCONSOLE.CONF(5)
All times are GMT -4. The time now is 05:17 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy