11-26-2013
http://www.gnu.org/software/parallel/ no sense reinventing a lesser wheel.
The real trick is to poll for modified files and maintain a registry file of file names, line counts and mod times, so the slow part is done ahead of time. Use fuser to ensure the file is fully written (not open for write) before counting lines.
Last edited by DGPickett; 11-26-2013 at 02:57 PM..
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
How can i add a character(#) in the beginning of every line in a .dat file (2 Replies)
Discussion started by: Cool Coder
2 Replies
2. Shell Programming and Scripting
Hi,
compare two .dat files and difference will be moved into separate file.if anybody having code for this please send asap. using diff command, i don't know how to write shell programming.
and my first file is like this including Header and trailer
10Ç20060323Ç01(Header)
01ÇIÇbabuÇ3000
01ÇIÇbaluÇ4000... (1 Reply)
Discussion started by: kirankumar
1 Replies
3. Shell Programming and Scripting
Hello,
I am a bit stuck on something I am sure is easy to most.
I have a dat file that has a list of server names.
sentra
blue
red
willy
clawcrab
I need to take each server name from the dat file in a shell script and attempt to ssh to it to run a script on that server. So I guess I... (1 Reply)
Discussion started by: LRoberts
1 Replies
4. Shell Programming and Scripting
Hi,
I have created one shell script in which it will count number of "~" tilda charactors from each line of the file.But the problem is that i need to count each line count individually, that means. if line one contains 14 "~"s and line two contains 15 "~"s then it should give an error msg.each... (3 Replies)
Discussion started by: Ganesh Khandare
3 Replies
5. UNIX for Dummies Questions & Answers
Could you please tell e how to find the total number of rows in a .dat file.
edit by bakunin: this is exactly why in "advanced and expert" section?? I transfer this thread to "Unix for Dummies Questions and Answers" (2 Replies)
Discussion started by: Deeptanshu
2 Replies
6. Shell Programming and Scripting
Hi all
i want to write program with shell script that able compare two file content and if one of lines of file have # at the first of string or nothing find same string in one of two file . remove the line in second file that have not the string in first file. for example:
file... (2 Replies)
Discussion started by: saleh67
2 Replies
7. Shell Programming and Scripting
What I'm trying to accomplish. I receive a Header and Detail file for daily processing. The detail file comes first which holds data, the header is a receipt of the detail file and has the detail files record count. Before processing the detail file I would like to put a wrapper around another... (4 Replies)
Discussion started by: pone2332
4 Replies
8. Shell Programming and Scripting
Hi all,
I have a log file say Test.log that gets updated continuously and it has data in pipe separated format. A sample log file would look like:
<date1>|<data1>|<url1>|<result1>
<date2>|<data2>|<url2>|<result2>
<date3>|<data3>|<url3>|<result3>
<date4>|<data4>|<url4>|<result4>
What I... (3 Replies)
Discussion started by: pat_pramod
3 Replies
9. Shell Programming and Scripting
Hi,
I have a .dat file with contents like the below:
Input file
============SEQ NO-1: COLUMN1==========
9835619
7152815
============SEQ NO-2: COLUMN2 ==========
7615348
7015548
9373086
============SEQ NO-3: COLUMN3===========
9373086
Expected Output: (I just... (1 Reply)
Discussion started by: MS06
1 Replies
10. UNIX for Beginners Questions & Answers
Hi All,
I need to compare a record in one file and find the matching across two file.
This is the master file. File name : CUST.dat
CUST_ID
9998
10000
10004
10005
DATAFILE1
9998;80000091;4;687582837443;;;;;;;;;
9998;80000091;4;687582841003;;;;;;;;;... (4 Replies)
Discussion started by: arunkumar_mca
4 Replies
WC(1) User Commands WC(1)
NAME
wc - print newline, word, and byte counts for each file
SYNOPSIS
wc [OPTION]... [FILE]...
wc [OPTION]... --files0-from=F
DESCRIPTION
Print newline, word, and byte counts for each FILE, and a total line if more than one FILE is specified. A word is a non-zero-length
sequence of characters delimited by white space.
With no FILE, or when FILE is -, read standard input.
The options below may be used to select which counts are printed, always in the following order: newline, word, character, byte, maximum
line length.
-c, --bytes
print the byte counts
-m, --chars
print the character counts
-l, --lines
print the newline counts
--files0-from=F
read input from the files specified by NUL-terminated names in file F; If F is - then read names from standard input
-L, --max-line-length
print the maximum display width
-w, --words
print the word counts
--help display this help and exit
--version
output version information and exit
AUTHOR
Written by Paul Rubin and David MacKenzie.
REPORTING BUGS
GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
Report wc translation bugs to <http://translationproject.org/team/>
COPYRIGHT
Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
SEE ALSO
Full documentation at: <http://www.gnu.org/software/coreutils/wc>
or available locally via: info '(coreutils) wc invocation'
GNU coreutils 8.28 January 2018 WC(1)