Sponsored Content
Top Forums Shell Programming and Scripting FASTEN count line of dat file and compare with the CTRL file Post 302876765 by DGPickett on Tuesday 26th of November 2013 01:32:04 PM
Old 11-26-2013
http://www.gnu.org/software/parallel/ no sense reinventing a lesser wheel.

The real trick is to poll for modified files and maintain a registry file of file names, line counts and mod times, so the slow part is done ahead of time. Use fuser to ensure the file is fully written (not open for write) before counting lines.

Last edited by DGPickett; 11-26-2013 at 02:57 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Adding a character in the beginning of every line in a .dat file

How can i add a character(#) in the beginning of every line in a .dat file (2 Replies)
Discussion started by: Cool Coder
2 Replies

2. Shell Programming and Scripting

compare two .dat files and if there is any difference pulled into a separate file

Hi, compare two .dat files and difference will be moved into separate file.if anybody having code for this please send asap. using diff command, i don't know how to write shell programming. and my first file is like this including Header and trailer 10Ç20060323Ç01(Header) 01ÇIÇbabuÇ3000 01ÇIÇbaluÇ4000... (1 Reply)
Discussion started by: kirankumar
1 Replies

3. Shell Programming and Scripting

Read dat file line by line

Hello, I am a bit stuck on something I am sure is easy to most. I have a dat file that has a list of server names. sentra blue red willy clawcrab I need to take each server name from the dat file in a shell script and attempt to ssh to it to run a script on that server. So I guess I... (1 Reply)
Discussion started by: LRoberts
1 Replies

4. Shell Programming and Scripting

Shell script to count number of ~ from each line and compare with next line

Hi, I have created one shell script in which it will count number of "~" tilda charactors from each line of the file.But the problem is that i need to count each line count individually, that means. if line one contains 14 "~"s and line two contains 15 "~"s then it should give an error msg.each... (3 Replies)
Discussion started by: Ganesh Khandare
3 Replies

5. UNIX for Dummies Questions & Answers

Count rows in .dat file

Could you please tell e how to find the total number of rows in a .dat file. edit by bakunin: this is exactly why in "advanced and expert" section?? I transfer this thread to "Unix for Dummies Questions and Answers" (2 Replies)
Discussion started by: Deeptanshu
2 Replies

6. Shell Programming and Scripting

Compare two string in two separate file and delete some line of file

Hi all i want to write program with shell script that able compare two file content and if one of lines of file have # at the first of string or nothing find same string in one of two file . remove the line in second file that have not the string in first file. for example: file... (2 Replies)
Discussion started by: saleh67
2 Replies

7. Shell Programming and Scripting

Compare file1 header count with file2 line count

What I'm trying to accomplish. I receive a Header and Detail file for daily processing. The detail file comes first which holds data, the header is a receipt of the detail file and has the detail files record count. Before processing the detail file I would like to put a wrapper around another... (4 Replies)
Discussion started by: pone2332
4 Replies

8. Shell Programming and Scripting

How to read file line by line and compare subset of 1st line with 2nd?

Hi all, I have a log file say Test.log that gets updated continuously and it has data in pipe separated format. A sample log file would look like: <date1>|<data1>|<url1>|<result1> <date2>|<data2>|<url2>|<result2> <date3>|<data3>|<url3>|<result3> <date4>|<data4>|<url4>|<result4> What I... (3 Replies)
Discussion started by: pat_pramod
3 Replies

9. Shell Programming and Scripting

Help with Getting distinct record count from a .dat file using UNIX command

Hi, I have a .dat file with contents like the below: Input file ============SEQ NO-1: COLUMN1========== 9835619 7152815 ============SEQ NO-2: COLUMN2 ========== 7615348 7015548 9373086 ============SEQ NO-3: COLUMN3=========== 9373086 Expected Output: (I just... (1 Reply)
Discussion started by: MS06
1 Replies

10. UNIX for Beginners Questions & Answers

Compare one file and get the count of multiple file

Hi All, I need to compare a record in one file and find the matching across two file. This is the master file. File name : CUST.dat CUST_ID 9998 10000 10004 10005 DATAFILE1 9998;80000091;4;687582837443;;;;;;;;; 9998;80000091;4;687582841003;;;;;;;;;... (4 Replies)
Discussion started by: arunkumar_mca
4 Replies
WC(1)								   User Commands							     WC(1)

NAME
wc - print newline, word, and byte counts for each file SYNOPSIS
wc [OPTION]... [FILE]... wc [OPTION]... --files0-from=F DESCRIPTION
Print newline, word, and byte counts for each FILE, and a total line if more than one FILE is specified. A word is a non-zero-length sequence of characters delimited by white space. With no FILE, or when FILE is -, read standard input. The options below may be used to select which counts are printed, always in the following order: newline, word, character, byte, maximum line length. -c, --bytes print the byte counts -m, --chars print the character counts -l, --lines print the newline counts --files0-from=F read input from the files specified by NUL-terminated names in file F; If F is - then read names from standard input -L, --max-line-length print the maximum display width -w, --words print the word counts --help display this help and exit --version output version information and exit AUTHOR
Written by Paul Rubin and David MacKenzie. REPORTING BUGS
GNU coreutils online help: <http://www.gnu.org/software/coreutils/> Report wc translation bugs to <http://translationproject.org/team/> COPYRIGHT
Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. SEE ALSO
Full documentation at: <http://www.gnu.org/software/coreutils/wc> or available locally via: info '(coreutils) wc invocation' GNU coreutils 8.28 January 2018 WC(1)
All times are GMT -4. The time now is 10:12 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy