Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Print number of lines for files in directory, also print number of unique lines Post 303036815 by Scrutinizer on Thursday 11th of July 2019 05:21:07 PM
Old 07-11-2019
Adaptation to post#6 that uses less memory, only unique lines for one file at a time (thanks Don):

Code:
awk '
  FNR==1 {
    filenr++
    Name[filenr]=FILENAME
    split("", Seen)
  }

  !Seen[$0]++ {
    Uniq[filenr]++
  } 

  {
    Total[filenr]++
  } 

  END {
    for(i in Name)
      print Total[i], Uniq[i], Name[i]
  }
' file* | sort -nk1,1 -nk2,2 -k3,3


Last edited by Scrutinizer; 07-11-2019 at 06:41 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to print number of lines with awk ?

Can some body tell me how to print number of line from a particular file, with sed. ? Input file format AAAA BBBB CCCC SDFFF DDDD DDDD Command to print line 2 and 3 ? BBBB CCCC And also please tell me how to assign column sum to variable. I user the following command it... (1 Reply)
Discussion started by: maheshsri
1 Replies

2. Shell Programming and Scripting

How do I print out lines with the same number in front using awk?

Hi, I need help in printing out the dates with the largest value in front of it using awk. 436 28/Feb/2008 436 27/Feb/2008 436 20/Feb/2008 422 13/Feb/2008 420 23/Feb/2008 409 21/Feb/2008 402 26/Feb/2008 381 22/Feb/2008 374 24/Feb/2008 360... (7 Replies)
Discussion started by: SIFA
7 Replies

3. SCO

Why? I can not change the number of lines to print

hi My problem now is that if shipping options as -o length = 88 it says the following: # lp -o length=88 -dhp4015 /etc/hosts UX:lp: ERROR: The following options can't be handled: -o length= TO FIX: The printer(s) that otherwise qualify for printing your request can't handle one or more of... (2 Replies)
Discussion started by: Edgar Guevara
2 Replies

4. Shell Programming and Scripting

print every 20 lines the lowest number

Hello all, How can I find the lowest number every 10 lines? For example i have a list name1 -0.1 name2 2 name3 3 name4 -3 name5 1 name6 2 name7 34 name8 34 (6 Replies)
Discussion started by: TheTransporter
6 Replies

5. Shell Programming and Scripting

print lines between line number

Hi, Anyone help me to print the lines from the flat file between 879th line number and 1424th line number. The 879 and 1424 should be passed as input to the shell script(It should be dynamic). Can any one give me using sed or awk? I tried using read, and print the lines..Its taking too... (3 Replies)
Discussion started by: senthil_is
3 Replies

6. Shell Programming and Scripting

Compare multiple files and print unique lines

Hi friends, I have multiple files. For now, let's say I have two of the following style cat 1.txt cat 2.txt output.txt Please note that my files are not sorted and in the output file I need another extra column that says the file from which it is coming. I have more than 100... (19 Replies)
Discussion started by: jacobs.smith
19 Replies

7. Shell Programming and Scripting

How to print lines that only have number lower than...

Hello guys, I am a beginner in Unix :wall: and was wondering if anyone could help me. I need a script that prints lines that only has Z-value lower than equals to (<=) 1.0e-02. Each column is seperated by a tab. 10009.fd Z-value = 3.62843e-03 10009.fd Z-value = 9.75489e-01... (3 Replies)
Discussion started by: narachaid
3 Replies

8. UNIX for Dummies Questions & Answers

Writing a script to print the number of lines in multiple files

Hi I have 1000 files labelled data1.txt through data1000.txt. I want to write a script that prints out the number of lines in each txt file and outputs it in the following format: Column 1: number of data file (1 through 1000) Column 2: number of lines in the text file Thanks! (2 Replies)
Discussion started by: evelibertine
2 Replies

9. Shell Programming and Scripting

How to print N number of lines before and after the grep?

Hi , My record file , need to print up to above (DATA array)(there may be n no lines ) , grep "myvalue" row now .....suggest me some options --- DATA Array--- record type xxxxx sequence type yyyyy 2 3---> data1 /dev/ --- DEVICE --- MAXIMUM_People= data_blocks= MY_value=2 xyz abc ... (0 Replies)
Discussion started by: Huvan
0 Replies

10. UNIX for Beginners Questions & Answers

Advise on how to print range of lines above and below a number?

Hi, I have attached an output file which is some kind of database file mapping. It is basically like an allocation mapping of a tablespace and its datafile/s. The output is generated by the SQL script that I found from 401 Authorization Required Excerpts of the file are as below: ... (2 Replies)
Discussion started by: newbie_01
2 Replies
Df(3pm) 						User Contributed Perl Documentation						   Df(3pm)

NAME
Filesys::Df - Perl extension for filesystem disk space information. SYNOPSIS
use Filesys::Df; #### Get information by passing a scalar directory/filename value my $ref = df("/tmp"); # Default output is 1K blocks if(defined($ref)) { print "Total 1k blocks: $ref->{blocks} "; print "Total 1k blocks free: $ref->{bfree} "; print "Total 1k blocks avail to me: $ref->{bavail} "; print "Total 1k blocks used: $ref->{used} "; print "Percent full: $ref->{per} "; if(exists($ref->{files})) { print "Total inodes: $ref->{files} "; print "Total inodes free: $ref->{ffree} "; print "Inode percent full: $ref->{fper} "; } } #### Get information by passing a filehandle open(FILE, "some_file"); # Get information for filesystem at "some_file" my $ref = df(*FILE); #### or my $ref = df(*FILE); #### or my $fhref = *FILE; my $ref = df($fhref); #### Get information in other than 1k blocks my $ref = df("/tmp", 8192); # output is 8K blocks my $ref = df("/tmp", 1); # output is bytes DESCRIPTION
This module provides a way to obtain filesystem disk space information. This is a Unix only distribution. If you want to gather this information for Unix and Windows, use "Filesys::DfPortable". The only major benefit of using "Filesys::Df" over "Filesys::DfPortable", is that "Filesys::Df" supports the use of open filehandles as arguments. The module should work with all flavors of Unix that implement the "statvfs()" and "fstatvfs()" calls, or the "statfs()" and "fstatfs()" calls. This would include Linux, *BSD, HP-UX, AIX, Solaris, Mac OS X, Irix, Cygwin, etc ... "df()" requires a argument that represents the filesystem you want to query. The argument can be either a scalar directory/file name or a open filehandle. There is also an optional block size argument so you can tailor the size of the values returned. The default block size is 1024. This will cause the function to return the values in 1k blocks. If you want bytes, set the block size to 1. "df()" returns a reference to a hash. The keys available in the hash are as follows: "{blocks}" = Total blocks on the filesystem. "{bfree}" = Total blocks free on the filesystem. "{bavail}" = Total blocks available to the user executing the Perl application. This can be different than "{bfree}" if you have per-user quotas on the filesystem, or if the super user has a reserved amount. "{bavail}" can also be a negative value because of this. For instance if there is more space being used then you have available to you. "{used}" = Total blocks used on the filesystem. "{per}" = Percent of disk space used. This is based on the disk space available to the user executing the application. In other words, if the filesystem has 10% of its space reserved for the superuser, then the percent used can go up to 110%. You can obtain inode information through the module as well, but you must call "exists()" on the "{files}" key first, to make sure the information is available. Some filesystems may not return inode information, for example some NFS filesystems. Here are the available inode keys: "{files}" = Total inodes on the filesystem. "{ffree}" = Total inodes free on the filesystem. "{favail}" = Total inodes available to the user executing the application. See the rules for the "{bavail}" key. "{fused}" = Total inodes used on the filesystem. "{fper}" = Percent of inodes used on the filesystem. See rules for the "{per}" key. There are some undocumented keys that are defined to maintain backwards compatibilty: "{su_blocks}", "{user_blocks}", etc ... If the "df()" call fails for any reason, it will return undef. This will probably happen if you do anything crazy like try to get information for /proc, or if you pass an invalid filesystem name, or if there is an internal error. "df()" will "croak()" if you pass it a undefined value. Requirements: Your system must contain "statvfs()" and "fstatvfs()", or "statfs()" and "fstatfs()" You must be running Perl 5.6 or higher. AUTHOR
Ian Guthrie IGuthrie@aol.com Copyright (c) 2006 Ian Guthrie. All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. SEE ALSO
statvfs(2), fstatvfs(2), statfs(2), fstatfs(2), df(1), Filesys::DfPortable perl(1). perl v5.14.2 2006-06-25 Df(3pm)
All times are GMT -4. The time now is 09:27 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy