Print filename and last line using awk


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Print filename and last line using awk
# 22  
Old 11-04-2012
Hi.

I would probably use perl, such as in this example. The steps to identify the missing NEWLINE are done in perhaps excruciating detail:
Code:
#!/usr/bin/env bash

# @(#) s1	Driver for perl detector.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C od perl

pl " Simple perl NEWLINE omission test code:"
cat p1

LIST="first.sh middle.sh last.sh"
rm -f $LIST
./create

for i in $LIST
do
  pl " Content of file $i from od:"
  od -bc $i
  ./p1 $i
done

exit 0

producing:
Code:
% ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0.8 (lenny) 
bash GNU bash 3.2.39
od (GNU coreutils) 6.10
perl 5.10.0

-----
 Simple perl NEWLINE omission test code:
#!/usr/bin/env perl

# @(#) p1	Demonstrate detection of missing NEWLINE at last line.

use strict;
use warnings;

my ($debug);
$debug = 1;
$debug = 0;

my ( $a, $b, $file, $f );

foreach $file (@ARGV) {
  if ( !open( $f, "<", $file ) ) {
    print STDERR " Cannot open file $file, skipping.\n";
    next;
  }
  while (<$f>) {
    $a = $_;
  }
  $b = substr( $a, -1, 1 );
  print " Last character of last line of file $file is :$b:\n" if $debug;
  if ( $b ne "\n" ) {
    print " File $file has no NEWLINE on last line.\n";
  }
}

exit 0;

-----
 Content of file first.sh from od:
0000000 143 165 164 012 157 146 146
          c   u   t  \n   o   f   f
0000007
 File first.sh has no NEWLINE on last line.

-----
 Content of file middle.sh from od:
0000000 167 151 156 144 157 167 163 015 012 163 164 171 154 145 015 012
          w   i   n   d   o   w   s  \r  \n   s   t   y   l   e  \r  \n
0000020

-----
 Content of file last.sh from od:
0000000 154 141 163 164 012 154 151 156 145 012
          l   a   s   t  \n   l   i   n   e  \n
0000012

See man pages and perldoc for details.

Best wishes ... cheers, drl
# 23  
Old 11-04-2012
Here's a very tight AND (hopefully) POSIX compliant version:
Code:
for i in *.sh; do echo "<#>"| cat $i - |awk -vfn="$i" '/.<#>/ {gsub("<#>",""); print fn, $0}'; done
a.sh apple

You need to find a "sentinel" string that doesn't interfere with your text.
This User Gave Thanks to RudiC For This Post:
# 24  
Old 11-04-2012
Here is another one:
Code:
find . -type f -name "*.sh" | 
while read f; do
  [ $(wc -l < "$f") -ne $(grep -c '' "$f") ] && echo "$f"
done

# 25  
Old 11-04-2012
Code:
for i in *.sh; do 
[ -s "$i" ] && [ $(tail -1 $i | wc -l) = 0 ] && echo "$i has a missing '\n'" 
done

# 26  
Old 11-04-2012
Quote:
Originally Posted by RudiC
Here's a very tight AND (hopefully) POSIX compliant version:
Code:
for i in *.sh; do echo "<#>"| cat $i - |awk -vfn="$i" '/.<#>/ {gsub("<#>",""); print fn, $0}'; done
a.sh apple

You need to find a "sentinel" string that doesn't interfere with your text.
Hi RudiC!

Thanks for posting this. As long as an incomplete line at the end of a file is no longer than {LINE_MAX}-4 bytes, this should work on any POSIX compliant system. This limit shouldn't be a problem given the way most of us write shell scripts and the submitter wanted this to process files matching the pattern *.sh.

I don't think we need to worry about this for this project, but note that the shell has no line length limit (other than available resources) when reading shell scripts. If a *.sh file has a valid shell script that contains one or more lines consisting of more than {LINE_MAX} bytes, the awk results would still be unspecified.

---------- Post updated at 15:36 ---------- Previous update was at 15:12 ----------

Quote:
Originally Posted by ctsgnb
Code:
for i in *.sh; do 
[ -s "$i" ] && [ $(tail -1 $i | wc -l) = 0 ] && echo "$i has a missing '\n'" 
done

and
Quote:
Originally Posted by Scrutinizer
Here is another one:
Code:
find . -type f -name "*.sh" | 
while read f; do
  [ $(wc -l < "$f") -ne $(grep -c '' "$f") ] && echo "$f"
done

Although both of these suggestions will probably work on many systems, they are not portable solutions. The standards say that the results are unspecified for both tail -n -1 "$i" and grep -c '' "$i" when $i names a non-empty file that does not have a <newline> character as the last byte of the file. (Note also that the 2008 revision of the POSIX standard and the Single UNIX Specification dropped the requirement to treat tail -1 file as a synonym for tail -n -1 file; this requirement was marked obsolete or deprecated in the 2004 revision of those standards.)
These 2 Users Gave Thanks to Don Cragun For This Post:
# 27  
Old 11-05-2012
Quote:
Originally Posted by Don Cragun
Hi RudiC!

Thanks for posting this. As long as an incomplete line at the end of a file is no longer than {LINE_MAX}-4 bytes, this should work on any POSIX compliant system. This limit shouldn't be a problem given the way most of us write shell scripts and the submitter wanted this to process files matching the pattern *.sh.

I don't think we need to worry about this for this project, but note that the shell has no line length limit (other than available resources) when reading shell scripts. If a *.sh file has a valid shell script that contains one or more lines consisting of more than {LINE_MAX} bytes, the awk results would still be unspecified. ...
Hi Don,

thank you for pointing this out! Although LINE_MAX may be far away, it may need to be considered, esp. as I was thinking of a looong sentinel code to make it unambiguous and separate it from the file contents.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk to print filename words along with delimiter

Hi, I have filename as: 010020001_S-FOR-Sort-SYEXC_20180109_062320_0100.x937 I need first 5 words of my filename along with the respective delimiters: I tried this: f=010020001_S-FOR-Sort-SYEXC_20180109_062320_0100.x937 echo $f | awk -F '' '{print $1$2$3$4$5}' 010020001SFORSortSYEXC ... (11 Replies)
Discussion started by: gnnsprapa
11 Replies

2. Shell Programming and Scripting

Use while loop to read file and use ${file} for both filename input into awk and as string to print

I have files named with different prefixes. From each I want to extract the first line containing a specific string, and then print that line along with the prefix. I've tried to do this with a while loop, but instead of printing the prefix I print the first line of the file twice. Files:... (3 Replies)
Discussion started by: pathunkathunk
3 Replies

3. Shell Programming and Scripting

awk to print the line that matches and the next if line is wrapped

I have a file and when I match the word "initiators" in the first column I need to be able to print the rest of the columns in that row. This is fine for the most part but on occasion the "initiators" line gets wrapped to the next line. Here is a sample of the file. caw-enabled ... (3 Replies)
Discussion started by: kieranfoley
3 Replies

4. UNIX for Advanced & Expert Users

Problem piping find output to awk, 1st line filename is truncated, other lines are fine.

Today I needed to take a look through a load of large backup files, so I wrote the following line to find them, order them by size, and print the file sizes in GB along with the filename. What happened was odd, the output was all as expected except for the first output line which had the filename... (4 Replies)
Discussion started by: gencon
4 Replies

5. Shell Programming and Scripting

awk script -print line when $2 > $2 of previous line

Hi all, From a while loop I am reading a sorted file where I want to print only the lines that have $1 match and $2 only when the difference from $2 from the previous line is > 30. Input would be like ... AN237 010 193019 0502 1 CSU Amoxycillin AN237 080 ... (2 Replies)
Discussion started by: gafoleyo73
2 Replies

6. Shell Programming and Scripting

Print filename with awk

I can got the filename with this script. it's only show "-" in result. cut -d , -f7 CSV_d.* | awk 'OFS=":"{print FILENAME,substr($1,1,8),substr($1,9,2),substr($1,11,2),substr($1,13,2)}' | sort |uniq (2 Replies)
Discussion started by: before4
2 Replies

7. Shell Programming and Scripting

awk to print on the same line

Hi all, I've a script that uses awk to parse the output of wget during a database update. The code I use is: wget -c ftp://ftpsite/file 2>&1 | awk '/0%/ {print}'But this spits out each progress line on a new line: 37250K .......... .......... .......... .......... .......... 80% 10.9M 1s ... (2 Replies)
Discussion started by: euval
2 Replies

8. Shell Programming and Scripting

Howto Print File Path or Print the Filename

I'm trying to clean up my samba share and need to print the found file or print the path of the image it tried to searched for. So far I have this but can't seem to get the logic right. Can anyone help point me in the right direction? for FILE in `cat list`; do if ; then ... (1 Reply)
Discussion started by: overkill
1 Replies

9. Shell Programming and Scripting

awk print the next line on the current line

Ok I have a file with hundreds of lines, four columns, space delimited, TESTB.TXT for example TESTB.TXT --- AA ZZ 12 34 BB YY 56 78 CC XX 91 23 DD VV 45 67 --- I want a new file that has 7 columns, the first four are identical, and the next 3 are the last three of the next line...so... (5 Replies)
Discussion started by: ajp7701
5 Replies

10. Shell Programming and Scripting

print any required line by its line no using awk and its NR variable

how to print any required line by its line no using awk and its NR variable for eg: ------------ 121343 adfdafd 21213sds dafadfe432 adf.adf%adf --------------- requied o/p if give num=3 it print: 21213sds -------------------------------------- (2 Replies)
Discussion started by: RahulJoshi
2 Replies
Login or Register to Ask a Question