Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Problem piping find output to awk, 1st line filename is truncated, other lines are fine.
Top Forums UNIX for Advanced & Expert Users Problem piping find output to awk, 1st line filename is truncated, other lines are fine. Post 302846663 by gencon on Sunday 25th of August 2013 11:21:08 AM
Old 08-25-2013
Problem piping find output to awk, 1st line filename is truncated, other lines are fine.
Today I needed to take a look through a load of large backup files, so I wrote the following line to find them, order them by size, and print the file sizes in GB along with the filename. What happened was odd, the output was all as expected except for the first output line which had the filename heavily truncated. I thought the problem might be with that particular file name, so I reversed the sort order, again the first filename was heavily truncated, this time a different file which had been listed correctly before I changed the ordering. [Note: I used '**' as a field seperator for awk, none of the filenames contain the string '**'.]
Code:
find . -type f -size +500M -printf '%s**%p\n' | sort -n | awk 'FS="**" {gb=$1/(2^30); printf("%f GB\t%s\n", gb, $2)}'

So I wrote the script below to create some dirs and files with different lengths and simplified the command line. Please have a look at what happens with the different find commands below, can someone explain why the first line always has the filename truncated as I can't work out why it is. Thanks.
Code:
#!/bin/bash

mkdir "Test Dir 1"
echo "Test File 1 extra chars so diff file lengths" > "Test Dir 1/Test File 1"

mkdir "Test Dir 2"
echo "Test File 2 fewer extra chars" > "Test Dir 2/Test File 2"

mkdir "Test Dir 3"
echo "Test File 3 even fewer" > "Test Dir 3/Test File 3"

mkdir "Test Dir 4"
echo "Test File 4 a few" > "Test Dir 4/Test File 4"

Code:
# Test 1 - no piping - All OK:

$ find . -type f -printf '%s**%p\n'
23**./Test Dir 3/Test File 3
30**./Test Dir 2/Test File 2
18**./Test Dir 4/Test File 4
45**./Test Dir 1/Test File 1


# Test 2 - pipe to sort - All OK:

$ find . -type f -printf '%s**%p\n' | sort -n
18**./Test Dir 4/Test File 4
23**./Test Dir 3/Test File 3
30**./Test Dir 2/Test File 2
45**./Test Dir 1/Test File 1


# Test 3 - pipe to awk - First line filename truncated:

$ find . -type f -printf '%s**%p\n' | awk 'FS="**" {printf("%d \t%s\n", $1, $2)}'
23     Dir
30     ./Test Dir 2/Test File 2
18     ./Test Dir 4/Test File 4
45     ./Test Dir 1/Test File 1


# Test 4 - pipe to sort, then to awk - First line filename truncated:

$ find . -type f -printf '%s**%p\n' | sort -n | awk 'FS="**" {printf("%d \t%s\n", $1, $2)}'
18     Dir
23     ./Test Dir 3/Test File 3
30     ./Test Dir 2/Test File 2
45     ./Test Dir 1/Test File 1


# Test 5 - pipe to reverse sort, then to awk - First line filename truncated:

$ find . -type f -printf '%s**%p\n' | sort -nr | awk 'FS="**" {printf("%d \t%s\n", $1, $2)}'
45     Dir
30     ./Test Dir 2/Test File 2
23     ./Test Dir 3/Test File 3
18     ./Test Dir 4/Test File 4

Thanks all.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

ps output truncated

Hi! I have some shell scripts receiving in input lots of parameters and I need to select the ones having a particular value in one parameter. A typical shell command line is: PROMPT > shell_name.ksh -avalue_a -bvalue_b -cvalue_c -dvalue_d ... I used a combinaton of ps and grep commands... (1 Reply)
Discussion started by: pciatto
1 Replies

2. UNIX for Dummies Questions & Answers

piping the output of find command to grep

Hi, I did not understand why the following did not work out as I expected: find . -name "pqp.txt" | grep -v "Permission" I thought I would be able to catch whichever paths containing my pqp.txt file without receiving the display of messages such as "find: cannot access... Permisson... (1 Reply)
Discussion started by: 435 Gavea
1 Replies

3. UNIX for Dummies Questions & Answers

Using awk to get a line number to delete, piping through sed

Alright, I'm sure there's a more efficient way to do this... I'm not an expert by any means. What I'm trying to do is search a file for lines that match the two input words (first name, last name) in order to remove that line. The removal part is what I'm struggling with. Here is my code: echo... (4 Replies)
Discussion started by: lazypeterson
4 Replies

4. Shell Programming and Scripting

awk find a string, print the line 2 lines below it

I am parsing a nagios config, searching for a string, and then printing the line 2 lines later (the "members" string). Here's the data: define hostgroup{ hostgroup_name chat-dev alias chat-dev members thisisahostname } define hostgroup{ ... (1 Reply)
Discussion started by: mglenney
1 Replies

5. Shell Programming and Scripting

Find in first column and replace the line with Awk, and output new file

Find in first column and replace the line with Awk, and output new file File1.txt"2011-11-02","Georgia","Atlanta","x","","" "2011-11-03","California","Los Angeles","x","","" "2011-11-04","Georgia","Atlanta","x","x","x" "2011-11-05","Georgia","Atlanta","x","x","" ... (4 Replies)
Discussion started by: charles33
4 Replies

6. Solaris

ps output truncated

Hi, I have Solaris-10 server. /usr/ucb/ps auxww is showing full path if I am running it from root. But if I run it from non-root user, its output is truncated. I don't want to use any other alternate command. Please suggest, what can be its solution. Terminal is set to term. (21 Replies)
Discussion started by: solaris_1977
21 Replies

7. Shell Programming and Scripting

awk problem two lines in the same line

Hi guy, I have an output command like this: Policy Name: NBU.POL.ORA.PROD Policy Type: Oracle Active: yes HW/OS/Client: Linux RedHat2.6 node1 Iclude: /usr/openv/netbackup/scripts/backup_ora1.bash I would like to parse the... (1 Reply)
Discussion started by: luca72m
1 Replies

8. Shell Programming and Scripting

(n)awk: print regex search output lines in one line

Hello. I have been looking high and low for the solution for this. I seems there should be a simple answer, but alas. I have a big xml file, and I need to extract certain information from specific items. The information I need can be found between a specific set of tags. let's call them... (2 Replies)
Discussion started by: Tobias-Reiper
2 Replies

9. Shell Programming and Scripting

Script using awk to find and replace a line, how to ignore comment lines

Hello, I have some code that works more or less. This is called by a make file to adjust some hard-coded definitions in the src code. The script generated some values by looking at some of the src files and then writes those values to specific locations in other files. The awk code is used to... (3 Replies)
Discussion started by: LMHmedchem
3 Replies

10. UNIX for Beginners Questions & Answers

Get an output of lines in pattern 1st line then 10th line then 11th line then 20th line and so on.

Input file: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 (6 Replies)
Discussion started by: Sagar Singh
6 Replies

LEARN ABOUT ULTRIX

bdiff

bdiff(1)						      General Commands Manual							  bdiff(1)

Name
       bdiff - big file differential comparator

Syntax
       bdiff file1 file2 [n] [-s]

Description
       The  command  is  used to find lines that must be changed in two files to bring them into agreement.  Its purpose is to allow processing of
       files that are too large for

       The command ignores lines common to the beginning of both files, splits the remainder of each file into n-line segments, and  invokes  upon
       corresponding  segments.   The  value of n is 3500 by default.  If the optional third argument is given and if it is numeric, it is used as
       the value for n.  This is useful in those cases in which 3500-line segments are too large for causing it to fail.

       The output of the command is the same as the output of the command: line numbers are adjusted to account for the segmenting of the files to
       make  it  look  as  if the files had been processed whole.  Note that because of the segmenting of the files, does not necessarily find the
       smallest sufficient set of file differences.

       If either file1 or file2 is -, the standard input is read.  The optional -s (silent) argument specifies	that  no  diagnostics  are  to	be
       printed by However, this does not suppress possible exclamations by If both optional arguments are specified, they must appear in the order
       indicated above.

Options
       -s		   Suppresses normal diagnostic messages.

Diagnostics
       Use for explanations.

Files
       /tmp/bd?????

See Also
       diff(1)

																	  bdiff(1)
All times are GMT -4. The time now is 04:57 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy