using awk for setting variable but change the output of this variable within awk


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting using awk for setting variable but change the output of this variable within awk
# 1  
Old 08-10-2011
using awk for setting variable but change the output of this variable within awk

Hi all,

Hope someone can help me out here.
I have this BASH script (see below)

My problem lies with the variable path.
The output of the command find will give me several fields. The 9th field is the path. I want to captured that and the I want to filter this to a specific level.

The output of the path is somewhat like this: /Volume/Server/department/otherlevels/....

The part I want to capture in the variable is the department.
I can do this in a single line with awk -F/ '{print $5}' But I need it in an array.

Using it like it is now will give me an error.

Any idea how to resolve this ?

Many regards,

Ronald



Code:
find . -iname $1 -exec ls -l {} \; | awk 'BEGIN {

# Initialize all Arrays
        size = "0";
        u_count[""]=0;
        all_count[""]=0;
}
{
# Assign field names
        sizes=$5
        path=$( echo $9 awk {-F/ '{print $5}'} )


Last edited by Scott; 08-10-2011 at 07:10 AM.. Reason: Please use code tags for code
# 2  
Old 08-10-2011
Something like below..? stores the $5 value in array a[]
Code:
find . -iname $1 -exec ls -l {} \; | awk -F/ '{a[NR]=$5;next}END{for(i in a) print i,a[i]}'


Last edited by michaelrozar17; 08-10-2011 at 08:37 AM.. Reason: above code is not tested..
# 3  
Old 08-10-2011
Quote:
Originally Posted by michaelrozar17
Something like below..? stores the $5 value in array a[]
Code:
find . -iname $1 -exec ls -l {} \; | awk -F/ '{a[NR]=$5;next}END{for(i in a) print i,a[i]}'

Not quite.

I do not want to change the find command because I am using multiple fields from that output. In fact I need to capture the 9th ($9) field and manipulate the output so that I only capture the department folder.

ex: the output is like:
-rwx------+ 1 admin staff 71680 Jun 12 2010 /Volumes/Sys_Data/Server One/Department/.../.../.../.../...

From this output I capture the $5 field for size counting and the $3 field for user tracking. Now I want to capture the $9 field with the path and only capture the Department field. So I can keep track of the departments as well.

---------- Post updated at 08:00 AM ---------- Previous update was at 06:44 AM ----------

is there any idea why this doesn't work?

Quote:
# Search Command where $1 is the type to find. Search will start at current location.
find . -iname $1 -exec ls -l {} \; | awk 'BEGIN {

# Initialize all Arrays
size = "0";
u_count[""]=0;
all_count[""]=0;
}
{
# Assign field names
sizes=$5
user=$3
{ path=`for (I=9 ; I<=NF ; I++) { printf "%s",$I } print`
}
# 4  
Old 08-10-2011
Quote:
Originally Posted by Cowardly
Not quite.

I do not want to change the find command because I am using multiple fields from that output. In fact I need to capture the 9th ($9) field and manipulate the output so that I only capture the department folder.

ex: the output is like:
-rwx------+ 1 admin staff 71680 Jun 12 2010 /Volumes/Sys_Data/Server One/Department/.../.../.../.../...

From this output I capture the $5 field for size counting and the $3 field for user tracking. Now I want to capture the $9 field with the path and only capture the Department field. So I can keep track of the departments as well.

---------- Post updated at 08:00 AM ---------- Previous update was at 06:44 AM ----------

is there any idea why this doesn't work?
The syntax is wrong for awk. Post the error you get. Alternatively can you try the below sed if the output from the find command is as you posted in post# 3..
Code:
find . -iname $1 -exec ls -l {} \; |  sed 's|[^/]*/[^/]*/[^/]*/[^/]*/\([^/]*\)/.*|\1|'

# 5  
Old 08-10-2011
Quote:
Originally Posted by michaelrozar17
The syntax is wrong for awk. Post the error you get. Alternatively can you try the below sed if the output from the find command is as you posted in post# 3..
Code:
find . -iname $1 -exec ls -l {} \; |  sed 's|[^/]*/[^/]*/[^/]*/[^/]*/\([^/]*\)/.*|\1|'


Thanks for the reply.

But I do not want to change my original find command because of the other fields I am using. Besides that with the sed you filter out the / but it is undefined how many /'s there are in the directory path. So this will not work completely

I will try to be more specific. Here is the complete code of my original script:
Code:
#!/usr/bin/env bash
#
# 
# Bash Script written by R. Blaas
#
# This script will find files of a specific type and displays the full path and size
# Also a total of found files and size is displayed per user and an overal total found files and size
# 
# if nothing is passed to the script, show usage and exit
[[ -n "$1" ]] || { echo “Usage: usage.sh [Variable]“; exit 0 ; }

# Make sure only root can run our script
if [ "$(id -u)" != "0" ]; then
    echo "This script must be run as root" 1>&2
    exit 1
fi

# Variables
POSTMASTER="my-email@domain.com"

DATE=`date +"%d%m%Y"` 
DATIME=`date +"%Y%m%d%H%M"`
DAGNAAM=`date +"%A"`

# Set current directory to variable $CURRENT
CURRENT=/opt/local/usage

# Set the log directory
DIR_LOG=$CURRENT"/log"

# Catch search variable
SEARCH=`echo $1 | sed 's/*//' | sed 's/.//'`

# Check if directory exists
if test ! -d "$DIR_LOG"
 then
    mkdir "$DIR_LOG"
fi

# Create temporary file for log messages
POSTMLOG=$DIR_LOG"/"$SEARCH"-usage.log"

# BEGIN
logger "Script start (usage.sh)"
echo `date` start of script usage.sh > $POSTMLOG
echo "" >> $POSTMLOG
echo Search variable = $SEARCH >> $POSTMLOG
echo "" >> $POSTMLOG

# Search Command where $1 is the type to find. Search will start at current location.
find . -iname $1 -exec ls -l {} \; | awk 'BEGIN { 

# Initialize all Arrays
    size = "0"; 
    u_count[""]=0;
    all_count[""]=0;
} 
{
# Assign field names
    sizes=$5
    user= $3

# Count of number of files
    u_count[user]++;
    all_count["* *"]++;

# Count disc space used
    u_size[user]+=sizes;
    all_size["* *"]+=sizes;
}
{
    for (I=9 ; I<=NF ; I++) { printf "%s",$I } print " :",$5}{ x++ } { size=size+$5 } 
END { print "\n" x " Files Found""\n" } 
END { hum[1024**4]="Tb"; hum[1024**3]="Gb"; hum[1024**2]="Mb"; hum[1024]="Kb"; 
    for (x=1024**4; x>=1024; x/=1024) { if (size>=x) { { printf "Total Size = ",NR } 
    printf "%.2f %s\n\n",size/x,hum[x];break } 
    } 
# Output
    { FS = ":";
    format = "%11s %6s %-16s\n";
    prinft "\n"
    printf ( format, "Size","Count","Who" ) }
    for (i in u_count) {
        if (i != "") {
        { hum[1024**4]="Tb"; hum[1024**3]="Gb"; hum[1024**2]="Mb"; hum[1024]="Kb";
        for (x=1024**4; x>=1024; x/=1024) { if (u_size[i]>=x) {
    usersize = sprintf ( "%.2f %s", u_size[i]/x,hum[x] )
        printf ( format,usersize, u_count[i], i);break } } } 
              } 
        } 
    for (i in all_count) {
        if (i != "") {
        { hum[1024**4]="Tb"; hum[1024**3]="Gb"; hum[1024**2]="Mb"; hum[1024]="Kb";
        for (x=1024**4; x>=1024; x/=1024) { if (all_size[i]>=x) {
    allsize = sprintf ( "%.2f %s", all_size[i]/x,hum[x] )
        printf ( format,allsize, all_count[i], "Total");break } } } 
            } 
    }      
} ' >> $POSTMLOG

echo "" >> $POSTMLOG
echo `date` end of search >> $POSTMLOG

echo `date` send report to POSTMASTER >> $POSTMLOG

# Mail report to POSTMASTER
mail -s " usage.sh REPORT `date`" $POSTMASTER < $POSTMLOG
 if [ "$?" == "0" ]; then
  echo Mail sent successful! >> $POSTMLOG
 else echo Mail sent unsuccesful! >> $POSTMLOG
 fi

echo "" >> $POSTMLOG
echo `date` end of script usage.sh >> $POSTMLOG
echo "" >> $POSTMLOG

# Preserving logfile
mv "$POSTMLOG" "$DIR_LOG/$DATIME-$SEARCH-usage.log"

logger "Script end (usage.sh)"

And here is a possible outcome:
Code:
wo 10 aug 2011 15:52:42 CEST start of script usage.sh

Search variable = doc
......

......
/Library/MailDownloads/ObtainingNetVaultLicenseKeysLATEST.doc : 36352
./Library/MailDownloads/Philips.doc : 43008
./Library/MailDownloads/PO-nov2010.doc : 110080
./Library/MailDownloads/UseragreementVPNServeraccess&Authentication(v-1.1).doc : 34304

99 Files Found

Total Size = 42,22 Mb

       Size  Count Who             
   42,22 Mb     99 ronald          
   42,22 Mb     99 Total           

wo 10 aug 2011 15:52:46 CEST end of search
wo 10 aug 2011 15:52:46 CEST send report to POSTMASTER

wo 10 aug 2011 15:52:46 CEST end of script usage.sh

Now the thing is that in stead of the user I want to see the department directory (these are on a different server, but it is on 1 level)

As you can see in the output I can see the complete path. But I am want the 5th value (the level of the departments dirs) in a variable.

Many thanks
# 6  
Old 08-10-2011
here is a possible solution. You may have to tweak it to the input.

Declare a new variable "dept" like shown below. See how I am using split on $9 which should have your dept name
Once you have done that, replace in your script anywhere you find u_count[user]++; with u_count[dept]++;

Quote:
# Assign field names
sizes=$5
user= $3
split($9,a,"/")
dept=a[2]
Here is the sample output. Like I said you may have to tweak. I am assuming my dept. names to be Documents and Downloads :).

Search variable = zip

./Documents/conkyscripts.zip : 4777
./Downloads/ebook.zip : 1561317
./Downloads/conkyscripts.zip : 4777
./Downloads/TEST.zip : 26494027

4 Files Found

Total Size = 26.76 Mb

Size Count Who
26.76 Mb 3 Downloads
4.67 Kb 1 Documents
26.76 Mb 4 Total
This User Gave Thanks to dude2cool For This Post:
# 7  
Old 08-11-2011
Quote:
Originally Posted by dude2cool
here is a possible solution. You may have to tweak it to the input.

split($9,a,"/")
dept=a[2]
This is really good.. And it works as well Smilie Many thanks!

Could you explain a bit more about the variable?
- What does the a mean in the split ?
- what does the [2] do in the variable ?

edit: Ok, done some google on it and this explains it pretty good:

Code:
split(string, array, fieldsep)
    This divides string into pieces separated by fieldsep, and stores the pieces in array.
    The first piece is stored in array[1], the second piece in array[2], and so forth. 
    The string value of the third argument, fieldsep, is a regexp describing where to split string (much as FS can be a regexp describing where to split input records).
    If the fieldsep is omitted, the value of FS is used. split returns the number of elements created. The split function, then,
    splits strings into pieces in a manner similar to the way input lines are split into fields. For example:

    split("auto-da-fe", a, "-")

    splits the string `auto-da-fe' into three fields using `-' as the separator. It sets the contents of the array a as follows:

    a[1] = "auto"
    a[2] = "da"
    a[3] = "fe"

again many thanks!

Still have to do some tweaking (as you said) because some Dirs have spaces in them..

Last edited by Cowardly; 08-11-2011 at 04:51 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk - Removing extra character when setting variable

I have a data file d0 that looks like this: $cat d0 server1 running -n-cv- 8G 3.1% 1435d 15h server2 running -n---- 8G 39% 660d 22h server3 running -n--v- 8G 2.5% 1173d 6h server4 running -n---- 8G 1.1% 1048d 20h... (2 Replies)
Discussion started by: jake0391S
2 Replies

2. Shell Programming and Scripting

awk question : system output to awk variable.

Hi Experts, I am trying to get system output to capture inside awk , but not working: Please advise if this is possible : I am trying something like this but not working, the output is coming wrong: echo "" | awk '{d=system ("date") ; print "Current date is:" , d }' Thanks, (5 Replies)
Discussion started by: rveri
5 Replies

3. UNIX for Dummies Questions & Answers

awk - change variable on fly

Hello All, I have csv file, where one of fields is date (yyyy/mm/dd 00:00:00). Using awk I am trying to find all records with date newer/older than specific date. My idea was to compare unix timestamps of both dates: start=`date +%s -d "$DateStart"` awk -v start="$start" -v current=`date +%s... (34 Replies)
Discussion started by: haczyk
34 Replies

4. Shell Programming and Scripting

help on awk---- need to assign the output of awk to a variable

hi i want to find the size of a folder and assign it to a variable and then compare if it is greater than 1 gb. i am doin this script, but it is throwing error.... #!/bin/ksh cd . | du -s | size = awk '{print $1}' if size >= 112000 then echo size high fi ERROR : (4 Replies)
Discussion started by: Nithz
4 Replies

5. Shell Programming and Scripting

setting a shell script variable in awk

The following is part of a larger shell script grep -v "Col1" my_test.log | grep -v "-" | awk '$5 == "Y" {print $1}' instead of printing, can I set set $1 to a variable that the rest of the shell script can read? if $5 == Y, I want to call another shell script and pass $1 as a... (2 Replies)
Discussion started by: guessingo
2 Replies

6. Shell Programming and Scripting

awk: change string variable

Hi. How to change string variable in awk? for example, I parse with awk script text file named some_name_with_extension.txt I want to print only some_name in my script .... varCompName = FILENAME print varCompName How to put not all symbols from FILENAME to variable? thank you This... (4 Replies)
Discussion started by: cintlt
4 Replies

7. Shell Programming and Scripting

awk: Eliminating white space while setting variable

Hi, I have a large flat file from host without delimiter. I'm transforming this file to a csv file using statements like # Row 03: Customer / field position 3059 +20 WOFABNAM=substr( $0, 3059, 20 ); and deleting the trailing whitespaces before and after with that sub( /^ +/, "",... (4 Replies)
Discussion started by: Celald
4 Replies

8. Shell Programming and Scripting

How to change awk and shell variable value?

In awk script, #!/bin/sh awk 'BEGIN{i=0;}{i=i+5;}END{print i}' in.txt vr=0; vr=$i; echo "$vr" How can i assign that value of i in $vr(variable) of shell script? (7 Replies)
Discussion started by: cola
7 Replies

9. Shell Programming and Scripting

setting a variable, using SSH and awk?

hi there I am trying to get a value from a remote machine into a local variable. To get this value i want to use awk but im having trouble getting it to run, am i escaping in the right places here and using the right quotes (i must have tried a million combinations :() # VAR=`ssh server1... (5 Replies)
Discussion started by: hcclnoodles
5 Replies

10. UNIX for Dummies Questions & Answers

setting environment variable in awk

Dear all, I have a data sample... Dose: Summed ROI: Bladder ************************** Bin Dose Volume 001 0.700 100.000 002 0.715 99.998 168 3.142 0.368 169 3.157 0.338 170 3.171 0.292 Dose: Summed ROI:... (2 Replies)
Discussion started by: tintin72
2 Replies
Login or Register to Ask a Question