Shell Programming and Scripting

Hi everyone,

I'm stuck with this scenario where our system creates files every night and puts them in several folders according from whom it came from.

I have managed to create a script which will list the folders in one column and the files that are in them in another column, now my problem is when some of these folders have multiple files created for the day I have a double entry in the folders column. Below is an example of what I mean.

Folder1 File1-1
Folder2 File2-1
Folder3 File3-1
Folder3 File3-2
Folder4 File4-1

This is how I would like it to show;

Folder1 File1-1
Folder2 File2-1
Folder3 File3-1, File3-2
Folder4 File4-1

How can I achieve this. Please guide me resolve this problem.

This is an example of my script,

#!/bin/sh
find /home/user/ -name testfile* | sort -k 4 | cut -c20-24 > folders
find /home/user/ -name testfile* | sort -k 4 | cut -c30-36 > files
paste folders files > final
cat final

Thank you all for your help.

In perl

Code:

open(FI, "<", $ARGV[0]);

while (<FI> ) {
  chomp;
  my ($folder, $file) = split(/ /);
  push( @{$data_hash{$folder}}, $file);
}

close(FI);

foreach my $k ( keys %data_hash ) {
  print "$k " , @{$data_hash{$k}}, "\n";
}

What are "folders"? Do you mean directories?

Code:

for dir in Folder*
do
  printf "%s" "$dir"
  (
    cd "$dir"
    set -- *
    if [ -f "$1" ]
    then
      printf "\t"
      ## For bash 3.1 or later:
      #  printf -v list "%s, " "$@"
      ## Otherwise:
      list=$( printf "%s, " "$@" )
      printf "%s" "${list%, }"
    fi
   )
  echo
done

Thank you matrixmadhan & cfajohnson, I wish I could create scripts as good as you guys but unfortunately I have started using the unix enviroment only recently, hence I cant make heads or tails out of your scripts.

I wonder if it would be to much to ask you guys to explain to me the scripts that you have provided.

And cfajohnson, yes, by folders I do mean directories.

Thank you all for your efforts.

Code:

for dir in Folder*
## The shell expands Folder* to a list
## of all files and directories beginning with "Folder"
##
## "for" assigns each member of the list in turn to the variable $dir
## and executes the code between "do" and "done" using that value
do
  printf "%s" "$dir" ## print the name of the directory with no newline
  (
    cd "$dir"  ## Change the working directory to the one in $dir
    ## That should have included a test that the cd was successful:
    ## cd "$dir" ||
    ## continue ## go to the next iteration of the loop if not successful

    ## Place all the [non-hidden] files into the positional parameters
    set -- *

    if [ -f "$1" ] ## check that there actually is a file
    then
      printf "\t" ## print a TAB character

      ## Store all the filenames, with commas, in $list
      list=$( printf "%s, " "$@" )

      ## Print list of filenames without newlines
      ## and without the trailing comma
      printf "%s" "${list%, }"
    fi
   )
  echo ## print a newline
done

here is the explanation for the perl code I posted

Code:

open(FI, "<", $ARGV[0]) or die "Unable to open file : $ARGV[0] < $! > \n";
# open the file in read mode, first argument to the script is input file
# FI is the file handle
# If unable to open the file, just terminate from the script

my %data_hash = ();
# define a hash data structure

while (<FI> ) {
# while contents from file are available, iterate through the file through the file handle obtained above

  chomp;
# remove new line character from the input read from file which is based on $/ as known as INPUT_RECORD_SEPERATOR

  my ($folder, $file) = split(/ /);
# split the input record with space as delimiter

  push( @{$data_hash{$folder}}, $file);
# key to the hash is folder
# value is an array reference containing list of files corresponding to a folder
# push the file value in an array against the corresponding folder name
}

close(FI) or die "Unable to close file : $ARGV[0] < $! > \n";
# close the file handle that is opened above

foreach my $k ( keys %data_hash ) {

# iterate through the above built hash data structure
  print "$k " , @{$data_hash{$k}}, "\n";

# print the key and values
# key - folder name
# value - list of files for a folder
}

Hi guys,

First of all I would like to apologize for not replying to your posts, I have been away on holiday then went for a training session.

cfajohnson, I tried your script and it listed all files in each folder horizontally line by line.

Perhaps I should explain further how I need the file formatted. Below is the script I use to generate a file called output.xls.

DAY=`TZ=MYT+16 date '+%b'`
DAY1=`TZ=MYT+16 date '+%e'`
ls -ltr /bscswork_bi2/WORK/UMOBILE/IR/IN/ALL/PROCESSED | grep "$DAY $DAY1" | awk '{print $9}' | sort | cut -c3-7 > output1
ls -ltr /bscswork_bi2/WORK/UMOBILE/IR/IN/ALL/PROCESSED | grep "$DAY $DAY1" | awk '{print $9}' | sort | cut -c13-17 > output2
paste output1 output2 > output.xls
rm output1 output2

This is how the output.xls looks like (partial file).

AREDU 772
AREDU 773
AREDU 774
ARM05 17
AUSOP 251
AZEAC 51
AZEAF 264
AZEAF 265
AZEAF 266
AZEAF 267
BELKO 172
BGDAK 258
BGDWT 199
BGRVA 745
BRNBR 186
BRNDS 79
CYPSC 129
CYPSC 130
DZAA1 16
ESPRT 55
ESTRE 308
ESTRE 309

There are identical directory name but the file sequence differ in the above example.

And this is how I would like the output.xls file to look like.

AREDU 772, 773, 774
ARM05 17
AUSOP 251
AZEAC 51
AZEAF 264, 265, 266, 267
BELKO 172
BGDAK 258
BGDWT 199
BGRVA 745
BRNBR 186
BRNDS 79
CYPSC 129, 130
DZAA1 16
ESPRT 55
ESTRE 308, 309

Is there a way I could accomplish this. Please advice.

Thank you all for your assistance.