Two questions about my script : apply to all my csv and recover filename ?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Two questions about my script : apply to all my csv and recover filename ?
# 1  
Old 07-08-2019
Two questions about my script : apply to all my csv and recover filename ?

Hello,

I allow me to writing here to seek your assistance about my script bash !

I have many csv files that looks like :

Code:
MO2PPC20,60,1.5,4
MO2PPC20,39,0.3,5
MO2PPC20,105,2.0,4
MO2PPC20,91,2.0,4
MO2PPC20,79,0.4,4
MO2PPC20,62,1.2,3
MO2PPC20,69,0.3,4
MO2PPC20,60,0.6,4
MO2PPC20,60,1.2,4
MO2PPC20,12,0.5,1
MO2PPC20,130,1.5,11
MO2PPC20,180,0.5,3

The first column is the name of my frame, the second the RAM, the third is the CPU1 value and the last column is the CPU2 value


I want to display thoses informations like that :
Code:
 
FRAME : MO2PPC20 ========================================

RAM : 60
CPU 1 : 1.5
CPU 2 : 4

RAM : 39
CPU 1 : 0.3
CPU 2 :5

etc...

So, to do that, I use this little script :

Code:
#!/bin/bash

OLDIFS=$IFS
IFS=","


while read FRAME RAM CPU1 CPU2
do
    if [[ $FRAME != $PREV ]]
    then
        PREV="$FRAME"
        echo -e "FRAME : $FRAME \
========================================\n"
fi

echo -e "RAM :\t$RAM\n\
CPU 1 :\t$CPU1\n\
CPU 2 :\t$CPU2\n"
echo ""
done < <(sort "my_csv_file.csv")

My script works well in the case that I have one file in process.


So my first question is :

How to apply this script to all my csv in my directory ?

I think something like this :

Code:
 
for i in ` ls *.csv`
do
./my_script $i
done

And change the last line of my script for something simple such as :

Code:
 #!/bin/bash

OLDIFS=$IFS
IFS=","


while read FRAME RAM CPU1 CPU2
do
    if [[ $FRAME != $PREV ]]
    then
        PREV="$FRAME"
        echo -e "FRAME : $FRAME \
========================================\n"
fi

echo -e "RAM :\t$RAM\n\
CPU 1 :\t$CPU1\n\
CPU 2 :\t$CPU2\n"
echo ""
done

But it doesn't work...


And, the second question :

As soon as I am able to apply my script to all my csv, the informations from my csv will be display without distinction... I want to be able to display the file in process by my script like that :

Code:
File : my_csv_1.csv

FRAME : MO2PPC20 ========================================

RAM : 60
CPU 1 : 1.5
CPU 2 : 4

RAM : 39
CPU 1 : 0.3
CPU 2 :5


File : my_csv_2.csv

FRAME : XXXXXX

RAM : XXX
CPU 1 : XXX
CPU 2 : XXX


etc ...

Do you have any solutions for theses two questions ?

Thank you ! Smilie
# 2  
Old 07-08-2019
You need to supply the read command with something to read from. Which you did in the first version, but which you missed in the second. Either redirect the last line from the first positional parameter when calling the script as you do now, or redirect the entire script's stdin from $i in the caller.
Same two options for the file name: Either echo $i in the caller, or echo $1 early in the script itself.
# 3  
Old 07-08-2019
Most efficient with awk:
Code:
awk -F, 'p1!=$1 { print "\nFRAME :", p1=$1i, "=====" } { print "\nRAM :", $2; for (i=3; i<=NF; i++) print "CPU ", i-2, ":", $i }'  *.csv

The same can be done with shell:
Code:
#!/bin/bash
pframe=
# loop over my arguments
for arg
do
  while IFS=, read -a col
  do
    if [[ $col != $pframe ]]
    then
      pframe=$col
      echo "
FRAME : $col ========================================"
    fi
    echo "
RAM : ${col[1]}"
    for ((i=2; i<${#col[@]}; i++))
    do
      echo "CPU $((i-1)) : ${col[i]}"
    done
  done <"$arg" #while
done #for

# 4  
Old 07-08-2019
Hello ! Smilie

Thank you for your answers !



Quote:
Originally Posted by MadeInGermany
Most efficient with awk:
Code:
awk -F, 'p1!=$1 { print "\nFRAME :", p1=$1i, "=====" } { print  "\nRAM :", $2; for (i=3; i<=NF; i++) print "CPU ", i-2, ":", $i }'   *.csv


The problem with your proposition is that your script display all FRAMES with all LPARS. At the begin, I used a similary script than yours but to be more efficient, I wanted to display one FRAME with his LPARSs. But thank you for your proposition ! Smilie


Quote:
Originally Posted by RudiC
You need to supply the read command with something to read from. Which you did in the first version, but which you missed in the second. Either redirect the last line from the first positional parameter when calling the script as you do now, or redirect the entire script's stdin from $i in the caller.
Same two options for the file name: Either echo $i in the caller, or echo $1 early in the script itself.
I try something like this :

Code:
#!/bin/bash

OLDIFS=$IFS
IFS=","

while [[ $# -gt 0 ]]
do
    while read FRAME LPARS RAM CPU1 CPU2
    do
        if [[ $FRAME != $PREV ]]
        then
            PREV="$FRAME"
            echo -e "FRAME : $FRAME \
            ==============================\n"
        fi
           echo -e "LPARS : $LPARS\n\
          RAM : \t$RAM\n\
          CPU1 : \t$CPU1\n\
          CPU2 : \t$CPU2\n"
          echo ""
          done < $1
          shift
done

And that works perfectly.

Still the second question as a problem. I don't know how to put the name of the file in process at the begin of each " Frame list ".
# 5  
Old 07-08-2019
Sorry, an extra i slipped into my awk code. Should be p1=$1 not p1=$1i.

With printing the file names:
Code:
awk -F, 'FILENAME!=pFN { printf "File : %s\n\n", pFN=FILENAME }  p1!=$1 { print "FRAME :", p1=$1, "=====\n" } { print "RAM :", $2; for  (i=3; i<=NF; i++) print "CPU " i-2, ":", $i; print "" }'  *.csv

The same with a bash script (run with: ./scriptname.sh *.csv or find ... -exec ./scriptname.sh {} +):
Code:
#!/bin/bash
pframe=
# loop over my arguments
for arg
do
  echo "File : $arg
"
  while IFS=, read -a col
  do
    if [[ $col != $pframe ]]
    then
      pframe=$col
      echo "\
FRAME : $col ========================================
"
    fi
    echo "RAM : ${col[1]}"
    for ((i=2; i<${#col[@]}; i++))
    do
      echo "CPU $((i-1)) : ${col[i]}"
    done
    echo
  done <"$arg" #while
done #for


Last edited by MadeInGermany; 07-08-2019 at 05:54 PM.. Reason: awk: use printf to avoid a trailing space after the file name
# 6  
Old 07-10-2019
Quote:
Originally Posted by Tim2424
Hello ! Smilie
Still the second question as a problem. I don't know how to put the name of the file in process at the begin of each " Frame list ".
Simply echo the parameter 1 at the top of your loop (change in red below).

Code:
#!/bin/bash

OLDIFS=$IFS
IFS=","

while [[ $# -gt 0 ]]
do
    echo "File : $1"
    while read FRAME LPARS RAM CPU1 CPU2
    do
        if [[ $FRAME != $PREV ]]
        then
            PREV="$FRAME"
            echo -e "FRAME : $FRAME \
            ==============================\n"
        fi
        echo "LPARS : \t$LPARS"
        echo "RAM : \t$RAM"
        echo "CPU1 : \t$CPU1"
        echo "CPU2 : \t$CPU2"
    done < $1
    shift
done

# 7  
Old 08-08-2019
Hello !

Finally I use this script :

Code:
awk -F',|;' -v test="$test" '
     NR==1 { 
        split(FILENAME ,a,"[-.]");
      }
      $0 ~ test {
          if(!header++){
              print "DATE ========================== : " a[4] 
          }
          print ""
          print "LPARS :" $2
          print "RAM : " $5
          print "CPU 1 : " $6
          print "CPU 2 : " $7
          print "" 
          print ""
      }' $fn;


That allow me to :

- Display the name of my csv file and only keep what I want
- Delete the useless/empty lines

Thank you ! Smilie
This User Gave Thanks to Tim2424 For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script to apply functions to multiple columns dynamically

Hello, I have a requirement to apply hashing algorithm on flat file on one or more columns dynamically based on header sample input file ID|NAME|AGE|GENDER 10|ABC|30|M 20|DEF|20|F say if i want multiple columns based on the header example id,name or id,age or name,gender and hash and... (13 Replies)
Discussion started by: mkathi
13 Replies

2. Shell Programming and Scripting

How can I apply 'date' command to specific columns, in a BASH script?

Hi everyone, I have a situation in which I have multiple (3 at last count) date columns in a CSV file (, delim), which need to be changed from: January 1 2017 (note, no comma after day) to: YYYY-MM-DD So far, I am able to convert a date using: date --date="January 12, 1990" +%Y-%m-%d ... (7 Replies)
Discussion started by: richardsantink
7 Replies

3. Shell Programming and Scripting

Apply md5 hash to a field in csv file

I have a .csv file and I want to md5 hash the second column for each row in the file. File is something like data1,foobar1,123,345 data2,foobar2,456,9393 data3,foobar3,1002,10109 Output would be like data1,6c81243028f8e455fa617dd5f0232ce1,123,345... (3 Replies)
Discussion started by: jjwags
3 Replies

4. Shell Programming and Scripting

[Beginner's questions] Filename Validation & Parsing

Hi !! I'm rather new both to the UNIX and scripting worlds, and I'm learning the ropes of scripting. Having said this, please excuse me if you notice certain basic errors. I'm working on a script that implements .jar and .war files for a WAS environment and I need to perform certain... (4 Replies)
Discussion started by: levaldez
4 Replies

5. Shell Programming and Scripting

Merge CSV files and create a column with the filename from the original file

Hello everyone!! I am not completely new to shell script but I havent been able to find the answer to my problem and I'm sure there are some smart brains here up for the challenge :D. I have several CSV files that I need to combine into one, but I also need to know where each row came from.... (7 Replies)
Discussion started by: fransanchezoria
7 Replies

6. Homework & Coursework Questions

Print questions from a questions folder in a sequential order

1.) I am to write scripts that will be phasetest folder in the home directory. 2.) The folder should have a set-up,phase and display files I have written a small script which i used to check for the existing users and their password. What I need help with: I have a set of questions in a... (19 Replies)
Discussion started by: moraks007
19 Replies

7. Shell Programming and Scripting

Script to find/apply Solaris 10 ACL's

This may be a question for a different forum, but as I will need a script I thought I would start here. We recently migrated from Solaris 8 to Solaris 10. The file system in question here is ZFS, meaning the method for listing and applying ACL's has changed dramatically. To make a long story... (3 Replies)
Discussion started by: Shoeless_Mike
3 Replies

8. Shell Programming and Scripting

Apply script to several archives in several Folders.

Hello. I'm here again. I have a script in python and bash, and I need execute this script over all files in all folders. Example: Folder: CMDB Subfolders: router1 router2 switch1 switch2 and in this folders exists a file called... (3 Replies)
Discussion started by: bobbasystem
3 Replies

9. Shell Programming and Scripting

Need to know how to recover the shell script!

Hi All, I have a very typical problem and need you guys help I was about to modify the content inside one shell script (sample.sh) and before doing this i took a backup of it. But the problem is this script didn't get any issue but while creating the new script i have given this content and... (1 Reply)
Discussion started by: kumar16
1 Replies

10. UNIX for Dummies Questions & Answers

Few script questions

Hi all, Just two quick questions about writting some scripts. The script I am writting has to be able to add users. Well I can work out the commands I need to put in for the user to be added. But how would I need to do to set the password for that user. Keeping in mind the script will be run... (1 Reply)
Discussion started by: merlin
1 Replies
Login or Register to Ask a Question