The UNIX and Linux Forums - Shell Programming and Scripting

looping through mysql select results in a shell script

vanessafan99 — Thu, 17 May 2012 01:47:32 GMT

my query is like this

Code:

  mysql -sN -e "select user_name, user_email, user_group from table name LIMIT 0, 10" |

  while read user_name user_email user_group ; do



 something





  done

basically want to save the usernames and email to a group
but the problem is if the email is empty, then
the output is like
username 2

and then it thinks the email is 2
when the email is actually blank

so when i do read user_name user_email user_group
and if the table is blank then the select returns
user_name and user_group
and then it thinks the email is the user_group

if that makes sense

do you know how to fix this?

thanks!!

Find a particular directory in multiple file systems and calculate total size

Sam1974 — Wed, 16 May 2012 21:50:46 GMT

Hello :

I need some help in writing a ksh script which will find a particular directory in all the file systems in a server and finally report the total size of the direcotry in all the file systems.

Some thing like this..

find /u*/app/oracle -type d -name "product" -prune

and then iteratively report the size of the product directory per file system and report the total size for the directory in all file systems in the server.

Any help would be greatly appreciated.

Thanks in Advance.

Script to report database home in unix

Sam1974 — Wed, 16 May 2012 21:03:35 GMT

Hello:

I am trying to build a ksh shell script which would summarize space occupied by "product" directory under /u*/app/oracle
and summarize and report the total size occupied by it in one server.

Environment is this...

HP UNIX 11i

Korn Shell

some thing like
/u402/app/oracle/product
du -s /u402/app/oracle/product
24991150 /u402/app/oracle/product - A
/u12/app/oracle/product
du -s /u12/app/oracle/product
12975244 /u12/app/oracle/product - B
/u25/app/oracle/product
du -s /u25/app/oracle/product
11699776 /u25/app/oracle/product - C

Total size in MB ==> A+B+C
Some thing like this.
Regards,
Sammy

Help needed in reading line value into variable

sarat949 — Wed, 16 May 2012 20:55:13 GMT

I have a file which has data in 8 lines:

Code:

I am trying to read each value into each different variable so that I use these variables in preparing my report.

Another option is to dynamically print each line values in the report like below:

Code:

users with privA: line 1 value

users with privB: line 2 value

users with privC: line 3 value

users with privD: line 4 value

users with privE: line 5 value

users with privF: line 6 value

users with privG: line 7 value

users with privH: line 8 value

Please help me out with the correct command

Thanks in advance.

Trying to extract domain and tld from list of urls.

chamb1 — Wed, 16 May 2012 19:23:52 GMT

I have done a fair amount of searching the threads, but I have not been able to cobble together a solution to my challenge. What I am trying to do is to line edit a file that will leave behind only the domain and tld of a long list of urls. The list looks something like this:

Code:

www.google.com

ja.wikipedia.org

bbc.co.uk

fr-fr.facebook.com

and I would like to end up with:

Code:

google.com

wikipedia.org

bbc.co.uk

facebook.com

I prefer bash, but am learning ruby and perl....though not very good at them yet. I have used ruby's URI function to extract the input links above...is there another ruby function I am overlooking for domain.tld?

Thanks!

top 10 highest and lowest percentile from a column

kylle345 — Wed, 16 May 2012 19:05:44 GMT

Hi, I want to extract the the top 10 and lowest 10 percentile for a column of values.

For example in column 2 for this file:

Code:

JOE   1 

JAY   5

JAM   6

JIL   8

JIB   4

JIH   3

JIG   2

JIT   7

JAM   9

MAR   10

The top 10 lowest will be:

Code:

JOE 1

and the top 10 highest will be:

Code:

MAR 10

If there is a quick way of doing this then it would be great

Thanks

average each column in a file

kylle345 — Wed, 16 May 2012 18:26:35 GMT

Hi, I tried to do this in excel but there is a limit to how many rows it can handle.

All I need to do is average each column in a file and get the final value.

My file looks something like this (obviously a lot larger):

Code:

Joe HHR + 1 2 3 4 5 6 7 8

Jor HHR - 1 2 3 4 5 6 7 8

the output would be (average values only):

Code:

1 2 3 4 5 6 7 8

Thanks

Extracting the last 10mins worth of data in a log file

Goldengreen — Wed, 16 May 2012 17:03:29 GMT

Hi all,
Hope someone here will be able to help me. Our system has some scripts that are run from a cron job every ten mins and is used to see how many error there are in that time frame. Problem is that in the scripts grep is used to find the data, but as the day goes on these log file grow to a huge size and the whole process is getting slower and slower.

What I need to do if figure out a smarter way of doing it. So what I was looking for is there some way I can extract the last 10 mins (even have this variable) worth of data from the logs, with out having to scan through the whole file.

I was think maybe of reading the file from the bottom up to 10 mins ago, some thing like the tac command, but the problem with the tac command is it will read the whole file which I want to avoid.

All the entries in the log file are timestamped in the following format:
2012-05-16 17:00:46

I am on a linuix system and using ksh.
Scripting is not a strong point of mine
Thanks all in advance.

pull out rows with specific criteria

phil_heath — Wed, 16 May 2012 16:44:55 GMT

Hi,
I have a file with multiple columns and for column 5 I want to extract that row into another file if it is equal to or greater than a certain value.

For example:

Code:

FAR  4  5  7  LOP

GAT  3  3  3  POL

I want values that are greater than 5 for column 4. So the final file will look like this.

Code:

FAR 4 5 7 LOP

Thanks

Phil

Monitoring log for FATAL in last 15mins only

stunnerz_84 — Wed, 16 May 2012 16:33:53 GMT

Hi all,
I need to develop the following script. Could anyone please provide me some inputs.

1) Monitor the system log file (sys.log) for any lines in the parameter file that will contain all the different phrases of FATAL errors that the process has to scan for in the sys.log.

If any FATAL messages are found then an email notification has to be sent.

2) The process should look for errors only from the point of time when the last monitoring took place. For eg; if the first monitoring process ran at 8:00am CT, the next one that runs at 8:15 am should look for these errors from 8:01am time mark

This is how the sys.log looks like:

Code:

May 15 11:10:41  xy_updserv[29245]: +++INFO+++ initialization complete

May 15 11:10:48  xy_qryserv[29243]: +++INFO+++ initialization complete

May 15 11:10:49  xy_qryserv[12345]: RPC: using socket receive timeout of 0 seconds

May 15 11:10:49  xy_updserv[13445]: RPC: using socket receive timeout of 0 seconds

May 15 11:12:32  xy_qryserv[12343]: +++ERROR+++ Warning: could not load '/abc/az/xyz/file.fe' 

May 15 11:27:21  xy_qryserv[10905]: +++ERROR+++ Info: FATAL - data missing

May 15 11:42:19  xy_qryserv[731]: +++FATAL+++ initialization failed. Aborting...

This is how the param file looks like:

Code:

+++ERROR+++ Info: FATAL - data missing

+++FATAL+++ initialization failed. Aborting...

Thanks all!!

Moderator's Comments:

Code tags for code, please.

Change numbers in a file, incrementing them

fzd — Wed, 16 May 2012 14:44:47 GMT

Hello,

Here's a file of mine:

Code:

key1:431

key2:159

key3:998

I need to change these keys to something bigger - and I actually need to shift them all by a range of 3.

The output would be:

Code:

key1:434

key2:162

key3:1001

I can't find the propper sed/awk line that would alter all my lines.

The plan-B would be to parse each line, get the value, increment it, and replace it. But I'm sure there's something smarter. .

Thanks

Moderator's Comments:

edit by bakunin: Please use [CODE]...[/CODE]-tags when posting code or terminal output. Thank you.

Multi-line filtering based on multi-line pattern in a file

Finja — Wed, 16 May 2012 14:12:46 GMT

I have a file with data records separated by multiple equals signs, as below.

==========
RECORD 1
==========
RECORD 2
DATA LINE
==========
RECORD 3
==========
RECORD 4
DATA LINE
==========
RECORD 5
DATA LINE
==========

I need to filter out all data from this file where the record has no data lines. So in the example above, I would need RECORD 1 and RECORD 3 to be filtered out, plus the extra rows of equals signs separating them. So I would like the output to be:

==========
RECORD 2
DATA LINE
==========
RECORD 4
DATA LINE
==========
RECORD 5
DATA LINE
==========

I can get done whan I need on single lines, but I am stuck trying to match the pattern over multiple lines, and then deleting multiple lines.

assign multiple rows value to a variable using eval

black_fender — Wed, 16 May 2012 13:39:10 GMT

background : Solaris, ksh

Code:

metresult="ooo

> pp"

ts=89

eval append_${ts}="$metresult"

bash: pp: command not found

I want to create a variable which has in a part of its name a dynamically-established number (stored in another variable) usually I do this with eval command. The problem I have now is that I want to assign to this variable a multiple row value and I get the error you see above ("pp" is the word from the second row from $metresult value.

If I want to assign a single-row-value to such a variable using eval everything works fine :

Code:

 metresult="ooo"

ts=89

eval append_${ts}="$metresult"

echo $append_89

ooo

I do something wrong syntactically speaking and I wonder if you have another approach. I do not want workarounds with arrays a.s.o. I only want to know this : how to assign multiple rows value to a variable using eval command

THX

Send correct exit code from child script back to parent

gvolpini — Wed, 16 May 2012 13:38:15 GMT

Hello all; hope someone can help me cause I am going crazy trying to find a solution for (what I think is simple) issue...looked hard up and down this forum and tried several "solutions" with no avail...so here's my issue:

I have this (parent) script: copylsofdcmcadefttosftpwithmove.sh

Inside this script I have this piece of code:

Code:

if [ "$flag" = "yes" ]

then

        /usr/local/bin/ssh $sftpserver mv $sftppath/$filenameshort $finaldirname  >> ${_ftpdebug} 2>&1

        ssh faunus:/home/dslmain/scripts/senddcmcadefttest.sh

        if [ $? -ne 0 ]

        then

           echo "EXIT: " $?

           exit

        fi

        echo "DONE"

else

        echo "FAILED"

fi

I call another script via ssh (red text).

This is the contents of the script: senddcmcadefttest.sh

Code:

#!/bin/sh -x

node=fid

NODE=FAUNUS

_Ymd=`date +%Y%m%d`

indir=/home/dslmain/export/${node}/eft/in

sentdir=/home/dslmain/export/${node}/eft/sent

LOG="/home/dslmain/export/$node/eft/logs/eft.log.`date +%m%d`"

_temp=/utemp/tmp_logs/eftexists

if [ -s $_temp ]

then

  rm $_temp

fi

_ym=`date +%m%d`



ls ${indir}/*${_ym}* > $_temp

if [ ! -s $_temp ]

then

   echo "No EFT files detected" >> ${LOG}

   exit 1

fi

cd ${indir}

for FILE in `cat $_temp|awk -F'/' '{print $NF}'`

do

  if [ -s ${FILE} ]

  then

    echo "Here _UPLOAD function would have run"

    echo "Snapshot of files of the day found on RBC FTP" >> ${LOG}

    A=1

    B=1

    echo $A >> ${LOG}

    echo $B >> ${LOG} 

    if [ ${A} -eq ${B} ]

    then

      echo "${FILE} was successfully uploaded to RBC" >> ${LOG}

      echo ""  >> ${LOG}

      mv ${indir}/${FILE} ${sentdir}/${FILE}

      exit 0

    else

      echo "${indir}/${FILE} failed to be loaded to RBC" >> ${LOG}

      echo " "  >> ${LOG}

      exit 2

    fi

  else

    echo "Sorry file dose not exist, or file is empty" >> ${LOG}

    exit 3

  fi

done

So in the child script I have 3 different scenarios (in red) where the exit code will be either 0, 2 or 3...I want to pass the correct exit code back to the parent script copylsofdcmcadefttosftpwithmove.sh so that it can be evaluated in the "if" statement...however $? is always coming back as "0"...

Thanks for reading and any assistance.

Regards
Giuliano

Moderator's Comments:

Please use next time code tags for your code and data

delete ftp accounts created in cpanel

vanessafan99 — Wed, 16 May 2012 13:28:42 GMT

i want to delete the extra ftp accounts that are created in cpanel
e.g., when you go to cpanel->ftp accounts, any that are not there by default

how can i do this via commandline?

i am looping through the users on the server
where is this info stored?

also - how can the password of an ftp account be set via the command line?
thanks