Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

Need to select files larger than 500Mb from servers

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Need to select files larger than 500Mb from servers
# 1  
Old 05-29-2018
Need to select files larger than 500Mb from servers
I need help modifying these two scripts to do the following:
- print files in (MB) instead of (KB)
- only select files larger than 500MB -> these will be mailed out daily
- Select all files regardless of size all in (MB) -> these will be mailed out once a week

this is what i have so far and it is now working well...help will be appreciated.


Code:
[set v_alert_command_db002 "; echo '            Alert Log (12cLNX): /oracle/UB1/saptrace/diag/rdbms/test/TEST/trace/alert_TEST.log - ' | tr '\\n' ' '; du -sk '/oracle/TEST/saptrace/diag/rdbms/test/TEST/trace/alert_TEST.log' | awk '{ print \$1 }' | tr '\\n' ' '; echo '(KB)' "]

[# add listener command for db003 server]
 [set v_listener_db003 "; echo '            Listener Log' | tr '\\n' ':'; echo '' | tr '\\n' ' ' ; lsnrctl status LISTENER_TEST01 | grep Log| awk '{ print \$4 }' | sed 's/alert\\/log.xml/trace\\/listener_test01.log/g' | tr '\\n' ' '; echo '-' | tr '\\n' ' ' ; du -sk `lsnrctl status LISTENER_TEST01 | grep Log | awk '{ print \$4 }' | sed 's/alert\\/log.xml/trace\\/listener_test01.log/g'` | awk '{ print \$1 }' | tr '\\n' ' ' ;  echo '(KB)'"]


Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!
Last edited by RudiC; 05-29-2018 at 12:51 PM.. Reason: Added CODE tags.
# 2  
Old 05-30-2018
Could you tell us what operating system you are using? The version of shell (bash3, bash4, ksh88, ksh93) would be useful too.

For instance, if I knew you were using a version of Linux I could tell you that you could replace
Code:
du -k

with
Code:
du -m

but if you are using Solaris that won't work. You could use wc -c to get the size of the file in bytes and then divide by 1048576 to convert to megabytes.

So in each case you want to suppress the output if the size of the file in question is smaller than 500Mb?

Andrew
# 3  
Old 05-30-2018
shell- #!/bin/ksh
operating system - 5.11 11.3 sun4v sparc sun4v
I would like the format for getting the size the same as in the example because thats how mgmt wants it and its part of an expect script. Yes i will like to suppress the output if the size of the file in question is smaller than 500Mb and -> these will be mailed out daily and again to:
- Select all files regardless of size all in (MB) -> these will be mailed out once a week
thanks for your input so far
# 4  
Old 05-31-2018
Do you have gdu on your system? It is the GNU version of du. If so, try this:
Code:
[set v_alert_command_db002 "; file='/oracle/TEST/saptrace/diag/rdbms/test/TEST/trace/alert_TEST.log'; size_M=$(gdu -m $file | nawk '{ print \$1 }'); (( size_M > 500 )) && printf \"            Alert Log (12cLNX): %s - %s (MB)\n\" $file $size_M"]
[set v_listener_db003 "; file=$(lsnrctl status LISTENER_TEST01 | nawk '/Log/ { print $4 }' | sed 's^alert/log.xml^trace/listener_test01.log/' ); size_M=$(gdu -m $file | nawk '{ print \$1 }'); (( size_M > 500 )) && printf \"            Listener Log: %s - %d (Mb)\n\" ${file} ${size_M}"]

You may have to add backslashes to some of the backslashes and other special characters. I don't know the tool you are using there so don't know the rules for escaping strings.

I've removed several pointless trs and greps and replaced the echos with printfs. Do you really need that
Code:
sed 's^alert/log.xml^trace/listener_test01.log/'

I don't think you do.

I haven't tested this as I cannot.

If you don't have gdu try replacing
Code:
size_M=$(gdu -m $file | nawk '{ print \$1 }')

with
Code:
size_M=$(( $(wc -c < ${file}) / 1048576 ))

Andrew
# 5  
Old 05-31-2018
[I've removed several pointless trs and greps and replaced the echos with printfs. Do you really need that!]

Thank you for the help i will test what you have provided and update the thread. I guess i do need them (trs) to stay in place.... i am just trying to mod the script that my lead gave me and he would not like that many changes made.

---------- Post updated at 10:58 AM ---------- Previous update was at 10:02 AM ----------

Code:
@apmcd47 ...Please see if you can help me figure this out....please.....!!!
This expect script is suppose to connect to SERVER DB01 that hosts two databases and pull the alert and listener log files sizes. The problem i am having is that the correct alert log/size file is pulled for both databases but the wrong listener log/size is pulled for testdb2...because in the output (below) the sid is set to testdb1 instead of testdb2. I need help in directing the script ot set the correct sid and to pull from the correct path for testdb2. I will also like the output to be in MB because i only want to print out the log files larger that 500MB and mail them out every day , then every friday i mail out the complete output.


[# command for gathering hostname
set v_host_command "echo hostname | tr '\\n' ':' ;hostname"]
[# Exception hostname
set v_host_null "echo''"]

[# command for gathering Listener Log
set v_listener_command "; echo ' Listener Log' | tr '\\n' ':'; echo '' | tr '\\n' ' ' ; lsnrctl status | grep Log | awk '{ print \$4 }' | sed 's/alert\\/log.xml/trace\\/listener.log/g' | tr '\\n' ' '; echo '-' | tr '\\n' ' ' ; du -sk `lsnrctl status | grep Log | awk '{ print \$4 }' | sed 's/alert\\/log.xml/trace\\/listener.log/g'` | awk '{ print \$1 }' | tr '\\n' ' ' ; echo '(KB)'"]


# add env setting command for db01 server TEST DB
[#set v_command_db01_testdb1 "export PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/home/users/oraclet/bin:/usr/bin/X11:/sbin:/usr/ccs/bin:/bin:/usr/local/bin:.:/oraclet/120/bin:/oraclet/120;export ORACLE_HOME=/oraclet/120;export ORACLE_SID=TESTDB1;"]

[# add env setting command for db01 server Catalog
set v_command_db01_testdb2 "export PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/home/users/oraclet/bin:/usr/bin/X11:/sbin:/usr/ccs/bin:/bin:/usr/local/bin:.:/oraclet/120/bin:/oraclet/120;export ORACLE_HOME=/oraclet/120;export ORACLE_SID=TESTDB2;"]

[# add env listener command for db01 server TEST DB
set v_listener_db01_1 "; echo ' Listener Log' | tr '\\n' ':'; echo '' | tr '\\n' ' ' ; echo '/oraclet/diag/tnslsnr/db01/listener/trace/listener.log' | tr '\\n' ' '; echo '-' | tr '\\n' ' ' ; du -sk /oraclet/diag/tnslsnr/db01/listener/trace/listener.log | awk '{print \$1 }' | tr '\\n' ' ' ; echo '(KB)'"]

[# add listener command for db01 server RMAN Catalog
set v_listener_db01_testdb1 "; echo ' Listener Log' | tr '\\n' ':'; echo '' | tr '\\n' ' ' ; echo '/oraclet/diag/tnslsnr/db01/listener_testdb2/trace/listener_TESTDB2.log' | tr '\\n' ' '; echo '-' | tr '\\n' ' ' ; du -sk /oraclet/diag/tnslsnr/db01/listener_testdb2/trace/listener_TESTDB2.log | awk '{print \$1 }' | tr '\\n' ' ' ; echo '(KB)'"]

[# now connect to remote UNIX box
} elseif {$v_server == $v_host_db01 && $v_user == $v_user_oraclet} {
spawn ssh $v_user@$v_server $v_command_db01_testdb1$v_host_command$v_exc_alert_command$v_listener_db01_1$v_end_command
match_max 100000]
[} elseif {$v_server == $v_host_db01 && $v_user == $v_user_oraclet} {
spawn ssh $v_user@$v_server $v_command_db01_testdb2$v_host_command$v_exc_alert_command$v_listener_db01_b$v_end_command
match_max 100000]

[# Look for passwod prompt
expect "*?assword:*"]

[# Send password
send "$v_password\r"]

[# send blank line (\r) to make sure we get back to gui
send "\r"
expect eof]


[running ssh oraclem@db01...
spawn ssh oraclem@db01 export PATH=/oraclet/120/bin:/usr/bin:/usr/ccs/bin:/usr/vac/bin:/usr/sbin:/sbin:/usr/bin/X11:/bin:/usr/local/bin:/usr/lbin:/usr/ucb:PATH:.;export ORACLE_HOME=/oraclet/120;export ORACLE_SID=TESTDB1;echo hostname | tr '\n' ':' ;hostname; echo ' Alert Log' | tr '\n' ':'; echo '' | tr '\n' ' ' ;echo /oraclet/diag/rdbms/testdb1/TESTDB1/trace/alert_TESTDB1.log | tr '\n' ' '; echo '-' | tr '\n' ' ' ; du -sk /oraclet/diag/rdbms/testdb1/TESTDB1/trace/alert_TESTDB1.log | awk '{ print $1 }'| tr '\n' ' ' ; echo '(KB)'; echo ' Listener Log' | tr '\n' ':'; echo '' | tr '\n' ' ' ; lsnrctl status | grep Log | awk '{ print $4 }' | sed 's/alert\/log.xml/trace\/listener.log/g' | tr '\n' ' '; echo '-' | tr '\n' ' ' ; du -sk `lsnrctl status | grep Log | awk '{ print $4 }' | sed 's/alert\/log.xml/trace\/listener.log/g'` | awk '{ print $1 }' | tr '\n' ' ' ; echo '(KB)';echo] [----------------------------------------------------------------------------------------------------------------------------------]
[Password:
hostname:db01
Alert Log: /oraclet/diag/rdbms/TESTDB1/TESTDB1/trace/alert_TESTDB1.log - 3 (KB)
Listener Log: /oraclet/diag/tnslsnr/db01/listener/trace/listener.log - 154255 (KB)]
[----------------------------------------------------------------------------------------------------------------------------------]
[running ssh oraclem@db01...(in this output its setting and connecting the the wrong sid ...testdb1...needs to be connecting to testdb2)
spawn ssh oraclem@db01 export PATH=/oraclet/120/bin:/usr/bin:/usr/ccs/bin:/usr/vac/bin:/usr/sbin:/sbin:/usr/bin/X11:/bin:/usr/local/bin:/usr/lbin:/usr/ucb:PATH:.;export ORACLE_HOME=/oraclet/120;export ORACLE_SID=TESTDB1;echo hostname | tr '\n' ':' ;hostname; echo ' Alert Log' | tr '\n' ':'; echo '' | tr '\n' ' ' ;echo /oraclet/diag/rdbms/testdb2/TESTDB2/trace/alert_TESTDB2.log | tr '\n' ' '; echo '-' | tr '\n' ' ' ; du -sk /oraclet/diag/rdbms/testdb2/TESTDB2/trace/alert_TESTDB2.log | awk '{ print $1 }'| tr '\n' ' ' ; echo '(KB)'; echo ' Listener Log' | tr '\n' ':'; echo '' | tr '\n' ' ' ; lsnrctl status | grep Log | awk '{ print $4 }' | sed 's/alert\/log.xml/trace\/listener.log/g' | tr '\n' ' '; echo '-' | tr '\n' ' ' ; du -sk `lsnrctl status | grep Log | awk '{ print $4 }' | sed 's/alert\/log.xml/trace\/listener.log/g'` | awk '{ print $1 }' | tr '\n' ' ' ; echo '(KB)';echo]
[----------------------------------------------------------------------------------------------------------------------------------]
[Password:
hostname:db01
Alert Log: /oraclet/diag/rdbms/testdb2/TESTDB2/trace/alert_TESTDB2.log - 770 (KB)
Listener Log: /oraclet/diag/tnslsnr/db01/listener/trace/listener.log - 154255 (KB)]
[Listener Log: /oraclet/diag/tnslsnr/db01/listener_testdb2/trace/listener_TESTDB2.log - 154255 (KB)]

# 6  
Old 06-01-2018
Quote:
Originally Posted by donpasscal
[I've removed several pointless trs and greps and replaced the echos with printfs. Do you really need that!]

Thank you for the help i will test what you have provided and update the thread. I guess i do need them (trs) to stay in place.... i am just trying to mod the script that my lead gave me and he would not like that many changes made.
Actually that question was aimed at the sed command, not the use of tr to suppress newline generation.
Quote:
@apmcd47 ...Please see if you can help me figure this out....please.....!!!
This expect script is suppose to connect to SERVER DB01 that hosts two databases and pull the alert and listener log files sizes. The problem i am having is that the correct alert log/size file is pulled for both databases but the wrong listener log/size is pulled for testdb2...because in the output (below) the sid is set to testdb1 instead of testdb2. I need help in directing the script ot set the correct sid and to pull from the correct path for testdb2. I will also like the output to be in MB because i only want to print out the log files larger that 500MB and mail them out every day , then every friday i mail out the complete output.
I'm sorry, I know nothing about expect and don't really know enough about Oracle to help you. My advice at this time would be to log onto the server and figure out how to get this to work interactively and then put what you have learnt into the expect script. If they won't let you do this, because it's a production server, then ask to use the development server.

I gave you two possible ways of getting the file size in MB in my last post; can you use either of them in your script?

And please, when posting code, use the CODE tags. It makes it easier to understand the code.

Andrew
This User Gave Thanks to apmcd47 For This Post:
donpasscal
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Help with Expect script for pulling log files size larger than 500Mb;

I am new at developing EXPECT scripts. I'm trying to create a script that will automatically connect to a several UNIX (sun solaris and HPUX) database server via FTP and pull the sizes of the listener/alert log files from specified server directory on the remote machines. 1. I want the script... (7 Replies)
Discussion started by: mikebantor
7 Replies

2. UNIX for Dummies Questions & Answers

Split larger files into smaller ones with Column names

Hi, I have one large files of 100000 rows with header column. Eg: Emp Code, Emp Name 101,xxx 102,YYY 103,zzz ... ... I want to split the files into smaller files with only 30000 rows each..File 1,2 and 3 must have 30000 rows and file 4 must contain 10000 rows. But the column... (1 Reply)
Discussion started by: Nivas
1 Replies

3. Shell Programming and Scripting

Backingup larger files with TAR command

I need to backup my database but the files are very large and the TAR command will not let me. I searched aids and found that I could do something with the mknod, COMPRESS and TAR command using them together. I appreciate your help. (10 Replies)
Discussion started by: frizcala
10 Replies

4. UNIX for Dummies Questions & Answers

7z command for files larger than 4GB ( unzip doesn't work)

My unzip command doesn't work for files that are greater than 4GB. Consider my file name is unzip -p -a filename.zip, the command doesn't work since the size of the file is larger. I need to know the corresponding 7z command for the same. This is my Unix shell script program: if then ... (14 Replies)
Discussion started by: chandraprakash
14 Replies

5. UNIX for Dummies Questions & Answers

Using UNIX Commands with Larger number of Files

Hello Unix Gurus, I am new to Unix so need some help on this. I am using the following commands: 1) mv -f Inputpath/*. outputpath 2) cp Inputpath/*. outputpath 3) rm -rf somepath/* 4) Find Inputpath/*. Now I get the following error with... (18 Replies)
Discussion started by: pchegoor
18 Replies

6. Shell Programming and Scripting

How to compare if the size of the directory is more than 500MB in unix

I use du -sk command to find the size of the directory but when i use the result of 'du -sk' into if statement its throwing error.. Could u solve with this..? (14 Replies)
Discussion started by: shaal89
14 Replies

7. AIX

Tar files larger than 2GB

Hi, Does anyone know if it is possible to tar files larger than 2GB? The reason being is they want me to dump a single file (which is around 20GB) to a tape drive and they will restore it on a Solaris box. I know the tar have a limitation of 2GB so I am thinking of a way how to overcome this.... (11 Replies)
Discussion started by: depam
11 Replies

8. UNIX for Advanced & Expert Users

read() wont allow me to read files larger than 2 gig (on a 64bit)

Hi the following c-code utilizing the 'read()' man 2 read method cant read in files larger that 2gig. Hi I've found a strange problem on ubuntu64bit, that limits the data you are allowed to allocate on a 64bit platform using the c function 'read()' The following program wont allow to allocate... (14 Replies)
Discussion started by: monkeyking
14 Replies

9. IP Networking

Select DNS Servers depending on the domain

Hello, I'm using CentOS 5.3, and I connect to a VPN in order to work. The problem is that I'm constantly accessing things on the local network and the remote network. But once I'm connected to the VPN I can't access local addresses by name, I have to use the ip-address. What I'd like is to... (4 Replies)
Discussion started by: martincastell
4 Replies

10. UNIX for Advanced & Expert Users

sending larger files via ftp

hi all, i am looking for ways to make ftp efficient by tuning the parameters currently, tcp_max_buf is 1 MB tcp_xmit_hiwat is 48 KB say to transmit multiple 2 gb files from unix server to mainframe sys, will increasing the window size or the send buffer size of the current TCP/IP... (6 Replies)
Discussion started by: matrixmadhan
6 Replies
Login or Register to Ask a Question