|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
Help with File Slow Processing
Post 302534593 by srattani on Tuesday 28th of June 2011 08:42:31 AM
06-28-2011
Hi methyl,
Thanks for look at my post I really appreciate it, I am new to Unix scripting so def. need guidance. Please see my answers
What Operating System and version are you running? It is sun solaris
What Shell is /bin/sh on your computer? How do I tell? I just know i am using sh
How many lines are processed from the 133 files? Is it definitely not the whole of each file? Each file has a number of records on very first line, I read that and process those many rows it can be anywhere from 10 to 200
Does the script work? Yes the script works but each file is taking approx 4 seconds to process and 133 files are taking 523 seconds which is almost 8 minutes for 133 files for 1 day and I have to process it for 400 days
which wud take 53 hours
What are these lines for? Is there a local reason for these complex redirects? I copied it from a colleague so if you think there is no reason for these redirections I would appreciate your guidance
Thanks for look at my post I really appreciate it, I am new to Unix scripting so def. need guidance. Please see my answers
What Operating System and version are you running? It is sun solaris
What Shell is /bin/sh on your computer? How do I tell? I just know i am using sh
How many lines are processed from the 133 files? Is it definitely not the whole of each file? Each file has a number of records on very first line, I read that and process those many rows it can be anywhere from 10 to 200
Does the script work? Yes the script works but each file is taking approx 4 seconds to process and 133 files are taking 523 seconds which is almost 8 minutes for 133 files for 1 day and I have to process it for 400 days
which wud take 53 hours What are these lines for? Is there a local reason for these complex redirects? I copied it from a colleague so if you think there is no reason for these redirections I would appreciate your guidance
|
|
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
Hello Guru,
I am using a Pro*C program to prepare some reports
usaually the report file size is greater than 1GB.
But nowadays program is very slow.
I found out the program is taking much time to write data to file .....
is there any unix related reason to be slow down, the file writting... (2 Replies)
Discussion started by: bhagyaraj.p
2 Replies
2. SCO
I have been a SCO UNIX user, never an administrator...so I am stumbling around looking for information.
I don't know too much about what is onboard in terms of hardware, however; I will try my best.
We have SCO 5.07 and have applied MP5.
We have a quad core processor with 4 250 GB... (1 Reply)
Discussion started by: atpbrownie
1 Replies
3. Red Hat
Hi guys,
I am trying something. I wrote a simple shell program to test something where continuous while loop writes on a file over the nfs. The time taken to write "hello" 3000 times take about 10 sec which is not right. Ideally it should take fraction of seconds. If I write on the local disk, it... (1 Reply)
Discussion started by: abhig
1 Replies
4. Shell Programming and Scripting
Dear All,
I am using the following script to find and replace the date format in a file. The field18 in the file has the following format: "01/26/2010 11:55:14 GMT+04:00" which I want to convert into the following format "20100126115514" for this purpose I am using the following lines of codes:... (5 Replies)
Discussion started by: bilalghazi
5 Replies
5. Shell Programming and Scripting
Please, I need help tuning my script. It works but it's too slow.
The code reads an acivity log file with 50.000 - 100.000 lines and filters error messages from it. The data in the actlog file look similar to this:
02/08/2011 00:25:01,ANR2034E QUERY MOUNT: No match found using this criteria.... (5 Replies)
Discussion started by: Miila
5 Replies
6. Shell Programming and Scripting
Hello everyone,
suppose there is a very big text file (>800 mb) that each line contains an article from wikipedia. Each article begins with a tag (<..>) containing its url. Currently there are 10^6 articles in the file.
I want to take random N articles, eliminate all non-alpharithmetic... (14 Replies)
Discussion started by: fedonMan
14 Replies
7. Red Hat
My code
Hi All,
I am having redhat linux 5.3 (Tikanga) with GFS file system and its very very slow for executing ls -ls command also.Please see the below for 2minits 12 second takes.
Please help me to fix the issue.
$ sudo time ls -la BadFiles |wc -l
0.01user 0.26system... (3 Replies)
Discussion started by: susindram
3 Replies
8. Programming
Hello,
I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this :
This is the output of ls command : I stored the output in a file filelist
1.1M... (5 Replies)
Discussion started by: ajayram
5 Replies
9. Shell Programming and Scripting
I have shell program as below
#!/bin/sh
echo ======= LogManageri start ==========
#This directory is getting the raw data from remote server
Raw_data=/opt/ftplogs
# This directory is ready for process the data
Processing_dir=/opt/processing_dir
# This directory is prcoessed files and... (4 Replies)
Discussion started by: Chenchireddy
4 Replies
10. Shell Programming and Scripting
I have 2 files
file 1 : contains
ALINE
ALINE BANG
B ON A
B.B.V.A.
BANG AMER CORG
BANG ON MORENA
BANG ON MORENAIC
BANG ON MORENAICA
BANG ON MORENAICA CORP
BANG ON MORENAICA N.A
file 2 contains and is seprated by ^ delimiter :
NATIO MARKET^345432534 (10 Replies)
Discussion started by: nikhil jain
10 Replies
LEARN ABOUT LINUX
service
service(8) System Manager's Manual service(8) NAME
service - run a System V init script SYNOPSIS
service SCRIPT COMMAND [OPTIONS] service --status-all service --help | -h | --version DESCRIPTION
service runs a System V init script in as predictable environment as possible, removing most environment variables and with current working directory set to /. The SCRIPT parameter specifies a System V init script, located in /etc/init.d/SCRIPT. The supported values of COMMAND depend on the invoked script, service passes COMMAND and OPTIONS it to the init script unmodified. All scripts should support at least the start and stop commands. As a special case, if COMMAND is --full-restart, the script is run twice, first with the stop command, then with the start command. service --status-all runs all init scripts, in alphabetical order, with the status command. EXIT CODES
service calls the init script and returns the status returned by it. FILES
/etc/init.d The directory containing System V init scripts. ENVIRONMENT
LANG, TERM The only environment variables passed to the init scripts. SEE ALSO
/etc/init.d/skeleton, update-rc.d(8), init(8), invoke-rc.d(8). Jan 2006 service(8)