We are facing some performance issue in UNIX. If someone had faced such kind of issue in past please provide your suggestions on this .
Problem Definition:
/Few of load processes of our Finance Application are facing issue in UNIX when they uses a shell script having below portion of code. The below portion of codes reads an input file and writes them into an .dat file. The performance issue arises when there is huge volume of data in the input file.
For example: For data volume having 200,000 records is taking 38 mins to get append/write into the .dat file which increases the complete load process timings. We need to increase the performance of this proces by reducing the time its taking to append/write the records.
With a file that size, you should really be using awk.
Quote:
/*****************************************
Portion of Code from Shell Script:
Please put code inside [code] tags.
Quote:
/****************************************
That cat is an unnecessary external command, but since it is only run once, eliminating it wll make very little difference.
Part of the slowness is due to calling multiple external commands (many of which are unnecessary: there's no need for expr as the shell can do its own arithmetic) for every line.
Quote:
What shell are you using? If it's bash or ksh93, you can replace the call to cut:
Quote:
An unnecessary subshell (here and later) can add a significant amount of time. Use:
Quote:
You don't need wc as well as grep:
Quote:
Please provide your valuable suggestions. Also is there any way by using SED command for appending the output in fast way?
Hi,
how does the Unix File System perform with large directories (containing ~30.000 files)?
What kind of structure is used for the organization of a directory's content, linear lists, (binary) trees?
I hope the description 'Unix File System' is exact enough, I don't know more about the file... (3 Replies)
Hi.
I want to attach a .xls or .dat file while sending mail thru unix.
I have come across diff attachments sending options, but allthose embeds the content in the mail. I want the attachement to be send as such.
Please help me out.
regards
Diwakar (1 Reply)
Hi All,
I have a .dat file named test.dat where I have stored some process IDs.
Now I need to pick a process ID, one by one and then fire kill -9 for each of those. The logic should be:
1. open file <filename.dat>
2. read until last line of file
3. if process ID is found fire kill -9... (5 Replies)
I have a .dat file in unix and it keeps failing file validation on line x. How do I delete a data string from a .dat file in UNIX?
I tried the following:
sed -e 'data string' -e file name
and it telling me unrecognized command (4 Replies)
Background
-------------
The Unix flavor can be any amongst Solaris, AIX, HP-UX and Linux. I have below 2 flat files.
File-1
------
Contains 50,000 rows with 2 fields in each row, separated by pipe.
Row structure is like Object_Id|Object_Name, as following:
111|XXX
222|YYY
333|ZZZ
... (6 Replies)
Hi,
The source system has created the file in the dat format and put into the linux directory as mentioned below. I want to do foloowing things.
a) Delete the Line started with <CR><LF> in the record
b)Also line
...........................................................<CR><LF>
... (1 Reply)
I have around 300 files(*.rdf,*.fmb,*.pll,*.ctl,*.sh,*.sql,*.prog) which are of large size.
Around 8000 keywords(which will be in the file $keywordfile) needed to be searched inside those files.
If a keyword is found in a file..I have to insert the filename,extension,catagoery,keyword,occurrence... (8 Replies)
Hi Guys,
I have a simple request. I have a file in w3c format. Each file has 2 header lines. Rest of the lines are 16 columns each. They are separated by Tab. I need to discard the first 2 lines and then write each column of the txt file into a seperate column of CSV. I tried the command below... (1 Reply)
i have two files , one is var.txt and another res.dat file
var.txt contains informaton like below
date,request,sales,item
20171015,1,123456,216
20171015,1,123456,217
20171015,2,345678,214
20171015,3,456789,218
and res.dat contains is a one huge file contains information like... (1 Reply)
i have a unix script which generates the csv file. the data in csv file is dynamic. how can i convert/move the data from csv file to xml. please suggest (1 Reply)