Sponsored Content
Top Forums Shell Programming and Scripting Performance issue in UNIX while generating .dat file from large text file Post 302309341 by KRAMA on Tuesday 21st of April 2009 04:43:30 PM
Old 04-21-2009
Hi John,

Please find the answers as below:

Please describe exactly what the script needs to do.
This script splits the data from Detail files (i.e which are the input files for the shell script in .txt format) . In this case the detal file located at ${m_recv_dir}/${m_glb_d92_nm}${m_glb_file_seq} . which is the starting portion of the code which i posted.

This script reads the data line by line from the text file and prepare output .DAT file.

Once the .DAT file is created it puts '*/' end of file character at the bottom of the output file generated. Once the .DAT output file is generated another shell script loads data from this .DAT files to work tables of the database using SQL Loader.

What files does it use for input? What is the format of those files?

The format of the input file is .txt

What is the format of the output?

The output format is .DAT

Please do let me know what else information you need so you can help me on this..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Unix File System performance with large directories

Hi, how does the Unix File System perform with large directories (containing ~30.000 files)? What kind of structure is used for the organization of a directory's content, linear lists, (binary) trees? I hope the description 'Unix File System' is exact enough, I don't know more about the file... (3 Replies)
Discussion started by: dive
3 Replies

2. Shell Programming and Scripting

How to attach an excel file/ dat file thru unix mails

Hi. I want to attach a .xls or .dat file while sending mail thru unix. I have come across diff attachments sending options, but allthose embeds the content in the mail. I want the attachement to be send as such. Please help me out. regards Diwakar (1 Reply)
Discussion started by: diwakar82
1 Replies

3. Shell Programming and Scripting

How to read from a .dat file in Unix

Hi All, I have a .dat file named test.dat where I have stored some process IDs. Now I need to pick a process ID, one by one and then fire kill -9 for each of those. The logic should be: 1. open file <filename.dat> 2. read until last line of file 3. if process ID is found fire kill -9... (5 Replies)
Discussion started by: Sibasish
5 Replies

4. UNIX for Dummies Questions & Answers

How do I delete a data string from a .dat file in unix

I have a .dat file in unix and it keeps failing file validation on line x. How do I delete a data string from a .dat file in UNIX? I tried the following: sed -e 'data string' -e file name and it telling me unrecognized command (4 Replies)
Discussion started by: supergirl3954
4 Replies

5. Shell Programming and Scripting

Severe performance issue while 'grep'ing on large volume of data

Background ------------- The Unix flavor can be any amongst Solaris, AIX, HP-UX and Linux. I have below 2 flat files. File-1 ------ Contains 50,000 rows with 2 fields in each row, separated by pipe. Row structure is like Object_Id|Object_Name, as following: 111|XXX 222|YYY 333|ZZZ ... (6 Replies)
Discussion started by: Souvik
6 Replies

6. Shell Programming and Scripting

Remove <CR><LF> from the dat file in unix

Hi, The source system has created the file in the dat format and put into the linux directory as mentioned below. I want to do foloowing things. a) Delete the Line started with <CR><LF> in the record b)Also line ...........................................................<CR><LF> ... (1 Reply)
Discussion started by: mr_harish80
1 Replies

7. Shell Programming and Scripting

Performance issue in Grepping large files

I have around 300 files(*.rdf,*.fmb,*.pll,*.ctl,*.sh,*.sql,*.prog) which are of large size. Around 8000 keywords(which will be in the file $keywordfile) needed to be searched inside those files. If a keyword is found in a file..I have to insert the filename,extension,catagoery,keyword,occurrence... (8 Replies)
Discussion started by: millan
8 Replies

8. UNIX for Dummies Questions & Answers

Generating a CSV file from a text file

Hi Guys, I have a simple request. I have a file in w3c format. Each file has 2 header lines. Rest of the lines are 16 columns each. They are separated by Tab. I need to discard the first 2 lines and then write each column of the txt file into a seperate column of CSV. I tried the command below... (1 Reply)
Discussion started by: tinkugadu
1 Replies

9. Answers to Frequently Asked Questions

How to split a dat file based on another file ni UNIX?

i have two files , one is var.txt and another res.dat file var.txt contains informaton like below date,request,sales,item 20171015,1,123456,216 20171015,1,123456,217 20171015,2,345678,214 20171015,3,456789,218 and res.dat contains is a one huge file contains information like... (1 Reply)
Discussion started by: pogo
1 Replies

10. Shell Programming and Scripting

Generating xml file from UNIX

i have a unix script which generates the csv file. the data in csv file is dynamic. how can i convert/move the data from csv file to xml. please suggest (1 Reply)
Discussion started by: archana25
1 Replies
DAT.CONF(5)															       DAT.CONF(5)

NAME
dat.conf - configuration file for static registration of user-level DAT rdma providers DESCRIPTION
The DAT (direct access transport) architecture supports the use of multiple DAT providers within a single consumer application. Consumers implicitly select a provider using the Interface Adapter name parameter passed to dat_ia_open(). The subsystem that maps Interface Adapter names to provider implementations is known as the DAT registry. When a consumer calls dat_ia_open(), the appropriate provider is found and notified of the consumer's request to access the IA. After this point, all DAT API calls acting on DAT objects are automatically directed to the appropriate provider entry points. A persistent, administratively configurable database is used to store mappings from IA names to provider information. This provider infor- mation includes: the file system path to the provider library object, version information, and thread safety information. The location and format of the registry is platform dependent. This database is known as the Static Registry (SR) and is provided via entries in the dat.conf file. The process of adding a provider entry is termed Static Registration. Registry File Format * All characters after # on a line are ignored (comments). * Lines on which there are no characters other than whitespace and comments are considered blank lines and are ignored. * Non-blank lines must have seven whitespace separated fields. These fields may contain whitespace if the field is quoted with double quotes. Within fields quoated with double quotes, the backslash or qoute are valid escape sequences: * Each non-blank line will contain the following fields: - The IA Name. - The API version of the library: [k|u]major.minor where "major" and "minor" are both integers in decimal format. User-level examples: "u1.2", and "u2.0". - Whether the library is thread-safe: [threadsafe|nonthreadsafe] - Whether this is the default section: [default|nondefault] - The library image, version included, to be loaded. - The vendor id and version of DAPL provider: id.major.minor - ia params, IA specific parameters - device name and port - platform params, (not used) Example netdev entries for OpenFabrics rdma_cm providers, both v1.2 and v2.0 OpenIB-cma u1.2 nonthreadsafe default libdaplcma.so.1 dapl.1.2 "ib0 0" "" ofa-v2-ib0 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 "ib0 0" "" NOTE: The OpenFabrics providers use <ia_params> to specify the device with one of the following: network address, network hostname, or netdev name; along with port number. The OpenIB- and ofa-v2- IA names are unique mappings. Reserved for OpenFabrics providers. The default location for this configuration file is /etc/dat.conf. The file location may be overridden with the environment variable DAT_OVERRIDE=/your_own_directory/your_dat.conf. SEE ALSO
25 March 2008 DAT.CONF(5)
All times are GMT -4. The time now is 03:25 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy