Generated csv files size are more than 3GB


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Generated csv files size are more than 3GB
# 1  
Old 08-03-2013
Generated csv files size are more than 3GB

Hi

i have a shell script which will generate the CSV files from a group of XML files.
but the XML files are in 50K plus. so the generated CSV files are more then 4 GB in size. is there any way i can take care in the scripting itself to make the shell script efficent to write data into CSV files to save the size of generating.


Thanks & Regards,
Sunny
# 2  
Old 08-04-2013
it would help to understand the problem if you supply ...

1. the code snippet that creates the csv files from xml
2. input file
3. desired output format
4. os and version
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Files generated today

I would like to find the Files which are generated today in the current directory: I use the commad ls -lrt * | egrep " `date "+%b"` * `date "+%d"` to acheive this. Is there any better way to acquire the same. Multiple answers will be great. Thanks (3 Replies)
Discussion started by: kusathy
3 Replies

2. Shell Programming and Scripting

Files generated by a particular user

Hi I have the following files generated by different users on a directory -rw-rw-r-- 1 NAME1 database03 809 Nov 17 10:41 PCAS_CARD_TRANS_OFF.1111171041.lg -rw-rw-r-- 1 richard ccsdba 10968411 Nov 17 10:43 load_123_RX0_0.1111171016.lg -rw-rw-r-- 1 DEV db03 10713 Nov 17... (5 Replies)
Discussion started by: bobby1015
5 Replies

3. Solaris

Script fails when generated output file reaches a particular size

Hi All, New to unix. Here is the problem. Running a script that extracts data from hyperion essbase and generates a file in unix. This script fails most of the times with a very low success rate. The data has increased a lot in the last few months resulting in the file being more than 2 gb. ... (2 Replies)
Discussion started by: noufalshaw
2 Replies

4. Shell Programming and Scripting

Script to Grep column 3 from csv file generated yesterday

Hello, Can any one please assist how to scirpt it: Every day a new log file is create and I want to process only the one generated yesterday and get the data of column 3 and 6. For example today's date is 24 then I want to get the data of log file created on 23rd. Log Files in... (7 Replies)
Discussion started by: sureshcisco
7 Replies

5. Solaris

core files not getting generated

Hi, We have an application ASPA . The application related processes are running in /ASPA/bin directory . now whenever a process terminates abruptly , a core file should be generated (correct me if i am wrong) in the /ASPA/bin directory . But i am not able to see any such files . The... (4 Replies)
Discussion started by: asalman.qazi
4 Replies

6. UNIX for Advanced & Expert Users

Type v for generated files

Hi All, I was checking some of the files and I got the following entries:- =============== v, 664, serv, serv, version.txt, exe L, 775, serv, serv, start.sh, eventserv ================ Could someone please tell me what does the type"v" and "L" represent to. I have not... (2 Replies)
Discussion started by: shubhranshu
2 Replies

7. Programming

how to add CRLF support for CSV file generated in unix

Helo, my server side system is running on a redhat linux o.s. I have c program on the server which export list file into CSV format. now, I want that my program on server side is able to add support for CRLF(carriage return Line feed)into csv file format. so how do i write C program whcih... (3 Replies)
Discussion started by: amitpansuria
3 Replies

8. Shell Programming and Scripting

Ftp all the generated files

Hi All, I'm working on a ftp shell script in which I'm tranfering files from one sever to another using ftp. Some program generates files at undefined time & throughout the day. I have to transfer the files time to time.. i.e. once the file is generated, it should be transfered at the very... (3 Replies)
Discussion started by: im_new
3 Replies

9. UNIX for Dummies Questions & Answers

How to list all the files which are not generated today

How to list all the files which are not generated today, and move all the above files to backup dir. (2 Replies)
Discussion started by: redlotus72
2 Replies

10. Shell Programming and Scripting

bash script working for small size files but not for big size files.

Hi, I have one file stat. Stat file contents are as follows: for example. H50768020040913,00260100,507680,13,0000000643,0000000643,00000,0000 H50769520040808,00260100,507695,13,0000000000,0000000000,00000,0000 H50770620040611,00260100,507706,13,0000000000,0000000000,00000,0000 Now i... (1 Reply)
Discussion started by: davidpreml
1 Replies
Login or Register to Ask a Question