Sponsored Content
Top Forums Shell Programming and Scripting Generated csv files size are more than 3GB Post 302839895 by sunny13392 on Saturday 3rd of August 2013 02:45:26 PM
Old 08-03-2013
Generated csv files size are more than 3GB

Hi

i have a shell script which will generate the CSV files from a group of XML files.
but the XML files are in 50K plus. so the generated CSV files are more then 4 GB in size. is there any way i can take care in the scripting itself to make the shell script efficent to write data into CSV files to save the size of generating.


Thanks & Regards,
Sunny
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

bash script working for small size files but not for big size files.

Hi, I have one file stat. Stat file contents are as follows: for example. H50768020040913,00260100,507680,13,0000000643,0000000643,00000,0000 H50769520040808,00260100,507695,13,0000000000,0000000000,00000,0000 H50770620040611,00260100,507706,13,0000000000,0000000000,00000,0000 Now i... (1 Reply)
Discussion started by: davidpreml
1 Replies

2. UNIX for Dummies Questions & Answers

How to list all the files which are not generated today

How to list all the files which are not generated today, and move all the above files to backup dir. (2 Replies)
Discussion started by: redlotus72
2 Replies

3. Shell Programming and Scripting

Ftp all the generated files

Hi All, I'm working on a ftp shell script in which I'm tranfering files from one sever to another using ftp. Some program generates files at undefined time & throughout the day. I have to transfer the files time to time.. i.e. once the file is generated, it should be transfered at the very... (3 Replies)
Discussion started by: im_new
3 Replies

4. Programming

how to add CRLF support for CSV file generated in unix

Helo, my server side system is running on a redhat linux o.s. I have c program on the server which export list file into CSV format. now, I want that my program on server side is able to add support for CRLF(carriage return Line feed)into csv file format. so how do i write C program whcih... (3 Replies)
Discussion started by: amitpansuria
3 Replies

5. UNIX for Advanced & Expert Users

Type v for generated files

Hi All, I was checking some of the files and I got the following entries:- =============== v, 664, serv, serv, version.txt, exe L, 775, serv, serv, start.sh, eventserv ================ Could someone please tell me what does the type"v" and "L" represent to. I have not... (2 Replies)
Discussion started by: shubhranshu
2 Replies

6. Solaris

core files not getting generated

Hi, We have an application ASPA . The application related processes are running in /ASPA/bin directory . now whenever a process terminates abruptly , a core file should be generated (correct me if i am wrong) in the /ASPA/bin directory . But i am not able to see any such files . The... (4 Replies)
Discussion started by: asalman.qazi
4 Replies

7. Shell Programming and Scripting

Script to Grep column 3 from csv file generated yesterday

Hello, Can any one please assist how to scirpt it: Every day a new log file is create and I want to process only the one generated yesterday and get the data of column 3 and 6. For example today's date is 24 then I want to get the data of log file created on 23rd. Log Files in... (7 Replies)
Discussion started by: sureshcisco
7 Replies

8. Solaris

Script fails when generated output file reaches a particular size

Hi All, New to unix. Here is the problem. Running a script that extracts data from hyperion essbase and generates a file in unix. This script fails most of the times with a very low success rate. The data has increased a lot in the last few months resulting in the file being more than 2 gb. ... (2 Replies)
Discussion started by: noufalshaw
2 Replies

9. Shell Programming and Scripting

Files generated by a particular user

Hi I have the following files generated by different users on a directory -rw-rw-r-- 1 NAME1 database03 809 Nov 17 10:41 PCAS_CARD_TRANS_OFF.1111171041.lg -rw-rw-r-- 1 richard ccsdba 10968411 Nov 17 10:43 load_123_RX0_0.1111171016.lg -rw-rw-r-- 1 DEV db03 10713 Nov 17... (5 Replies)
Discussion started by: bobby1015
5 Replies

10. Shell Programming and Scripting

Files generated today

I would like to find the Files which are generated today in the current directory: I use the commad ls -lrt * | egrep " `date "+%b"` * `date "+%d"` to acheive this. Is there any better way to acquire the same. Multiple answers will be great. Thanks (3 Replies)
Discussion started by: kusathy
3 Replies
XMLSTARLET(1)							 xmlstarlet Manual						     XMLSTARLET(1)

NAME
xmlstarlet - command line XML/XSLT toolkit SYNOPSIS
xmlstarlet [<options>] [<command>] [<cmd-options>] INTRODUCTION
XMLStarlet is a set of command line utilities (tools) which can be used to transform, query, validate, and edit XML documents and files us- ing simple set of shell commands in similar way it is done for plain text files using UNIX grep, sed, awk, diff, patch, join, etc commands. This set of command line utilities can be used by those who deal with many XML documents on UNIX shell command prompt as well as for auto- mated XML processing with shell scripts. OPTIONS
--version Display the version of xmlstarlet. --help Display help. COMMANDS
Type: xml <command> --help <ENTER> for command help Available commands include: ed (or edit) Edit/update XML document(s). sel (or select) Select data or query XML document(s) (XPATH, etc). tr (or transform) Transform XML documents(s) using XSLT. val (or validate) Validate XML document(s) (well-formed/DTD/XSD/RelaxNG). fo (or format) Format XML document(s). el (or elements) Display element structure of XML document. c14n (or canonic) XML canonicalization. ls (or list) List directory as XML. esc (or escape) Escape special XML characters. unesc (or unescape) Unescape special XML characters. pyx (or xmln) Convert XML into PYX format (based on ESIS - ISO 8879). p2x (or depyx) Convert PYX into XML. REFERENCES
XMLStarlet is a command line toolkit to query/edit/check/transform XML documents (for more information see http://xmlstar.source- forge.net/). AUTHOR
Mikhail Grushinskiy. XMLSTARLET(1)
All times are GMT -4. The time now is 10:59 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy