Sponsored Content
Full Discussion: Parsing Bulk Data
Top Forums Shell Programming and Scripting Parsing Bulk Data Post 302891883 by jojo123 on Sunday 9th of March 2014 01:43:38 AM
Old 03-09-2014
Parsing Bulk Data

Hi All, Smilie

Actullay I am looking for a smart way Smilie to parse files in a directory whose count is around 2000000 Smilie in a single day.

Find is working with me but taking a lot of times Smilie, sometimes even a day which is not helping me.Smilie

So anyone can help me know a smart way to get my desired data from a 2000000 files. Smilie
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing the data in a file

Hi, I have file (FILE.tmp) having contents, FILE.tmp ======== filename=menudata records=0000000000037 ldbname=pinsys timestamp=2005/05/14-18:32:33 I want to parse it bring a new file which will look like, filename records ldbname timestamp... (2 Replies)
Discussion started by: Omkumar
2 Replies

2. Shell Programming and Scripting

Parsing the data

Hi I need to parse the following data using shell script Table ----- stage4n_abc 48 stage4o_abcd 4 adashpg_abc_HeartBeat 1 stage4l_asc 168 Can anyone gimme the solution. I want each value to get stored in an array or variable and want the value to be greped from another file.... (1 Reply)
Discussion started by: Archana.Dheepan
1 Replies

3. Shell Programming and Scripting

Parsing data

Hi all , I have a file with billing CDR records in it. I need to parse that information (row format) . The purpose is to compare full content. The example I have given below is a single line record but it has two portions, (1) the line start with “!” and end with “1.2.1.8” and (2) second part... (5 Replies)
Discussion started by: jaygamini
5 Replies

4. Shell Programming and Scripting

Creating Header & Trailer for bulk volume data file

Hi all, I have a requirement to create a Header &Trailer for a file which is having 20 millions of records. If I use the following method, i think it will take more time. cat "Header"> file1.txt cat Data_File>>file1.txt cat "Trailer">>file1.txt since second CAT command has to read all... (4 Replies)
Discussion started by: Raamc
4 Replies

5. Shell Programming and Scripting

Parsing the data

Hi friends, I need to parse the following data in the given format and get the desired output. I need a function, which takes the input as a parameter and the desired output will be returned from the function. INPUT(single parameter as complete string) A;BCF;DFG;FD ... (3 Replies)
Discussion started by: sumesh.1988
3 Replies

6. Shell Programming and Scripting

Help in Parsing data

I have below string Transaction_ID:SDP-DM-151204679 , Transaction_DateTime:2011-02-11 00:00:15 GMT+05:30 , Transaction_Mode:WAP , Circle_ID:4 , Circle_Name:BJ ,Zone: , CustomerID:B_31563486 , MSISDN:7870904329 , IMSI:405876122068099 , IMEI: , Sub_Profile:Pre-Paid , CPID:Nazara , CPNAME:Nazara ,... (6 Replies)
Discussion started by: poweroflinux
6 Replies

7. Shell Programming and Scripting

Data parsing

Hi, I do have a data file which is divided into compartments by ---------. I would like to extract (parse) some of the data and numbers either using awk or sed The file has the format: CATGC Best GO enrichment: Genes/ORF that have the motifs (genes are sorted by max(pa+pd+po)): ... (6 Replies)
Discussion started by: Lucky Ali
6 Replies

8. Shell Programming and Scripting

Help with parsing data with awk , eliminating unwanted data

Experts , Below is the data: --- Physical volumes --- PV Name /dev/dsk/c1t2d0 VG Name /dev/vg00 PV Status available Allocatable yes VGDA 2 Cur LV 8 PE Size (Mbytes) 8 Total PE 4350 Free PE 2036 Allocated PE 2314 Stale PE 0 IO Timeout (Seconds) default --- Physical volumes ---... (5 Replies)
Discussion started by: rveri
5 Replies

9. Shell Programming and Scripting

Parsing file data

Hey Guys, I'm a novice at shell scripts and i need some help parsing file data. Basically, I want to write a script that retrieves URLs. Here is what I have so far. #!/bin/bash echo "Please enter start date (format: yyyy-mm-dd):\c" read STARTDATE echo "Please enter end date... (7 Replies)
Discussion started by: silverdust
7 Replies

10. Shell Programming and Scripting

Parsing XML (and insert data) then output data (bash / Solaris)

Hi folks I have a script I wrote that basically parses a bunch of config and xml files works out were to add in the new content then spits out the data into a new file. It all works - apart from the xml and config file format in the new file with XML files the original XML (that ends up in... (2 Replies)
Discussion started by: dfinch
2 Replies
AIDA2FLAT(1)							   User Commands						      AIDA2FLAT(1)

NAME
aida2flat - Convert AIDA data files to a flat format SYNOPSIS
aida2flat [options] aidafile [aidafile2 ...] DESCRIPTION
Convert AIDA data files to a flat format which is more human-readable then the XML (and by default also plottable directly using make-plots). The output is by default written out to standard output unless the --split, --smart-output, --gnuplot, or --output options are specified. When specifying either input or output filenames, a '-' is used to refer to stdin or stdout as appropriate. Histograms can also be filtered by AIDA path, using the -m or -M options for a positive or negative regex pattern patch respectively. OPTIONS
-h, --help show this help message and exit -o OUTPUT, --output=OUTPUT Write all histos to a single output file, rather than the default writing to stdout. stdout can be explicitly specified by setting '-' as the output filename. This option will be disregarded if --split, --smart-output, or --gnuplot is specified. -s, --split Write each histo to a separate output file, with names based on the histo path -S, --smart-output Write to output files with names based on the corresponding input filename. This option will be disregarded if --split is specified. -g, --gnuplot Provide output suitable for Gnuplot's 'plot "foo.dat" with xye'. This option implies --split and will override --output or --smart-output --plotinfodir=PLOTINFODIR directory which may contain plot header information -m PATHPATTERNS, --match=PATHPATTERNS Only write out histograms whose $path/$name string matches these regexes -M PATHUNPATTERNS, --unmatch=PATHUNPATTERNS Exclude histograms whose $path/$name string matches these regexes AUTHOR
This manual page was written by Lifeng Sun <lifongsun@gmail.com> for the Debian system (but may be used by others). Rivet June 2012 AIDA2FLAT(1)
All times are GMT -4. The time now is 04:38 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy