Sponsored Content
Full Discussion: Parsing Bulk Data
Top Forums Shell Programming and Scripting Parsing Bulk Data Post 302891928 by Scrutinizer on Sunday 9th of March 2014 04:26:43 PM
Old 03-09-2014
Perhaps that could be speeded up a little. What is the format of those files? Could you provide a sample?
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing the data in a file

Hi, I have file (FILE.tmp) having contents, FILE.tmp ======== filename=menudata records=0000000000037 ldbname=pinsys timestamp=2005/05/14-18:32:33 I want to parse it bring a new file which will look like, filename records ldbname timestamp... (2 Replies)
Discussion started by: Omkumar
2 Replies

2. Shell Programming and Scripting

Parsing the data

Hi I need to parse the following data using shell script Table ----- stage4n_abc 48 stage4o_abcd 4 adashpg_abc_HeartBeat 1 stage4l_asc 168 Can anyone gimme the solution. I want each value to get stored in an array or variable and want the value to be greped from another file.... (1 Reply)
Discussion started by: Archana.Dheepan
1 Replies

3. Shell Programming and Scripting

Parsing data

Hi all , I have a file with billing CDR records in it. I need to parse that information (row format) . The purpose is to compare full content. The example I have given below is a single line record but it has two portions, (1) the line start with “!” and end with “1.2.1.8” and (2) second part... (5 Replies)
Discussion started by: jaygamini
5 Replies

4. Shell Programming and Scripting

Creating Header & Trailer for bulk volume data file

Hi all, I have a requirement to create a Header &Trailer for a file which is having 20 millions of records. If I use the following method, i think it will take more time. cat "Header"> file1.txt cat Data_File>>file1.txt cat "Trailer">>file1.txt since second CAT command has to read all... (4 Replies)
Discussion started by: Raamc
4 Replies

5. Shell Programming and Scripting

Parsing the data

Hi friends, I need to parse the following data in the given format and get the desired output. I need a function, which takes the input as a parameter and the desired output will be returned from the function. INPUT(single parameter as complete string) A;BCF;DFG;FD ... (3 Replies)
Discussion started by: sumesh.1988
3 Replies

6. Shell Programming and Scripting

Help in Parsing data

I have below string Transaction_ID:SDP-DM-151204679 , Transaction_DateTime:2011-02-11 00:00:15 GMT+05:30 , Transaction_Mode:WAP , Circle_ID:4 , Circle_Name:BJ ,Zone: , CustomerID:B_31563486 , MSISDN:7870904329 , IMSI:405876122068099 , IMEI: , Sub_Profile:Pre-Paid , CPID:Nazara , CPNAME:Nazara ,... (6 Replies)
Discussion started by: poweroflinux
6 Replies

7. Shell Programming and Scripting

Data parsing

Hi, I do have a data file which is divided into compartments by ---------. I would like to extract (parse) some of the data and numbers either using awk or sed The file has the format: CATGC Best GO enrichment: Genes/ORF that have the motifs (genes are sorted by max(pa+pd+po)): ... (6 Replies)
Discussion started by: Lucky Ali
6 Replies

8. Shell Programming and Scripting

Help with parsing data with awk , eliminating unwanted data

Experts , Below is the data: --- Physical volumes --- PV Name /dev/dsk/c1t2d0 VG Name /dev/vg00 PV Status available Allocatable yes VGDA 2 Cur LV 8 PE Size (Mbytes) 8 Total PE 4350 Free PE 2036 Allocated PE 2314 Stale PE 0 IO Timeout (Seconds) default --- Physical volumes ---... (5 Replies)
Discussion started by: rveri
5 Replies

9. Shell Programming and Scripting

Parsing file data

Hey Guys, I'm a novice at shell scripts and i need some help parsing file data. Basically, I want to write a script that retrieves URLs. Here is what I have so far. #!/bin/bash echo "Please enter start date (format: yyyy-mm-dd):\c" read STARTDATE echo "Please enter end date... (7 Replies)
Discussion started by: silverdust
7 Replies

10. Shell Programming and Scripting

Parsing XML (and insert data) then output data (bash / Solaris)

Hi folks I have a script I wrote that basically parses a bunch of config and xml files works out were to add in the new content then spits out the data into a new file. It all works - apart from the xml and config file format in the new file with XML files the original XML (that ends up in... (2 Replies)
Discussion started by: dfinch
2 Replies
Time::ParseDate(3pm)					User Contributed Perl Documentation				      Time::ParseDate(3pm)

NAME
Time::ParseDate -- date parsing both relative and absolute SYNOPSIS
use Time::ParseDate; $seconds_since_jan1_1970 = parsedate("12/11/94 2pm", NO_RELATIVE => 1) $seconds_since_jan1_1970 = parsedate("12/11/94 2pm", %options) OPTIONS
Date parsing can also use options. The options are as follows: FUZZY -> it's okay not to parse the entire date string NOW -> the "current" time for relative times (defaults to time()) ZONE -> local timezone (defaults to $ENV{TZ}) WHOLE -> the whole input string must be parsed GMT -> input time is assumed to be GMT, not localtime UK -> prefer UK style dates (dd/mm over mm/dd) DATE_REQUIRED -> do not default the date TIME_REQUIRED -> do not default the time NO_RELATIVE -> input time is not relative to NOW TIMEFIRST -> try parsing time before date [not default] PREFER_PAST -> when year or day of week is ambiguous, assume past PREFER_FUTURE -> when year or day of week is ambiguous, assume future SUBSECOND -> parse fraction seconds VALIDATE -> only accept normal values for HHMMSS, YYMMDD. Otherwise days like -1 might give the last day of the previous month. DATE FORMATS RECOGNIZED
Absolute date formats Dow, dd Mon yy Dow, dd Mon yyyy Dow, dd Mon dd Mon yy dd Mon yyyy Month day{st,nd,rd,th}, year Month day{st,nd,rd,th} Mon dd yyyy yyyy/mm/dd yyyy-mm-dd (usually the best date specification syntax) yyyy/mm mm/dd/yy mm/dd/yyyy mm/yy yy/mm (only if year > 12, or > 31 if UK) yy/mm/dd (only if year > 12 and day < 32, or year > 31 if UK) dd/mm/yy (only if UK, or an invalid mm/dd/yy or yy/mm/dd) dd/mm/yyyy (only if UK, or an invalid mm/dd/yyyy) dd/mm (only if UK, or an invalid mm/dd) Relative date formats: count "days" count "weeks" count "months" count "years" Dow "after next" Dow "before last" Dow (requires PREFER_PAST or PREFER_FUTURE) "next" Dow "tomorrow" "today" "yesterday" "last" dow "last week" "now" "now" "+" count units "now" "-" count units "+" count units "-" count units count units "ago" Absolute time formats: hh:mm:ss[.ddd] hh:mm hh:mm[AP]M hh[AP]M hhmmss[[AP]M] "noon" "midnight" Relative time formats: count "minutes" (count can be franctional "1.5" or "1 1/2") count "seconds" count "hours" "+" count units "+" count "-" count units "-" count count units "ago" Timezone formats: [+-]dddd GMT[+-]d+ [+-]dddd (TZN) TZN Special formats: [ d]d/Mon/yyyy:hh:mm:ss [[+-]dddd] yy/mm/dd.hh:mm DESCRIPTION
This module recognizes the above date/time formats. Usually a date and a time are specified. There are numerous options for controlling what is recognized and what is not. The return code is always the time in seconds since January 1st, 1970 or undef if it was unable to parse the time. If a timezone is specified it must be after the time. Year specifications can be tacked onto the end of absolute times. If "parsedate()" is called from array context, then it will return two elements. On sucessful parses, it will return the seconds and what remains of its input string. On unsucessful parses, it will return "undef" and an error string. EXAMPLES
$seconds = parsedate("Mon Jan 2 04:24:27 1995"); $seconds = parsedate("Tue Apr 4 00:22:12 PDT 1995"); $seconds = parsedate("04.04.95 00:22", ZONE => PDT); $seconds = parsedate("Jan 1 1999 11:23:34.578", SUBSECOND => 1); $seconds = parsedate("122212 950404", ZONE => PDT, TIMEFIRST => 1); $seconds = parsedate("+3 secs", NOW => 796978800); $seconds = parsedate("2 months", NOW => 796720932); $seconds = parsedate("last Tuesday"); $seconds = parsedate("Sunday before last"); ($seconds, $remaining) = parsedate("today is the day"); ($seconds, $error) = parsedate("today is", WHOLE=>1); LICENSE
Copyright (C) 1996-2010 David Muir Sharnoff. Copyright (C) 2011 Google, Inc. License hereby granted for anyone to use, modify or redistribute this module at their own risk. Please feed useful changes back to cpan@dave.sharnoff.org. perl v5.12.3 2011-05-20 Time::ParseDate(3pm)
All times are GMT -4. The time now is 01:22 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy