How to open large datafie in awk? Post: 302777801

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Reading large file, awk and cut

Hello all, I have 2 files, the first (indexFile1) contains start offset and length for each record inside the second file. The second file can be very large, each actual record start offset and length is defined by the entry in indexFile1. Since there are no records separators wc-l returns 0 for...

2. Shell Programming and Scripting

AWK Shell Program to Split Large Files

Hi, I need some help creating a tidy shell program with awk or other language that will split large length files efficiently. Here is an example dump: <A001_MAIL.DAT> 0001 Ronald McDonald 01 H81 0002 Elmo St. Elmo 02 H82 0003 Cookie Monster 01 H81 0004 Oscar ...

3. Programming

Is there a system call other than 'open' for opening very large files?

Dear all, Inside a C program, I want to open a very big file (about 12 GB) in order to read its content. Here is the code: /* argv contains the path to the file. */ inputFileDescriptor = open(argv, O_RDONLY); if (inputFileDescriptor < 0) { ...

4. Shell Programming and Scripting

Help needed for parsing large XML with awk.

My XML structure looks like: <?xml version="1.0" encoding="UTF-8"?> <SearchRepository> <SearchItems> <SearchItem> ... </SearchItem> <SearchItem> ... ...

5. Shell Programming and Scripting

sed and awk not working on a large record file

Hi All, I have a very large single record file. abc;date||bcd;efg|......... pqr;stu||record_count;date when i do wc -l on this file it gives me "0" records, coz of missing line feed. my problem is there is an extra pipe that is coming at the end of this record like...

6. UNIX for Dummies Questions & Answers

Find common numbers from two very large files using awk or the like

I've got two files that each contain a 16-digit number in positions 1-16. The first file has 63,120 entries all sorted numerically. The second file has 142,479 entries, also sorted numerically. I want to read through each file and output the entries that appear in both. So far I've had no...

7. Shell Programming and Scripting

awk input large file

Hi...Does anyone know how to input huge file about 25 GB to awk if single file then this works awk '{print}' <hugefile suppose if have to use something like this awk FNR==NR{x=$0;next}{print $0,x}' hugefile1 hugefile2 then how to redirect ? and is there any provision to assign memory...

8. Shell Programming and Scripting

awk output yields error: awk:can't open job_name (Autosys)

Good evening, Im newbie at unix specially with awk From an scheduler program called Autosys i want to extract some data reading an inputfile that comprises jobs names, then formating the output to columns for example 1. This is the inputfile: $ more MapaRep.txt ds_extra_nikira_usuarios...

9. Shell Programming and Scripting

Process multiple large files with awk

Hi there, I'm camor and I'm trying to process huge files with bash scripting and awk. I've got a dataset folder with 10 files (16 millions of row each one - 600MB), and I've got a sorted file with all keys inside. For example: a sample_1 200 a.b sample_2 10 a sample_3 10 a sample_1 10 a...

10. Shell Programming and Scripting

Reversing large data set with awk?

Hello! I have quite a bit of map data that I have to edit. I originally had a DOS script that would reverse x1, y1 coordinates in order to change the direction of a particular segment in a map file. It worked wonderfully and all was well, but my bossman told me that there is a boatload of nodes...

LEARN ABOUT DEBIAN

ltsconvert

LTSCONVERT(1)							   User Commands						     LTSCONVERT(1)

NAME

       ltsconvert - convert and optionally minimise an LTS

SYNOPSIS

       ltsconvert [OPTION]... [INFILE [OUTFILE]]

DESCRIPTION

       Convert the labelled transition system (LTS) from INFILE to OUTFILE in the requested format after applying the selected minimisation method
       (default is none). If OUTFILE is not supplied, stdout is used. If INFILE is not supplied, stdin is used.

       The output format is determined by the extension of OUTFILE, whereas the input format is determined by the content of INFILE. Options  --in
       and --out can be used to force the input and output formats. The supported formats are:
	 'aut' for the Aldebaran format (CADP),
	 'dot' for the GraphViz format,
	 'fsm' for the Finite State Machine format,
	 'lts' for the mCRL2 LTS format (default), or
	 'svc' for the SVC format

OPTIONS

       OPTION can be any of the following:

       -D, --determinise
	      determinise LTS

       -eNAME, --equivalence=NAME
	      generate an equivalent LTS, preserving equivalence NAME:
		'bisim' for strong bisimilarity,
		'branching-bisim' for branching bisimilarity,
		'dpbranching-bisim' for divergence preserving branching bisimilarity,
		'sim' for strong simulation equivalence,
		'tau-star' for tau star reduction,
		'trace' for strong trace equivalence, or
		'weak-trace' for weak trace equivalence

       -iFORMAT, --in=FORMAT
	      use FORMAT as the input format

       -lFILE, --lps=FILE
	      use FILE as the LPS from which the input LTS was generated; this might be needed to store the correct parameter names of states when
	      saving in fsm format and to convert non-mCRL2 LTSs to a mCRL2 LTS

       --no-reach
	      do not perform a reachability check on the input LTS

       -n, --no-state
	      leave out state information when saving in dot format

       -oFORMAT, --out=FORMAT
	      use FORMAT as the output format

       --tau=ACTNAMES
	      consider actions with a name in the comma separated list ACTNAMES to be internal (tau) actions in addition to those defined as  such
	      by the input

       --timings[=FILE]
	      append timing measurements to FILE. Measurements are written to standard error if no FILE is provided

       Standard options:

       -q, --quiet
	      do not display warning messages

       -v, --verbose
	      display short intermediate messages

       -d, --debug
	      display detailed intermediate messages

       --log-level=LEVEL
	      display intermediate messages up to and including level

       -h, --help
	      display help information

       --version
	      display version information

AUTHOR

       Written by Muck van Weerdenburg, Jan Friso Groote.

REPORTING BUGS

       Report bugs at <http://www.mcrl2.org/issuetracker>.

COPYRIGHT

       Copyright (C) 2012 Technische Universiteit Eindhoven.
       This    is    free   software.	 You   may   redistribute   copies   of   it   under   the   terms   of   the	Boost	Software   License
       <http://www.boost.org/LICENSE_1_0.txt>.	There is NO WARRANTY, to the extent permitted by law.

SEE ALSO

       See also the manual at <http://www.mcrl2.org/mcrl2/wiki/index.php/User_manual/ltsconvert>.

ltsconvert mCRL2 toolset 201202.0 (Release)			    April 2012							     LTSCONVERT(1)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Reading large file, awk and cut

Discussion started by: gio001

2. Shell Programming and Scripting

AWK Shell Program to Split Large Files

Discussion started by: mkastin

3. Programming

Is there a system call other than 'open' for opening very large files?

Discussion started by: dariyoosh

4. Shell Programming and Scripting

Help needed for parsing large XML with awk.

Discussion started by: jasonjustice