How to extract data from a huge file? Post: 302159600

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

search and grab data from a huge file

folks, In my working directory, there a multiple large files which only contain one line in the file. The line is too long to use "grep", so any help? For example, if I want to find if these files contain a string like "93849", what command I should use? Also, there is oder_id number...

2. Shell Programming and Scripting

How to extract a piece of information from a huge file

Hello All, I need some assistance to extract a piece of information from a huge file. The file is like this one : database information ccccccccccccccccc ccccccccccccccccc ccccccccccccccccc ccccccccccccccccc os information cccccccccccccccccc cccccccccccccccccc...

3. Shell Programming and Scripting

insert a header in a huge data file without using an intermediate file

I have a file with data extracted, and need to insert a header with a constant string, say: H|PayerDataExtract if i use sed, i have to redirect the output to a seperate file like sed ' sed commands' ExtractDataFile.dat > ExtractDataFileWithHeader.dat the same is true for awk and...

4. Shell Programming and Scripting

How to extract a subset from a huge dataset

Hi, All I have a huge file which has 450G. Its tab-delimited format is as below x1 A 50020 1 x1 B 50021 8 x1 C 50022 9 x1 A 50023 10 x2 D 50024 5 x2 C 50025 7 x2 F 50026 8 x2 N 50027 1 : : Now, I want to extract a subset from this file. In this subset, column 1 is x10, column 2 is...

5. Shell Programming and Scripting

Three Difference File Huge Data Comparison Problem.

I got three different file: Part of File 1 ARTPHDFGAA . . Part of File 2 ARTGHHYESA . . Part of File 3 ARTPOLYWEA . .

6. Shell Programming and Scripting

Help- counting delimiter in a huge file and split data into 2 files

I’m new to Linux script and not sure how to filter out bad records from huge flat files (over 1.3GB each). The delimiter is a semi colon “;” Here is the sample of 5 lines in the file: Name1;phone1;address1;city1;state1;zipcode1 Name2;phone2;address2;city2;state2;zipcode2;comment...

7. Shell Programming and Scripting

Extract header data from one file and combine it with data from another file

Hi, Great minds, I have some files, in fact header files, of CTD profiler, I tried a lot C programming, could not get output as I was expected, because my programming skills are very poor, finally, joined unix forum with the hope that, I may get what I want, from you people, Here I have attached...

8. Shell Programming and Scripting

Extract few content from a huge list of files

I have a huge list of files (about 300,000) which have a pattern like this. .I 1 .U 87049087 .S Am J Emerg .M Allied Health Personnel/*; Electric Countershock/*; .T Refibrillation managed by EMT-Ds: .P ARTICLE. .W Some patients converted from ventricular fibrillation to organized...

9. UNIX for Advanced & Expert Users

Need Optimization shell/awk script to aggreagte (sum) for all the columns of Huge data file

Optimization shell/awk script to aggregate (sum) for all the columns of Huge data file File delimiter "|" Need to have Sum of all columns, with column number : aggregation (summation) for each column File not having the header Like below - Column 1 "Total Column 2 : "Total ... ......

10. UNIX for Advanced & Expert Users

File comaprsons for the Huge data files ( around 60G) - Need optimized and teh best way to do this

I have 2 large file (.dat) around 70 g, 12 columns but the data not sorted in both the files.. need your inputs in giving the best optimized method/command to achieve this and redirect the not macthing lines to the thrid file ( diff.dat) File 1 - 15 columns File 2 - 15 columns Data is...

LEARN ABOUT CENTOS

amarchiver

AMARCHIVER(8)						  System Administration Commands					     AMARCHIVER(8)

NAME

       amarchiver - Create, extract or list amanda archive

SYNOPSIS

       amarchiver --version|--create|--extract|--list [--verbose] [--file file] [filename]...

DESCRIPTION

       Amarchiver manipulates amanda archive file. On creation, amarchiver doesn't recurse into directories. If a filename specifies a directory,
       it will be ignored. If it specifies a character device, amarchiver reads the device and archives the data. If a filename specifies a named
       pipe, amarchiver reads the named pipe and archives the data.

       Note that this tool is more limited than the Amanda archive library, and may not be appropriate for some archive files -- particularly
       those which use non-strings in their filenames.

OPTIONS

       --version
	   print the amarchiver version

       --create
	   Create an amanda archive. Only the supplied filenames are included. With one --verbose, lists the filenames. With two, lists the
	   filenames and sizes.

       --list
	   List the filenames in an amanda archive. No additional filenames are allowed on the command line.

       --extract
	   Extract an amanda archive. If filenames are supplied, only those files are extracted. Files are created in the current directory,
	   suffixed with a dot ('.') and the attribute ID.

       --verbose
	   Give more information.

       --file file
	   Create, list or extract from the given file instead of stdin/stdout.

SEE ALSO

       amanda(8), amanda-archive-format(5)

       The Amanda Wiki: : http://wiki.zmanda.com/

AUTHORS

       Dustin J. Mitchell <dustin@zmanda.com>
	   Zmanda, Inc. (http://www.zmanda.com)

       Jean-Louis Martineau <martineau@zmanda.com>
	   Zmanda, Inc. (http://www.zmanda.com)

Amanda 3.3.3							    01/10/2013							     AMARCHIVER(8)