Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: How to extract data from a huge file?
Top Forums Shell Programming and Scripting How to extract data from a huge file? Post 302159592 by dennis.jacob on Friday 18th of January 2008 03:43:27 AM
Old 01-18-2008
Try this:

Code:
cat test| grep item_ | xargs mkdir
for each in $(cat test| grep item_)
do
awk '/'$each'/,/dublin_core>/ {print }' test > ./${each}/dublin_core.xml
echo "license.txt(tab \t)bundle:LICENSE" >./${each}/contents
done

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

search and grab data from a huge file

folks, In my working directory, there a multiple large files which only contain one line in the file. The line is too long to use "grep", so any help? For example, if I want to find if these files contain a string like "93849", what command I should use? Also, there is oder_id number... (1 Reply)
Discussion started by: ting123
1 Replies

2. Shell Programming and Scripting

How to extract a piece of information from a huge file

Hello All, I need some assistance to extract a piece of information from a huge file. The file is like this one : database information ccccccccccccccccc ccccccccccccccccc ccccccccccccccccc ccccccccccccccccc os information cccccccccccccccccc cccccccccccccccccc... (2 Replies)
Discussion started by: Marcor
2 Replies

3. Shell Programming and Scripting

insert a header in a huge data file without using an intermediate file

I have a file with data extracted, and need to insert a header with a constant string, say: H|PayerDataExtract if i use sed, i have to redirect the output to a seperate file like sed ' sed commands' ExtractDataFile.dat > ExtractDataFileWithHeader.dat the same is true for awk and... (10 Replies)
Discussion started by: deepaktanna
10 Replies

4. Shell Programming and Scripting

How to extract a subset from a huge dataset

Hi, All I have a huge file which has 450G. Its tab-delimited format is as below x1 A 50020 1 x1 B 50021 8 x1 C 50022 9 x1 A 50023 10 x2 D 50024 5 x2 C 50025 7 x2 F 50026 8 x2 N 50027 1 : : Now, I want to extract a subset from this file. In this subset, column 1 is x10, column 2 is... (3 Replies)
Discussion started by: cliffyiu
3 Replies

5. Shell Programming and Scripting

Three Difference File Huge Data Comparison Problem.

I got three different file: Part of File 1 ARTPHDFGAA . . Part of File 2 ARTGHHYESA . . Part of File 3 ARTPOLYWEA . . (4 Replies)
Discussion started by: patrick87
4 Replies

6. Shell Programming and Scripting

Help- counting delimiter in a huge file and split data into 2 files

I’m new to Linux script and not sure how to filter out bad records from huge flat files (over 1.3GB each). The delimiter is a semi colon “;” Here is the sample of 5 lines in the file: Name1;phone1;address1;city1;state1;zipcode1 Name2;phone2;address2;city2;state2;zipcode2;comment... (7 Replies)
Discussion started by: lv99
7 Replies

7. Shell Programming and Scripting

Extract header data from one file and combine it with data from another file

Hi, Great minds, I have some files, in fact header files, of CTD profiler, I tried a lot C programming, could not get output as I was expected, because my programming skills are very poor, finally, joined unix forum with the hope that, I may get what I want, from you people, Here I have attached... (17 Replies)
Discussion started by: nex_asp
17 Replies

8. Shell Programming and Scripting

Extract few content from a huge list of files

I have a huge list of files (about 300,000) which have a pattern like this. .I 1 .U 87049087 .S Am J Emerg .M Allied Health Personnel/*; Electric Countershock/*; .T Refibrillation managed by EMT-Ds: .P ARTICLE. .W Some patients converted from ventricular fibrillation to organized... (1 Reply)
Discussion started by: shoaibjameel123
1 Replies

9. UNIX for Advanced & Expert Users

Need Optimization shell/awk script to aggreagte (sum) for all the columns of Huge data file

Optimization shell/awk script to aggregate (sum) for all the columns of Huge data file File delimiter "|" Need to have Sum of all columns, with column number : aggregation (summation) for each column File not having the header Like below - Column 1 "Total Column 2 : "Total ... ...... (2 Replies)
Discussion started by: kartikirans
2 Replies

10. UNIX for Advanced & Expert Users

File comaprsons for the Huge data files ( around 60G) - Need optimized and teh best way to do this

I have 2 large file (.dat) around 70 g, 12 columns but the data not sorted in both the files.. need your inputs in giving the best optimized method/command to achieve this and redirect the not macthing lines to the thrid file ( diff.dat) File 1 - 15 columns File 2 - 15 columns Data is... (9 Replies)
Discussion started by: kartikirans
9 Replies

LEARN ABOUT OPENSOLARIS

file-roller

file-roller(1)							   User Commands						    file-roller(1)

NAME

       file-roller - create, view, modify, or unpack an archive

SYNOPSIS

       file-roller  [--add]  [--add-to=archive]  [--default-dir=folder]  [--extract]  [--extract-here] [--extract-to=folder] [--force] [gnome-std-
       options]

DESCRIPTION

       The File Roller application enables you to create, view, modify, or unpack an archive. An archive is a file that acts as  a  container  for
       other files. An archive can contain many files, folders, and subfolders, usually in compressed form.

OPTIONS

       The following options are supported:

       -d, --add		       Ask for the name of the archive, then add the files, then quit the application.

       -a, --add-to=archive	       Add files to the specified archive, then quit the application.

       --default-dir=folder	       Specify the default folder for the --add and --extract options.

       -f, --extract		       Ask for the name of the destination folder, then extract the files, then quit the application.

       -h, --extract-here	       Extract archives using the archive name as destination folder and quit the program.

       -e, --extract-to=folder	       Extract archives to the specified folder, then quit the application.

       --force			       Create the destination folder without asking for confirmation.

       gnome-std-options	       Standard  options available for use with most GNOME applications.  See gnome-std-options(5) for more infor-
				       mation.

EXAMPLES

       Example 1: Unpacking an Archive

       example% file-roller --extract archive

EXIT STATUS

       The following exit values are returned:

       0	Application exited successfully

       >0	Application exited with failure

FILES

       The following files are used by this application:

       /usr/bin/file-roller    Executable for File Roller application.

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |Availability		     |SUNW			   |
       +-----------------------------+-----------------------------+
       |Interface stability	     |Volatile			   |
       +-----------------------------+-----------------------------+

SEE ALSO

       File Roller Manual

       ar(1), tar(1), unzip(1), zip(1), attributes(5), gnome-std-options(5)

NOTES

       Written by Laszlo Kovacs, Sun Microsystems Inc., 2003, 2004, 2007.

SunOS 5.11							    08 Nov 2007 						    file-roller(1)
All times are GMT -4. The time now is 10:52 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy