Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Split the file based on pattern
Top Forums Shell Programming and Scripting Split the file based on pattern Post 302757709 by sol_nov on Friday 18th of January 2013 12:42:54 AM
Old 01-18-2013
Power
Quote:
Originally Posted by Chubler_XL
How about using sed:

Code:
sed 's:|[^|]*pi=\[\([^]]*\)],uid=\[\([^]]*\)].*:|\1|\2:' scenario_1.txt > scenario_1_n.txt
sed 's:|[^|]*390=\([^,]*\),391=\([^,]*\),.*:|\1|\2:' scenario_2.txt > scenario_2_n.txt
sed 's:|[^|~]*~[^~]*~\([^~]*\)~\([^~]*\)~.*:|\1|\2:' scenario_3.txt > scenario_3_n.txt


But it is failing for row size around 10k bytes.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split a file based on pattern in awk, grep, sed or perl

Hi All, Can someone please help me write a script for the following requirement in awk, grep, sed or perl. Buuuu xxx bbb Kmmmm rrr ssss uuuu Kwwww zzzz ccc Roooowwww eeee Bxxxx jjjj dddd Kuuuu eeeee nnnn Rpppp cccc vvvv cccc Rhhhhhhyyyy tttt Lhhhh rrrrrssssss Bffff mmmm iiiii Ktttt... (5 Replies)
Discussion started by: kumarn
5 Replies

2. Shell Programming and Scripting

Split File Based on Line Number Pattern

Hello all. Sorry, I know this question is similar to many others, but I just can seem to put together exactly what I need. My file is tab delimitted and contains approximately 1 million rows. I would like to send lines 1,4,& 7 to a file. Lines 2, 5, & 8 to a second file. Lines 3, 6, & 9 to... (11 Replies)
Discussion started by: shankster
11 Replies

3. Shell Programming and Scripting

Split a file based on a pattern

Dear all, I have a large file which is composed of 8000 frames, what i would like to do is split the file into 8000 single files names file.pdb.1, file.pdb.2 etc etc each frame in the large file is seperated by a "ENDMDL" flag so my thinking is to use this flag a a point to split the files... (4 Replies)
Discussion started by: Mish_99
4 Replies

4. Shell Programming and Scripting

Split a file into multiple files based on the input pattern

I have a file with lines something like. ...... 123_start ...... ....... 123_end .... ..... 456_start ...... ..... 456_end .... ..... 789_start .... .... 789_end (6 Replies)
Discussion started by: abinash
6 Replies

5. Shell Programming and Scripting

split XML file into multiple files based on pattern

Hello, I am using awk to split a file into multiple files using command: nawk '{ if ( $1 == "<process" ) { n=split($2, arr, "\""); file=arr } print > file }' processes.xml <process name="Process1.process"> ... (3 Replies)
Discussion started by: chiru_h
3 Replies

6. Shell Programming and Scripting

Split a file based on pattern and size

Hello, I have a large file (2GB) that I would like to split based on pattern and size. I've used the following command to split the file (token is "HELLO") awk '/HELLO/{i++}{print > "file"i}' input.txt and the output is similar to the following (i included filesize in KB): 10 ... (2 Replies)
Discussion started by: jl487
2 Replies

7. UNIX for Dummies Questions & Answers

Split a huge 7 GB File Based on Pattern into 4 files

Hi, I have a Huge 7 GB file which has around 1 million records, i want to split this file into 4 files to contain around 250k messages each. Please help me as Split command cannot work here as it might miss tags.. Format of the file is as below <!--###### ###### START-->... (6 Replies)
Discussion started by: KishM
6 Replies

8. Shell Programming and Scripting

How to split a file based on pattern line number?

Hi i have requirement like below M <form_name> sdasadasdMklkM D ...... D ..... M form_name> sdasadasdMklkM D ...... D ..... D ...... D ..... M form_name> sdasadasdMklkM D ...... M form_name> sdasadasdMklkM i want split file based on line number by finding... (10 Replies)
Discussion started by: bhaskar v
10 Replies

9. Shell Programming and Scripting

Split a text file into multiple pages based on pattern

Hi, I have a text file (attached the sample). I have also, attached the way the way the files need to be split. We get this file, that will either have 24 Jurisdictions, or will miss some and retain some. Like in the attached sample file, there are only Jurisdictions 03,11,14,15, 20 and 30.... (3 Replies)
Discussion started by: ebsus
3 Replies

10. UNIX for Advanced & Expert Users

Split one file to many based on pattern

Hello All, I have records in a file in a pattern A,B,B,B,B,K,A,B,B,K Is there any command or simple logic I can pull out records into multiple files based on A record? I want output as File1: A,B,B,B,B,K File2: A,B,B,K (9 Replies)
Discussion started by: deal1dealer
9 Replies

LEARN ABOUT DEBIAN

h5jam

h5jam(1)						      General Commands Manual							  h5jam(1)

NAME

       h5jam - Add a user block to a HDF5 file

SYNOPSIS

       h5jam -u user_block -i in_file.h5 [-o out_file.h5] [--clobber]

DESCRIPTION

       h5jam  concatenates  a  user_block  file  and an HDF5 file to create an HDF5 file with a user block. The user block can be either binary or
       text. The output file is padded so that the HDF5 header begins on byte 512, 1024, etc.. (See the HDF5 File Format.)

       If out_file.h5 is given, a new file is created with the user_block followed by the contents of in_file.h5.   In	this  case,  infile.h5	is
       unchanged.

       If out_file.h5 is not specified, the user_block is added to in_file.h5.

       If  in_file.h5  already	has  a	user  block,  the contents of user_block will be added to the end of the existing user block, and the file
       shifted to the next boundary. If --clobber is set, any existing user block will be overwritten.

EXAMPLE USAGE

       Create new file, newfile.h5, with the text in file mytext.txt as the user block for the HDF5 file file.h5.

	    h5jam -u mytext.txt -i file.h5 -o newfile.h5

       Add text in file mytext.txt to front of HDF5 dataset, file.h5.

	    h5jam -u mytext.txt -i file.h5

       Overwrite the user block (if any) in file.h5 with the contents of mytext.txt.

	    h5jam -u mytext.txt -i file.h5 --clobber

RETURN VALUE

       h5jam returns the size of the output file, or -1 if an error occurs.

CAVEATS

       This tool copies all the data (sequentially) in the file(s) to new offsets. For a large file, this copy will take a long time.

       The most efficient way to create a user block is to create the file with a user block (see H5Pset_user_block), and  write  the  user  block
       data into that space from a program.

       The  user block is completely opaque to the HDF5 library and to the h5jam and h5unjam tools.  The user block is simply read or written as a
       string of bytes, which could be text or any kind of binary data.  It is up to the user to know what the contents of the	user  block  means
       and how to process it.

       When the user block is extracted, all the data is written to the output, including any padding or unwritten data.

       This tool moves the HDF5 file through byte copies, i.e., it does not read or interpret the HDF5 objects.

SEE ALSO

       h5dump(1), h5ls(1), h5diff(1), h5import(1), gif2h5(1), h52gif(1), h5perf(1), h5unjam(1).

																	  h5jam(1)
All times are GMT -4. The time now is 06:07 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy