Sponsored Content
Top Forums Shell Programming and Scripting Help- counting delimiter in a huge file and split data into 2 files Post 302497369 by kurumi on Thursday 17th of February 2011 04:20:26 AM
Old 02-17-2011
Code:
 $ ruby -ne '$_.count(";")==5 && print ' file >> good

 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split a huge data into few different files?!

Input file data contents: >seq_1 MSNQSPPQSQRPGHSHSHSHSHAGLASSTSSHSNPSANASYNLNGPRTGGDQRYRASVDA >seq_2 AGAAGRGWGRDVTAAASPNPRNGGGRPASDLLSVGNAGGQASFASPETIDRWFEDLQHYE >seq_3 ATLEEMAAASLDANFKEELSAIEQWFRVLSEAERTAALYSLLQSSTQVQMRFFVTVLQQM ARADPITALLSPANPGQASMEAQMDAKLAAMGLKSPASPAVRQYARQSLSGDTYLSPHSA... (7 Replies)
Discussion started by: patrick87
7 Replies

2. Shell Programming and Scripting

Perl script error to split huge data one by one.

Below is my perl script: #!/usr/bin/perl open(FILE,"$ARGV") or die "$!"; @DATA = <FILE>; close FILE; $join = join("",@DATA); @array = split( ">",$join); for($i=0;$i<=scalar(@array);$i++){ system ("/home/bin/./program_name_count_length MULTI_sequence_DATA_FILE -d... (5 Replies)
Discussion started by: patrick87
5 Replies

3. Shell Programming and Scripting

renaming files using split with a delimiter

I have a directory of files that I need to rename by splitting the first and second halves of the filenames using the delimiter "-O" and then renaming with the second half first, followed by two underscores and then the first half. For example, natfinal1995annvol1_14.pdf -O filenum-20639 will be... (2 Replies)
Discussion started by: swimulator
2 Replies

4. Shell Programming and Scripting

Implement in one line sed or awk having no delimiter and file size is huge

I have file which contains around 5000 lines. The lines are fixed legth but having no delimiter.Each line line contains nearly 3000 characters. I want to delete the lines a> if it starts with 1 and if 576th postion is a digit i,e 0-9 or b> if it starts with 0 or 9(i,e header and footer) ... (4 Replies)
Discussion started by: millan
4 Replies

5. Shell Programming and Scripting

Split file into multiple files using delimiter

Hi, I have a file which has many URLs delimited by space. Now i want them to move to separate files each one holding 10 URLs per file. http://3276.e-printphoto.co.uk/guardian http://abdera.apache.org/ http://abdera.apache.org/docs/api/index.html I have used the below code to arrange... (6 Replies)
Discussion started by: vel4ever
6 Replies

6. UNIX for Dummies Questions & Answers

Split a huge 7 GB File Based on Pattern into 4 files

Hi, I have a Huge 7 GB file which has around 1 million records, i want to split this file into 4 files to contain around 250k messages each. Please help me as Split command cannot work here as it might miss tags.. Format of the file is as below <!--###### ###### START-->... (6 Replies)
Discussion started by: KishM
6 Replies

7. Shell Programming and Scripting

Split a folder with huge number of files in n folders

We have a folder XYZ with large number of files (>350,000). how can i split the folder and create say 10 of them XYZ1 to XYZ10 with 35,000 files each. (doesnt matter which files go where). (12 Replies)
Discussion started by: AlokKumbhare
12 Replies

8. UNIX for Advanced & Expert Users

File comaprsons for the Huge data files ( around 60G) - Need optimized and teh best way to do this

I have 2 large file (.dat) around 70 g, 12 columns but the data not sorted in both the files.. need your inputs in giving the best optimized method/command to achieve this and redirect the not macthing lines to the thrid file ( diff.dat) File 1 - 15 columns File 2 - 15 columns Data is... (9 Replies)
Discussion started by: kartikirans
9 Replies

9. UNIX for Beginners Questions & Answers

Shell script to Split matrix file with delimiter into multiple files

I have a large semicolon delimited file with thousands of columns and many thousands of line. It looks like: ID1;ID2;ID3;ID4;A_1;B_1;C_1;A_2;B_2;C_2;A_3;B_3;C_3 AA;ax;ay;az;01;02;03;04;05;06;07;08;09 BB;bx;by;bz;03;05;33;44;15;26;27;08;09 I want to split this table in to multiple files: ... (1 Reply)
Discussion started by: trymega
1 Replies
ERB(1)							 Ruby Programmers Reference Guide						    ERB(1)

NAME
erb -- Ruby Templating SYNOPSIS
erb [--version] [-UPdnvx] [-E ext[:int]] [-S level] [-T mode] [-r library] [--] [file ...] DESCRIPTION
erb is a command line front-end for ERB library, which is an implementation of eRuby. ERB provides an easy to use but powerful templating system for Ruby. Using ERB, actual Ruby code can be added to any plain text document for the purposes of generating document information details and/or flow control. erb is a part of Ruby. OPTIONS
--version Prints the version of erb. -E external[:internal] --encoding external[:internal] Specifies the default value(s) for external encodings and internal encoding. Values should be separated with colon (:). You can omit the one for internal encodings, then the value (Encoding.default_internal) will be nil. -P Evaluates lines starting with % as Ruby code and removes the tailing EOLs. -S level Specifies the safe level in which eRuby script will run. -T mode Specifies trim mode (default 0). mode can be one of 0 EOL remains after the embedded ruby script is evaluated. 1 EOL is removed if the line ends with %>. 2 EOL is removed if the line starts with <% and ends with %>. - EOL is removed if the line ends with -%>. And leading whitespaces are removed if the erb directive starts with <%-. -U can be one of Sets the default value for internal encodings (Encoding.default_internal) to UTF-8. -d --debug Turns on debug mode. $DEBUG will be set to true. -h --help Prints a summary of the options. -n Used with -x. Prepends the line number to each line in the output. -v Enables verbose mode. $VERBOSE will be set to true. -x Converts the eRuby script into Ruby script and prints it without line numbers. EXAMPLES
Here is an eRuby script <?xml version="1.0" ?> <% require 'prime' -%> <erb-example> <calc><%= 1+1 %></calc> <var><%= __FILE__ %></var> <library><%= Prime.each(10).to_a.join(", ") %></library> </erb-example> Command % erb -T - example.erb prints <?xml version="1.0" ?> <erb-example> <calc>2</calc> <var>example.erb</var> <library>2, 3, 5, 7</library> </erb-example> SEE ALSO
ruby(1). And see ri(1) documentation for ERB class. REPORTING BUGS
Security vulnerabilities should be reported via an email to <security@ruby-lang.org>. Reported problems will be published after being fixed. And you can report other bugs and feature requests via the Ruby Issue Tracking System (http://bugs.ruby-lang.org). Do not report security vulnerabilities via the system because it publishes the vulnerabilities immediately. AUTHORS
Written by Masatoshi SEKI. UNIX
November 7, 2012 UNIX
All times are GMT -4. The time now is 01:03 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy