Sponsored Content
Top Forums Shell Programming and Scripting Implement in one line sed or awk having no delimiter and file size is huge Post 302489239 by Ygor on Wednesday 19th of January 2011 10:54:43 PM
Old 01-19-2011
In awk...
Code:
awk '!/^[09]/&&!(/^1/&&substr($0,576,1)~/[0-9]/)' A.DAT > B.DAT

---------- Post updated at 11:54 AM ---------- Previous update was at 11:49 AM ----------

In perl...
Code:
perl -n -e 'print $_ if !/^[09]/ && !(/^1/ && substr($_, 576, 1) =~ /[0-9]/)' A.DAT > B.DAT

These 2 Users Gave Thanks to Ygor For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Insert a line as the first line into a very huge file

Hello, I need to insert a line (like a header) as the first line of a very huge file (about 3 ml rows). I am able to do it with sed, but redirecting the output and creating a new file takes quite some time. I was wondering if there was a more efficient way of doing it? Any help would be... (3 Replies)
Discussion started by: shriek
3 Replies

2. Shell Programming and Scripting

Split a huge line into multiple 120 characters lines with sed?

Hello , I'm trying to split a file which contains a single very long line. My aim is to split this single line each 120 characters. I tried with the sed command : `cat ${MYPATH}/${FILE}|sed -e :a -e 's/^.\{1,120\}$/&\n/;ta' >{MYPATH}/${DEST}` but when I wc -l the destination file it is... (2 Replies)
Discussion started by: jerome_1664
2 Replies

3. Shell Programming and Scripting

awk,nawk,sed, delimiter |~|

RECORD=NEW|~|VENDORN=LUCENT|~|VENDORM=CBX500_REAR|~|NETWORK=ATM|~|SUBNETWORK=N/A|~|SITE=CIL|~|REGION=KN|~|COUNTRY=PS|~|SWITCH=SWITCH1|~|E THERNET=N/A|~|LOOPBACK=N/A|~|SHELF=N/A|~|SLOT=14|~|SUBSLOT=N/A|~|STSCHAN=N/A|~|PORT=S14|~|DS1SLOT=N/A|~|LINE=N/A|~|LPORTID=N/A|~|CARDDESC=N/A|~|CARDTYPE=BAC2RT0... (7 Replies)
Discussion started by: knijjar
7 Replies

4. Shell Programming and Scripting

how to implement in one-line awk in a fixed file having no delimiter

Hi, I have a file a.txt having no delimiter. I want to exclude the line which contains 435th character as 1 or 2 and redirect the rest of the lines to another file b. Can you pls suggest how to do this in one liner awk. Following is just one line of the input file a:- 120110116 ... (10 Replies)
Discussion started by: millan
10 Replies

5. Shell Programming and Scripting

Help- counting delimiter in a huge file and split data into 2 files

I’m new to Linux script and not sure how to filter out bad records from huge flat files (over 1.3GB each). The delimiter is a semi colon “;” Here is the sample of 5 lines in the file: Name1;phone1;address1;city1;state1;zipcode1 Name2;phone2;address2;city2;state2;zipcode2;comment... (7 Replies)
Discussion started by: lv99
7 Replies

6. Shell Programming and Scripting

FTP a huge Size file

Dear All, Good Evening!! I have a requirement to ftp a 220GB backup file to a remote backup server. I wrote a script for this purpose. But it takes more than 8 hours to transfer this file. Is there any other method to do it in less time??? Thanks in Advance!!! ---------- Post updated... (5 Replies)
Discussion started by: Naga06
5 Replies

7. Shell Programming and Scripting

Optimised way for search & replace a value on one line in a very huge file (File Size is 24 GB).

Hi Experts, I had to edit (a particular value) in header line of a very huge file so for that i wanted to search & replace a particular value on a file which was of 24 GB in Size. I managed to do it but it took long time to complete. Can anyone please tell me how can we do it in a optimised... (7 Replies)
Discussion started by: manishkomar007
7 Replies

8. HP-UX

Performance issue with 'grep' command for huge file size

I have 2 files; one file (say, details.txt) contains the details of employees and another file (say, emp.txt) has some selected employee names. I am extracting employee details from details.txt by using emp.txt and the corresponding code is: while read line do emp_name=`echo $line` grep -e... (7 Replies)
Discussion started by: arb_1984
7 Replies

9. Shell Programming and Scripting

Need next line as a space delimiter in awk

Hi,Below is the output for p3fi_dev services 1/app/oracle> . ./oraprofile_p3fi_dev p3fi_dev_01 (P):/devoragridcn_01/app/oracle> srvctl config service -d p3fi_dev p3fi_p3fi_dev.world PREF: p3fi_dev_01 AVAIL: p3fi_dev_02 pplnet_p3fidev PREF: p3fi_dev_01 AVAIL: p3fi_dev_02 nexus_p3fidev PREF:... (3 Replies)
Discussion started by: Vishal_dba
3 Replies

10. Shell Programming and Scripting

Join the line on delimiter using sed/awk in UNIX

I've input as , abcd| ef 123456| 78| 90 Desired output as, abcdef 1234567890 Anyone please give the solution. (5 Replies)
Discussion started by: jinixvimal
5 Replies
paste(1)							   User Commands							  paste(1)

NAME
paste - merge corresponding or subsequent lines of files SYNOPSIS
paste [-s] [-d list] file... DESCRIPTION
The paste utility will concatenate the corresponding lines of the given input files, and write the resulting lines to standard output. The default operation of paste will concatenate the corresponding lines of the input files. The NEWLINE character of every line except the line from the last input file will be replaced with a TAB character. If an EOF (end-of-file) condition is detected on one or more input files, but not all input files, paste will behave as though empty lines were read from the files on which EOF was detected, unless the -s option is specified. OPTIONS
The following options are supported: -d list Unless a backslash character () appears in list, each character in list is an element specifying a delimiter character. If a backslash character appears in list, the backslash character and one or more characters following it are an element specifying a delimiter character as described below. These elements specify one or more delimiters to use, instead of the default TAB charac- ter, to replace the NEWLINE character of the input lines. The elements in list are used circularly. That is, when the list is exhausted, the first element from the list is reused. When the -s option is specified: o The last newline character in a file will not be modified. o The delimiter will be reset to the first element of list after each file operand is processed. When the option is not specified: o The NEWLINE characters in the file specified by the last file will not be modified. o The delimiter will be reset to the first element of list each time a line is processed from each file. If a backslash character appears in list, it and the character following it will be used to represent the following delimiter characters: Newline character. Tab character. \ Backslash character. Empty string (not a null character). If is immediately followed by the character x, the character X, or any character defined by the LC_CTYPE digit keyword, the results are unspecified. If any other characters follow the backslash, the results are unspecified. -s Concatenate all of the lines of each separate input file in command line order. The NEWLINE character of every line except the last line in each input file will be replaced with the TAB character, unless otherwise specified by the -d option. OPERANDS
The following operand is supported: file A path name of an input file. If - is specified for one or more of the files, the standard input will be used. The standard input will be read one line at a time, circularly, for each instance of -. Implementations support pasting of at least 12 file operands. USAGE
See largefile(5) for the description of the behavior of paste when encountering files greater than or equal to 2 Gbyte ( 2**31 bytes). EXAMPLES
Example 1: Listing a directory in one column example% ls | paste -d" " - Example 2: Listing a directory in four columns example% ls | paste - - - - Example 3: Combining pairs of lines from a file into single lines example% paste -s -d" t n" file ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of paste: LANG, LC_ALL, LC_CTYPE, LC_MES- SAGES, and NLSPATH. EXIT STATUS
The following exit values are returned: 0 Successful completion. >0 An error occurred. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWesu | +-----------------------------+-----------------------------+ |CSI |Enabled | +-----------------------------+-----------------------------+ |Interface Stability |Standard | +-----------------------------+-----------------------------+ SEE ALSO
cut(1), grep(1), pr(1), attributes(5), environ(5), largefile(5), standards(5) DIAGNOSTICS
"line too long" Output lines are restricted to 511 characters. "too many files" Except for -s option, no more than 12 input files may be specified. "no delimiters" The -d option was specified with an empty list. "cannot open file" The specified file cannot be opened. SunOS 5.10 20 Dec 1996 paste(1)
All times are GMT -4. The time now is 06:15 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy