Sponsored Content
Top Forums Shell Programming and Scripting Implement in one line sed or awk having no delimiter and file size is huge Post 302489239 by Ygor on Wednesday 19th of January 2011 10:54:43 PM
Old 01-19-2011
In awk...
Code:
awk '!/^[09]/&&!(/^1/&&substr($0,576,1)~/[0-9]/)' A.DAT > B.DAT

---------- Post updated at 11:54 AM ---------- Previous update was at 11:49 AM ----------

In perl...
Code:
perl -n -e 'print $_ if !/^[09]/ && !(/^1/ && substr($_, 576, 1) =~ /[0-9]/)' A.DAT > B.DAT

These 2 Users Gave Thanks to Ygor For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Insert a line as the first line into a very huge file

Hello, I need to insert a line (like a header) as the first line of a very huge file (about 3 ml rows). I am able to do it with sed, but redirecting the output and creating a new file takes quite some time. I was wondering if there was a more efficient way of doing it? Any help would be... (3 Replies)
Discussion started by: shriek
3 Replies

2. Shell Programming and Scripting

Split a huge line into multiple 120 characters lines with sed?

Hello , I'm trying to split a file which contains a single very long line. My aim is to split this single line each 120 characters. I tried with the sed command : `cat ${MYPATH}/${FILE}|sed -e :a -e 's/^.\{1,120\}$/&\n/;ta' >{MYPATH}/${DEST}` but when I wc -l the destination file it is... (2 Replies)
Discussion started by: jerome_1664
2 Replies

3. Shell Programming and Scripting

awk,nawk,sed, delimiter |~|

RECORD=NEW|~|VENDORN=LUCENT|~|VENDORM=CBX500_REAR|~|NETWORK=ATM|~|SUBNETWORK=N/A|~|SITE=CIL|~|REGION=KN|~|COUNTRY=PS|~|SWITCH=SWITCH1|~|E THERNET=N/A|~|LOOPBACK=N/A|~|SHELF=N/A|~|SLOT=14|~|SUBSLOT=N/A|~|STSCHAN=N/A|~|PORT=S14|~|DS1SLOT=N/A|~|LINE=N/A|~|LPORTID=N/A|~|CARDDESC=N/A|~|CARDTYPE=BAC2RT0... (7 Replies)
Discussion started by: knijjar
7 Replies

4. Shell Programming and Scripting

how to implement in one-line awk in a fixed file having no delimiter

Hi, I have a file a.txt having no delimiter. I want to exclude the line which contains 435th character as 1 or 2 and redirect the rest of the lines to another file b. Can you pls suggest how to do this in one liner awk. Following is just one line of the input file a:- 120110116 ... (10 Replies)
Discussion started by: millan
10 Replies

5. Shell Programming and Scripting

Help- counting delimiter in a huge file and split data into 2 files

I’m new to Linux script and not sure how to filter out bad records from huge flat files (over 1.3GB each). The delimiter is a semi colon “;” Here is the sample of 5 lines in the file: Name1;phone1;address1;city1;state1;zipcode1 Name2;phone2;address2;city2;state2;zipcode2;comment... (7 Replies)
Discussion started by: lv99
7 Replies

6. Shell Programming and Scripting

FTP a huge Size file

Dear All, Good Evening!! I have a requirement to ftp a 220GB backup file to a remote backup server. I wrote a script for this purpose. But it takes more than 8 hours to transfer this file. Is there any other method to do it in less time??? Thanks in Advance!!! ---------- Post updated... (5 Replies)
Discussion started by: Naga06
5 Replies

7. Shell Programming and Scripting

Optimised way for search & replace a value on one line in a very huge file (File Size is 24 GB).

Hi Experts, I had to edit (a particular value) in header line of a very huge file so for that i wanted to search & replace a particular value on a file which was of 24 GB in Size. I managed to do it but it took long time to complete. Can anyone please tell me how can we do it in a optimised... (7 Replies)
Discussion started by: manishkomar007
7 Replies

8. HP-UX

Performance issue with 'grep' command for huge file size

I have 2 files; one file (say, details.txt) contains the details of employees and another file (say, emp.txt) has some selected employee names. I am extracting employee details from details.txt by using emp.txt and the corresponding code is: while read line do emp_name=`echo $line` grep -e... (7 Replies)
Discussion started by: arb_1984
7 Replies

9. Shell Programming and Scripting

Need next line as a space delimiter in awk

Hi,Below is the output for p3fi_dev services 1/app/oracle> . ./oraprofile_p3fi_dev p3fi_dev_01 (P):/devoragridcn_01/app/oracle> srvctl config service -d p3fi_dev p3fi_p3fi_dev.world PREF: p3fi_dev_01 AVAIL: p3fi_dev_02 pplnet_p3fidev PREF: p3fi_dev_01 AVAIL: p3fi_dev_02 nexus_p3fidev PREF:... (3 Replies)
Discussion started by: Vishal_dba
3 Replies

10. Shell Programming and Scripting

Join the line on delimiter using sed/awk in UNIX

I've input as , abcd| ef 123456| 78| 90 Desired output as, abcdef 1234567890 Anyone please give the solution. (5 Replies)
Discussion started by: jinixvimal
5 Replies
DAT.CONF(5)															       DAT.CONF(5)

NAME
dat.conf - configuration file for static registration of user-level DAT rdma providers DESCRIPTION
The DAT (direct access transport) architecture supports the use of multiple DAT providers within a single consumer application. Consumers implicitly select a provider using the Interface Adapter name parameter passed to dat_ia_open(). The subsystem that maps Interface Adapter names to provider implementations is known as the DAT registry. When a consumer calls dat_ia_open(), the appropriate provider is found and notified of the consumer's request to access the IA. After this point, all DAT API calls acting on DAT objects are automatically directed to the appropriate provider entry points. A persistent, administratively configurable database is used to store mappings from IA names to provider information. This provider infor- mation includes: the file system path to the provider library object, version information, and thread safety information. The location and format of the registry is platform dependent. This database is known as the Static Registry (SR) and is provided via entries in the dat.conf file. The process of adding a provider entry is termed Static Registration. Registry File Format * All characters after # on a line are ignored (comments). * Lines on which there are no characters other than whitespace and comments are considered blank lines and are ignored. * Non-blank lines must have seven whitespace separated fields. These fields may contain whitespace if the field is quoted with double quotes. Within fields quoated with double quotes, the backslash or qoute are valid escape sequences: * Each non-blank line will contain the following fields: - The IA Name. - The API version of the library: [k|u]major.minor where "major" and "minor" are both integers in decimal format. User-level examples: "u1.2", and "u2.0". - Whether the library is thread-safe: [threadsafe|nonthreadsafe] - Whether this is the default section: [default|nondefault] - The library image, version included, to be loaded. - The vendor id and version of DAPL provider: id.major.minor - ia params, IA specific parameters - device name and port - platform params, (not used) Example netdev entries for OpenFabrics rdma_cm providers, both v1.2 and v2.0 OpenIB-cma u1.2 nonthreadsafe default libdaplcma.so.1 dapl.1.2 "ib0 0" "" ofa-v2-ib0 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 "ib0 0" "" NOTE: The OpenFabrics providers use <ia_params> to specify the device with one of the following: network address, network hostname, or netdev name; along with port number. The OpenIB- and ofa-v2- IA names are unique mappings. Reserved for OpenFabrics providers. The default location for this configuration file is /etc/dat.conf. The file location may be overridden with the environment variable DAT_OVERRIDE=/your_own_directory/your_dat.conf. SEE ALSO
25 March 2008 DAT.CONF(5)
All times are GMT -4. The time now is 03:13 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy