Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Implement in one line sed or awk having no delimiter and file size is huge
Top Forums Shell Programming and Scripting Implement in one line sed or awk having no delimiter and file size is huge Post 302489239 by Ygor on Wednesday 19th of January 2011 10:54:43 PM
Old 01-19-2011
In awk...
Code:
awk '!/^[09]/&&!(/^1/&&substr($0,576,1)~/[0-9]/)' A.DAT > B.DAT

---------- Post updated at 11:54 AM ---------- Previous update was at 11:49 AM ----------

In perl...
Code:
perl -n -e 'print $_ if !/^[09]/ && !(/^1/ && substr($_, 576, 1) =~ /[0-9]/)' A.DAT > B.DAT

These 2 Users Gave Thanks to Ygor For This Post:
Chubler_XL millan
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Insert a line as the first line into a very huge file

Hello, I need to insert a line (like a header) as the first line of a very huge file (about 3 ml rows). I am able to do it with sed, but redirecting the output and creating a new file takes quite some time. I was wondering if there was a more efficient way of doing it? Any help would be... (3 Replies)
Discussion started by: shriek
3 Replies

2. Shell Programming and Scripting

Split a huge line into multiple 120 characters lines with sed?

Hello , I'm trying to split a file which contains a single very long line. My aim is to split this single line each 120 characters. I tried with the sed command : `cat ${MYPATH}/${FILE}|sed -e :a -e 's/^.\{1,120\}$/&\n/;ta' >{MYPATH}/${DEST}` but when I wc -l the destination file it is... (2 Replies)
Discussion started by: jerome_1664
2 Replies

3. Shell Programming and Scripting

awk,nawk,sed, delimiter |~|

RECORD=NEW|~|VENDORN=LUCENT|~|VENDORM=CBX500_REAR|~|NETWORK=ATM|~|SUBNETWORK=N/A|~|SITE=CIL|~|REGION=KN|~|COUNTRY=PS|~|SWITCH=SWITCH1|~|E THERNET=N/A|~|LOOPBACK=N/A|~|SHELF=N/A|~|SLOT=14|~|SUBSLOT=N/A|~|STSCHAN=N/A|~|PORT=S14|~|DS1SLOT=N/A|~|LINE=N/A|~|LPORTID=N/A|~|CARDDESC=N/A|~|CARDTYPE=BAC2RT0... (7 Replies)
Discussion started by: knijjar
7 Replies

4. Shell Programming and Scripting

how to implement in one-line awk in a fixed file having no delimiter

Hi, I have a file a.txt having no delimiter. I want to exclude the line which contains 435th character as 1 or 2 and redirect the rest of the lines to another file b. Can you pls suggest how to do this in one liner awk. Following is just one line of the input file a:- 120110116 ... (10 Replies)
Discussion started by: millan
10 Replies

5. Shell Programming and Scripting

Help- counting delimiter in a huge file and split data into 2 files

I’m new to Linux script and not sure how to filter out bad records from huge flat files (over 1.3GB each). The delimiter is a semi colon “;” Here is the sample of 5 lines in the file: Name1;phone1;address1;city1;state1;zipcode1 Name2;phone2;address2;city2;state2;zipcode2;comment... (7 Replies)
Discussion started by: lv99
7 Replies

6. Shell Programming and Scripting

FTP a huge Size file

Dear All, Good Evening!! I have a requirement to ftp a 220GB backup file to a remote backup server. I wrote a script for this purpose. But it takes more than 8 hours to transfer this file. Is there any other method to do it in less time??? Thanks in Advance!!! ---------- Post updated... (5 Replies)
Discussion started by: Naga06
5 Replies

7. Shell Programming and Scripting

Optimised way for search & replace a value on one line in a very huge file (File Size is 24 GB).

Hi Experts, I had to edit (a particular value) in header line of a very huge file so for that i wanted to search & replace a particular value on a file which was of 24 GB in Size. I managed to do it but it took long time to complete. Can anyone please tell me how can we do it in a optimised... (7 Replies)
Discussion started by: manishkomar007
7 Replies

8. HP-UX

Performance issue with 'grep' command for huge file size

I have 2 files; one file (say, details.txt) contains the details of employees and another file (say, emp.txt) has some selected employee names. I am extracting employee details from details.txt by using emp.txt and the corresponding code is: while read line do emp_name=`echo $line` grep -e... (7 Replies)
Discussion started by: arb_1984
7 Replies

9. Shell Programming and Scripting

Need next line as a space delimiter in awk

Hi,Below is the output for p3fi_dev services 1/app/oracle> . ./oraprofile_p3fi_dev p3fi_dev_01 (P):/devoragridcn_01/app/oracle> srvctl config service -d p3fi_dev p3fi_p3fi_dev.world PREF: p3fi_dev_01 AVAIL: p3fi_dev_02 pplnet_p3fidev PREF: p3fi_dev_01 AVAIL: p3fi_dev_02 nexus_p3fidev PREF:... (3 Replies)
Discussion started by: Vishal_dba
3 Replies

10. Shell Programming and Scripting

Join the line on delimiter using sed/awk in UNIX

I've input as , abcd| ef 123456| 78| 90 Desired output as, abcdef 1234567890 Anyone please give the solution. (5 Replies)
Discussion started by: jinixvimal
5 Replies

LEARN ABOUT CENTOS

datadm

datadm(1M)																datadm(1M)

NAME

       datadm - maintain DAT static registry file

SYNOPSIS

       /usr/bin/datadm [-v] [-u] [-a service_provider.conf] [-r service_provider.conf]

       The datadm utility maintains the DAT static registry file, dat.conf(4).

       This administrative configuration program allows uDAPL service providers to add and remove themselves to the dat.conf file.

       You  can  add  or  remove  interface  adapters that a service provider supports from a system after its installation. You can use datadm to
       update the dat.conf file to reflect the current state of the system. A new set of interface adapters for all  the  service  providers  cur-
       rently installed is regenerated.

       The following options are supported:

       -a service_provider.conf

	   Enumerate  each  device  entry  in the service_provider.conf(4) file into a list of interface adapters, that is, interfaces to external
	   network that are available to uDAPL consumers.

       -r service_provider.conf

	   Remove the list of interface adapters that corresponds to the device entry in the service_provider.conf(4) file.

       -u

	   Update the dat.conf to reflect the current state of the system with an up to date set of interface adapters for the	service  providers
	   that are currently listed in the DAT static registry.

       -v

	   Display the DAT static registry file, dat.conf.

       Example 1: Enumerating a Device Entry

       The following example enumerates a device entry in the service_provider.conf(4) file into interface adapters in the dat.conf(4) file.

       Assume  that  SUNW  has	a service provider library that supports the device tavor. It has a service_provider.conf(4) file installed in the
       directory /usr/share/dat/SUNWudaplt.conf with a single entry as follows:

       driver_name=tavor u1.2 nonthreadsafe default udapl_tavor.so.1 SUNW.1.0 ""

       tavor is an Infiniband Host Channel Adapter with two ports. Both IB ports exist in a single IB partition, 0x8001. If  an  IP  interface	is
       plumbed to each port, there are two IPoIB device instances, ibd0 and ibd1:

       # ls -l /dev/ibd*
       /dev/ibd0 -> /devices/pci@1/pci15b3,5a44@0/ibport@1,8001,ipib:ibd0
       /dev/ibd1 -> /devices/pci@1/pci15b3,5a44@0/ibport@2,8001,ipib:ibd1

       Running	the  command,  datadm  -a  /usr/share/dat/SUNWudaplt.conf  appends  two  new  entries  (if  they  do  not  already exists]) in the
       /etc/dat/dat.conffile:

       ibd0 u1.2 nonthreadsafe default udapl_tavor.so.1 SUNW.1.0 ""
       "driver_name=tavor"
       ibd1 u1.2 nonthreadsafe default udapl_tavor.so.1 SUNW.1.0 ""
       "driver_name=tavor"

       Example 2: Updating the dat.conf to Reflect the Current State of the System

       A new IB partition, 0x8002 is added to the above example covering port 1 of the Host Channel Adapter. If a new IP interface is  plumbed	to
       port 1/partition 0x8002, there is a third IPoIB device instance: ibd2.

       # ls -l /dev/ibd*
	/dev/ibd0 -> /devices/pci@1/pci15b3,5a44@0/ibport@1,8001,ipib:ibd0
	/dev/ibd1 -> /devices/pci@1/pci15b3,5a44@0/ibport@2,8001,ipib:ibd1
	/dev/ibd2 -> /devices/pci@1/pci15b3,5a44@0/ibport@1,8002,ipib:ibd2

       Running datadm -u command, updates the /etc/dat/dat.conf file with a new entry added reflecting the current state of the system.

       datadm -v shows that there are now three entries in the /etc/dat/dat.conf file:

       ibd0 u1.2 nonthreadsafe default udapl_tavor.so.1 SUNW.1.0 ""
       "driver_name=tavor"
       ibd1 u1.2 nonthreadsafe default udapl_tavor.so.1 SUNW.1.0 ""
       "driver_name=tavor"
       ibd2 u1.2 nonthreadsafe default udapl_tavor.so.1 SUNW.1.0 ""
       "driver_name=tavor"

       /etc/dat/dat.conf       DAT static registry file

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |Availability		     |SUNWudaplu		   |
       +-----------------------------+-----------------------------+
       |Interface Stability	     |Evolving			   |
       +-----------------------------+-----------------------------+

       pkgadd(1M), pkgrm(1M), libdat(3LIB), dat.conf(4), service_provider.conf(4). attributes(5)

								    18 Jun 2004 							datadm(1M)
All times are GMT -4. The time now is 08:51 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy