Sponsored Content
Top Forums Shell Programming and Scripting Comparing two fixed width file Post 302523251 by anshul_er on Wednesday 18th of May 2011 08:53:22 AM
Old 05-18-2011
Error Comparing two fixed width file

Hi Guys

I am checking the treads to get the answer but i am not able to get the answer for my question.

I have two files. First file is a pattern file and the second file is the file i want to search in it. Output will be the lines from file2.
File1:
Code:
P2797f12af                    44751228
P2b1204d0f                    33470964
P2b1205f76                    35815429
P2797f0250                    8219027

File 2:
Code:
P2797ea6c0                    1942611  SAN   SAN
P2797f12af                    44751228 SAN   SAN
P2b1204d0f                    33470964 SAN   SAN
P2b1205f76                    35815429 SAN   SAN
P2797f0250                    8219027  SAN   SAN

Output:
Code:
P2797f12af                    44751228 SAN   SAN
P2b1204d0f                    33470964 SAN   SAN
P2b1205f76                    35815429 SAN   SAN
P2797f0250                    8219027  SAN   SAN

I am able to do this using below command:
Code:
fgrep -f file1 file2

But it is giving me an error of out or memory as my file size is more than 1 million.
I also tried splitting it:

Code:
split -l 10000 file1 file1.split.
for CHUNK in file1.split.* ; do
        fgrep -f "$CHUNK" file2
done
rm file1.split.*

It is also taking a lot of time. First loop is done really quick but for the next loop to start it is taking long time. Smilie

Can you please let me know if i am doing something wrong here. Or can you please provide me any awk command to do this stuff.

You guys are great... looking forward for your reply.

Last edited by radoulov; 05-18-2011 at 10:11 AM.. Reason: Code tags, please!
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Fixed Width file using AWK

I am using the following command at the Unix prompt to make my 'infile' into a fixed width file of 100 characters. awk '{printf "%-100s\n",$0}' infile > outfile However, there are some records with a special character "©" These records are using 3 characters in place of one and my record... (2 Replies)
Discussion started by: alok.benjwal
2 Replies

2. Shell Programming and Scripting

adding delimiter to a fixed width file

Hi , I have a file : CSCH74000.00 CSCH74000.00 CSCH74100.00 CSCH74000.00 CSCH74100.00 CSCH74000.00 CSCH74000.00 CSCH74100.00 CSCH74100.00 CSCH74100.00 I have to put a delimiter( say comma) in between after 6th character: CSCH74,000.00 CSCH74,000.00 CSCH74,100.00 (2 Replies)
Discussion started by: sumeet
2 Replies

3. Shell Programming and Scripting

Changing particular field in fixed width file

I have a fixed width file and i need to change 36th field to "G" in for about random 20 records? How can I do it? (4 Replies)
Discussion started by: dsravan
4 Replies

4. UNIX Desktop Questions & Answers

Help with Fixed width File Parsing

I am trying to parse a Fixed width file with data as below. I am trying to assign column values from each record to variables. When I parse the data, the spaces in all coumns are dropped. I would like to retain the spaces as part of the dat stored in the variables. Any help is appreciated. I... (4 Replies)
Discussion started by: sate911
4 Replies

5. Shell Programming and Scripting

Comparing column of variable length anf fixed width file

Hi, I have two input files. File1: ID Name Place 1-234~name1~Newyork 1-34~name2~Boston 1-2345~name3~Hungary File1 is a variable length file where each column is seperated by delimitter "~". File2: ID Country 1-34<<11 SPACES>>USA<<7 spaces>> 1-234<<10 SPACES>>UK<<8... (5 Replies)
Discussion started by: manneni prakash
5 Replies

6. Shell Programming and Scripting

Fixed-Width file from Oracle

Hi All, I have created a script which generates FIXED-WIDTH file by executing Oracle query. SELECT RPAD(NVL(col1,CHR(9)),20)||NVL(col2,CHR(9))||NVL(col3,CHR(9) FROM XYZ It generates the data file with proper alignment. But if same file i transfer to windows server or Mainframe... (5 Replies)
Discussion started by: Amit.Sagpariya
5 Replies

7. Shell Programming and Scripting

sorting a fixed width seq file

I have a file like this... 2183842512010-11-25 15379043 453130325 2386225062010-11-30 4946518 495952336 2386225062010-11-30 4946518 495952345 2386225062010-11-25 262066688 -516224026 2679350512010-11-25 262066688 -516224124 3196089062010-11-25 262066688 203238229... (5 Replies)
Discussion started by: issaq84mohd
5 Replies

8. UNIX for Dummies Questions & Answers

Length of a fixed width file

I have a fixed width file of length 53. when is try to get the lengh of the record of that file i get 2 different answers. awk '{print length;exit}' <File_name> The above code gives me length 50. wc -L <File_name> The above code gives me length 53. Please clarify on... (2 Replies)
Discussion started by: Amrutha24
2 Replies

9. Shell Programming and Scripting

Alter Fixed Width File

Thank u so much .Its working fine as expected. ---------- Post updated at 03:41 PM ---------- Previous update was at 01:46 PM ---------- I need one more help. I have another file(fixed length) that will get negative value (ex:-00000000003000) in postion (98 - 112) then i have to... (6 Replies)
Discussion started by: vinus
6 Replies

10. Answers to Frequently Asked Questions

Fixed width file issue and resolutions

I have a fixed width file with no separators , but the file has JUNK characters in it and i know how to remove it. but in few cases these junk characters having created a mess by splitting single row into 2 or 3 rows. I need to put them back to a single line like rest of the rows. This fixed... (2 Replies)
Discussion started by: santoshkm
2 Replies
fp(7d)								      Devices								    fp(7d)

NAME
fp - Sun Fibre Channel port driver DESCRIPTION
The fp driver is a Sun fibre channel nexus driver that enables fibre channel topology discovery, device discovery, fibre channel adapter port management and other capabilities through well-defined fibre channel adapter driver interfaces. The fp driver requires the presence of a fabric name server in fabric and public loop topologies to discover fibre channel devices. In pri- vate loop topologies, the driver discovers devices by performing PLOGI to all valid AL_PAs, provided that devices do not participate in LIRP and LILP stages of loop initialization. CONFIGURATION
The fp driver is configured by defining properties in the fp.conf file. The fp driver supports the following properties: mpxio-disable Solaris I/O multipathing is enabled or disabled on fibre channel devices with the mpxio-disable property. Specifying mpxio-disable="no" activates I/O multipathing, while mpxio-disable="yes" disables the feature. Solaris I/O multipathing may be enabled or disabled on a per port basis. Per port settings override the global setting for the specified ports. The following example shows how to disable multipathing on port 0 whose parent is /pci@8,600000/SUNW,qlc@4: name="fp" parent="/pci@8,600000/SUNW,qlc@4" port=0 mpxio-disable="yes"; manual_configuration_only Automatic configuration of SCSI devices in the fabric is enabled by default and thus allows all devices discovered in the SAN zone to be enumerated in the kernel's device tree automatically. The manual_configuration_only property may be configured to disable the default behavior and force the manual configuration of the devices in the SAN. Specifying manual_configuration_only=1 disables the automatic con- figuration of devices. FILES
/kernel/drv/fp 32-bit ELF kernel driver (x86) /kernel/drv/amd64/fp 64-bit ELF kernel driver (x86) /kernel/drv/sparcv9/fp 64-bit ELF kernel driver (SPARC) /kernel/drv/fp.conf fp driver configuration file. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |mpxio-disable |Unstable | +-----------------------------+-----------------------------+ |manual_configuration_only |Obsolete | +-----------------------------+-----------------------------+ |Availability |SUNWfctl | +-----------------------------+-----------------------------+ SEE ALSO
cfgadm_fp(1M), prtconf(1M), stmsboot(1M), driver.conf(4), attributes(5), fcp(7D), fctl(7D), scsi_vhci(7D) Writing Device Drivers Fibre Channel Physical and Signaling Interface (FC-PH) ANSI X3.230: 1994 Fibre Channel Generic Services (FC-GS-2) Project 1134-D Fibre Channel Arbitrated Loop (FC-AL) ANSI X3.272-1996 Fibre Channel Protocol for SCSI (FCP) ANSI X3.269-1996 SCSI-3 Architecture Model (SAM) Fibre Channel Private Loop SCSI Direct Attach (FC-PLDA) ANSI X3.270-1996 SCSI Direct Attach (FC-PLDA) ANSI X3.270-1996 SCSI Direct Attach (FC-PLDA) NCITS TR-19:1998 Fabric Loop Attachment (FC-FLA), NCITS TR-20:1998 SunOS 5.10 2 Dec 2004 fp(7d)
All times are GMT -4. The time now is 10:28 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy