Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Extracting 482/300k columns no's with respective info. listed in file2 from file1
Top Forums UNIX for Dummies Questions & Answers Extracting 482/300k columns no's with respective info. listed in file2 from file1 Post 302375463 by sogi on Friday 27th of November 2009 10:09:01 PM
Old 11-27-2009
jsmithstl;

Code:
#!/usr/bin/perl

use strict;

my @a_column;
my $outline;
my $line;
my $key;

open COLFILE, "<column.lst"
  or die "can't open file: $!";

while(<COLFILE>)
{
   chomp($_);
   push (@a_column, "$_");
}

close COLFILE
  or die "can't close file: $!";

print "@a_column\n";

open DATFILE, "<column.dat"
  or die "can't open file: $!";

while($line = <DATFILE>)
{
   undef $outline;
   chomp($line);
   foreach $key (@a_column)
   {
      if ( defined $outline )
      {
         $outline = $outline . " ";
      }
      $outline = $outline . (split / /, $line) [$key-1];
   }
   print "$outline\n";
}

close DATFILE

This code does not work. I tried it and I ran out of memory. My large file is 1.6 GB to start with.

---------- Post updated at 08:00 PM ---------- Previous update was at 07:55 PM ----------

radoulov;

the two perl codes don't return anything. The file2 is not modified at all. It seems the programs quit right away.

I tried also this code:

Code:
awk 'END { print r, "}\47 file1" }
  { r = r ? r ", $" $1 : "awk \47{ print $" $1 }
  ' file2 | sh

This gives me a weird output. All the column numbers listed in file 2 repeated thousands of times horizontally. The numerical information listed under the desired columns of file1 is lost. It fetches only the column numbers with no information.

---------- Post updated at 08:09 PM ---------- Previous update was at 08:00 PM ----------

File 1 is tab separated

Code:
1 2 3 4 5 6 .......etc until column 300K
1 23 21 24 12 22
1 23 21 24 12 22
1 23 21 24 12 22
1 23 21 24 12 22
1 23 21 24 12 22
1 23 21 24 12 22
1 23 21 24 12 22
.
.
etc until row 1411

The desired output, for example, would be (in this message tabs don't work)
Code:
123 955 .........etc all column numbers from file2 (482 columns)
12   22
21   23
24   12
33   11
21   41
34   32
12   22
12   11
12   31
12   22
12   22
.
.
.
.
.
etc until row 1411


Note to administrators: Thank you for formatting my original message. I'm also clicking on the # icon to this reply posts but it does not work. What is the issue here? Do other users have the same problem?
Last edited by Franklin52; 11-29-2009 at 10:00 AM.. Reason: Correcting code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

extracting lines from a file1 which maches a pattern in file2

Hi guys, Can you help me in solving ths problem? I have two files file1 and file2 as following: ===FILE1==== >LOC21 MASSKFCTVLSLALFLVLLTHANSAELFSFNFQTFNAANLILQGNASVSSSGQLRLTEVKSNGEPKVASL VASFATAFTFNILAPILSNSADGLAFALVPVGSQPKFNGGFLGLFQNVTYDP >LOC05... (11 Replies)
Discussion started by: smriti_shridhar
11 Replies

2. Shell Programming and Scripting

awk/sed search lines in file1 matching columns in file2

Hi All, as you can see I'm pretty new to this board. :D I'm struggling around with small script to search a few fields in another file. Basically I have file1 looking like this: 15:38:28 sz:10001 pr:14.16 15:38:28 sz:10002 pr:18.41 15:38:29 sz:10003 pr:19.28 15:38:30 sz:10004... (1 Reply)
Discussion started by: floripoint
1 Replies

3. Shell Programming and Scripting

grep -f file1 file2

Wat does this command do? fileA is a subset of fileB..now, i need to find the lines in fileB that are not in fileA...i.e fileA - fileB. diff fileA fileB gives the ouput but the format looks no good.... I just need the contents alone not the line num etc. (7 Replies)
Discussion started by: vijay_0209
7 Replies

4. UNIX for Dummies Questions & Answers

Replace columns from File1 with columns from File2

Hi all, I would like to replace some columns from file1 with columns from file2. Currently, I'm able to do it with the following command: awk 'NR==FNR{a=$1;b=$2;c=$3;next;} {$2=a;$4=b;$5=c;print}' file2 file1 > temp mv -f temp file1 First, i make the changes and save it as a temp... (1 Reply)
Discussion started by: seijihiko
1 Replies

5. Shell Programming and Scripting

Get values from different columns from file2 when match values of file1

Hi everyone, I have file1 and file2 comma separated both. file1 is: Header1,Header2,Header3,Header4,Header5,Header6,Header7,Header8,Header9,Header10 Code7,,,,,,,,, Code5,,,,,,,,, Code3,,,,,,,,, Code9,,,,,,,,, Code2,,,,,,,,,file2... (17 Replies)
Discussion started by: cgkmal
17 Replies

6. UNIX for Dummies Questions & Answers

if matching strings in file1 and file2, add column from file1 to file2

I have very limited coding skills but I'm wondering if someone could help me with this. There are many threads about matching strings in two files, but I have no idea how to add a column from one file to another based on a matching string. I'm looking to match column1 in file1 to the number... (3 Replies)
Discussion started by: pathunkathunk
3 Replies

7. Shell Programming and Scripting

look for line from FILE1 at FILE2

Hi guys! I'm trying to write something to find each line of file1 into file2, if line is found return YES, if not found return NO. The result can be written to a new file. Can you please help me out? FILE1 INPUT: WATER CAR SNAKE (in reality this file has about 600 lines each with a... (2 Replies)
Discussion started by: demmel
2 Replies

8. UNIX for Dummies Questions & Answers

Compare file1 and file2, print matching lines in same order as file1

I want to print only the lines in file2 that match file1, in the same order as they appear in file 1 file1 file2 desired output: I'm getting the lines to match awk 'FNR==NR {a++}; FNR!=NR && a' file1 file2 but they are in sorted order, which is not what I want: Can anyone... (4 Replies)
Discussion started by: pathunkathunk
4 Replies

9. Shell Programming and Scripting

awk to search field2 in file2 using range of fields file1 and using match to another field in file1

I am trying to use awk to find all the $2 values in file2 which is ~30MB and tab-delimited, that are between $2 and $3 in file1 which is ~2GB and tab-delimited. I have just found out that I need to use $1 and $2 and $3 from file1 and $1 and $2of file2 must match $1 of file1 and be in the range... (6 Replies)
Discussion started by: cmccabe
6 Replies

10. Shell Programming and Scripting

Mapping the values of ids of two columns of file1 from file2

I have of two space separated files: ==> File1 <== PT|np_496075.1 st|K92748.1 st|K89648.1 PT|np_001300561.1 PT|np_497284.1 st|K90752.1 st|K90279.1 PT|np_740775.1 PT|np_497749.1 st|K90752.1 st|K92038.1 PT|np_490856.1 PT|np_497284.1 st|K90752.1 st|K88095.1 PT|np_494764.1 ==> File 2 <==... (2 Replies)
Discussion started by: sammy777888
2 Replies

LEARN ABOUT OPENSOLARIS

getusershell

getusershell(3C)					   Standard C Library Functions 					  getusershell(3C)

NAME

       getusershell, setusershell, endusershell - get legal user shells

SYNOPSIS

       #include <unistd.h>

       char *getusershell(void);

       void setusershell(void);

       void endusershell(void);

DESCRIPTION

       The  getusershell()  function  returns  a  pointer  to  a  legal  user  shell  as defined by the system manager in the file /etc/shells. If
       /etc/shells does not exist, the following locations of the standard system shells are used in its place:

	 /bin/bash		    /bin/csh
	 /bin/jsh		    /bin/ksh
	 /bin/ksh93		    /bin/pfcsh
	 /bin/pfksh		    /bin/pfsh
	 /bin/sh		    /bin/tcsh
	 /bin/zsh		    /sbin/jsh
	 /sbin/pfsh		    /sbin/sh
	 /usr/bin/bash		    /usr/bin/csh
	 /usr/bin/jsh		    /usr/bin/ksh
	 /usr/bin/ksh93 	    /usr/bin/pfcsh
	 /usr/bin/pfksh 	    /usr/bin/pfsh
	 /usr/bin/sh		    /usr/bin/tcsh
	 /usr/bin/zsh		    /usr/sfw/bin/zsh
	 /usr/xpg4/bin/sh

       The getusershell() function opens the file /etc/shells, if it exists, and returns the next entry in the list of shells.

       The setusershell() function rewinds the file or the list.

       The endusershell() function closes the file, frees any memory used by getusershell() and setusershell(), and rewinds the file /etc/shells.

RETURN VALUES

       The getusershell() function returns a null pointer on EOF.

BUGS

       All information is contained in memory that may be freed with a call to endusershell(), so it must be copied if it is to be saved.

NOTES

       Restricted shells should not be listed in /etc/shells.

SunOS 5.11							    1 Nov 2007							  getusershell(3C)
All times are GMT -4. The time now is 03:00 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy