Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Copying number by looking a large file
Top Forums Shell Programming and Scripting Copying number by looking a large file Post 302561679 by shoaibjameel123 on Tuesday 4th of October 2011 11:53:37 PM
Old 10-05-2011
Copying number by looking a large file
Hi All,

I have a big file which looks like this:

Code:
abc 34.32
cdf 343.45
computer 1.34
ladder 2.3422

I have some 100000 .TXT files which look like this:

Code:
computer
cdf
align

I have to open each of the text files and read the words from the text files. Then I have to look into that dictionary for the occurrence of the word and copy on the number in 1.num file. This is what I mean:

Contents of 1.num
Code:
1.34
343.45
0

As you can see, I have copied the numbers matching with words in 1.txt from that big file. 0 for no-occurrence.

I have to do this for all 100000 .txt files.

I have tried some lookup techniques but not working though.

Code:
for txt in *.txt; do                                                         
  awk '
     NR == FNR { a[$1]=$1 }
     NR != FNR { print $1, a[$1] }
  ' big_file.dat $txt >${txt%txt}num
done

I am doing this in Linux with BASH
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

copying a large filesystem

Hi there In my organisation we have a solaris network with /home being automounted from /export/home on a central file server (usual stuff) however, the guy who originally set this up only allocated 3gb to /export/home and now we are really struggling for space. I have a new 18gb disk installed... (3 Replies)
Discussion started by: hcclnoodles
3 Replies

2. Filesystems, Disks and Memory

Strange difference in file size when copying LARGE file..

Hi, Im trying to take a database backup. one of the files is 26 GB. I am using cp -pr to create a backup copy of the database. after the copying is complete, if i do du -hrs on the folders i saw a difference of 2GB. The weird fact is that the BACKUP folder was 2 GB more than the original one! ... (1 Reply)
Discussion started by: 0ktalmagik
1 Replies

3. Shell Programming and Scripting

script to splite large file to number of small files

Dear All, Could you please help me to split a file contain around 240,000,000 line to 4 files all equally likely , note that we need to maintain that the end of each file should started by start flage (MSISDN) and ended by end flag (End), also the number of the line between the... (10 Replies)
Discussion started by: ahmed.gad
10 Replies

4. UNIX for Dummies Questions & Answers

Copying large file problem on SVR4 Unix

We have 3 Unix servers all running SVR4 Unix 1.4. I have no problems copying files to and from 2 of the servers using either the rcp command or ftp but when i come to transfer large files to the third server the copy gives up part way through and crashes this server. Copying smaller files using RCP... (7 Replies)
Discussion started by: coatesd
7 Replies

5. Shell Programming and Scripting

Copying of large files fail

Hi, I have a process which duplicates files for different environments. As the files arrive, my script (korn shell) makes copies of them (giving a unique name) and then renames the original file so that my process won't get triggered again. I don't like it either, but it's what we were told to... (4 Replies)
Discussion started by: GoldenEye4ever
4 Replies

6. UNIX for Dummies Questions & Answers

Copying a Large File

I have a large file that I append entries to the end of every few seconds. Its grown to >150MB. Its basically a log file but a perl script is writing to it. I need to make a copy of it to a new directory. I realize the latest entries occuring while the copy is taking place will not be recorded... (1 Reply)
Discussion started by: lforum
1 Replies

7. Shell Programming and Scripting

Find line number of bad data in large file

Hi Forum. I was trying to search the following scenario on the forum but was not able to. Let's say that I have a very large file that has some bad data in it (for ex: 0.0015 in the 12th column) and I would like to find the line number and remove that particular line. What's the easiest... (3 Replies)
Discussion started by: pchang
3 Replies

8. Shell Programming and Scripting

Start copying large file while its still being restored from tape

Hello, I need to copy a 700GB tape-image file over a network. I want to start the copy process before the tape-image has finished being restored from the tape. The tape restore speed is about 78 Mbps and the file transfer speed over the network is about 45 Mbps I don't want to use a pipe, since... (7 Replies)
Discussion started by: swamik
7 Replies

9. UNIX for Dummies Questions & Answers

Delete large number of columns rom file

Hi, I have a data file that contains 61 columns. I want to delete all the columns except columns, 3,6 and 8. The columns are tab de-limited. How would I achieve this on the terminal? Thanks (2 Replies)
Discussion started by: lost.identity
2 Replies

10. UNIX for Advanced & Expert Users

Count number of lines between a pattern in a large file

1000CUS E Y4NYRETAIL 10010004HELIOPOLIS 110000500022360591000056XX EG 1101DEBY XXAD ZSSKY TSSROS 1102HANYNNYY@HOTMAIL.COM 210030/05/201301/06/2013AED 3100 OPE 3100 CLO 3100 The 1000CUS E Y NYCORPORATE 10010004HELIOPOLIS 110000500025270504550203XX EG 1101XXXQ FOR CTING AND... (1 Reply)
Discussion started by: john2022
1 Replies

LEARN ABOUT CENTOS

locale::currency

Locale::Currency(3)					User Contributed Perl Documentation				       Locale::Currency(3)

NAME

       Locale::Currency - standard codes for currency identification

SYNOPSIS

	   use Locale::Currency;

	   $curr = code2currency('usd');     # $curr gets 'US Dollar'
	   $code = currency2code('Euro');    # $code gets 'eur'

	   @codes   = all_currency_codes();
	   @names   = all_currency_names();

DESCRIPTION

       The "Locale::Currency" module provides access to standard codes used for identifying currencies and funds, such as those defined in ISO
       4217.

       Most of the routines take an optional additional argument which specifies the code set to use. If not specified, the default ISO 4217
       three-letter codes will be used.

SUPPORTED CODE SETS

       There are several different code sets you can use for identifying currencies. A code set may be specified using either a name, or a
       constant that is automatically exported by this module.

       For example, the two are equivalent:

	  $curr = code2currency('usd','alpha');
	  $curr = code2currency('usd',LOCALE_CURR_ALPHA);

       The codesets currently supported are:

       alpha, LOCALE_CURR_ALPHA
	   This is a set of three-letter (uppercase) codes from ISO 4217 such as EUR for Euro.

	   Two of the codes specified by the standard (XTS which is reserved for testing purposes and XXX which is for transactions where no
	   currency is involved) are omitted.

	   This is the default code set.

       num, LOCALE_CURR_NUMERIC
	   This is the set of three-digit numeric codes from ISO 4217.

ROUTINES

       code2currency ( CODE [,CODESET] )
       currency2code ( NAME [,CODESET] )
       currency_code2code ( CODE ,CODESET ,CODESET2 )
       all_currency_codes ( [CODESET] )
       all_currency_names ( [CODESET] )
       Locale::Currency::rename_currency  ( CODE ,NEW_NAME [,CODESET] )
       Locale::Currency::add_currency  ( CODE ,NAME [,CODESET] )
       Locale::Currency::delete_currency  ( CODE [,CODESET] )
       Locale::Currency::add_currency_alias  ( NAME ,NEW_NAME )
       Locale::Currency::delete_currency_alias	( NAME )
       Locale::Currency::rename_currency_code  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Currency::add_currency_code_alias  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Currency::delete_currency_code_alias  ( CODE [,CODESET] )
	   These routines are all documented in the Locale::Codes::API man page.

SEE ALSO

       Locale::Codes
	   The Locale-Codes distribution.

       Locale::Codes::API
	   The list of functions supported by this module.

       http://www.iso.org/iso/support/currency_codes_list-1.htm
	   The ISO 4217 data.

AUTHOR

       See Locale::Codes for full author history.

       Currently maintained by Sullivan Beck (sbeck@cpan.org).

COPYRIGHT

	  Copyright (c) 1997-2001 Canon Research Centre Europe (CRE).
	  Copyright (c) 2001	  Michael Hennecke
	  Copyright (c) 2001-2010 Neil Bowers
	  Copyright (c) 2010-2013 Sullivan Beck

       This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

perl v5.16.3							    2013-06-03						       Locale::Currency(3)
All times are GMT -4. The time now is 04:03 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy