Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Copying number by looking a large file
Top Forums Shell Programming and Scripting Copying number by looking a large file Post 302561679 by shoaibjameel123 on Tuesday 4th of October 2011 11:53:37 PM
Old 10-05-2011
Copying number by looking a large file
Hi All,

I have a big file which looks like this:

Code:
abc 34.32
cdf 343.45
computer 1.34
ladder 2.3422

I have some 100000 .TXT files which look like this:

Code:
computer
cdf
align

I have to open each of the text files and read the words from the text files. Then I have to look into that dictionary for the occurrence of the word and copy on the number in 1.num file. This is what I mean:

Contents of 1.num
Code:
1.34
343.45
0

As you can see, I have copied the numbers matching with words in 1.txt from that big file. 0 for no-occurrence.

I have to do this for all 100000 .txt files.

I have tried some lookup techniques but not working though.

Code:
for txt in *.txt; do                                                         
  awk '
     NR == FNR { a[$1]=$1 }
     NR != FNR { print $1, a[$1] }
  ' big_file.dat $txt >${txt%txt}num
done

I am doing this in Linux with BASH
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

copying a large filesystem

Hi there In my organisation we have a solaris network with /home being automounted from /export/home on a central file server (usual stuff) however, the guy who originally set this up only allocated 3gb to /export/home and now we are really struggling for space. I have a new 18gb disk installed... (3 Replies)
Discussion started by: hcclnoodles
3 Replies

2. Filesystems, Disks and Memory

Strange difference in file size when copying LARGE file..

Hi, Im trying to take a database backup. one of the files is 26 GB. I am using cp -pr to create a backup copy of the database. after the copying is complete, if i do du -hrs on the folders i saw a difference of 2GB. The weird fact is that the BACKUP folder was 2 GB more than the original one! ... (1 Reply)
Discussion started by: 0ktalmagik
1 Replies

3. Shell Programming and Scripting

script to splite large file to number of small files

Dear All, Could you please help me to split a file contain around 240,000,000 line to 4 files all equally likely , note that we need to maintain that the end of each file should started by start flage (MSISDN) and ended by end flag (End), also the number of the line between the... (10 Replies)
Discussion started by: ahmed.gad
10 Replies

4. UNIX for Dummies Questions & Answers

Copying large file problem on SVR4 Unix

We have 3 Unix servers all running SVR4 Unix 1.4. I have no problems copying files to and from 2 of the servers using either the rcp command or ftp but when i come to transfer large files to the third server the copy gives up part way through and crashes this server. Copying smaller files using RCP... (7 Replies)
Discussion started by: coatesd
7 Replies

5. Shell Programming and Scripting

Copying of large files fail

Hi, I have a process which duplicates files for different environments. As the files arrive, my script (korn shell) makes copies of them (giving a unique name) and then renames the original file so that my process won't get triggered again. I don't like it either, but it's what we were told to... (4 Replies)
Discussion started by: GoldenEye4ever
4 Replies

6. UNIX for Dummies Questions & Answers

Copying a Large File

I have a large file that I append entries to the end of every few seconds. Its grown to >150MB. Its basically a log file but a perl script is writing to it. I need to make a copy of it to a new directory. I realize the latest entries occuring while the copy is taking place will not be recorded... (1 Reply)
Discussion started by: lforum
1 Replies

7. Shell Programming and Scripting

Find line number of bad data in large file

Hi Forum. I was trying to search the following scenario on the forum but was not able to. Let's say that I have a very large file that has some bad data in it (for ex: 0.0015 in the 12th column) and I would like to find the line number and remove that particular line. What's the easiest... (3 Replies)
Discussion started by: pchang
3 Replies

8. Shell Programming and Scripting

Start copying large file while its still being restored from tape

Hello, I need to copy a 700GB tape-image file over a network. I want to start the copy process before the tape-image has finished being restored from the tape. The tape restore speed is about 78 Mbps and the file transfer speed over the network is about 45 Mbps I don't want to use a pipe, since... (7 Replies)
Discussion started by: swamik
7 Replies

9. UNIX for Dummies Questions & Answers

Delete large number of columns rom file

Hi, I have a data file that contains 61 columns. I want to delete all the columns except columns, 3,6 and 8. The columns are tab de-limited. How would I achieve this on the terminal? Thanks (2 Replies)
Discussion started by: lost.identity
2 Replies

10. UNIX for Advanced & Expert Users

Count number of lines between a pattern in a large file

1000CUS E Y4NYRETAIL 10010004HELIOPOLIS 110000500022360591000056XX EG 1101DEBY XXAD ZSSKY TSSROS 1102HANYNNYY@HOTMAIL.COM 210030/05/201301/06/2013AED 3100 OPE 3100 CLO 3100 The 1000CUS E Y NYCORPORATE 10010004HELIOPOLIS 110000500025270504550203XX EG 1101XXXQ FOR CTING AND... (1 Reply)
Discussion started by: john2022
1 Replies

LEARN ABOUT ULTRIX

ppmtosixel

ppmtosixel(1)						      General Commands Manual						     ppmtosixel(1)

NAME

       ppmtosixel - convert a portable pixmap into DEC sixel format

SYNOPSIS

       ppmtosixel [-raw] [-margin] [ppmfile]

DESCRIPTION

       Reads  a  portable  pixmap  as input.  Produces sixel commands (SIX) as output.	The output is formatted for color printing, e.g. for a DEC
       LJ250 color inkjet printer.

       If RGB values from the PPM file do not have maxval=100, the RGB values are rescaled.  A printer control header and a color assignment table
       begin the SIX file.  Image data is written in a compressed format by default.  A printer control footer ends the image file.

OPTIONS

       -raw   If  specified,  each  pixel  will  be explicitly described in the image file.  If -raw is not specified, output will default to com-
	      pressed format in which identical adjacent pixels are replaced by "repeat pixel" commands.  A raw file is often an order	of  magni-
	      tude larger than a compressed file and prints much slower.

       -margin
	      If  -margin  is not specified, the image will be start at the left margin (of the window, paper, or whatever).  If -margin is speci-
	      fied, a 1.5 inch left margin will offset the image.

PRINTING

       Generally, sixel files must reach the printer unfiltered.  Use the lpr -x option or cat filename > /dev/tty0?.

BUGS

       Upon rescaling, truncation of the least significant bits of RGB values may result in poor color conversion.  If the original PPM maxval was
       greater	than  100,  rescaling  also reduces the image depth.  While the actual RGB values from the ppm file are more or less retained, the
       color palette of the LJ250 may not match the colors on your screen.  This seems to be a printer limitation.

SEE ALSO

       ppm(5)

AUTHOR

       Copyright (C) 1991 by Rick Vinci.

								   26 April 1991						     ppmtosixel(1)
All times are GMT -4. The time now is 05:21 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy