Sponsored Content
Top Forums Shell Programming and Scripting replace issue with large files Post 302375365 by tootles564 on Friday 27th of November 2009 12:25:36 PM
Old 11-27-2009
replace issue with large files

I have the following problem:
I have two files: S containing sentences (one in each row) and W containing files (one in each row). It might look like this:

S:
a b c apple d.
e f orange g.
h banana i j.

W:
orange
banana
apple

My task is to replace in S all words that appear in W with "X". How can I do that efficiently, if both W and S are large (100K lines)?

The output should look like this:
a b c X d.
e f X g.
h X i j.

Could you help me please?
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Large files

I am trying to understand the webserver log file for an error which has occured on my live web site. The webserver access file is very big in size so it's not possible to open this file using vi editor. I know the approximate time the error occured, so i am interested in looking for the log file... (4 Replies)
Discussion started by: sehgalniraj
4 Replies

2. UNIX for Dummies Questions & Answers

large files?

How do we check 'large files' is enabled on a Unix box -- HP-UX B11.11 (2 Replies)
Discussion started by: ranj@chn
2 Replies

3. Shell Programming and Scripting

highly specific search and replace for a large number of files

hey guys, I have a directory with about 600 files. I need to find a specific word inside a command and replace only that instance of the word in many files. For example, lets say I have a command called 'foo' in many files. One of the input arguments of the 'foo' call is 'bar'. The word 'bar'... (5 Replies)
Discussion started by: ksubrama
5 Replies

4. Shell Programming and Scripting

Divide large data files into smaller files

Hello everyone! I have 2 types of files in the following format: 1) *.fa >1234 ...some text... >2345 ...some text... >3456 ...some text... . . . . 2) *.info >1234 (7 Replies)
Discussion started by: ad23
7 Replies

5. Solaris

How to safely copy full filesystems with large files (10Gb files)

Hello everyone. Need some help copying a filesystem. The situation is this: I have an oracle DB mounted on /u01 and need to copy it to /u02. /u01 is 500 Gb and /u02 is 300 Gb. The size used on /u01 is 187 Gb. This is running on solaris 9 and both filesystems are UFS. I have tried to do it using:... (14 Replies)
Discussion started by: dragonov7
14 Replies

6. Shell Programming and Scripting

Severe performance issue while 'grep'ing on large volume of data

Background ------------- The Unix flavor can be any amongst Solaris, AIX, HP-UX and Linux. I have below 2 flat files. File-1 ------ Contains 50,000 rows with 2 fields in each row, separated by pipe. Row structure is like Object_Id|Object_Name, as following: 111|XXX 222|YYY 333|ZZZ ... (6 Replies)
Discussion started by: Souvik
6 Replies

7. UNIX for Dummies Questions & Answers

Large file data handling issue

I have a single record large file, semicolon ';' and pipe '|' separated. I am doing a vi on the file. It is throwing an error "File to long" I need to actually remove the last | symbol from this file. sed -e 's/\|*$//' filename is working fine for small files. But not working on this big... (13 Replies)
Discussion started by: Gurkamal83
13 Replies

8. Shell Programming and Scripting

Performance issue in Grepping large files

I have around 300 files(*.rdf,*.fmb,*.pll,*.ctl,*.sh,*.sql,*.prog) which are of large size. Around 8000 keywords(which will be in the file $keywordfile) needed to be searched inside those files. If a keyword is found in a file..I have to insert the filename,extension,catagoery,keyword,occurrence... (8 Replies)
Discussion started by: millan
8 Replies

9. Shell Programming and Scripting

Large search replace using sed results in memory problem.

I have one big file of size 9GB (big_file.txt). This big file has sentences and paragraphs like any usual English document. I have another file consisting of replacement strings for sed to use. The file name is replace.sed and each entry in one line looks like this: s/\<shout\>/shout/g s/\<b is... (2 Replies)
Discussion started by: shoaibjameel123
2 Replies

10. Shell Programming and Scripting

Large File masking incorrectly happening Ç delimeter issue

The OS version is Red Hat Enterprise Linux Server release 6.10 I have a script to mask some columns with **** in a data file which is delimeted with Ç , I am using awk for the masking , when I try to mask a small file the awk works fine and masks the required column , but when the file is... (6 Replies)
Discussion started by: LinuxUser8092
6 Replies
replace(1)							  MySQL database							replace(1)

NAME
replace - A utility program that is used by msql2mysql, but that has more general applicability as well. replace changes strings in place in files or on the standard input. Uses a finite state machine to match longer strings first. Can be used to swap strings. USAGE
replace [-?svIV] from to from to ... -- [files] or replace [-?svIV] from to from to ... < fromfile > tofile SYNOPSIS
replace [-?|-I] [-s] [-v] DESCRIPTION
replace -?|-I info -s silent -v verbose EXTRA INFO
Special characters in from string: ^ Match start of line. $ Match end of line.  Match space-character, start of line or end of line. For a end  the next replace starts locking at the end space-character. A  alone in a string matches only a space-character. EXAMPLE
this command swaps a and b in the given files: shell> replace a b b a -- file1 file2 ... SEE ALSO
isamchk(1), isamlog(1), mysql(1), mysqlaccess(1), mysqladmin(1), mysqld(1), mysqld_multi(1), mysqld_safe(1), mysqldump(1), mysql_fix_privi- lege_tables(1), mysqlshow(1), mysql_zap(1), perror(1), For more information please refer to the MySQL reference manual, which may already be installed locally and which is also available online at http://www.mysql.com/doc/en/ BUGS
Please refer to http://bugs.mysql.com/ to report bugs. AUTHOR
Ver 1.0, distribution 4.0.24 Michael (Monty) Widenius (monty@mysql.com), MySQL AB (http://www.mysql.com/). This software comes with no warranty. Manual page by L. (Kill-9) Pedersen (kill-9@kill-9.dk), Mercurmedia Data Model Architect / system developer (http://www.mercur- media.com) ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +--------------------+------------------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +--------------------+------------------------------------+ |Availability | SUNWmysqlr, SUNWmysqlu, SUNWmysqlt | +--------------------+------------------------------------+ |Interface Stability | External | +--------------------+------------------------------------+ NOTES
Source for mysql is available on http://opensolaris.org. MySQL 4.0 19 December 2000 replace(1)
All times are GMT -4. The time now is 07:49 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy