Sponsored Content
Top Forums Shell Programming and Scripting Search for patterns in thousands of files Post 302785695 by danish0909 on Tuesday 26th of March 2013 07:36:15 AM
Old 03-26-2013
I have run it again but I am getting the same error. There are subdirectories under it so I have given a patter for it.
Code:
/comptel4/elink/backup1/output/vas/NG0> ./ms.sh
./ms.sh: xrealloc: subst.c:5173: cannot allocate 268435456 bytes (536936448 bytes allocated)
/comptel4/elink/backup1/output/vas/NG0> more ms.sh
#!/usr/local/bin/bash

for i in `cat msisdn_u.txt`
do

find /comptel4/elink/backup1/output/vas/NG0/201303* -name "*GPX.Z" -print | xargs zgrep -il $i
done


Last edited by Franklin52; 03-26-2013 at 08:41 AM.. Reason: Please use code tags for data and code samples
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Finding a specific pattern from thousands of files ????

Hi All, I want to find a specific pattern from approximately 400000 files on solaris platform. Its very heavy for me to grep that pattern to each file individually. Can anybody suggest me some way to search for specific pattern (alpha numeric) from these forty thousand files. Please note that... (6 Replies)
Discussion started by: aarora_98
6 Replies

2. UNIX Desktop Questions & Answers

how to search files efficiently using patterns

hi friens, :) if i need to find files with extension .c++,.C++,.cpp,.Cpp,.CPp,.cPP,.CpP,.cpP,.c,.C wat is the pattern for finding them :confused: (2 Replies)
Discussion started by: arunsubbhian
2 Replies

3. UNIX for Advanced & Expert Users

Copying Thousands of Tiny or Empty Files?

There is a procedure I do here at work where I have to synchronize file systems. The source file system always has three or four directories of hundreds of thousands of tiny (1k or smaller) or empty files. Whenever my rsync command reaches these directories, I'm waiting for hours for those files... (3 Replies)
Discussion started by: deckard
3 Replies

4. UNIX for Advanced & Expert Users

Best way to search for patterns in huge text files

I have the following situation: a text file with 50000 string patterns: abc2344536 gvk6575556 klo6575556 .... and 3 text files each with more than 1 million lines: ... 000000 abc2344536 46575 0000 000000 abc2344536 46575 4444 000000 abc2344555 46575 1234 ... I... (8 Replies)
Discussion started by: andy2000
8 Replies

5. Shell Programming and Scripting

help to parallelize work on thousands of files

I need to find a smarter way to process about 60,000 files in a single directory. Every night a script runs on each file generating a output on another directory; this used to take 5 hours, but as the data grows it is taking 7 hours. The files are of different sizes, but there are 16 cores... (10 Replies)
Discussion started by: vhope07
10 Replies

6. UNIX for Dummies Questions & Answers

script to search patterns inside list of files

>testfile while read x do if then echo $x >> testfile else fi if then echo $x >> testfile else fi done < list_of_files is there any efficient way to search abc.dml and xyz.dml ? (2 Replies)
Discussion started by: dr46014
2 Replies

7. Shell Programming and Scripting

to read two files, search for patterns and store the output in third file

hello i have two files temp.txt and temp_unique.text the second file consists the unique fields from the temp.txt file the strings stored are in the following form 4,4 17,12 15,65 4,4 14,41 15,65 65,89 1254,1298i'm able to run the following script to get the total count of a... (3 Replies)
Discussion started by: vaibhavkorde
3 Replies

8. SuSE

Search all files based on first and in all listed files search the second patterns

Hello Linux Masters, I am not a linux expert therefore i need help from linux gurus. Well i have a requirement where i need to search all files based on first patterns and after seraching all files then serach second pattern in all files which i have extracted based on first pattern.... (1 Reply)
Discussion started by: Black-Linux
1 Replies

9. Shell Programming and Scripting

Bash-awk to process thousands of files

Hi to all, I have thousand of files in a folder with names with format "FILE-YYYY-MM-DD-HHMM" for what I want to send the following AWK command awk '/Code.*/' FILE-2014* I'd like to separate all files that have the same date to a folder named with the corresponding date. For example, if I... (7 Replies)
Discussion started by: Ophiuchus
7 Replies

10. Shell Programming and Scripting

Bash - Find files excluding file patterns and subfolder patterns

Hello. For a given folder, I want to select any files find $PATH1 -f \( -name "*" but omit any files like pattern name ! -iname "*.jpg" ! -iname "*.xsession*" ..... \) and also omit any subfolder like pattern name -type d \( -name "/etc/gconf/gconf.*" -o -name "*cache*" -o -name "*Cache*" -o... (2 Replies)
Discussion started by: jcdole
2 Replies
BACKUP_ADDHOST(8)					       AFS Command Reference						 BACKUP_ADDHOST(8)

NAME
backup_addhost - Adds a Tape Coordinator entry to the Backup Database SYNOPSIS
backup addhost -tapehost <tape machine name> [-portoffset <TC port offset>] [-localauth] [-cell <cell name>] [-help] backup addh -t <tape machine name> [-p <TC port offset>] [-l] [-c <cell name>] [-h] DESCRIPTION
The backup addhost command creates a Tape Coordinator entry in the Backup Database. The entry records o The host name of the Tape Coordinator machine where the Tape Coordinator (butc) process runs, as specified with the -tapehost argument. o The Tape Coordinator's port offset number, as specified with the -portoffset argument. An entry for the port offset must also appear in the /var/lib/openafs/backup/tapeconfig file on the Tape Coordinator machine, where it is mapped to a UNIX device name (for a tape device) or pathname (for a backup data file). Each Tape Coordinator must have its own port offset number, and the command fails if a Backup Database entry already exists for the requested port offset number. To display existing Tape Coordinator entries, use the backup listhosts command. OPTIONS
-tapehost <tape machine name> Specifies the fully-qualified hostname of the machine for which to create a Tape Coordinator entry in the Backup Database. The machine must have an entry in either the cell's naming service (such as the Domain Name Service) or the host file (/etc/hosts or equivalent) on the machine where the command is issued. -portoffset <TC port offset> Specifies the Tape Coordinator's port offset number. Provide an integer from the range 0 through 58510, or omit this argument to use the default value of 0 (zero). The value must match the port offset number recorded for the same combination of Tape Coordinator and tape device or file in the /var/lib/openafs/backup/tapeconfig file on the Tape Coordinator machine named by the -tapehost argument. -localauth Constructs a server ticket using a key from the local /etc/openafs/server/KeyFile file. The backup command interpreter presents it to the Backup Server, Volume Server and VL Server during mutual authentication. Do not combine this flag with the -cell argument. For more details, see backup(8). -cell <cell name> Names the cell in which to run the command. Do not combine this argument with the -localauth flag. For more details, see backup(8). -help Prints the online help for this command. All other valid options are ignored. EXAMPLES
The following command creates an entry in the Backup Database that assigns port offset number 4 to a Tape Coordinator running on the machine "backup1.abc.com": % backup addhost -tapehost backup1.abc.com -portoffset 4 The following command creates a Backup Database entry that assigns port offset number 0 to a Tape Coordinator on the machine "backup3.abc.com": % backup addhost backup3.abc.com PRIVILEGE REQUIRED
The issuer must be listed in the /etc/openafs/server/UserList file on every machine where the Backup Server is running, or must be logged onto a server machine as the local superuser "root" if the -localauth flag is included. SEE ALSO
backup(8), backup_delhost(8), backup_listhosts(8) COPYRIGHT
IBM Corporation 2000. <http://www.ibm.com/> All Rights Reserved. This documentation is covered by the IBM Public License Version 1.0. It was converted from HTML to POD by software written by Chas Williams and Russ Allbery, based on work by Alf Wachsmann and Elizabeth Cassell. OpenAFS 2012-03-26 BACKUP_ADDHOST(8)
All times are GMT -4. The time now is 06:03 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy