01-01-2004
Help with searching a text file
Hello all!
I've been working for days on this and it is really bugging me!!
Here's my dilemma:
Say I have a very large text file which contains fields delimited my a ':' which logs various records. Each record is separated by a newline character, therefore I can search for lines with 'grep' to find the results i want. The problem is, there may be as many as 30 or so fields with associated values per entry...not an issue...but, if a certain field is not used for a particular record, that field is not included in the file entry (not even a '::' for an empty field...i didn't write the software).
So the problem arrises that yes i can search for lines containing whatever i want, but sometimes I don't want to view 30 or so fields, sometimes i am only interested in one or two fields...now some of you may say use something like '# cut -d: -f22,24' or '# awk -F: {print $1, $2}' but, since not every field is used for ever record, certain data does not always appear in the same field!!!! I can guess which field until i get the correct one, but this is often waaaaayyy too time consuming. Is there a way to somehow use a string or expression to delimit a field (as opposed to a character which cut or awk only seem to do)?? Or is there a way to do this with awk?? Thank you so much...
Sorry if it was really long winded...i'm flustered!!!
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
what command can i use to search the files in a directory for a text. the output would list the files containing the text. ive tried this but it is not exactly what im looking to do: find . -name "*.xml" -exec agrep searchstring {} \; (2 Replies)
Discussion started by: jim majors
2 Replies
2. UNIX for Dummies Questions & Answers
If I go into a directory and type in ..
more * | grep foo
I get the lines of text that contain foo in all of the files in the directory out of all of the files that are there.
How do I make it so I can find out what the names of the files are that contain that text "foo"? By doing what I... (4 Replies)
Discussion started by: LordJezo
4 Replies
3. UNIX for Dummies Questions & Answers
Hi
I am trying to search a firewall syslog space delimeted file for all of the different tcp and udp destination ports.
I know that grep will find lines that contain specific text. And I have tried using the the the cut command to cut out of the file certain colums. However the test I am... (6 Replies)
Discussion started by: andyblaylock
6 Replies
4. Shell Programming and Scripting
Looking for a bit of help. I need to search for a string of words, but unfortunately these words are located on separate lines.
for example the text output is:
United
Chanmpions
Ronaldo
Liverpool
Losers
Torres
and my script code is
print("DEBUG - checking file message");
while... (15 Replies)
Discussion started by: meevagh
15 Replies
5. UNIX for Dummies Questions & Answers
Hi Gurus,
I am new to unix.I have a requirement as below
I have text file like a.txt which contains
a.txt
hi hello
process update
status
Ok to Proceed no issues good
data arrangement
My requirement here is i need to read the file and check for the words
"OK to Proceed" and if... (2 Replies)
Discussion started by: pssandeep
2 Replies
6. Shell Programming and Scripting
Hi,
I'm very new to UNIX scripting and find quite difficult to understand simple UNIX syntax. Really appreciat if somebody could help me to give simple codes for my below problems:-
1) I need to search for a string "TTOH 8031950001" in a files which filename will be "*host*'. For example, the... (3 Replies)
Discussion started by: cuji
3 Replies
7. Shell Programming and Scripting
Hi,
I need help searching through a large text file. I need to find a certain string within the text, and copy each line until another string appears.
The file looks like this:
>scf15164843
ATTAAAGGNNNGGAATTTCCCCAA
ATTACCGGCTTTAAANNNTTACCC
>scf15154847
CCGGGNNNTTTAAACCCGNGNGCC... (2 Replies)
Discussion started by: repiv
2 Replies
8. Shell Programming and Scripting
without using conventional file searching commands like find etc, is it possible to locate a file if i just know that the file that i'm searching for contains a particular text like "Hello world" or something? (5 Replies)
Discussion started by: arindamlive
5 Replies
9. Shell Programming and Scripting
Hi Gurus,
i have 8 zipped files and each file is having more than 100,000 records or more.
issue :-
i want to search the missing text from each zipped files
i have stuck here, the below command works fine if i give the value 10 for the deptno.
if i have more than 1 records... (6 Replies)
Discussion started by: SeenuGuddu
6 Replies
10. Shell Programming and Scripting
I'm trying to get one specific number out of a text file, in order to use as part of an algorithm in a shell script. It will always come after a specific string and that string won't appear anywhere else in the file.
So I'm trying to search through the file for that one string, then grab the... (2 Replies)
Discussion started by: mikedigornio
2 Replies
JOIN(1) General Commands Manual JOIN(1)
NAME
join - relational database operator
SYNOPSIS
join [ options ] file1 file2
DESCRIPTION
Join forms, on the standard output, a join of the two relations specified by the lines of file1 and file2. If file1 is `-', the standard
input is used.
File1 and file2 must be sorted in increasing ASCII collating sequence on the fields on which they are to be joined, normally the first in
each line.
There is one line in the output for each pair of lines in file1 and file2 that have identical join fields. The output line normally con-
sists of the common field, then the rest of the line from file1, then the rest of the line from file2.
Fields are normally separated by blank, tab or newline. In this case, multiple separators count as one, and leading separators are dis-
carded.
These options are recognized:
-an In addition to the normal output, produce a line for each unpairable line in file n, where n is 1 or 2.
-e s Replace empty output fields by string s.
-jn m Join on the mth field of file n. If n is missing, use the mth field in each file.
-o list
Each output line comprises the fields specifed in list, each element of which has the form n.m, where n is a file number and m is a
field number.
-tc Use character c as a separator (tab character). Every appearance of c in a line is significant.
SEE ALSO
sort(1), comm(1), awk(1)
BUGS
With default field separation, the collating sequence is that of sort -b; with -t, the sequence is that of a plain sort.
The conventions of join, sort, comm, uniq, look and awk(1) are wildly incongruous.
JOIN(1)