Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

Grep returning partial lines due to special characters

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Grep returning partial lines due to special characters
# 1  
Old 09-15-2011
Grep returning partial lines due to special characters
Hey guys,

I have a file with an ID which I'm using to grep out the original record from another file. Problem is I have special characters in the original file, and grep is returning only a partial record. How can I get around this?

Appreciate your help!
Pete
# 2  
Old 09-15-2011
Could you post an example of the data? cat -v filename should show it with ^M and so forth instead of nonprinting chars.

I suspect it's not grep that's truncating the results, but your terminal. A carriage return in the middle of the value could send the cursor back to the beginning of the line and overwrite previous content.
# 3  
Old 09-15-2011
Thanks for the prompt reply! I changed some of the data to protect client obligations, but the format is the same. This is the results of cat -v FILE | grep "0000000000000000001":

Code:
0000000000000000001XX 0000000000000000 000002011-09-12 00000000000000 00000 00000 000000 00 0000000.00 0000 0000 0000010.00 00000.00000000000000N 00000000000000 000                                        ^@0 0000000000000000   0000000000000000         ^@ 0000000000000000  0000000000000000  0000000000                                                                       Y^


Problem is I have 50,000 rejected records out of a 5MM record file, and it would be impractical to loop through a file 50,000 times to get the data out with the cat -v | grep method.

Original method I was attempting to use is:
Code:
cat $reject_file | grep ID_NO | awk '{print $2}' | xargs -I {} grep {} SLS_TBL_20110914_001.dat

Moderator's Comments:
Mod Comment Please use code tags!
# 4  
Old 09-15-2011
cat -v doesn't seem so impractical when you've got one useless use of cat in there already. Smilie

Just seeing the record doesn't tell me what's going wrong. I wasn't suggesting using cat -v to fix the file -- I need to see what the strange characters are.

How about:

Code:
cat -v datafile | grep "\^" | head

That should capture a few lines with strange chars in them.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Lines starting with special characters

Hi I have a file and need to extract lines starting with "grep ^" I tried with quotes single/double before/after but no luck. suggestion pls, thanks! (2 Replies)
Discussion started by: magnus29
2 Replies

2. Shell Programming and Scripting

Fetch the lines which contains special characters

Hi All, My source file contains special characters(Latin characters).I need to fetch only the lines which contains the special characters. The problem is i don't know which all latin/special characters can come in the source. Is there anyway to extract the lines which contain letters other... (3 Replies)
Discussion started by: joe!!
3 Replies

3. Shell Programming and Scripting

grep lines having special characters

Hi, I have a file which has numerous lines and some of the lines having special characters in it. i want to grep the lines which are having special characters. say, one line looks like - %*()$#@"", | acbd antoher line looks like ***##^%! | efcg so these kind of lines are present... (5 Replies)
Discussion started by: rbalaj16
5 Replies

4. Shell Programming and Scripting

Is there anyway to grep any special characters from a file ?

Is there any command or shell script to grep any special character from a file ? I have a huge file containing millions of user names; the requirement is to find names containing special characters. #!/bin/bash for i in `cat username.txt` do #COMMAND to grep special character done ... (3 Replies)
Discussion started by: poga
3 Replies

5. Shell Programming and Scripting

Grep with special Characters

Need Help For GREP I have a file say g1.txt and content of file is below REG ADD "HKCU\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer" /v NoDrives /t REG_DWORD /d 4 /f , REG ADD "HKCU\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer" /v NoClose /t REG_DWORD /d 1 /f ,... (4 Replies)
Discussion started by: jalpasoni
4 Replies

6. UNIX for Dummies Questions & Answers

locate special characters and digits using grep

Hello, i have a file called test hello1 "how" are you4 good"bye" good7bye i am trying to print all lines from test that either end with a digit or contain a double quote character anywhere on the line. i did grep -n '$' test and was able to print lines ending with digits. i also did... (2 Replies)
Discussion started by: hobiwhenuknowme
2 Replies

7. UNIX for Dummies Questions & Answers

Using GREP for special characters

Hi folks I am issuing the following command: grep "" * Looking for the characters \/:*?"<>|#+%& within all files in a directory, but the command fails being unhappy with pipe: ksh: 0403-057 Syntax error: `|' is not expected. How do I force the command to take the pipe | ? I guess... (2 Replies)
Discussion started by: daveaasmith
2 Replies

8. Shell Programming and Scripting

Grep not working - special characters??

I have a file that I am processing with a while loop from, in come cases the grep/sed command (strings record | grep “errorDetail” | sed 's&*errorDetail\(.*)\(/errorDetail\).*&\1&') works and produces the data I am after and in some it does not. I have inspected the data within the failing... (3 Replies)
Discussion started by: gugs
3 Replies

9. UNIX Desktop Questions & Answers

grep with special characters

Hi there I need to grep for a detail from a file. The pattern to search for involves escape sequences in it. This causes for the problem. grep "P\_SOME\_STRING\_SEARCH" filename Note, I have line like below in the file and expect it to grep. select * from my_system_param ... (3 Replies)
Discussion started by: guruparan18
3 Replies

10. Shell Programming and Scripting

Grep with Special Characters

I need to sort a file, the sort is not a alphabetical sort, it's based on a predefined order which is read from a file called fSortOrder. The format of the fSortOrder file is : STARTPATH" .... .... The file that needs to be sorted is called tmpUnsorted and contains data in the format : ... (6 Replies)
Discussion started by: Vashj
6 Replies
Login or Register to Ask a Question