Sponsored Content
Full Discussion: Getting rid of ^M
Top Forums Shell Programming and Scripting Getting rid of ^M Post 302581584 by teledon on Tuesday 13th of December 2011 12:22:53 PM
Old 12-13-2011
Getting rid of ^M

I have a text file with hundreds of 32-character hash codes in it, each terminated with a linefeed (/l, or ^M).
Code:
185ead08e45a5cbb51e9f7b0b384aaa2
57643e1a17252a9fc746d49c3da04168
60cba11d09221d52aaabb5db30f408a2
2b75ee6e5c2efc31b4ee9a190d09a4df

...... etc.

I want to create a file for each hash code, with the .txt extension.

I ran a test shell script called test1.sh to see if I could pull out each line in the file and create a file name from it:
Code:
#!/bin/sh
   while read line
   do
      echo $line.txt
  done < hashlist.txt
echo "done"

What I got was
Code:
$ ./test1.sh | less

  185ead08e45a5cbb51e9f7b0b384aaa2^M.txt
57643e1a17252a9fc746d49c3da04168^M.txt
60cba11d09221d52aaabb5db30f408a2^M.txt
2b75ee6e5c2efc31b4ee9a190d09a4df^M.txt
...... etc.

How can I create these file names without the ^M?
It seems if I remove the ^M and replace it with a carriage return (/r) then the while loop won't work.

Teledon

Last edited by vbe; 12-13-2011 at 01:32 PM.. Reason: code tags
 

10 More Discussions You Might Find Interesting

1. Solaris

getting rid of ^H

Hello everybody I have a very annoying problem on my Solaris (Unix in general) servers. When I open a shell and press the backspace button, it results in a ^H character being printed on screen. I can resolve it by typing stty erase <backspace>, but does anyone know how I can prevent the... (3 Replies)
Discussion started by: soliberus
3 Replies

2. Shell Programming and Scripting

How to get rid of last line

I have to process a data file in Ab Initio. This data file is pipe delimited. BUt the file may have a Disclaimer line at the end. So before picking it for processing, I need to check if this line is there I need to remove it. ANy suggestions. Thanks Shalu (1 Reply)
Discussion started by: shalua
1 Replies

3. Shell Programming and Scripting

How to get rid of the last period (.)

Hello, I'm able to trim down the log message to "192.168.1.0.", but can't get rid off the last period. Can any one help me on the syntax to get rid of the last period? Basically, I want "192.168.1.0" instead of "192.168.1.0." Thanks, (4 Replies)
Discussion started by: dannytrinh
4 Replies

4. UNIX for Dummies Questions & Answers

how to get rid of ==>

ok the assignment question: That English paper you were writing on the works of Lewis Carroll is due in a few hours and you have forgeotten the name of the text file in which you has written a number of quotations to use in your paper. Luckily, you know that the file is somewhere in your... (1 Reply)
Discussion started by: mek86
1 Replies

5. UNIX for Dummies Questions & Answers

How to get rid of ^[[D

Hi All, Im selecting a large record from a table and putting it in a file in the unix box. The file has a hidden character "^[[D " present in it. Can any one help me in getting rid of the character Thanks in advance, (4 Replies)
Discussion started by: madhan@29
4 Replies

6. Shell Programming and Scripting

how to get rid of ' from variable?

hi, I want to get rid of ' from my variable value in ksh, and I also want to read the value on the right hand side of = into a variable thanks (10 Replies)
Discussion started by: JamesByars
10 Replies

7. Shell Programming and Scripting

Getting rid of whitespace

Hello I am working aon script, that tells me how many users or on the system when i run it. The script is #!/bin/bash w | cut -f 1 -d ' ' |sort -u | wc -l When ran it shows 16 users including myself and a line of white space. I was wondering what I need to add to remove my user... (2 Replies)
Discussion started by: mosdojaf
2 Replies

8. UNIX for Dummies Questions & Answers

how to get rid of last _ in the files name?

ex: I have list of files in a folder. abc_def_geh_.txt abc_.txt abc_def_geh_12345_.txt ab134c_d345345ef_444geh_12345_.txt i need to rename all files to get rid of the _ before .txt result should look like this: abc_def_geh.txt abc.txt abc_def_geh_12345.txt... (2 Replies)
Discussion started by: lv99
2 Replies

9. Linux

How to get rid of ^m

Hi all, I am new to unix....pls help me with this. I have a binary file generating output by passing arguments in bash.when i open the output file in VI i can see that ^m is included in between most of lines,as a result when i pass this file to my java application it dosent parse the data... (3 Replies)
Discussion started by: asheshrocky
3 Replies

10. AIX

How to get rid of GPFS ?

Hi We are doing the migration of DMX3 disks to DMX4 disks using migratepv. We are not using GPFS but we have gpfs disks present in the server. Can anyone advise how to get rid of GPFS in both the servers cbspsrdb01 and cbspsrdb02. I will do migratepv for the other disks present in the servers... (2 Replies)
Discussion started by: newtoaixos
2 Replies
HFIND(1)						      General Commands Manual							  HFIND(1)

NAME
hfind - Lookup a hash value in a hash database SYNOPSIS
hfind [-i db_type ] [-f lookup_file ] [-eq] db_file [hashes] DESCRIPTION
hfind looks up hash values in a database using a binary search algorithm. This allows one to easily create a hash database and identify if a file is known or not. It works with the NIST National Software Reference Library (NSRL) and the output of 'md5sum'. Before the database can be used by 'hfind', an index file must be created with the '-i' option. This tool is needed for efficiency. Most text-based databases do not have fixed length entries and are sometimes not sorted. The hfind tool will create an index file that is sorted and has fixed-length entries. This allows for fast lookups using a binary search algorithm instead of a linear search such as 'grep'. ARGUMENTS
-i db_type Create an index file for the database. This step must be done before a lookup can be performed. The 'db_type' argument specifies the database type (i.e. nsrl-md5 or md5sum). See section below. -f lookup_file Specify the location of a file that contains one hash value per line. These hashes will be looked up in the database. -e Extended mode. Additional information besides just the name is printed. (Does not apply for all hash database types). -q Quick mode. Instead of displaying the corresponding information with the hash, just display 0 if the hash was not found and 1 if it was. If this flag is used, then only one hash can be given at a time. -V Display version db_file The location of the hash database file. [hashes] The hashes to lookup. If they are not supplied on the command line, STDIN is used. If index files exist for both SHA-1 and MD5 hashes, then both types of hashes can be given at runtime. INDEX FILE
hfind uses an index file to perform a binary search for a hash value. This is much faster than using 'grep', which will do a linear search. Before a hash database is used, a corresponding index file must be created. This is done with the '-i' option to hfind. The resulting index file will be named based on the database file name. The name will have the original name following by the hash type (sha1 or md5) followed by '.idx'. For example, creating an MD5 hash index of the NIST NSRL results in 'NSRLFile.txt-md5.idx' and the SHA-1 index results in 'NSRLFile.txt-sha1.idx'. The file has two columns. Each entry is sorted by the first column, which is the hash value. The second column has the byte offset of the corresponding entry in the original file. So, when a hash is found in the index, the offset is recorded and then 'hfind' seeks to the entry in the original database. The following input types are valid. For NSRL, 'nsrl-md5' and 'nsrl-sha1' can be used. The difference is which hash value the index is sorted by. The 'md5sum' value can also be used to sort and index "home made" databases. 'hfind' can take data in both common formats: MD5 (test.txt) = 76b1f4de1522c20b67acc132937cf82e and 76b1f4de1522c20b67acc132937cf82e test.txt EXAMPLES
To create an MD5 index file for NIST NSRL: # hfind -i nsrl-md5 /usr/local/hash/nsrl/NSRLFile.txt To lookup a value in the NSRL: # hfind /usr/local/hash/nsrl/NSRLFile.txt 76b1f4de1522c20b67acc132937cf82e 76b1f4de1522c20b67acc132937cf82e Hash Not Found You can even do both SHA-1 and MD5 if you want: # hfind -i nsrl-sha1 /usr/local/hash/nsrl/NSRLFile.txt # hfind /usr/local/hash/nsrl/NSRLFile.txt 76b1f4de1522c20b67acc132937cf82e 80001A80B3F1B80076B297CEE8805AAA04E1B5BA 76b1f4de1522c20b67acc132937cf82e Hash Not Found 80001A80B3F1B80076B297CEE8805AAA04E1B5BA thrdcore.cpp To make a database of critical binaries of a trusted system, use 'md5sum': # md5sum /bin/* /sbin/* /usr/bin/* /usr/bin/* /usr/local/bin/* /usr/local/sbin/* > system.md5 # hfind -i md5sum system.md5 To look entries up, the following will work: # hfind system.md5 76b1f4de1522c20b67acc132937cf82e 76b1f4de1522c20b67acc132937cf82e Hash Not Found or # md5sum -q /bin/* | hfind system.md5 928682269cd3edb1acdf9a7f7e606ff2 /bin/bash <...> or # md5sum -q /bin/* > bin.md5 # hfind -f bin.md5 system.md5 928682269cd3edb1acdf9a7f7e606ff2 /bin/bash <...> SEE ALSO
sorter(1) The NIST National Software Reference Library (NSRL) can be found at www.nsrl.nist.gov. LICENSE
Distributed under the Common Public License, found in the cpl1.0.txt file in the The Sleuth Kit licenses directory. AUTHOR
Brian Carrier <carrier at sleuthkit dot org> Send documentation updates to <doc-updates at sleuthkit dot org> HFIND(1)
All times are GMT -4. The time now is 01:38 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy