Sponsored Content
Top Forums Shell Programming and Scripting Stripping unwanted characters in field Post 302949274 by dagamier on Wednesday 8th of July 2015 04:14:43 PM
Old 07-08-2015
Stripping unwanted characters in field

I wrote myself a small little shell script to clean up a file I have issues with. In particular, I am stripping down a fully qualified host/domain name to just the hostname itself. The script works, but from a performance standpoint, it's not very fast and I will be working with large data sets.

Here is a sample dataset:
Code:
Field1|hostname|field3|field4.....
Field1|hostname.f.q.d.n|field3|field4......

My code is below:
Code:
while read LINE
do
        CUST=`echo $LINE | cut -d\| -f1`
        SERVER=`echo $LINE | cut -d\| -f2 | sed 's/\..[^.]*//g'`
        REST=`echo $LINE | cut -d\| -f3-`
        echo "$CUST|$SERVER|$REST" >> tmp1
        mv tmp1 $1
done < $1

As you can see, not an elegant solution, but it creates the wanted output (strip FQDN from field 2). My awk is a bit rusty and my perl is basic. If someone has a faster, cleaner way of doing this, i'm all ears.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Diff output, unwanted characters

I've got a diff command running in a shell script that writes the ouput to a new file. In the new file there is a ">" at the beginning of each line. The output file is going to be used by another program and that character makes the file useless. What I'm getting in the new file: > 2007-09-27... (5 Replies)
Discussion started by: scanner248
5 Replies

2. Shell Programming and Scripting

stripping leftmost characters from string

Hi there, if i have some strings ie test_324423 test_242332 test_767667 but I only want the number part (the bolded bit) how do I strip the leftmost 5 characters from the output so that i will have just 324423 242332 767667 any help would be greatly appreciated Gary (5 Replies)
Discussion started by: hcclnoodles
5 Replies

3. Shell Programming and Scripting

stripping certain characters in at the middle of a string

I am trying to strip out certain characters from a string on both (left & right) sides. For example, line=see@hear|touch, i only want to echo the "hear" part. Well i have tried this approach: line=see@hear|touch templine=${line#*@} #removed "see@" echo ${templine%%\|*} #removed... (4 Replies)
Discussion started by: mcoblefias
4 Replies

4. Shell Programming and Scripting

Unwanted field separation in awk

Hi everyone, My problem is strange, I cannot think of why this is happening. I have a set of data that looks like this: Although it does not look it, the fields are tab delimited. I have made sure of this, and awk does recognize them as such. However, it divides what I would expect... (2 Replies)
Discussion started by: ccox85
2 Replies

5. Shell Programming and Scripting

Bash script - stripping away characters that can't be used in filenames

I want to create a temp file which is named based on a search string. The search string may contain spaces or characters that aren't supposed to be used in filenames so I want to strip those out. My thought was to use 'tr' with but the result is the opposite of what I want: $ echo "test... (5 Replies)
Discussion started by: mglenney
5 Replies

6. Shell Programming and Scripting

removing unwanted characters from a file

i have a file like this 1111_2222#$#$dudgfdk 11111111_343434#$#$334 1111_22222#43445667 i want to remove all those charachetrs from # how can i do this Thank in advance Saravanan (4 Replies)
Discussion started by: saravanan71184
4 Replies

7. Shell Programming and Scripting

Stripping characters from a variable

I'm using a shell script to get user input with this command: read UserInput I would then like to take the "UserInput" variable and strip out all of the following characters, regardless of where they appear in the variable or how many occurrences there are: \/":|<>+=;,?*@ I'm not sure... (5 Replies)
Discussion started by: nrogers64
5 Replies

8. IP Networking

How do delete unwanted characters from lsof?

Hi. I need to trace on Unix level number of connections to an Oracle database. The listener runs on port 1521. The following is run: oracle@server03 >lsof -Pni |grep ".1521" |grep IPv4 | awk {'print $5'}|cut -d: -f 1|sort|uniq -c|sort -nk 1 87 IPv4 oracle@server03 > I need to append... (2 Replies)
Discussion started by: grigorianvlad
2 Replies

9. Shell Programming and Scripting

Stripping characters from a file and reformatting according to another one

Dear experts, my problem is pretty tricky. I want to change a file (see attached input.txt), according to another file (help.txt). The output that is desired is in output.txt. The example is attached. Note that -dashes should not be treated specially, they are considered normal characters,... (2 Replies)
Discussion started by: TheTransporter
2 Replies

10. Shell Programming and Scripting

sed to get rid of unwanted characters

so i have strings such as this: 'postfix/local#2,5#|CRON.*12062.*root.*CMD#2,5#|roice.*NQN1#1,2#|toysprc#1,4#' i need to get rid of the "#" and the numbers between them for each of the strings above. so the desired output should be: ... (1 Reply)
Discussion started by: SkySmart
1 Replies
HOSTNAME(1)                                                  Linux Programmer's Manual                                                 HOSTNAME(1)

NAME
hostname - show or set the system's host name domainname - show or set the system's NIS/YP domain name ypdomainname - show or set the system's NIS/YP domain name nisdomainname - show or set the system's NIS/YP domain name dnsdomainname - show the system's DNS domain name SYNOPSIS
hostname [-a|--alias] [-d|--domain] [-f|--fqdn|--long] [-A|--all-fqdns] [-i|--ip-address] [-I|--all-ip-addresses] [-s|--short] [-y|--yp|--nis] hostname [-b|--boot] [-F|--file filename] [hostname] hostname [-h|--help] [-V|--version] domainname [nisdomain] [-F file] ypdomainname [nisdomain] [-F file] nisdomainname [nisdomain] [-F file] dnsdomainname DESCRIPTION
Hostname is used to display the system's DNS name, and to display or set its hostname or NIS domain name. GET NAME When called without any arguments, the program displays the current names: hostname will print the name of the system as returned by the gethostname(2) function. domainname will print the NIS domainname of the system. domainname uses the gethostname(2) function, while ypdomainname and nisdomainname use the getdomainname(2). dnsdomainname will print the domain part of the FQDN (Fully Qualified Domain Name). The complete FQDN of the system is returned with host- name --fqdn (but see the warnings in section THE FQDN below). SET NAME When called with one argument or with the --file option, the commands set the host name or the NIS/YP domain name. hostname uses the sethostname(2) function, while all of the three domainname, ypdomainname and nisdomainname use setdomainname(2). Note, that this is effec- tive only until the next reboot. Edit /etc/hostname for permanent change. Note, that only the super-user can change the names. It is not possible to set the FQDN or the DNS domain name with the dnsdomainname command (see THE FQDN below). The host name is usually set once at system startup in /etc/init.d/hostname.sh (normally by reading the contents of a file which contains the host name, e.g. /etc/hostname). THE FQDN The FQDN (Fully Qualified Domain Name) of the system is the name that the resolver(3) returns for the host name, such as, ursula.exam- ple.com. It is usually the hostname followed by the DNS domain name (the part after the first dot). You can check the FQDN using hostname --fqdn or the domain name using dnsdomainname. You cannot change the FQDN with hostname or dnsdomainname. The recommended method of setting the FQDN is to make the hostname be an alias for the fully qualified name using /etc/hosts, DNS, or NIS. For example, if the hostname was "ursula", one might have a line in /etc/hosts which reads 127.0.1.1 ursula.example.com ursula Technically: The FQDN is the name getaddrinfo(3) returns for the host name returned by gethostname(2). The DNS domain name is the part after the first dot. Therefore it depends on the configuration of the resolver (usually in /etc/host.conf) how you can change it. Usually the hosts file is parsed before DNS or NIS, so it is most common to change the FQDN in /etc/hosts. If a machine has multiple network interfaces/addresses or is used in a mobile environment, then it may either have multiple FQDNs/domain names or none at all. Therefore avoid using hostname --fqdn, hostname --domain and dnsdomainname. hostname --ip-address is subject to the same limitations so it should be avoided as well. OPTIONS
-a, --alias Display the alias name of the host (if used). This option is deprecated and should not be used anymore. -A, --all-fqdns Displays all FQDNs of the machine. This option enumerates all configured network addresses on all configured network interfaces, and translates them to DNS domain names. Addresses that cannot be translated (i.e. because they do not have an appropriate reverse IP entry) are skipped. Note that different addresses may resolve to the same name, therefore the output may contain duplicate entries. Do not make any assumptions about the order of the output. -b, --boot Always set a hostname; this allows the file specified by -F to be non-existant or empty, in which case the default hostname local- host will be used if none is yet set. -d, --domain Display the name of the DNS domain. Don't use the command domainname to get the DNS domain name because it will show the NIS domain name and not the DNS domain name. Use dnsdomainname instead. See the warnings in section THE FQDN above, and avoid using this option. -f, --fqdn, --long Display the FQDN (Fully Qualified Domain Name). A FQDN consists of a short host name and the DNS domain name. Unless you are using bind or NIS for host lookups you can change the FQDN and the DNS domain name (which is part of the FQDN) in the /etc/hosts file. See the warnings in section THE FQDN above und use hostname --all-fqdns instead wherever possible. -F, --file filename Read the host name from the specified file. Comments (lines starting with a `#') are ignored. -i, --ip-address Display the network address(es) of the host name. Note that this works only if the host name can be resolved. Avoid using this option; use hostname --all-ip-addresses instead. -I, --all-ip-addresses Display all network addresses of the host. This option enumerates all configured addresses on all network interfaces. The loopback interface and IPv6 link-local addresses are omitted. Contrary to option -i, this option does not depend on name resolution. Do not make any assumptions about the order of the output. -s, --short Display the short host name. This is the host name cut at the first dot. -V, --version Print version information on standard output and exit successfully. -y, --yp, --nis Display the NIS domain name. If a parameter is given (or --file name ) then root can also set a new NIS domain. -h, --help Print a usage message and exit. NOTES
The address families hostname tries when looking up the FQDN, aliases and network addresses of the host are determined by the configuration of your resolver. For instance, on GNU Libc systems, the resolver can be instructed to try IPv6 lookups first by using the inet6 option in /etc/resolv.conf. FILES
/etc/hostname Historically this file was supposed to only contain the hostname and not the full canonical FQDN. Nowadays most software is able to cope with a full FQDN here. This file is read at boot time by the system initialization scripts to set the hostname. /etc/hosts Usually, this is where one sets the domain name by aliasing the host name to the FQDN. AUTHORS
Peter Tobias, <tobias@et-inf.fho-emden.de> Bernd Eckenfels, <net-tools@lina.inka.de> (NIS and manpage). Michael Meskes, <meskes@debian.org> net-tools 2009-09-16 HOSTNAME(1)
All times are GMT -4. The time now is 10:56 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy