Sponsored Content
Top Forums UNIX for Beginners Questions & Answers How to extract string between two specified characters and end of line? Post 303044863 by MadeInGermany on Thursday 5th of March 2020 12:29:09 PM
Old 03-05-2020
Code:
sed -n "s/$key/$value/gp" $input

only prints lines where a substitution occurred.

But it won't print any existing lines that have not be substituted.
The following bash-4 script is a universal fix:
Code:
#!/bin/bash
# bash 4+ or ksh 93+ required

input=test.txt

# aa[] is an associative (string-indexed) array
typeset -A aa
#declare -A aa # is bash-only

# loop through the key/value file, store in aa[]
while IFS="=" read key value
do
  aa[$key]=$value
done < bbb.txt

# loop through the target file, do not split on whitespace
while IFS= read line
do
  # for each $line
  # loop through the array(-indices)
  for key in "${!aa[@]}"
  do
    # get the corresponding value
    value=${aa[$key]}
    # do the substitutions $key -> $value
    line=${line//$key/$value}
  done
  # printf is more robust than echo
  printf "%s\n" "$line"
done < $input

I have put some comments that explain how it works.
Compared to the sed solution that processes the input file many times, this solution causes less I/O (but uses more memory: the whole bbb.txt file must fit into memory).
It even fixes the potential problem that the values may not contain a / character because it clashes with the / dividers in sed.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing characters from end of $string

I am writing a script to search PCL output and append more PCL data to the end accordingly. I need to remove the last 88 bytes from the string. I have searched for a few hours now and am coming up with nothing. I can't use head or tail because the PCL output is all on one line. awk crashes on... (3 Replies)
Discussion started by: craig2k
3 Replies

2. Shell Programming and Scripting

Extract digits at end of string

I have a string like xxxxxx44. What's the best way to extract the digits (one or more) in a ksh script? Thanks (6 Replies)
Discussion started by: offirc
6 Replies

3. AIX

CUT command - cutting characters from end of string

Hello, I need to delete the final few characters from a parameter leaving just the first few. However, the characters which need to remain will not always be a string of the same length. For instance, the parameter will be passed as BN_HSBC_NTRS/hub_mth_ifce.sf. I only need the bit before the... (2 Replies)
Discussion started by: JWilliams
2 Replies

4. Shell Programming and Scripting

Get the 1st 99 characters and add new line feed at the end of the line

I have a file with varying record length in it. I need to reformat this file so that each line will have a length of 100 characters (99 characters + the line feed). AU * A01 EXPENSE 6990370000 CWF SUBC TRAVEL & MISC MY * A02 RESALE 6990788000 Y... (3 Replies)
Discussion started by: udelalv
3 Replies

5. Shell Programming and Scripting

adding characters end of line where line begins with..

Hi all, using VI, can anyone tell me how to add some characters onto the end of a line where the line begins with certain charactars eg a,b,c,......., r,s,t,........, a,b,c,......., all lines in the above example starting with a,b,c, I want to add an x at the end of the line so the... (6 Replies)
Discussion started by: satnamx
6 Replies

6. UNIX for Dummies Questions & Answers

Removing characters from end of string

Hello, I have records like below that I want to remove any five characters from the end of the string before the double quotes unless it is only an asterik. 3919,5020 ,04/17/2012,0000000000006601.43,,0000000000000000.00,, 132, 251219,"*" 1668,0125 ... (2 Replies)
Discussion started by: jyoung
2 Replies

7. UNIX for Dummies Questions & Answers

How to specify beginning-of-line/end-of-line characters inside a regex range

How can I specify special meaning characters like ^ or $ inside a regex range. e.g Suppose I want to search for a string that either starts with '|' character or begins with start-of-line character. I tried the following but it does not work: sed 's/\(\)/<do something here>/g' file1 ... (3 Replies)
Discussion started by: jawsnnn
3 Replies

8. Shell Programming and Scripting

How to extract text from STRING to end of line?

Hi I have a very large data file with several hundred columns and millions of lines. The important data is in the last set of columns with variable numbers of tab delimited fields in front of it on each line. Im currently trying sed to get the data out - I want anything beetween :RES and... (4 Replies)
Discussion started by: Manchesterpaul
4 Replies

9. Red Hat

How to add a new string at the end of line by searching a string on the same line?

Hi, I have a file which is an extract of jil codes of all autosys jobs in our server. Sample jil code: ************************** permission:gx,wx date_conditions:yes days_of_week:all start_times:"05:00" condition: notrunning(appDev#box#ProductLoad)... (1 Reply)
Discussion started by: raghavendra
1 Replies

10. Shell Programming and Scripting

Grep a particular string from column eliminating characters at the end.

Hi, So basically I have this file containing query output in seperated columns. In particular column I have the below strings: news news-prio I am trying to grep the string news without listing news-prio aswell. I tried grep "$MSG_TYPE" , grep -w "$MSG_TYPE" , grep... (4 Replies)
Discussion started by: nms
4 Replies
KDB(1)								  [FIXME: manual]							    KDB(1)

NAME
kdb - Elektra key database command line administration tool SYNOPSIS
kdb get [-dlr] key/name kdb set [-t type] [-d] [-c "A comment about this key"] [-m mode] [-u uid] [-g gid] key/name "the value" kdb set [-t type] [-m mode] [-c "A comment"] key/name -- "the value" kdb set [-t type] [-b file] key/name kdb ls [-lRfvs] [key/dir | key/name] kdb ls [-lRfvx] [key/dir | key/name] > keys.xml kdb edit [-R] [key/dir | key/name] kdb rm key/name kdb mv key/src key/dest kdb ln key/src key/dest kdb export [-f] system/some/tree.root > [file.xml] kdb import < file.xml kdb import file.xml kdb monitor some/key/name Description.PP The kdb command provide ways to manipulate the Elektra keys database. The subcommands implemented are very similar to regular UNIX commands like ls, and rm, specially in their output and options. Subcommands.PP get Get the value from the specified key. Accepts options: -d, -l, -f, -s set Set the value to the specified key. Accepts options: -c, -t, -d, -m, -b ls As the ls(1) command, list key names for the specified key, or children keys, if specified a folder key. The -v argument will make it show also the values of each key. The -d (descriptive) will make it show the comment, key name and its value, as you are watching a plain text file. Accepts options: -x, -d, -l, -f, -v, -R, -s ln Creates a key that is a symbolic links to another key. mv Move, or renames a key. Currently it can't move keys across different filesystems. rm As the rm(1) command, removes the key specified. edit A very powerful subcommand that lets you edit an XML representation of the keys. The parameters it accepts is usually a parent key, so its child keys will be gathered. Can be used with the -R flag to work recursively. The editor used is the one set in the $EDITOR environment variable, or vi. After editing the keys, kdb edit will analyze them and commit only the changed keys, remove the keys removed, and add the keys added. This command is only available when /usr/lib/libelektratools.so is available. export, save Export a subtree of keys to XML. If no subtree is defined right after the export command, system and current user trees will be exported. Output is written to standard output. The output encoding will allways be UTF-8, regardeless of your system encoding. UTF-8 is the most universal charset you can get when exchanging data between multiple systems. Accepts -f. import, load Import an XML representation of keys and save it to the keys database. If no filename is passed right after the import command, standard input is used. This command is only available when /usr/lib/libelektratools.so is available. monitor, mon Monitor a key for some value change. It will block your command line until a change in the key value is detected, then return its new value. Options.PP -R Causes to work recursively. In ls, will list recursively. -x Makes ls output an XML representation of the keys, instead of an ls-compatible output. -l Causes to display long results. With ls, will generate lists similar to ls -l. With get, will show also the key name. -a Causes ls to display also inactive keys. Generate lists similar to ls -a. Inactive keys are keys which basename begins with a '.' (dot). An example of inactive key: system/sw/XFree/current/Monitor/.Monitor1 -f Causes to work with full key names. A full key name makes sense only on user/* keys, and differentiate from the regular key names in specifying the owner user. If the current user is someuser, the user/some/key full name is user:someuser/some/key. Makes effect in ls, export and get subcommands. -d Causes get to work descriptivelly. When requesting a key it will show the comment, key name and its value in a fancy format. Causes set to mark the key as a directory key. -s Causes get and ls to be more friendly to Shell scripts. For example, when requesting user/env/env2/PATH, the output will be PATH="the value", that is, only the basename of the key will be showed and the value will be surrounded by ' " '. -t type When setting a key's value, you can specify the type with this switch. Currently accepted types are string for plain text, bin for binary as-is values, dir to create folder keys and link to create symbolic links between keys. Plain text are always stored as UTF-8(7) in Elektra, regardeless of your current encoding ($LANG). If you want to force a value to be stored without the UTF-8(7) encoding (a bad idea), you can set it as binary. Binary values should be avoided, because they are black boxes for system administrators. -b filename Set the key value as the content of file filename. This option is more useful when setting binary keys. -m mode For the set command. Will set the key access permission to mode, which must be an octal number as for chmod(1). -u uid Create the key with uid user ID. It can be a user name or a uid number. -g gid Create the key with gid group ID. It can be a group name or a gid number -c comment When setting keys, you can use this argument to set a descriptive comment for it. This comment is exactly as a comment in a plain text configuration file. The comment is stored as UTF-8(7) regardeless of your current encoding ($LANG). -v With the ls subcommand, will make it show also the value stored in the key. -- With the set subcommand, everything after it will be considered the value, even text with dashes (-). Best Practices When Creating Keys.PP When using Elektra to store your application's configuration and state, please keep in mind the following rules: o You are not allowed to create keys right under system or user. o You are not allowed to create folder keys right under system or user. They are reserved for very essential OS subsystems. o The keys for your application, called say MyApp, should be created under system/sw/MyApp and/or user/sw/MyApp. ENVIRONMENT
KDB_ROOT if defined, prepends it to key names. KDB_BACKEND defines the name of another backend plugin library to use ExamplesSetting Keys.PP bash$kdb set -c "My first key" user/example/key "Some nice value" bash$kdb set user:luciana/example/key -- "Some - nice - value with dashes" bash#KDB_ROOT=user:http/sw/httpd kdb set -u nobody -g http key "Some value" bash$kdb set -b image.png -t bin user/example/binaryKey bash$kdb set -b file.txt user/example/regularKey bash#kdb set -t link system/sw/XFree/current system/sw/XFree/handmade Getting Keys.PP bash$KDB_ROOT=user/example kdb get some/key/name bash$eval `kdb get -s user/env/env1/PS1` bash$KDB_BACKEND=gconf kdb get user/sw/gnome-terminal/global/active_encodings Listing.PP bash$kdb ls -laR user:valeria bash$kdb ls -lR system/sw/xorg/current bash$KDB_ROOT=system/sw kdb ls -lR xorg bash$KDB_BACKEND=fstab kdb ls -Rv system/filesystems bash$eval `kdb ls -Rvs user/env/env2` Miscelaneous.PP bash#kdb ln system/sw/xorg/handmade system/sw/xorg/current bash#kdb mv system/sw/xorg/current system/sw/xorg/old bash#kdb rm system/inittab/rc4 bash$KDB_BACKEND=gconf kdb rm user/gconfKey XML Import and Export.PP bash#kdb export user/sw/app | sed -e 's|/app/|/app2/|g' | kdb import bash#KDB_ROOT=system/sw kdb export myapp > myappconf.xml bash#kdb import myappconf.xml bash$KDB_BACKEND=gconf kdb export user/sw SEE ALSO
elektra(7), elektra(5) AUTHOR
Avi Alkalay <avi at unix.sh> Linux Market Developer, Senior IT and Software Architect, IBM Linux Impact Team :: ibm.com/linux Author. COPYRIGHT
Copyright (C) 2004 Avi Alkalay Elektra Initiative March 2004 KDB(1)
All times are GMT -4. The time now is 07:17 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy