One problem might be that your input data do have DOS line terminators (<CR> = 0x0D = ^M = \r); did you try without?
These are unix files, so there shouldn't be an issue with EOL. I am a bit mystified as to why it works from the command line but not from my script.
Quote:
Originally Posted by RudiC
BTW, your approach seems somewhat complicated. Does it do anything else or is its sole purpose to add a counter to the first instance of duplicates?
There are a number of things that need to be done. I need to identify and re-name duplicates in several files. Every name needs to be unique, so I am finding the dups and adding an indexed prefix to each instance. There could be more than one duplicate string.
Quote:
Originally Posted by MadeInGermany
Attention: sed uses RE, so any RE-special character or the / separator will cause a malfunction.
I suspect that something like this may be the issue but I am not sure why RudiC is able to run it.
Quote:
Originally Posted by MadeInGermany
Is the goal to index all the duplicates?
I think your code would work well if I only had one file to change. I need to change the name and then look up the name in several other files and propagate the change so that all files have the revised name.
This is the current script
Code:
#!/bin/bash# base file to check for duplicate names
base_file=$1
# set input field separator to newline so each line is stored in an array element
IFS=$'\n'
# use sort and uniq to output duplicate lines in to array
dup_list=( $(cat "$base_file" | sort -k2 | uniq -f1 -D) )
# count for indexed name
name_count=0
# to identify when we have a new duplicate
current_dup=''
# loop on duplicate names
for dup_name in "${dup_list[@]}"
do
# use second field for name
unset FIELD; IFS=$'\t' read -a FIELD <<< "$dup_name"
# set name value
current_name="${FIELD[1]}"
# if no current dup has been set
if [ "$current_dup" == "" ]; then
# set base to check for new duplicate
current_dup=$current_name
#create new dup name
# name count is already 0 so no need to increment
new_name='dup_'$name_count'_'$current_name
# if the current name matches the current dup, increment counter
elif [ "$current_dup" == "$current_name" ]; then
# increment counter
name_count=$((name_count+1))
# create name based on incremented counter
new_name='dup_'$name_count'_'$current_name
# if there is a new dup series
elif [ "$current_dup" != "$current_name" ]; then
# set base to new duplicate
current_dup=$current_name
# reset name counter
name_count=0
#create new dup name
new_name='dup_'$name_count'_'$current_name
fi
# test print
echo $new_name
# find first instance of dup name in base file and replace
sed "0,/$current_name/s//$new_name/" $base_file > 'revised_'$base_file
# make changes in other files
done
When I run this on the attached file test_base.txt, I get the printed output I expect,
All of the duplicates are identified and renamed with an indexed prefix. This works very fast and I have each duplicate name in scope in the do loop where I can work on other files.
At this point I am not able to make changes in other files, which is annoying. I am sure that the logic above is overly complex.
Could the problem also be how I am reading the data into the array?
The following seems quite basic but does not seem to work. Anybody know why?
$ g=1
$ echo $g
1
$ echo abc$g
abc1
$ abc$g=hello
ksh: abc1=hello: not found
$ echo $abc1
ksh: abc1: parameter not set
It works when I specify the full variable name
$ abc1=hello
$ echo $abc1
hello
... (2 Replies)
I am trying to write a simple script which will take a variable with sed to take a line out of a text and display it
#!/bin/sh
exec 3<list
while read list<&3
do
echo $list
sed -n '$list p'<list2
done
this does not work, yet when I replace the $list variable from the sed command and... (1 Reply)
Hi....
cd command is not working when dual string drive/volume name is passed to cd through variables.......
For Ex....
y=/Volumes/Backup\ vipin/
cd $y
the above command gives error.......
anyone with a genuine solution ? (16 Replies)
GNU sed version 4.1.4 on Windows XP SP3 from GnuWin32
I think that I've come across a seemingly simple text file change problem on a INI formatted file that I can't do with SED without side effects edge cases biting me. I've tried to think of various ways of doing this elegantly and quickly... (5 Replies)
I am writing a script with a sed call that needs to use a variable and still have quotations be present in the substitution.
Example:
sed -i "s/Replacable.\+$/Replaced="root@$VAR"/g"
this outputs:
where $VAR = place
Replaced=root@place
and i need
Replaced="root@place"
... (2 Replies)
...
declare vINIFILE
vINIFILE=$1
...
echo "The name of the File is $vINIFILE" >>mail_tmp
echo "" >> mail_tmp.$$
...
grep RUNJOB=0 $vINIFILE >>tmp_filter
...
So the strange is in echo-statement I get the correct output for $vINIFILE wrtitten into the file mail_tmp. But the... (2 Replies)
I am writing perl script to configure Cisco device but Variables inside Net::Telnet::Cisco Module doesn't work and passed to device without resolving.
Please advise.
here is a sample of script:
use Net::Telnet::Cisco;
$device = "10.14.199.1";
($o1, $o2, $o3, $o4) = split(/\./,$device);... (5 Replies)
$ x="/home/guru/temp/f1.txt"
$ echo $x | sed 's^.*/^^'
This will give the absolute path f1.txt. I don't understand WHY it works. How is it determining the last "/" character exactly? (7 Replies)
Hello!
I have a problem to insert variables with sed... And I can't find the solution. :confused:
I would like to display one/few line(s) between 2 values.
This line works well
sed -n '/Dec 12 10:42/,/Dec 12 10:47/p'
Thoses lines with variables doesn't work and I don't find the... (2 Replies)
Bash version 4.4.20 / Ubuntu 16.0.4
Hello,
I tried to write a script that gathers some data and passes them to an executable.
The executed application answers with an error. The echo output in the script returns correct values.
If I copy/paste the last echo command, it get's executed... (2 Replies)