Also one last comment, when I need to extract word by word from large multi-line files the "read -d "\0" -a array < <(some-input-here) always just "works" when other read commands fail, so I should be avoiding this? I think I need to revisit the read command entirely.
Missed this question.
Without knowing what your input was and what your code was, I can't say why it didn't work. Common pitfalls with read involve backslashes and carriage returns, though.
First of all, as has been shown in this thread, an array is NOT always necessary to process lists of data. It might be worthwhile to consider other means as described above.
Quote:
Originally Posted by Riker1204
. . . when I need to extract word by word from large multi-line files the "read -d "\0" -a array < <(some-input-here) always just "works" when other read commands fail, . . .
That's not a read problem, but has to do with the fact that the data are separated by <NL> (\n, 0x0A, ^J) characters, causing read to only get one single item per line, not assigning the array but just the first element. For assigning the array's elements, the items must be in one line separated by chars from the $IFS variable.
A caveat: even though a simple ls prints file names across the screen, it separates them by \n when not outputting to screen, e. g. being redirected. A solution to your special problem - besides suppressing \n in your overcomplicated awk pipe - could be
Hey guys I added most of your suggestions and it works great.
I just now realized the beauty of that grep pipe. I can simply use it to find the highest DN count file from the get go, which is the ultimate goal anyway.
I still need to fix the case statement and substitute a 0 in case the either one of the get_ldif comparison variables are empty. I will add and test that shortly.
Code:
#! /bin/bash
systemctl stop crond
make_dir () {
if ! mkdir -p "/var/lib/ldap" "/var/lib/ldap/accesslog" "/var/tmp/bdb-log" "/etc/openldap/slapd.d"
then
printf "%b\n" "Missing directories, unable to create them: [ \e[91mFAILED\e[0m ]"
return 1
fi
printf "%b\n" "Checking directories: [ \e[92mOK\e[0m ]"
cp /usr/share/openldap-servers/DB_CONFIG.example /var/lib/ldap/DB_CONFIG
}
set_permissions () {
if chown -R ldap:ldap "/var/lib/ldap" "/var/tmp/bdb-log" "/etc/openldap/slapd.d"
then
printf "%b\n" "Setting permissions: [ \e[92mOK\e[0m ]"
return 0
else
printf "%b\n" "Unable to set permissions: [ \e[91mFAILED\e[0m ]"
return 1
fi
}
stop_ldap () {
pid_number=$(pgrep slapd)
systemctl stop slapd
sleep 3
for SIG in TERM INT HUP KILL # Increasingly severe signals to kill it with
do
pgrep slapd >/dev/null || break
kill -$SIG "$pid_number"
sleep 3
done
}
start_ldap () {
systemctl start slapd
if [ $? -eq 0 ] && pgrep slapd ; then
printf "%b\n" "slapd started: [ \e[92mOK\e[0m ]"
return 0
else
printf "%b\n" "Unable to start slapd: [ \e[91mFAILED\e[0m ]"
return 1
fi
}
get_ldif () {
TOP_LOCAL_LDIF=$(grep -Hc "dn: " /localbackup/ldap/* | sort -t: -rn -k 2,2 | head -n 1)
TOP_REMOTE_LDIF=$(grep -Hc "dn: " /backup/ldap/* | sort -t: -rn -k 2,2 | head -n 1)
LOCAL_COUNT=$(echo "${TOP_LOCAL_LDIF}" | cut -d ":" -f 2 )
REMOTE_COUNT=$(echo "${TOP_REMOTE_LDIF}" | cut -d ":" -f 2 )
if [ ${LOCAL_COUNT} -ge ${REMOTE_COUNT} ] ; then
FILE_PATH=$(echo "${TOP_LOCAL_LDIF}" | cut -d ":" -f 1)
printf "%b\n%b\n" "Found the following LDIF with DN count:" "${TOP_LOCAL_LDIF} [ \e[92mOK\e[0m ]"
else
FILE_PATH=$(echo "${TOP_REMOTE_LDIF}" | cut -d ":" -f 1)
printf "%b\n%b\n" "Found the following LDIF with DN count:" "${TOP_REMOTE_LDIF} [ \e[92mOK\e[0m ]"
fi
sleep 7s
}
case $1 in
repair) repair=true
;;
testdb) testdb=true
;;
restore) restore=true
;;
rebuild) rebuild=true
;;
*) printf "%s\n\n" "Usage ldap.sh <repair|restore|rebuild|test>"
printf "%s\n" "repair ---> try to repair existing db via db_recover"
printf "%s\n" "restore ---> try to restore from latest backup via slapadd"
printf "%s\n" "rebuild ---> rebuild everything and restore latest backup"
printf "%s\n\n" "testdb ---> test service health; return a search"
exit 1
;;
esac
if [ "$repair" = true ] ; then
stop_ldap
db_recover -v -f -h /var/lib/ldap
if [ $? -eq 0 ] ; then
printf "%b\n" "db recovery complete: [ \e[92mOK\e[0m ]"
else
printf "%b\n" "db recovery could not complete: [ \e[91mFAILED\e[0m ]"
fi
set_permissions
start_ldap
fi
if [ "$restore" = true ] ; then
stop_ldap
printf "%s\n" "backing up db directory"
tar czf /localbackup/ldap-db-backup.tar /var/lib/ldap
rm -rf /var/lib/ldap/*
rm -rf /etc/openldap/slapd.d/*
rm -rf /var/tmp/bdb-log/*
make_dir
set_permissions
slaptest -f /localbackup/slapd.conf -F /etc/openldap/slapd.d
set_permissions
start_ldap
sleep 5s
stop_ldap
get_ldif
slapadd -n 1 -l "${FILE_PATH}"
if [ $? -eq 0 ] ; then
printf "%b\n" "ldif import successful: [ \e[92mOK\e[0m ]"
else
printf "%b\n" "ldif import was not successful: [ \e[91mFAILED\e[0m ]"
fi
set_permissions
start_ldap
fi
if [ "$rebuild" = true ] ; then
stop_ldap
printf "%s" "backing up database"
tar czf /localbackup/ldap-db-backup.tar /var/lib/ldap
yum -y remove openldap-servers
yum -y install openldap-servers
rm -rf /var/lib/ldap/*
rm -rf /etc/openldap/slapd.d/*
rm -rf /var/tmp/bdb-log/*
make_dir
set_permissions
slaptest -f /localbackup/slapd.conf -F /etc/openldap/slapd.d
set_permissions
start_ldap
sleep 5s
stop_ldap
get_ldif
slapadd -n 1 -l "${FILE_PATH}"
if [ $? -eq 0 ] ; then
printf "%b\n" "ldif import successful: [ \e[92mOK\e[0m ]"
else
printf "%b\n" "ldif import was not successful: [ \e[91mFAILED\e[0m ]"
fi
set_permissions
start_ldap
fi
if [ "$testdb" = true ] ; then
for i in 'cn=Standard' ; do
if ldapsearch -h localhost -p 389 -D "cn=root,dc=removed,dc=com" -w 'removed' -b "dc=removed,dc=com" \
"(${i})" 2> /dev/null | grep -E "numEntries: 1"
then
printf "%b\n" "found ${i}: [ \e[92mOK\e[0m ]"
ldapsearch -h localhost -p 389 -D "cn=root,dc=removed,dc=com" -w 'removed' -b "dc=removed,dc=com" \
"(${i})" | grep -E "(dn: )|(cn: )|(uid: )"
elif ldapsearch -h localhost -p 389 -D "cn=root,dc=removed,dc=com" -w 'removed' -b "dc=removed,dc=com" \
"(${i})" 2>&1 | grep -E "Can't contact LDAP server"
then
printf "%b\n" "Can not find ${i}: [ \e[91mFAILED\e[0m ]"
fi
done
fi
systemctl start crond
Last edited by Riker1204; 09-30-2017 at 02:54 PM..
Now, with your nice set of functions, why don't you stay with those, i.e. write a "repair" , a "restore" etc. function and just call those in the case statement?
The problem with empty variables will arise if files aren't found or directories are empty, i.e. grep fails. In this case, check for the exit code, deploy the shell's "parameter expansion - use default value", or use a trick like adding a "0" upfront:
Code:
[ 0"${LOCAL_COUNT}" -ge 0"${REMOTE_COUNT}" ]
As this is not an "arithmetic evaluation", no problem with "leading 0 are interpreted as octal numbers" should occur. Should your shell interpret the values as octal values (mine didn't), add a base descriptor like "10#".
Hi RudiC, that's exactly the plan. And I like that read function, very concise. My only point of consternation was I've never used functions inside other functions in bash, but I'm assuming there is no issue correct?
I have to "time slice" between various projects and when I make a change to the script I have to test every logical case that it works (I'm usually good for a typo or two) because my co -workers may have to use it when I'm not there, and it if doesn't work they have no idea about LDAP. No biggie for me as I can do it manually. Point being I will have to implement it when I get a little bit more time.
Hello together,
i make something wrong... I want an array that contains information to associate it for further processing.
Here is something from my bash... You will know, what I'm trying to do.
I have to point out in advance, that the variable $SYSOS is changing and not as static as in my... (2 Replies)
Continuing my quest to learn BASH, Bourne, Awk, Grep, etc. on my own through the use of a few books. I've come to an exercise that has me absolutely stumped.
The specifics:
1. Using ONLY BASH scripting commands (not sed, awk, etc.), write a script to convert a string on the command line to... (22 Replies)
Hi all
I have a problem where i have a large list ( up to 1000 of items) and need to have 2 items pulled from it into variables in a bash script
my list is like the following and I could have it as an array or possibly an external text file maintained separately. Every line is different and... (6 Replies)
I have an associative array named table
declare -A table
table="fruit"
table="veggie"
table="GT"
table="eminem"
Now say I have a variable returning the value highway
How do I find corresponding value GT ??
(this value that I find (GT in this case) is supposed to be the name of a mysql... (1 Reply)
Hi
Input:
{ committed = 782958592; init = 805306368; max = 1051394048; used = 63456712; }
Result:
A map (maybe Associative Array) where I can iterate through the key/value. Something like this:
for key in $map
do
echo key=$key value=$map
done
Sample output from the map:
... (2 Replies)
Hi All,
I got stuck up with shell script where i use awk. The scenario which i am working on is as below.
I have a file text.txt with contents
COL1 COL2 COL3 COL4
1 A 500 400
1 B 500 400
1 A 500 200
2 A 290 300
2 B 290 280
3 C 100 100
I could able to sum col 3 and col4 based on... (3 Replies)
brothers why inode index starts from 1 unlike array inex which starts from 0
its a question from the design of unix operating system of maurice j.bach
i need to know the answer urgently...someone help please (1 Reply)
why do inode indices starts from 1 unlike array indexes which starts from 0
its a question from "the design of unix operating system" of maurice j bach
id be glad if i get to know the answer quickly
:) (0 Replies)
Hi,
When using sort on an associative array:
foreach $key (sort(keys(%opalfabet))){
$value = $opalfabet{$key};
$result .= $value;
}
How does it handle double values?
It seems to me that it removes them, is that true? If so, is there a way to get... (2 Replies)
Hi,
I am trying to make an associative array to use in a popup_menu on a website. Here is what i have:
foreach $entr ( @entries )
{
$temp_uid = $entr->get_value(uid);
$temp_naam = $entr->get_value(sn);
$s++;
}
This is the popup_menu i want to use it in.
popup_menu(-name=>'modcon',... (4 Replies)
