BASH script to read external file to perform text replacements?
Hi all,
I have a moderate size (300 lines) BASH Shell script that performs various tasks on different source reports (CSV files). One of the tasks that it performs, is to use SED to replace 'non-conforming' titles with conformant ones. For example "How to format a RAW Report" needs to become "how_to_format_a_raw_report".
Right now, I include the SED commands within the BASH script. However, new titles are added almost daily, and we don't want any Joe (no offense to all the Joes out there..) going into the BASH script to add new SED commands.
I do have a delimited file that contains just two columns; SourceTitle:::OutputTitle (with ::: as the delimiter).
What would be the best way to incorporate a command (say SED, or AWK, or any other common-use command that doesn't need to be installed separately) to read each line of the delimited text file to get its information?
This is what the delimited text file looks like (titles.text):
This is what the actual source report the BASH script works upon (source.csv):
I can't unfortunately, include the BASH script here, but suffice it to say, if I had an idea of how to include a command that would step out and read the titles.text file and make the changes to source.csv, I'd be set.
Rich
P.S. At first glance the titles look like you can simply apply commands such as tr, sed, etc. to perform tasks such as 'replace whitespace with underscore', 'change all to lowercase', etc. but it isn't the case. Each title is evaluated on an individual basis to determine what the output will be. This is how the titles.text is generated.
P.P.S. The environment is Cygwin & Solaris using BASH, and AIX 7.1 using KSH. The Cygwin/Solaris environment is a must; the AIX is gravy..
How about
I'm inclined to bet that your ~300 lines bash can be boiled down to 20 - 30 lines awk.
hi RudiC,
Thanks for the response/solution! I'm just trying to incorporate it now, and I'm wondering if you can explain how the awk command knows to replace the first field in the mapping.txt with the second field in my CSV file. I don't quite follow the logic.
It fills translation table T with correct headings ($NF from file1) indexed by non-conforming ones ($1). When done with file1 (titles.txt), it switches the field separator to comma, and works on file2. If a line in $3 contains a non-conforming header that is found in T, it is replaced by the correct one.
Hello,
I have a text file (say, declarevars.txt) that contains multiple lines that are essentially meant to be variable declarations:
set arr1 = (var1a var1b var1c)
set arr2 = (var2a var2b var2c)
.
.
.
I want to be able to read this text file within a csh (sorry) script and have that... (2 Replies)
1. The problem statement, all variables and given/known data:
Hello!
I need help with this problem bash shell scripting that basically just reads the data in a tab delimited file and does the following below
1. Read in the data file Survey.txt and assign the column values to variables of... (6 Replies)
Hello,
I am trying to make a bash script that can pull data from a file and then change one part of said data.
I want to search by username and pull the full line. That way there is a way to replace just one part of that line then return it back to the file.
My Data is stored like:
... (1 Reply)
Hi ,
I am new to this scripting , I am facing an issue like how to read different values from external file by using different variables,
In script I supposed to declare
var 1
var 2
var 3
I know how to call this variables from external file (I am using awk command by giving same... (3 Replies)
HI all,
I want to script where all the server names will be in a text file like
server1
server2
server3 . and the script should take servernames from a text file and perform copy of files if the files are not present on those servers.after which it should take next servername till the end of... (0 Replies)
I'm writing a bash script that reads a file location from a user, and I'm wondering how to get the script to accept tab to auto complete the directories that are input. (8 Replies)
Hi all,
I'm trying to import some functions I have saved in a file named
functions.sh into a different script (protocol.sh). Can anybody show
me how to do this? I tried source functions.sh as I would do at the
terminal but source command cannot be found from within the script
protocol.sh.
... (2 Replies)
I am having one text file and i need to read that data from my shell script.
I will expain you the scenario:
Script look like:
For name type 1:
For age type 2:
For Salary type3:
echo "Enter the input:"
read the data
if input is 1 then go to the Text file and print the... (2 Replies)
Hi Guys,
I am new to unix scripting and I am tasked to parse through a CSV file delimited by #.
Sample:
sample.csv
H#A#B#C
D#A#B#C
T#A#B#C
H = Header
D = Detail Record
T = Tail
What I need is to read the file and parse through it to get the columns.
I have no idea on how... (8 Replies)
Hello,
I have got a script that reads a text file, and have got three problems that I an struggling with.
1. The script does not read the last line in the text file
2. within the second 'elif' within the script I included a 'break' - the script runs successfully (except for the first... (2 Replies)