Use of Variables in a sed/awk script Post: 302874975

Sponsored Content

Top Forums UNIX for Dummies Questions & Answers Use of Variables in a sed/awk script Post 302874975 by Marc_Camoc on Monday 18th of November 2013 07:24:04 AM

11-18-2013

Registered User

Use of Variables in a sed/awk script

Hi,

After looking at the differents post on this forum, I am convinced that I will benefit from the experience of advanced Unix user on some script I have already done for an aeronautical study. Here is one of them :

Step 1 :

Code:

sed -e "s/??/00/g" Base_Awk.txt > Awk_Cut_00.txt4;
sed -e "s/??/01/g" Base_Awk.txt > Awk_Cut_01.txt4;
sed -e "s/??/02/g" Base_Awk.txt > Awk_Cut_02.txt4;
cat *.txt4 > Awk_Cut_All.sh;rm *txt4;

with Base_Awk.txt :

Code:

awk 'NR==1{system("cat Cut_1??.txt")}1' Fichier_Base.txt >   Fichier_Base_1.txt;
awk 'NR==5001{system("cat Cut_2??.txt")}1' Fichier_Base_1.txt > Fichier_Base_2.txt;
...
awk 'NR==30001{system("cat Cut_7??.txt")}1' Fichier_Base_6.txt > Statique_6oc_Cut_??.txt;

Step 2

Code:

./Awk_Cut_All.sh

To summarize, I have 7 x 70 files Cut_1??.txt,Cut_2??.txt,...,Cut_7??.txt and the output is 70 files ordered with the awk command (line 1 file Cut_101.txt line 5001 Cut_2??.txt and so on to Cut_7??.txt line 30001).

I have obtained this results with the code above, I was wondering if i could bypass the script with the sed replacement and using variables in the awk command, something like that (I know it won(t work just for the idea) :

Code:

i=[1-70];
for i in [1-70];
do 
awk 'NR==1{system("cat Cut_1$I.txt")}1' Fichier_Base.txt > Fichier_Base_1$i.txt
...
awk 'NR==1{system("cat Cut_7$I.txt")}1' Fichier_Base.txt > Fichier_Base_7$i.txt
cat Fichier_Base_?$i.txt > output_$i.txt5;

I have seen on this forum the use of variable in Awk but have been unable to use it for this script.

If anyone can lend me a hand on this one, I would be much grateful.

Marc

Marc_Camoc

View Public Profile for Marc_Camoc

Find all posts by Marc_Camoc

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

passing variables to sed inside script

I am trying to pass a regular expression variable from a simple script to sed to remove entries from a text file e.g. a='aaaa bbbb cccc ...|...:' then executing sed from the script sed s'/"'$a"'//g <$FILE > $FILE"_"1 my output file is always the same as the input file !! any...

2. Shell Programming and Scripting

Manipulating awk $variables using sed?

I have been searching around the forums here trying to find a solution to my problem but not getting anywhere but closer to baldness. I have a 20 column pipe "|" seperated text file. The 14th variable doesnt always exist, but will have the format of YYYYMM or YYYY if it does. I need to take...

3. Shell Programming and Scripting

passing variables to sed function in a script ....

Hello , I have a script named testscript.sh wherein I have two variables $var and $final (both of which contain a number) I have a sed write function inside this script as follows: sed '1,2 w somefile.txt' fromfile.txt Now , in the above i want to pass $var and $final instead of...

4. Shell Programming and Scripting

using sed on bash variables (or maybe awk?)

Hi all- I've been fooling with this for a few days, but I'm rather new at this... I have a bash variable containing a long string of various characters, for instance: JUNK=this that the other xyz 1234 56 789 I don't know what "xyz" actually is, but I know that: START=he other and ...

5. Shell Programming and Scripting

Accessing Shell Variables in awk or sed

Hello, I wonder if it is possible to pass and use variables from shell environment into sed or awk. I am trying to achieve something similar to the following using sed or awk: var=some_regular_expression grep "$var" filename # Will extract lines from filename The following code,...

6. Shell Programming and Scripting

Using variables within awk/sed commands

Can I use my own variables within awk and sed for example: I've written a while loop with a counter $i and I want to use the value of $i within sed and awk to edit certain lines of text within a data file. I want to use : sed '1s/$/texthere/g' data.csv Like this: sed '$is/$/$age/g' data.csv...

7. Shell Programming and Scripting

sed command using variables in shell script

hi guys, The following command doesn't seem to work in my shell script: tag=$(sed -n '/${line}/ s/.*\.*/\1/p' myfile.txt) When i replace the ${line} with an actual value, it works fine. So, how do i use the ${line} in this sed command? Thanks in advance, Zaff

8. Shell Programming and Scripting

print pattern between two variables awk sed

I am trying to print text between two variables in a file I have tried the following things but none seem to work: awk ' /'$a'/ {flag=1;next} /'$b'/{flag=0} flag { print }' file and also sed "/$a/,/$b/p" file But none seem to work Any Ideas? Thanks in Advance

9. Shell Programming and Scripting

Variables into SED or AWK and multiple commands

Hello I am hoping you may help. I am not sure how to go about this exactly, I know the tools but not sure how to make them work together. I have two SED commands that I would like to run in a shell script. I would like to take the manual input of a user (types in when prompted) to be used...

10. Shell Programming and Scripting

Updating variables using sed or awk

Hi, I have a file(testfile.txt) that contains list of variables as shown below. T $$FirstName=James $$LastName=Fox $$Dateofbirth=1980-02-04 ……and so on there are 50 different variables. I am writing a script(script1.sh) that will update the above three variable one by one with the values...

LEARN ABOUT DEBIAN

pyp

PYP(1)							      General Commands Manual							    PYP(1)

NAME

       pyp - The Pyed Piper: A Modern Python Alternative to awk, sed and Other Unix Text Manipulation Utilities

SYNOPSIS

       pyp [options] files ...

DESCRIPTION

       pyp,  the  Pyed Piper, is a command line tool for text manipulation. It is similar to awk and sed in functionality, but its subcommands are
       Python based, and thus more familiar to many programmers.

       It can operate both on a per-line base and on the complete input stream.  Different features can be pipelined in a single command by  using
       the pipe character familiar from shell commands.

       pyp  backs  up  its  input  for reruns with modified commands, and can save commands as macros. On the downside, the rerun feature makes it
       unsuitable for continuous pipe operation.

OPTIONS

       These programs follow the usual GNU command line syntax, with long options starting with  two  dashes  (`-').   A  summary  of  options	is
       included below.	For a complete description, use --manual.

       -h, --help
	      Show this help message and exit.

       -m, --manual
	      Prints out extended help.

       -l, --macro_list
	      Lists all available macros.

       -s MACRO_SAVE_NAME, --macro_save=MACRO_SAVE_NAME
	      Saves current command as macro. use "#" for adding
	      comments	EXAMPLE:
	      pyp -s "great_macro # prints first letter" "p[1]".

       -f MACRO_FIND_NAME, --macro_find=MACRO_FIND_NAME
	      Searches for macros with keyword or user name.

       -d MACRO_DELETE_NAME, --macro_delete=MACRO_DELETE_NAME
	      Deletes specified public macro.

       -g, --macro_group
	      Specify group macros for save and delete; default is user.

       -t TEXT_FILE, --text_file=TEXT_FILE
	      Specify text file to load. For advanced users,
	      you should typically cat a file into pyp.

       -x, --execute
	      Execute all commands.

       -c, --turn_off_color
	      Prints raw, uncolored output.

       -u, --unmodified_config
	      Prints out generic PypCustom.py config file.

       -b BLANK_INPUTS, --blank_inputs=BLANK_INPUTS
	      Generate this number of blank input lines; useful for
	      generating numbered lists with variable 'n'.

       -n, --no_input
	      Use with command that generates output with no input;
	      same as --dummy_input 1.

       -k, --keep_false
	      Print blank lines for lines that test as False.
	      default is to filter out False lines from the output.

       -r, --rerun
	      Rerun based on automatically cached data from the last run.
	      Use this after executing "pyp", pasting input into the shell,
	      and hitting CTRL-D.

SEE ALSO

       awk(1), grep(1), sed(1).

AUTHOR

       pyp was written by Toby Rosen <tobyrosen@gmail.com>.

       This manual page was written by Khalid El Fathi <khalid@elfathi.fr>, for the Debian project (and may be used by others).

								  March 19, 2012							    PYP(1)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

passing variables to sed inside script

Discussion started by: Daniel234

2. Shell Programming and Scripting

Manipulating awk $variables using sed?

Discussion started by: r0sc0

3. Shell Programming and Scripting

passing variables to sed function in a script ....

Discussion started by: shweta_d

4. Shell Programming and Scripting

using sed on bash variables (or maybe awk?)

Discussion started by: rev66

5. Shell Programming and Scripting

Accessing Shell Variables in awk or sed

Discussion started by: nasersh

6. Shell Programming and Scripting

Using variables within awk/sed commands

Discussion started by: mustaine85

7. Shell Programming and Scripting

sed command using variables in shell script

Discussion started by: zaff

8. Shell Programming and Scripting

print pattern between two variables awk sed

Discussion started by: forumbaba

9. Shell Programming and Scripting

Variables into SED or AWK and multiple commands

Discussion started by: lostincashe

10. Shell Programming and Scripting

Updating variables using sed or awk

Discussion started by: Saanvi1

LEARN ABOUT DEBIAN

pyp