Shell Programming and Scripting

Linux 6.X environments (RHEL, Oracle Linux )

I could write basic shell scripts in bash.
In my spare time, I was planning to learn awk or sed to deal with regular expression tasks I have to deal with. But, I gather that python is gaining popularity these days and I came to know that python has a decent regular expression functionality.

I haven't worked with awk , sed or python before and I want to learn one of these to deal with regular expression tasks. So, I would like hear from you guys on which is powerful and simple .

Most, if not all, coding languages have at least some sort of 'REGEX' = Regular Expressions'.

And the basic REGEX is usualy the same, at least on POSIX systems.
Some tools have/provide their own Extended REGEX.

Once you got the basic REGEX understood, there is not much of a difference with AWK or SED, as they are both CLU's.
Python however, is a programming language, and therefor much more powerfull (in a general apporach/usage carease) than the other two.

Know thy tools.
Awk is almost a programming language, stream editor.
Python is a programming language.
Sed is a tool to add/replace strings, stream editor

hth

To a degree I agree with sea.

It all depends on whether you are an amateur or (semi-)professional at this coding lark.

Awk is a programming language with the ability to stream...

I started coding in Python from version 1.4.0 for tha AMIGA A1200 to current versions on various platforms.

As it has evolved it has become a serious contender for THE definitive scripting language with a plethora of libraries for just about everything you need to do in the computing world.

However I switched to UNIX scripting about 2.5 years ago and all but abandoned other languages, including Python, as shell scripting, IMO, is just SOOOOO flexible.

Yes it is cumbersome at times and is slow at doing some tasks but if you can shell script then expand on it and try some serious app of your own to find its limits. I keep finding what I consider a limit at something and then the big guns on here often show me the way in/out...

Regular expressions are part of the shell scripting make-up and with the extra tools, e.g. grep, sed and the like you have just about everything you need.

I know that some big guns on here will probably disagree with me on some points but I do look at it from an amateur POV, not professional...

Hope this helps...

Python is a language with a severe case of library-itis. Everything is libraries to such an absurd extreme that Python code is seldom portable unless you habitually use the exact same libraries and version as someone else. Otherwise you're in for a few rounds of installation whack-a-mole.

It also has an eyebrow-raising amount of overhead, witness the number of opened files it takes for various languages to sit there and do absolutely nothing:

Code:

$ strace python /dev/null 2>&1 | grep "^open" | egrep -v 'ENOENT|[.]so' | wc -l
42

$ strace php /dev/null 2>&1 | grep "^open" | egrep -v 'ENOENT|[.]so' | wc -l
6

$ strace bash /dev/null 2>&1 | grep "^open" | egrep -v 'ENOENT|[.]so' | wc -l
5

$ strace perl /dev/null 2>&1 | grep "^open" | egrep -v 'ENOENT|[.]so' | wc -l
3

$ strace awk '{}' /dev/null 2>&1 | grep "^open" | egrep -v 'ENOENT|[.]so' | wc -l
3

$

I would suggest starting with awk for a few reasons. For one thing it's a small, relatively simple language, it's native to UNIX everywhere (unlike perl and python), and for the most part quite portable (use nawk on solaris). sed is useful sometimes but not actually a language (some argue otherwise, but it's pretty far down in the turing tarpit).

Hi.

As with many aspects of life, it depends.

I agree with Corona wrt Python. I bought the 5th edition of Learning Python earlier this year -- 1500 pages !

However, for pros, it is growing in use -- it's hard to beat a number-4-slot: Interactive: The Top Programming Languages 2015 - IEEE Spectrum

When I was designing and presenting training courses, I included awk in the Intermediate unix-like classes (Solaris, Linux, etc.). It is easy to use, fast, and you will get a few of the ideas of C from the advanced syntax. In Solaris 11, Oracle has provided gawk (3.1.8) in their repository (plus a number of other GNU utilities).

I tend to dislike Java for the same reason that Corona mentioned -- our Java instructor often commented that it was the only language that one needed a library manual open while coding. However, Java was designed with portability as a primary concern -- write once, run anywhere.

For writing unix-like utilities without resorting to C/C++, I usually use perl. The Intro to perl courses, were usually 3-4 8-hour days with a LOT of hands-on, as was the Intermediate Unix.

If you are going to be working with big data, extraordinarily long lines, or have compute-intensive operations, you may need other tools, but for most people, awk is the right tool for most things. When a task calls for anything to do with fields, I usually reach for awk, if there is not a utility that will do the job.

Good luck ... cheers, drl

Thanks Sea, Wisecracker, Corona, drl