Understanding an example of perl map() function


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Understanding an example of perl map() function
# 1  
Old 09-15-2017
Understanding an example of perl map() function

Hello,
I have many folders under which there is always a file with the same name, which contains the data I need to process later. A perl oneliner was borrowed
Code:
perl -e 'print "gene_id\t", join("\t", map {/(.*)\//; $1} @ARGV),"\n";' *_test.trim/level.csv

to make a header so that each column corresponding to the respective folder to distinguish the same file names for later processing. The directory structure looks like this:
Code:
1_test.trim/level.csv
15_test.trim/level.csv
17_test.trim/level.csv
30_test.trim/level.csv
34_test.trim/level.csv
8_test.trim/level.csv

The output is:
Code:
gene_id    1_test    15_test    17_test    30_test    34_test    8_test

I had hard time to understand the $1 within the map() function in the oneliner.
I think I understand what the map() and join() functions in perl, but this $1 tripped me quite hard.
(.*)\/ is the regex which is to get rid of the .trim/ part, I believe, but then comes the $1. Maybe, the whole part of map {/(.*)\//; $1} is doing something that I did not catch.
I appreciate any explanation for me.

Last edited by yifangt; 09-15-2017 at 01:31 PM..
# 2  
Old 09-15-2017
map is confusing because it's actually a kind of loop. $1 is the bracketed part of /(.*)\// in this context.

The whole thing means, "For each item in @ARGV[N], do { /(.*)\//; output[N]=$1 }"

Then the whole thing is crammed into a "join" which returns them tab-separated.

I have no idea why it removes the ".trim", the .*\/ is a regex meaning "several of any character, followed by a forward slash". It just stops at the last forward slash in the string ( not the first, because of greedy matching. )
# 3  
Old 09-15-2017
the regex and the curly bracket with map()

Thanks!
Let's skip the @ARGV and the foreach loop which seems not that confusing to me.
I am aware of perl normally omits the default $_, so that I dissect the map{} as complex statements because map{/(.*)\//, $_; $1} is with the curly bracket {}, not the round ().
Take the first folder "1_test.trim/level.csv" as example. I tried to understand it with two steps.
I am not quite clear with your output[N] = $1.
First is map(/(.*)\//, $_). I thought
$0 = 1_test.trim/level.csv # similar to $0 in awk, not accurate but to give some idea for parsing. So I changed to give the correct part
$1 = 1_test.trim/ #"Not "1_test" at the original post after I struggled with this part that level.csv is skipped/omitted!
Is this correct?



Last edited by yifangt; 01-23-2018 at 12:09 PM..
# 4  
Old 09-15-2017
Yes. Perl allows you to call things without the () if you really want to, and they did so here. How it would with them is:

Code:
map( {code block}, @ARGV)

So @ARGV is the input array, and {code block} is what it does to every element of the array in turn. It's given @ARGV[N] as its argument, and returns whatever you want to transform it into.

I'm not sure that's valid syntax for perl in general - shoving entire code blocks wherever - especially since that code block isn't executed immediately, but repeatedly called by map(). This seems like special behavior.

The steps in the { } block are:
  • Match $_ (i.e. @ARGV[N]) against /(.*)\//
  • Return $1, i.e. the bracketed section matched by the regex
  • Assign the returned value to output[N] ( implied, done internally by map() )

Once every element is parsed, it's returned as a list into join().
This User Gave Thanks to Corona688 For This Post:
# 5  
Old 09-15-2017
You know what? When I execute this, it does return the ".trim" part. So that's one mystery solved. That trimming is either done before or after perl.

Code:
$ perl -e 'print "gene_id\t", join("\t", map {/(.*)\//; $1} @ARGV),"\n";' 1_test.trim/level.csv
gene_id 1_test.trim

$

This User Gave Thanks to Corona688 For This Post:
# 6  
Old 09-15-2017
back reference of regex in perl map() function

It does print ".trim" ---That's my bad!
Thank you so much!!!!!! $1 is the back reference from the regex.
Modified version gives expected output in post #1.
Code:
perl -e 'print "gene_id\t", join("\t", map {/(.*_test).trim\//; $1} @ARGV),"\n";' *_test.trim/level.csv

Wish perldoc had explicitly addressed this.

Last edited by yifangt; 09-15-2017 at 05:13 PM.. Reason: Adding new code
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Problem with (Understanding) function

I have this code #!/bin/bash LZ () { RETVAL="\n$(date +%Y-%m-%d_%H-%M-%S) --- " return RETVAL } echo -e $LZ"Test" sleep 3 echo -e $LZ"Test" which I want to use to make logentrys on my NAS. I expect of this code that there would be output like 2017-03-07_11-00-00 --- Test (4 Replies)
Discussion started by: matrois
4 Replies

2. Shell Programming and Scripting

Perl combine multiple map statements

I have a file like file. file.TODAY.THISYEAR file.TODAY.LASTYEARI want to substitute the words in caps with their actual values so that output should look like file.140805 file.140805.2014 file.140805.2013For this I am reading the file line bye line in an array and using multiple map... (1 Reply)
Discussion started by: sam05121988
1 Replies

3. Shell Programming and Scripting

Need help understanding a function

Hello, I recently started going in depth with the shell, so I started learning from Linux Shell Scripting CookBook, 2nd edition. I am at the first chapter atm, and the author tells to define a function in the ~/.bashrc. The function is below. prepend() { && eval $1=\"$2':'\$$1\" && export... (1 Reply)
Discussion started by: Vaseer
1 Replies

4. Shell Programming and Scripting

Looping in Perl based on defined keys in Map

Hello All, I am writing the below script where it will connect to database and returns the results. #!/sw/gcm/perl510/bin/perl use SybaseC; &openConnection; &loadvalues; sub openConnection { $dbproc = new SybaseC(SYDB}, $ENV{DBDFLTUSR}, $ENV{DBDFLTPWD}); if... (2 Replies)
Discussion started by: filter
2 Replies

5. Shell Programming and Scripting

understanding thread in perl

Hi all, I am trying to build threads which will go to localhost and list the files in given folder. #!/usr/bin/perl use threads; my $t1 = threads->new(\&sub1, 1); my $t2 = threads->new(\&sub2, 2); push(@threads,$t1); push(@threads,$t2); foreach... (5 Replies)
Discussion started by: PranavEcstasy
5 Replies

6. Shell Programming and Scripting

Help understanding some Perl code.

Well, I found myself trying to fix some Perl code (Ive never done any Perl in my life) and I pinpointed the place where the bug could be. But to be sure I have to know what does a few line of code mean: $files_lim =~ (/^\d*$/) $files_lim =~ (/^\d*h$/) $files_age =~ s/h// The code where... (2 Replies)
Discussion started by: RedSpyder
2 Replies

7. Shell Programming and Scripting

How to map the values of an array in perl?

Hi, I have 2 arrays: @names=qw(amith veena chaitra); @files=qw(file.txt file1.txt file3.txt); There is one to one relationship between names and files. There needs to be mapping created between names and files. The output should be like this: amith --> file.txt veena --->... (3 Replies)
Discussion started by: vanitham
3 Replies

8. Shell Programming and Scripting

Grep and map in perl

Hi guys, I'm trying to learn grep and map and having a little problem. Let's say I have a file which contains: Apple: abcdcabdadddbac I want to replace any combinations of three of abcd, thus when I do this: print grep {s/{3}/X/g} <F>; # will do the subtitution fine, output XXXX ... (1 Reply)
Discussion started by: new bie
1 Replies

9. Shell Programming and Scripting

Help needed for understanding a function

There is a function called start: start() { echo -n $"Sending Startup Email: " echo "${RESTARTBODY}" | mutt -s "${RESTARTSUBJECT}" ${EMAIL} RETVAL=$? if ; then touch ${LOCKFILE} success else failure fi echo return ${RETVAL} } Can anyone explain what the bold part of the... (3 Replies)
Discussion started by: proactiveaditya
3 Replies

10. Shell Programming and Scripting

Perl map doubt

Hello , Please can someone tell me what exactly happens when the below filehandler is chomped into an array and later mapped. $lcpLog="logcopy\@".getTimestamp."\log"; open CFg ,"< $lcpcfg"; chomp(@cfg = <CFG>); close CFG; @cfg=grep { $_ ne ' ' } map { lc + (split /\s*\/\//) }... (0 Replies)
Discussion started by: rmv
0 Replies
Login or Register to Ask a Question