Home Man
Today's Posts

Linux & Unix Commands - Search Man Pages
Man Page or Keyword Search:
Select Section of Man Page:
Select Man Page Repository:

Linux 2.6 - man page for ms_transform (linux section 3erl)

ms_transform(3erl)		     Erlang Module Definition		       ms_transform(3erl)

       ms_transform - Parse_transform that translates fun syntax into match specifications.

       This  module  implements  the  parse_transform  that makes calls to ets and dbg : fun2ms/1
       translate into literal match specifications. It also implements the back end for the  same
       functions when called from the Erlang shell.

       The  translations from fun's to match_specs is accessed through the two "pseudo functions"
       ets:fun2ms/1 and dbg:fun2ms/1 .

       Actually this introduction is more or less an introduction to the whole concept	of  match
       specifications.	Since  everyone  trying  to use ets:select or dbg seems to end up reading
       this page, it seems in good place to explain a little more  than  just  what  this  module

       There  are  some caveats one should be aware of, please read through the whole manual page
       if it's the first time you're using the transformations.

       Match specifications are used more or less as filters. They resemble usual Erlang matching
       in  a  list comprehension or in a fun used in conjunction with lists:foldl etc. The syntax
       of pure match specifications is somewhat awkward though, as they are  made  up  purely  by
       Erlang  terms and there is no syntax in the language to make the match specifications more

       As the match specifications execution and structure is quite like that of a fun, it  would
       for  most  programmers  be more straight forward to simply write it using the familiar fun
       syntax and having that translated into a match specification automatically.  Of	course	a
       real fun is more powerful than the match specifications allow, but bearing the match spec-
       ifications in mind, and what they can do, it's still more convenient to write it all as	a
       fun.  This  module contains the code that simply translates the fun syntax into match_spec

       Let's start with an ets example. Using ets:select and a match specification, one can  fil-
       ter  out  rows  of a table and construct a list of tuples containing relevant parts of the
       data in these rows. Of course one could use ets:foldl instead, but the select call is  far
       more efficient. Without the translation, one has to struggle with writing match specifica-
       tions  terms  to  accommodate  this,  or  one  has  to  resort  to   the   less	 powerful
       ets:match(_object)  calls,  or  simply  give  up  and  use  the more inefficient method of
       ets:foldl . Using the ets:fun2ms transformation, a ets:select call is at least as easy  to
       write as any of the alternatives.

       As an example, consider a simple table of employees:

       -record(emp, {empno,	%Employee number as a string, the key
		     surname,	%Surname of the employee
		     givenname, %Given name of employee
		     dept,	%Department one of {dev,sales,prod,adm}
		     empyear}). %Year the employee was employed

       We create the table using:


       Let's also fill it with some randomly chosen data for the examples:


       Now,  the  amount  of  data  in the table is of course to small to justify complicated ets
       searches, but on real tables, using select to get exactly the data you want will  increase
       efficiency remarkably.

       Lets  say for example that we'd want the employee numbers of everyone in the sales depart-
       ment. One might use ets:match in such a situation:

       1> ets:match(emp_tab, {'_', '$1', '_', '_', sales, '_'}).

       Even though ets:match does not require a full match specification,  but	a  simpler  type,
       it's  still somewhat unreadable, and one has little control over the returned result, it's
       always a list of lists. OK, one might use ets:foldl or ets:foldr instead:

       ets:foldr(fun(#emp{empno = E, dept = sales},Acc) -> [E | Acc];
		    (_,Acc) -> Acc

       Running that would result in ["011103","076324"] , which at least gets rid  of  the  extra
       lists.  The  fun  is  also quite straightforward, so the only problem is that all the data
       from the table has to be transferred from the table to the calling process for  filtering.
       That's inefficient compared to the ets:match call where the filtering can be done "inside"
       the emulator and only the result is transferred to the process. Remember that  ets  tables
       are  all  about efficiency, if it wasn't for efficiency all of ets could be implemented in
       Erlang, as a process receiving requests and sending answers back. One uses ets because one
       wants  performance,  and  therefore  one wouldn't want all of the table transferred to the
       process for filtering. OK, let's look at  a  pure  ets:select  call  that  does	what  the
       ets:foldr does:

       ets:select(emp_tab,[{#emp{empno = '$1', dept = sales, _='_'},[],['$1']}]).

       Even though the record syntax is used, it's still somewhat hard to read and even harder to
       write. The first element of the tuple, #emp{empno = '$1', dept = sales, _='_'} tells  what
       to  match,  elements  not  matching  this will not be returned at all, as in the ets:match
       example. The second element, the empty list is a list of guard expressions, which we  need
       none,  and  the third element is the list of expressions constructing the return value (in
       ets this almost always is a list containing one single term). In our case '$1' is bound to
       the  employee  number  in  the head (first element of tuple), and hence it is the employee
       number that is returned. The result is ["011103","076324"] ,  just  as  in  the	ets:foldr
       example, but the result is retrieved much more efficiently in terms of execution speed and
       memory consumption.

       We have one efficient but hardly readable way of doing it and one inefficient  but  fairly
       readable  (at  least  to  the  skilled Erlang programmer) way of doing it. With the use of
       ets:fun2ms , one could have something that is as efficient as possible but still is  writ-
       ten as a filter using the fun syntax:


       % ...

       ets:select(emp_tab, ets:fun2ms(
			     fun(#emp{empno = E, dept = sales}) ->

       This  may  not be the shortest of the expressions, but it requires no special knowledge of
       match specifications to read. The fun's head should simply match what you want  to  filter
       out and the body returns what you want returned. As long as the fun can be kept within the
       limits of the match specifications, there is no need to transfer all data of the table  to
       the  process  for  filtering as in the ets:foldr example. In fact it's even easier to read
       then the ets:foldr example, as the select call in itself discards  anything  that  doesn't
       match,  while the fun of the foldr call needs to handle both the elements matching and the
       ones not matching.

       It's worth noting in the above ets:fun2ms example that  one  needs  to  include	ms_trans-
       form.hrl  in  the  source  code,  as this is what triggers the parse transformation of the
       ets:fun2ms call to a valid match specification. This also implies that the  transformation
       is  done  at compile time (except when called from the shell of course) and therefore will
       take no resources at all in runtime. So although you use the more intuitive fun syntax, it
       gets as efficient in runtime as writing match specifications by hand.

       Let's  look at some more ets examples. Let's say one wants to get all the employee numbers
       of any employee hired before the year 2000. Using ets:match isn't an alternative  here  as
       relational  operators  cannot  be  expressed  there.  Once again, an ets:foldr could do it
       (slowly, but correct):

       ets:foldr(fun(#emp{empno = E, empyear = Y},Acc) when Y < 2000 -> [E | Acc];
			 (_,Acc) -> Acc

       The result will be ["052341","076324","535216","789789","989891"] , as expected.  Now  the
       equivalent  expression  using  a handwritten match specification would look something like

       ets:select(emp_tab,[{#emp{empno = '$1', empyear = '$2', _='_'},
			    [{'<', '$2', 2000}],

       This gives the same result, the [{'<', '$2', 2000}] is in the  guard  part  and	therefore
       discards anything that does not have a empyear (bound to '$2' in the head) less than 2000,
       just as the guard in the foldl example. Lets jump on to writing it using ets:fun2ms


       % ...

       ets:select(emp_tab, ets:fun2ms(
			     fun(#emp{empno = E, empyear = Y}) when Y < 2000 ->

       Obviously readability is gained by using the parse transformation.

       I'll show some more examples without the tiresome comparing-to-alternatives  stuff.  Let's
       say  we'd  want	the whole object matching instead of only one element. We could of course
       assign a variable to every part of the record and build it up once again in  the  body  of
       the fun , but it's easier to do like this:

       ets:select(emp_tab, ets:fun2ms(
			     fun(Obj = #emp{empno = E, empyear = Y})
				when Y < 2000 ->

       Just  as  in ordinary Erlang matching, you can bind a variable to the whole matched object
       using a "match in then match", i.e. a = . Unfortunately	this  is  not  general	in  fun's
       translated  to  match  specifications,  only  on  the "top level", i.e. matching the whole
       object arriving to be matched into a separate variable, is it allowed. For the one's  used
       to  writing  match  specifications  by hand, I'll have to mention that the variable A will
       simply be translated into '$_'. It's not general, but it has very common usage, why it  is
       handled	as  a  special, but useful, case. If this bothers you, the pseudo function object
       also returns the whole matched object, see the part about caveats and limitations below.

       Let's do something in the fun 's body too: Let's say that someone realizes that there  are
       a  few  people  having  an employee number beginning with a zero ( 0 ), which shouldn't be
       allowed. All those should have their numbers changed to begin with a one ( 1 ) instead and
       one wants the list [{<Old empno>,<New empno>}] created:

       ets:select(emp_tab, ets:fun2ms(
			     fun(#emp{empno = [$0 | Rest] }) ->

       As a matter of fact, this query hits the feature of partially bound keys in the table type
       ordered_set , so that not the whole table need be searched, only the  part  of  the  table
       containing keys beginning with 0 is in fact looked into.

       The  fun  of  course  can have several clauses, so that if one could do the following: For
       each employee, if he or she is hired prior to 1997, return the tuple {inventory, <employee
       number>}  ,  for each hired 1997 or later, but before 2001, return {rookie, <employee num-
       ber>} , for all others return {newbie, <employee number>} . All except for the ones  named
       Smith  as  they	would  be  affronted by anything other than the tag guru and that is also
       what's returned for their numbers; {guru, <employee number>} :

       ets:select(emp_tab, ets:fun2ms(
			     fun(#emp{empno = E, surname = "Smith" }) ->
				(#emp{empno = E, empyear = Y}) when Y < 1997  ->
				     {inventory, E};
				(#emp{empno = E, empyear = Y}) when Y > 2001  ->
				     {newbie, E};
				(#emp{empno = E, empyear = Y}) -> % 1997 -- 2001
				     {rookie, E}

       The result will be:


       and so the Smith's will be happy...

       So, what more can you do? Well, the simple answer would be; look in the	documentation  of
       match specifications in ERTS users guide. However let's briefly go through the most useful
       "built in functions" that you can use when the fun is to be translated into a match speci-
       fication  by ets:fun2ms (it's worth mentioning, although it might be obvious to some, that
       calling other functions than the one's allowed in match specifications cannot be done.  No
       "usual"	Erlang	code  can  be executed by the fun being translated by fun2ms , the fun is
       after all limited exactly to the power of the match specifications, which is  unfortunate,
       but  the  price	one  has  to  pay  for	the  execution speed of an ets:select compared to
       ets:foldl/foldr ).

       The head of the fun is obviously a head	matching  (or  mismatching)  one  parameter,  one
       object  of the table we select from. The object is always a single variable (can be _ ) or
       a tuple, as that's what's in ets, dets and mnesia tables (the match specification returned
       by  ets:fun2ms  can  of	course be used with dets:select and mnesia:select as well as with
       ets:select ). The use of = in the head is allowed (and encouraged) on the top level.

       The guard section can contain any guard expression of Erlang. Even the "old" type test are
       allowed	on  the toplevel of the guard ( integer(X) instead of is_integer(X) ). As the new
       type tests (the is_ tests) are in practice just guard bif's they can also be  called  from
       within  the  body of the fun, but so they can in ordinary Erlang code. Also arithmetics is
       allowed, as well as ordinary guard bif's. Here's a list of bif's and expressions:

	 * The type  tests:  is_atom,  is_constant,  is_float,	is_integer,  is_list,  is_number,
	   is_pid, is_port, is_reference, is_tuple, is_binary, is_function, is_record

	 * The boolean operators: not, and, or, andalso, orelse

	 * The relational operators: >, >=, <, =<, =:=, ==, =/=, /=

	 * Arithmetics: +, -, *, div, rem

	 * Bitwise operators: band, bor, bxor, bnot, bsl, bsr

	 * The guard bif's: abs, element, hd, length, node, round, size, tl, trunc, self

	 * The obsolete type test (only in guards): atom, constant, float, integer, list, number,
	   pid, port, reference, tuple, binary, function, record

       Contrary to the fact with "handwritten" match specifications, the is_record guard works as
       in ordinary Erlang code.

       Semicolons  ( ; ) in guards are allowed, the result will be (as expected) one "match_spec-
       clause" for each semicolon-separated part of the guard. The semantics being  identical  to
       the Erlang semantics.

       The  body  of the fun is used to construct the resulting value. When selecting from tables
       one usually just construct a suiting term here, using ordinary Erlang  term  construction,
       like  tuple  parentheses, list brackets and variables matched out in the head, possibly in
       conjunction with the occasional constant. Whatever expressions are allowed in  guards  are
       also allowed here, but there are no special functions except object and bindings (see fur-
       ther down), which returns the whole matched object and all known variable bindings respec-

       The dbg variants of match specifications have an imperative approach to the match specifi-
       cation body, the ets dialect hasn't. The fun body for ets:fun2ms returns the result  with-
       out  side  effects,  and  as matching ( = ) in the body of the match specifications is not
       allowed (for performance reasons) the only thing left, more or  less,  is  term	construc-

       Let's  move  on to the dbg dialect, the slightly different match specifications translated
       by dbg:fun2ms .

       The same reasons for using the parse transformation applies to dbg , maybe even more so as
       filtering  using Erlang code is simply not a good idea when tracing (except afterwards, if
       you trace to file). The concept is similar to that of ets:fun2ms except that  you  usually
       use it directly from the shell (which can also be done with ets:fun2ms ).

       Let's manufacture a toy module to trace on


       -export([start/1, store/2, retrieve/1]).

       start(Args) ->
	   toy_table = ets:new(toy_table,Args).

       store(Key, Value) ->

       retrieve(Key) ->
	   [{Key, Value}] = ets:lookup(toy_table,Key),

       During  model  testing,	the  first test bails out with a {badmatch,16} in {toy,start,1} ,

       We suspect the ets call, as we match hard on the return value, but want only the  particu-
       lar new call with toy_table as first parameter. So we start a default tracer on the node:

       1> dbg:tracer().

       And  so	we turn on call tracing for all processes, we are going to make a pretty restric-
       tive trace pattern, so there's no need to call trace only  a  few  processes  (it  usually

       2> dbg:p(all,call).

       It's  time  to  specify	the  filter.  We want to view calls that resemble ets:new(toy_ta-
       ble,<something>) :

       3> dbg:tp(ets,new,dbg:fun2ms(fun([toy_table,_]) -> true end)).

       As can be seen, the fun 's used with dbg:fun2ms takes a single list as  parameter  instead
       of  a  single  tuple.  The list matches a list of the parameters to the traced function. A
       single variable may also be used of course. The body of the fun expresses in a more imper-
       ative  way  actions  to	be  taken if the fun head (and the guards) matches. I return true
       here, but it's only because the body of a fun cannot be empty, the return  value  will  be

       When we run the test of our module now, we get the following trace output:

       (<0.86.0>) call ets:new(toy_table,[ordered_set])

       Let's play we haven't spotted the problem yet, and want to see what ets:new returns. We do
       a slightly different trace pattern:

       4> dbg:tp(ets,new,dbg:fun2ms(fun([toy_table,_]) -> return_trace() end)).

       Resulting in the following trace output when we run the test:

       (<0.86.0>) call ets:new(toy_table,[ordered_set])
       (<0.86.0>) returned from ets:new/2 -> 24

       The call to return_trace , makes a trace message appear	when  the  function  returns.  It
       applies	only to the specific function call triggering the match specification (and match-
       ing the head/guards of the match specification). This is the by far the most  common  call
       in the body of a dbg match specification.

       As  the	test now fails with {badmatch,24} , it's obvious that the badmatch is because the
       atom toy_table does not match the number returned for an unnamed table. So we spotted  the
       problem, the table should be named and the arguments supplied by our test program does not
       include named_table . We rewrite the start function to:

       start(Args) ->
	   toy_table = ets:new(toy_table,[named_table |Args]).

       And with the same tracing turned on, we get the following trace output:

       (<0.86.0>) call ets:new(toy_table,[named_table,ordered_set])
       (<0.86.0>) returned from ets:new/2 -> toy_table

       Very well. Let's say the module now passes all testing and goes into the system.  After	a
       while someone realizes that the table toy_table grows while the system is running and that
       for some reason there are a lot of elements with atom's as keys.  You  had  expected  only
       integer	keys  and  so does the rest of the system. Well, obviously not all of the system.
       You turn on call tracing and try to see calls to your module with an atom as the key:

       1> dbg:tracer().
       2> dbg:p(all,call).
       3> dbg:tpl(toy,store,dbg:fun2ms(fun([A,_]) when is_atom(A) -> true end)).

       We use dbg:tpl here to make sure to catch local calls (let's  say  the  module  has  grown
       since  the  smaller  version  and  we're  not  sure  this  inserting  of atoms is not done
       locally...). When in doubt always use local call tracing.

       Let's say nothing happens when we trace in this way. Our function  is  never  called  with
       these parameters. We make the conclusion that someone else (some other module) is doing it
       and we realize that we must trace on ets:insert and want to see the calling function.  The
       calling function may be retrieved using the match specification function caller and to get
       it into the trace message, one has to use the match spec function  message  .  The  filter
       call looks like this (looking for calls to ets:insert ):

       4> dbg:tpl(ets,insert,dbg:fun2ms(fun([toy_table,{A,_}]) when is_atom(A) ->

       The  caller  will  now appear in the "additional message" part of the trace output, and so
       after a while, the following output comes:

       (<0.86.0>) call ets:insert(toy_table,{garbage,can}) ({evil_mod,evil_fun,2})

       You have found out that the function evil_fun of the module evil_mod , with arity 2  ,  is
       the one causing all this trouble.

       This  was  just	a toy example, but it illustrated the most used calls in match specifica-
       tions for dbg The other, more esotheric calls are listed and explained in the Users  guide
       of the ERTS application , they really are beyond the scope of this document.

       To end this chatty introduction with something more precise, here follows some parts about
       caveats and restrictions concerning the fun's used  in  conjunction  with  ets:fun2ms  and
       dbg:fun2ms :

       To use the pseudo functions triggering the translation, one has to include the header file
       ms_transform.hrl in the source code. Failure to do so  will  possibly  result  in  runtime
       errors  rather than compile time, as the expression may be valid as a plain Erlang program
       without translation.

       The fun has to be literally constructed inside the parameter list to the pseudo functions.
       The  fun cannot be bound to a variable first and then passed to ets:fun2ms or dbg:fun2ms ,
       i.e this will work: ets:fun2ms(fun(A) -> A end) but  not  this:	F  =  fun(A)  ->  A  end,
       ets:fun2ms(F)  .  The later will result in a compile time error if the header is included,
       otherwise a runtime error. Even if the later construction would ever appear  to	work,  it
       really doesn't, so don't ever use it.

       Several	restrictions  apply to the fun that is being translated into a match_spec. To put
       it simple you cannot use anything in the fun that you cannot use  in  a	match_spec.  This
       means that, among others, the following restrictions apply to the fun itself:

	 * Functions  written  in  Erlang cannot be called, neither local functions, global func-
	   tions or real fun's

	 * Everything that is written as a function call will be  translated  into  a  match_spec
	   call  to  a	builtin  function,  so	that  the  call  is_list(X) will be translated to
	   {'is_list', '$1'} ( '$1' is just an example, the numbering may vary). If one tries  to
	   call a function that is not a match_spec builtin, it will cause an error.

	 * Variables occurring in the head of the fun will be replaced by match_spec variables in
	   the order of occurrence, so that the fragment fun({A,B,C}) will be replaced by  {'$1',
	   '$2',  '$3'}  etc. Every occurrence of such a variable later in the match_spec will be
	   replaced by a match_spec variable in the same way, so that  the  fun  fun({A,B})  when
	   is_atom(A) -> B end will be translated into [{{'$1','$2'},[{is_atom,'$1'}],['$2']}] .

	 * Variables  that  are  not  appearing in the head are imported from the environment and
	   made into match_spec const expressions. Example from the shell:

	 1> X = 25.
	 2> ets:fun2ms(fun({A,B}) when A > X -> B end).

	 * Matching with = cannot be used in the body. It can only be used on the  top	level  in
	   the head of the fun. Example from the shell again:

	 1> ets:fun2ms(fun({A,[B|C]} = D) when A > B -> D end).
	 2> ets:fun2ms(fun({A,[B|C]=D}) when A > B -> D end).
	 Error: fun with head matching ('=' in head) cannot be translated into
	 3> ets:fun2ms(fun({A,[B|C]}) when A > B -> D = [B|C], D end).
	 Error: fun with body matching ('=' in body) is illegal as match_spec

	   All	variables  are bound in the head of a match_spec, so the translator can not allow
	   multiple bindings. The special case when matching is done on the top level  makes  the
	   variable  bind  to  '$_'  in  the  resulting match_spec, it is to allow a more natural
	   access to the whole matched	object.  The  pseudo  function	object()  could  be  used
	   instead, see below. The following expressions are translated equally:

	 ets:fun2ms(fun({a,_} = A) -> A end).
	 ets:fun2ms(fun({a,_}) -> object() end).

	 * The	special  match_spec  variables	'$_'  and '$*' can be accessed through the pseudo
	   functions object() (for '$_' ) and bindings() (for '$*' ). as an  example,  one  could
	   translate the following ets:match_object/2 call to a ets:select call:

	 ets:match_object(Table, {'$1',test,'$2'}).

	   ...is the same as...

	 ets:select(Table, ets:fun2ms(fun({A,test,B}) -> object() end)).

	   (This  was  just  an  example,  in  this simple case the former expression is probably
	   preferable in terms of readability). The ets:select/2 call will conceptually look like
	   this in the resulting code:

	 ets:select(Table, [{{'$1',test,'$2'},[],['$_']}]).

	   Matching on the top level of the fun head might feel like a more natural way to access
	   '$_' , see above.

	 * Term constructions/literals are translated as much as is needed to get them into valid
	   match_specs,  so  that tuples are made into match_spec tuple constructions (a one ele-
	   ment tuple containing the tuple) and constant  expressions  are  used  when	importing
	   variables from the environment. Records are also translated into plain tuple construc-
	   tions, calls to element etc. The guard test is_record/2 is translated into  match_spec
	   code  using	the  three  parameter  version	that's	built  into  match_specs, so that
	   is_record(A,t) is translated into {is_record,'$1',t,5} given that the record  size  of
	   record type t is 5.

	 * Language  constructions like case , if , catch etc that are not present in match_specs
	   are not allowed.

	 * If the header file ms_transform.hrl is not included,  the  fun  won't  be  translated,
	   which may result in a runtime error (depending on if the fun is valid in a pure Erlang
	   context). Be  absolutely  sure  that  the  header  is  included  when  using  ets  and
	   dbg:fun2ms/1 in compiled code.

	 * If  the  pseudo  function  triggering the translation is ets:fun2ms/1 , the fun's head
	   must contain a  single  variable  or  a  single  tuple.  If	the  pseudo  function  is
	   dbg:fun2ms/1 the fun's head must contain a single variable or a single list.

       The  translation from fun's to match_specs is done at compile time, so runtime performance
       is not affected by using these pseudo functions. The compile time might be somewhat longer

       For more information about match_specs, please read about them in ERTS users guide .

       parse_transform(Forms,_Options) -> Forms

	      Types  Forms = Erlang abstract code format, see the erl_parse module description
		     _Options = Option list, required but not used

	      Implements  the  actual  transformation at compile time. This function is called by
	      the compiler to do the source code transformation if and when the  ms_transform.hrl
	      header  file  is included in your source code. See the ets and dbg : fun2ms/1 func-
	      tion manual pages for documentation on how to use  this  parse_transform,  see  the
	      match_spec chapter in ERTS users guide for a description of match specifications.

       transform_from_shell(Dialect,Clauses,BoundEnvironment) -> term()

	      Types  Dialect = ets | dbg
		     Clauses = Erlang abstract form for a single fun
		     BoundEnvironment = [{atom(), term()}, ...], list of variable bindings in the
		     shell environment

	      Implements the actual transformation when the fun2ms functions are called from  the
	      shell.  In  this case the abstract form is for one single fun (parsed by the Erlang
	      shell), and all imported variables should be in the key-value list passed as Bound-
	      Environment . The result is a term, normalized, i.e. not in abstract format.

       format_error(Errcode) -> ErrMessage

	      Types  Errcode = term()
		     ErrMessage = string()

	      Takes  an  error code returned by one of the other functions in the module and cre-
	      ates a textual description of the error. Fairly uninteresting function actually.

Ericsson AB				  stdlib 1.17.3 		       ms_transform(3erl)

All times are GMT -4. The time now is 01:30 AM.

Unix & Linux Forums Content Copyrightę1993-2018. All Rights Reserved.
Show Password