Sponsored Content
Top Forums Shell Programming and Scripting Fastest alternatives to flattening a non-uniform nested array with regex? Post 303015272 by f77hack on Saturday 31st of March 2018 05:37:10 PM
Old 03-31-2018
Quote:
Originally Posted by jim mcnamara
My opinion:

As a general solution that seems fine. I have not validated it. Are you encountering performance problems?

You could write a simple single purpose C program....
Thanks for the comments.

Yes, I tried some long, really off-the-wall, random arrays that would be equivalent to a messy json structure and the simple regex easily beat the recursion.

Good idea about C code, what do you think about the performance gain and memory utilization of using a threaded version? versus openmp/mpi? or versus GPU, if the incoming array is a continuous stream from a web socket? or similar.

For single precision, Int16 or UInt16, it seems like GPU would crush this problem but I'm unsure of the overhead required for the memory allocation.

Last edited by f77hack; 03-31-2018 at 06:52 PM.. Reason: add memory stuff
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

perl: storing regex in array variables trouble

hi this is an example of code: use strict; use warnings; open FILE, "/tmp/result_2"; my $regex="\\ Starting program ver. (.*)"; my $res="Program started, version <$1> - OK.\n"; while (<FILE>) { if ($_ =~ /($regex)/) { print "$res"; } } close FILE; This finds $regex and print... (3 Replies)
Discussion started by: xist
3 Replies

2. Shell Programming and Scripting

PHP: Search Multi-Dimensional(nested) array and export values of currenly worked on array.

Hi All, I'm writing a nagios check that will see if our ldap servers are in sync... I got the status data into a nested array, I would like to search key of each array and if "OK" is NOT present, echo other key=>values in the current array to a variable so...eg...let take the single array... (1 Reply)
Discussion started by: zeekblack
1 Replies

3. Shell Programming and Scripting

Perl nested array problem

I have a array reference which has some number of array references inside it.The nested array references also contains the array references. my $Filename = "sample.xml"; my $Parser = new XML::Parser( Style => 'tree' ); my $Tree = $Parser->parsefile( $Filename ); Here the $Tree is the... (6 Replies)
Discussion started by: karthigayan
6 Replies

4. Shell Programming and Scripting

Grep: Searching with a regex that contains a variable from an array

I'm attempting to grep for lines formatted like this: grep -e '^\\",' Any suggestions as to why this isn't working? ---------- Post updated at 05:03 PM ---------- Previous update was at 04:17 PM ---------- This was my solution: grep -e '^\'\",' It's hard to read, but basically I... (4 Replies)
Discussion started by: AcerAspirant
4 Replies

5. Shell Programming and Scripting

Attempting to pass my array as a nested parameter to loop. A little lost.

I want to pass this array as a parameter. IFS=$'\n' fortune_lines=($(fortune | fold -w 30 )) Inside of this line screen -p 0 -S ${SCREEN_SESSION} -X stuff "`printf "say ${fortune_lines}\r"`" And I am lost at this point. I am thinking something like this? Then make it loop.. ... (7 Replies)
Discussion started by: briandanielz
7 Replies

6. AIX

Uniform LUN size

Hi, Is there any advantage is making all my storage LUNS ( hdisk ) of uniform size. Currently the LUN's are having different size () eg: 50G / 60G / 75G etc ). I am planning for a storage migration....so should i go for uniform lun size or with current LUN size pattern ? Regards, jibu (3 Replies)
Discussion started by: jibujacob
3 Replies

7. Shell Programming and Scripting

Bash 3.2 - Array / Regex - IF 3rd member in array ends in 5 digits then do somthing...

Trying to do some control flow parsing based on the index postion of an array member. Here is the pseudo code I am trying to write in (preferably in pure bash) where possible. I am thinking regex with do the trick, but need a little help. pesudo code if == ENDSINFIVEINTS ]]; then do... (4 Replies)
Discussion started by: briandanielz
4 Replies

8. Shell Programming and Scripting

How to insert an array element within regex?

Hello to all, I'm trying to separate the string "str" using a regex within match function. The substrings that I want to separate, begin with 22, 23, 24 or 25 and followed by 12 or 14 characters. And I want to replace 22 with MJS, 23 with UYT, 24 with WER and 25 with PIL. For this string... (4 Replies)
Discussion started by: Ophiuchus
4 Replies

9. Shell Programming and Scripting

Uniform Spacing in the message

Hello, I am running a script which sends an output as an email; I am having issues with the spacing being not uniform in the message. Snippet of the code and email message below: if ] then echo "$Hostname\tMISSING\tHMCBackup" >> $BackupMsg else if ] then echo... (12 Replies)
Discussion started by: hasn318
12 Replies
mtmalloc(3MALLOC)                                                                                                                mtmalloc(3MALLOC)

NAME
mtmalloc, mallocctl - MT hot memory allocator SYNOPSIS
#include <mtmalloc.h> cc -o a.out -lthread -lmtmalloc void *malloc(size_t size); void free(void *ptr); void *memalign(size_t alignment, size_t size); void *realloc(void *ptr, size_t size); void *valloc(size_t size); void mallocctl(int cmd, long value); The malloc() and free() functions provide a simple general-purpose memory allocation package that is suitable for use in high performance multithreaded applications. The suggested use of this library is in multithreaded applications; it can be used for single threaded appli- cations, but there is no advantage in doing so. This library cannot be dynamically loaded with dlopen(3C) during runtime because there must be only one manager of the process heap. The malloc() function returns a pointer to a block of at least size bytes suitably aligned for any use. The argument to free() is a pointer to a block previously allocated by malloc() or realloc(). After free() is performed this space is available for further allocation. If ptr is a null pointer, no action occurs. The free() function does not set errno. Undefined results will occur if the space assigned by malloc() is overrun or if a random number is handed to free(). A freed pointer that is passed to free() will send a SIGABRT signal to the calling process. This behavior is controlled by mallocctl(). The memalign() function allocates size bytes on a specified alignment boundary and returns a pointer to the allocated block. The value of the returned address is guaranteed to be an even multiple of alignment. Note that the value of alignment must be a power of two, and must be greater than or equal to the size of a word. The realloc() function changes the size of the block pointed to by ptr to size bytes and returns a pointer to the (possibly moved) block. The contents will be unchanged up to the lesser of the new and old sizes. If the new size of the block requires movement of the block, the space for the previous instantiation of the block is freed. If the new size is larger, the contents of the newly allocated portion of the block are unspecified. If ptr is NULL, realloc() behaves like malloc() for the specified size. If size is 0 and ptr is not a null pointer, the space pointed to is freed. The valloc() function has the same effect as malloc(), except that the allocated memory will be aligned to a multiple of the value returned by sysconf(_SC_PAGESIZE). After possible pointer coercion, each allocation routine returns a pointer to a space that is suitably aligned for storage of any type of object. The malloc(), realloc(), memalign(), and valloc() functions will fail if there is not enough available memory. The mallocctl() function controls the behavior of the malloc library. The options fall into two general classes, debugging options and per- formance options. MTDOUBLEFREE Allows double free of a pointer. Setting value to 1 means yes and 0 means no. The default behavior of double free results in a core dump. MTDEBUGPATTERN Writes misaligned data into the buffer after free(). When the buffer is reallocated, the contents are verified to ensure that there was no access to the buffer after the free. If the buffer has been dirtied, a SIGABRT signal is delivered to the process. Setting value to 1 means yes and 0 means no. The default behavior is to not write mis- aligned data. The pattern used is 0xdeadbeef. Use of this option results in a performance penalty. MTINITBUFFER Writes misaligned data into the newly allocated buffer. This option is useful for detecting some accesses before initialization. Setting value to 1 means yes and 0 means no. The default behavior is to not write misaligned data to the newly allocated buffer. The pattern used is 0xbaddcafe. Use of this option results in a performance penalty. MTCHUNKSIZE This option changes the size of allocated memory when a pool has exhausted all available memory in the buffer. Increasing this value allocates more memory for the application. A substantial performance gain can occur because the library makes fewer calls to the OS for more memory. Acceptable number values are between 9 and 256. The default value is 9. This value is multiplied by 8192. If there is no available memory, malloc(), realloc(), memalign(), and valloc() return a null pointer. When realloc() is called with size > 0 and returns NULL, the block pointed to by ptr is left intact. If size, nelem, or elsize is 0, either a null pointer or a unique pointer that can be passed to free() is returned. If malloc() or realloc() returns unsuccessfully, errno will be set to indicate the error. The malloc() and realloc() functions will fail if: ENOMEM The physical limits of the system are exceeded by size bytes of memory which cannot be allocated. EAGAIN There is not enough memory available to allocate size bytes of memory; but the application could try again later. Comparative features of the various allocation libraries can be found in the umem_alloc(3MALLOC) manual page. See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |MT-Level |Safe | +-----------------------------+-----------------------------+ brk(2), getrlimit(2), bsdmalloc(3MALLOC), dlopen(3C), malloc(3C), malloc(3MALLOC), mapmalloc(3MALLOC), signal.h(3HEAD), umem_alloc(3MAL- LOC), watchmalloc(3MALLOC), attributes(5) WARNINGS
Undefined results will occur if the size requested for a block of memory exceeds the maximum size of a process's heap. This information may be obtained using getrlimit(). 21 Mar 2005 mtmalloc(3MALLOC)
All times are GMT -4. The time now is 08:00 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy