Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Awk/bash one liner replacement for a if condition
Top Forums UNIX for Beginners Questions & Answers Awk/bash one liner replacement for a if condition Post 303041697 by rbatte1 on Monday 2nd of December 2019 08:56:10 AM
Old 12-02-2019
If you are hitting memory limits, you could almost go very old-school:-
Code:
awk '/pattern/ { print $2 }' file1 | sort > $workdir/file1_col2
awk '/pattern/ { print $2 }' file2 | sort > $workdir/file2_col2
diff $workdir/file1_col2$workdir/file2_col2

Probably slow because of the disk access and if it's too big, then diff may also struggle.

Not the best, but a clunky way to be sure it will actually work. Perhaps this will give you a good basis to see what output from a better (more efficient) process should be.




Robin
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bad day !! test condition failed --need a one liner to do --help

Hi all this is simple but bad day for me nothing work out .. Problem is that I wan to check the argument passed to my script and accordignly exit or setup ENV variable I have a script name src_cpcp_preproc.sh i want to pass 2 argumet from command line argumet and check it in the script... (13 Replies)
Discussion started by: jambesh
13 Replies

2. UNIX for Dummies Questions & Answers

Awk/Sed One liner for text replacement

Hi group, I want to replace the occurance of a particular text in a paragraph.I tried with Sed,but Sed only displays the result on the screen.How can i update the changes in the original file??? The solution should be a one liner using awk and sed. Thanks in advance. (5 Replies)
Discussion started by: bishnu.bhatta
5 Replies

3. Shell Programming and Scripting

Bash one liner to CSH

Hello all, I have a Bash command I'm using on one system that replaces text in filenames, I'ts not working on another system that uses the Csh shell. Can anyone tell me what I need to do when i run for f in *;do mv $f ${f/text1/text2};done on the CSH shell i get ""Missing }."" (7 Replies)
Discussion started by: mike171562
7 Replies

4. Shell Programming and Scripting

Bash replacement to getchar

There's a replacement in bash for getchar or get functions of C and C++?Those functions read the next char avaliable in the input stream. I've tried something like: OLD_STTY=`stty -g` stty cbreak -echo look=`dd if=/dev/tty bs=1 count=1 2>/dev/null` stty $OLD_STTY But it is not working... (3 Replies)
Discussion started by: Asafe
3 Replies

5. UNIX for Dummies Questions & Answers

awk one liner

I need a one liner to" find /pattern/ print from x lines before "pattern" to y lines after "pattern" (3 Replies)
Discussion started by: kenneth.mcbride
3 Replies

6. Shell Programming and Scripting

Bash string replacement - how to use regex?

Hello I have a bash script where I need to do a substring replacement like this: variable2=${variable1/foo/bar} However, I only want "foo" replaced if it is at the end of the line. However, this does not work: variable2=${variable1/foo$/bar} as you can see I'm using the $ regex for... (2 Replies)
Discussion started by: Ubuntu-UK
2 Replies

7. Shell Programming and Scripting

Search & Replace regex Perl one liner to AWK one liner

Thanks for giving your time and effort to answer questions and helping newbies like me understand awk. I have a huge file, millions of lines, so perl takes quite a bit of time, I'd like to convert these perl one liners to awk. Basically I'd like all lines with ISA sandwiched between... (9 Replies)
Discussion started by: verge
9 Replies

8. Shell Programming and Scripting

HELP with AWK one-liner. Need to employ an If condition inside AWK to check for array variable ?

Hello experts, I'm stuck with this script for three days now. Here's what i need. I need to split a large delimited (,) file into 2 files based on the value present in the last field. Samp: Something.csv bca,adc,asdf,123,12C bca,adc,asdf,123,13C def,adc,asdf,123,12A I need this split... (6 Replies)
Discussion started by: shell_boy23
6 Replies

9. Programming

Resample time series data with replacement any way to do this in awk or just bash script

I have some time series data that I need to resample or downsample at some specific time intervals. The firs column is time in decimal hours. I am tryiong to resample this data every 3 minutse. So I need a data value ever 0.05. Here is the example data and as you can see, there time slot for 0.1500... (3 Replies)
Discussion started by: malandisa
3 Replies

10. UNIX for Beginners Questions & Answers

Perl one liner in bash script not replacing hours and minutes [HH:MM]

Hi I want to replace time stamp in the following line PROCNAME.Merge.exchMon.CODE.T_QSTART 08:45 read assuming the new time stamp is 09:45 ; the line is getting replaced as below :45 read I'm trying to use the perl one liner in bash script perl -pi... (4 Replies)
Discussion started by: charlie87
4 Replies

LEARN ABOUT DEBIAN

flow-expire

flow-expire(1)						      General Commands Manual						    flow-expire(1)

NAME

       flow-expire -- Manage storage of flow file archives by expiring old data.

SYNOPSIS

       flow-expire [-h]  [-d debug_level]  [-e expire_count]  [-E expire_size] -w workdir

DESCRIPTION

       The flow-expire utility will remove the oldest flow files in a directory based on either a count of files or space utilization.	The direc-
       tory is recursively searched for flow files.  Files that do not have a flow-tools signature will be ignored.   The  internal  timestamp	is
       used  so backups or copies of the flow files that do not retain the original timestamp will not impact the operation of flow-expire.  flow-
       expire is typically used to manage storage in a distributed environment where flows are collected on  a	different  server  than  they  are
       archived.

OPTIONS

       -d debug_level
		 Enable debugging.

       -e expire_count
		 Retain the maximum number of files so that the total file count is less than expire_count.

       -E expire_size
		 Retain the maximum number of files so that the total storage is less than expire_size.  The letters b,K,M,G can be used as multi-
		 pliers, ie 16 Megabytes is 16M.

       -h	 Display help.

       -w workdir
		 Work in workdir.

EXAMPLES

       Remove the oldest flow files in /flows/krc4 until the total storage is less then 2 Gigabytes.

	 flow-expire -E2Gig -w /flows/krc4

       Remove the oldest flow files in /flows/krc4 until the total number of files is less then 100.

	 flow-expire -e100 -w /flows/krc4

BUGS

       Empty directories are not removed.

AUTHOR

       Mark Fullmer maf@splintered.net

SEE ALSO

       flow-tools(1)

																    flow-expire(1)
All times are GMT -4. The time now is 12:32 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy