I know it is a really short script and if i am not mistaken you even wrote it .
I just timed both of them and the results are as follows. It looks like there is alot of improvement using the grep line except that i dont know if it removes the filtered lines from the file like the awk solution.
Code:
awk -F";" prijslijst_filter.csv lowercase_winnaar.csv 260,73s user 0,50s system 99% cpu 4:21,84 total
Code:
grep --color=auto -f prijslijst_filter.csv lowercase_winnaar.csv 45,13s user 0,52s system 99% cpu 45,679 total
Moderator's Comments:
Please use CODE tags as required by forum rules!
(sorry for the code part)
I just tested your last bit and it makes a huge difference.
Code:
awk prijslijst_filter.csv lowercase_winnaar.csv 9,51s user 0,13s system 99% cpu 9,647 total
I will have to check the files but this would help enormous with all scripts.
Last edited by SDohmen; 03-28-2019 at 08:53 AM..
Reason: extra data
hi someone tell me which ways i can improve disk I/O and system process performance.kindly refer some commands so i can do it on my test machine.thanks, Mazhar (2 Replies)
I have a data file of 2 gig
I need to do all these, but its taking hours, any where i can improve performance, thanks a lot
#!/usr/bin/ksh
echo TIMESTAMP="$(date +'_%y-%m-%d.%H-%M-%S')"
function showHelp {
cat << EOF >&2
syntax extreme.sh FILENAME
Specify filename to parse
EOF... (3 Replies)
Hi Friends,
I wrote the below shell script to generate a report on alert messages recieved on a day. But i for processing around 4500 lines (alerts) the script is taking aorund 30 minutes to process.
Please help me to make it faster and improve the performace of the script. i would be very... (10 Replies)
Hi All,
I have written a script as follows which is taking lot of time in executing/searching only 3500 records taken as input from one file in log file of 12 GB Approximately.
Working of script is read the csv file as an input having 2 arguments which are transaction_id,mobile_number and search... (6 Replies)
I have around 300 files(*.rdf,*.fmb,*.pll,*.ctl,*.sh,*.sql,*.prog) which are of large size.
Around 8000 keywords(which will be in the file $keywordfile) needed to be searched inside those files.
If a keyword is found in a file..I have to insert the filename,extension,catagoery,keyword,occurrence... (8 Replies)
Hi ,
i wrote a script to convert dates to the formate i want .it works fine but the conversion is tkaing lot of time . Can some one help me tweek this script
#!/bin/bash
file=$1
ofile=$2
cp $file $ofile
mydates=$(grep -Po '+/+/+' $ofile) # gets 8/1/13
mydates=$(echo "$mydates" | sort |... (5 Replies)
Hello,
I'm new to this forum and like to first of all say hello to everyone.
I've got a really annoying problem at the moment.
I'm trying to rsync some files (about 200MB with one file of 120MB) from a Raspberry PI with raspbian to a debian server via rsync.
This procedure is stored in a... (3 Replies)
Hello,
Attached is my very simple C++ code to remove any substrings (DNA sequence) of each other, i.e. any redundant sequence is removed to get unique sequences. Similar to sort | uniq command except there is reverse-complementary for DNA sequence. The program runs well with small dataset, but... (11 Replies)
Discussion started by: yifangt
11 Replies
LEARN ABOUT DEBIAN
sysprofile
SYSPROFILE(8) System Manager's Manual SYSPROFILE(8)NAME
sysprofile - modular centralized shell configuration
DESCRIPTION
sysprofile is a generic approach to configure shell settings in a modular and centralized way mostly aimed at avoiding work for lazy sysad-
mins. It has only been tested to work with the bash shell.
It basically consists of the small /etc/sysprofile shell script which invokes other small shell scripts having a .bash suffix which are
contained in the /etc/sysprofile.d/ directory. The system administrator can drop in any script he wants without any naming convention
other than that the scripts need to have a .bash suffix to enable automagic sourcing by /etc/sysprofile.
This mechanism is set up by inserting a small shell routine into /etc/profile for login shells and optionally into /etc/bashrc and/or
/etc/bash.bashrc for non-login shells from where the actual /etc/sysprofile script is invoked:
if [ -f /etc/sysprofile ]; then
. /etc/sysprofile
fi
For using "sysprofile" under X11, one can source it in a similar way from /etc/X11/Xsession or your X display manager's Xsession file to
provide the same shell environment as under the console in X11. See the example files in /usr/share/doc/sysprofile/ for illustration.
For usage of terminal emulators with a non-login bash shell under X11, take care to enable sysprofile via /etc/bash.bashrc. If not set
this way, your terminal emulators won't come up with the environment defined by the scripts in /etc/sysprofile.d/.
Users not wanting /etc/sysprofile to be sourced for their environment can easily disable it's automatic mechanism. It can be disabled by
simply creating an empty file called $HOME/.nosysprofile in the user's home directory using e.g. the touch(1) command.
Any single configuration file in /etc/sysprofile.d/ can be overridden by any user by creating a private $HOME/.sysprofile.d/ directory
which may contain a user's own version of any configuration file to be sourced instead of the system default. It's names have just to
match exactly the system's default /etc/sysprofile.d/ configuration files. Empty versions of these files contained in the $HOME/.syspro-
file.d/ directory automatically disable sourcing of the system wide version.
Naturally, users can add and include their own private script inventions to be automagically executed by /etc/sysprofile at login time.
OPTIONS
There are no options other than those dictated by shell conventions. Anything is defined within the configuration scripts themselves.
SEE ALSO
The README files and configuration examples contained in /etc/sysprofile.d/ and the manual pages bash(1), xdm(1x), xdm.options(5), and
wdm(1x). Recommended further reading is everything related with shell programming.
If you need a similar mechanism for executing code at logout time check out the related package syslogout(8) which is a very close compan-
ion to sysprofile.
BUGS
sysprofile in its current form is mainly restricted to bash(1) syntax. In fact it is actually a rather embarrassing quick and dirty hack
than anything else - but it works. It serves the practical need to enable a centralized bash configuration until something better
becomes available. Your constructive criticism in making this into something better" is very welcome. Before i forget to mention it: we
take patches... ;-)
AUTHOR
sysprofile was developed by Paul Seelig <pseelig@debian.org> specifically for the Debian GNU/Linux system. Feel free to port it to and use
it anywhere else under the conditions of either the GNU public license or the BSD license or both. Better yet, please help to make it into
something more worthwhile than it currently is.
SYSPROFILE(8)
03-28-2019
. 