10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi
I need to do a patten match between files .
I am new to shell scripting and have come up with this so far. It take 50 seconds to process files of 2mb size . I need to tune this code as file size will be around 50mb and need to save time.
Main issue is that I need to search the pattern from... (2 Replies)
Discussion started by: nitin_daharwal
2 Replies
2. Shell Programming and Scripting
HI,
I have File1 which contains :-
admins = anand,satheesha,user1,user2,user3,user4,user5,user10,vishal
nonadmins = read-only
@admins = rw
@nonadmins = r
One shell script, using that I want to change the File1 as per user input (let's say $1) which have value as 'John', so now I want to... (6 Replies)
Discussion started by: Vishal Mishra
6 Replies
3. Shell Programming and Scripting
i have little challenge, help me out.i have a file where i have a value declared and and i have to replace the value when called. for example i have the value for abc and ccc. now i have to substitute the value of
value abc and ccc in the place of them.
Input File:
go to &abc=ddd;
if... (16 Replies)
Discussion started by: saaisiva
16 Replies
4. Shell Programming and Scripting
Hi,
I have two files file1.txt and file2.txt. Please see the attachments.
In file2.txt (which actually is a diff output between two versions of file1.txt.), I extract the pattern corresponding to 1172c1172. Now ,In file1.txt I have to search for this pattern 1172c1172 and if found, I have to... (9 Replies)
Discussion started by: saurabh kumar
9 Replies
5. Shell Programming and Scripting
I am trying to search a file for a patterns ERR- in a file and return a count for each of the error reported
Input file is a free flowing file without any format
example of output
ERR-00001=5
....
ERR-01010=10
.....
ERR-99999=10 (4 Replies)
Discussion started by: swayam123
4 Replies
6. Shell Programming and Scripting
Hello All,
I have a file say
cat test.txt
sathya sxk opt
Sandy sxr omg
Ram rxm opt
Now I would like to change opt to omr only for the line in my file
sathya sxk opt
How can i do it.
I know we can do it with sed -i option, but its not present in my system, please help me some... (13 Replies)
Discussion started by: sathyaonnuix
13 Replies
7. Shell Programming and Scripting
Hi,
I got a file which looks like this:
Value A
Status: -
Other: -
Value B
Status: -
Other: -
Value C
Status: -
Other: -
I would like to change only the first line which includes the "Status:" string after matching the line containing "Value B", so the output file should look... (5 Replies)
Discussion started by: wenclu
5 Replies
8. Shell Programming and Scripting
Hi, I have 2 files that I have modified to basically match each other, however I want to determine what (if any) line in file 1 does not exist in file 2. I need to match column $1 and $2 as a single string in file1 to $1 and $2 in file2 as these two columns create a match.
I'm stuck in an AWK... (9 Replies)
Discussion started by: right_coaster
9 Replies
9. Shell Programming and Scripting
Hi All,
Please can someone assist in the script I have made that searches a pattern in a file and delete the whole line containing the pattern.
#!bin/sh
# The pattern that user want to add to the files
echo "Enter the pattern of the redirect"
read value
# check if the user has... (1 Reply)
Discussion started by: Shazin
1 Replies
10. Shell Programming and Scripting
Hi,
I have a file with contents in this format....
... (3 Replies)
Discussion started by: kripssmart
3 Replies
DEVELOPMENT(7) BSD Miscellaneous Information Manual DEVELOPMENT(7)
NAME
development -- introduction to development with the FreeBSD codebase
DESCRIPTION
This manual page describes how an ordinary system operator, UNIX administrator, or developer can, without any special permission, obtain,
maintain, and modify the FreeBSD codebase as well as how to maintain a master build which can then be exported to other machines in your net-
work. This manual page is targeted to system operators, programmers, and developers.
Please note that what is being described here is based on a complete FreeBSD environment, not just the FreeBSD kernel. The methods described
here are as applicable to production installations as it is to development environments.
SETTING UP THE ENVIRONMENT ON THE MASTER SERVER
Your master server should always run a stable, production version of the FreeBSD operating system. This does not prevent you from doing
-CURRENT builds or development. The last thing you want to do is to run an unstable environment on your master server which could lead to a
situation where you lose the environment and/or cannot recover from a mistake.
Create a partition called /FreeBSD. Approximately 20GB is recommended. This partition will contain nearly all the development environment,
including the subversion tree, broken-out source, and possibly even object files. You are going to export this partition to your other
machines via a READ-ONLY NFS export so do not mix it with other more security-sensitive partitions.
You have to make a choice in regards to /usr/obj. You can put /usr/obj in /FreeBSD or you can make /usr/obj its own partition. I recommend
making it a separate partition for several reasons. First, as a safety measure since this partition is written to a great deal. Second,
because you typically do not have to back it up. Third, because it makes it far easier to mix and match the development environments which
are described later in this document. I recommend a /usr/obj partition of at least 12GB.
On the master server, use svn(1) to pull down and maintain the FreeBSD source. The first pull will take a long time, it is several giga-
bytes, but once you have it, the updates will be quite small. Run all svn(1) operations as ``root''
Keeping the broken-out source and ports in /FreeBSD allows you to export it to other machines via read-only NFS. This also means you only
need to edit/maintain files in one place and all your clients automatically pick up the changes.
mkdir /FreeBSD
cd /FreeBSD
svn co https://svn.freebsd.org/ports/head ports
svn co https://svn.freebsd.org/doc/head doc
svn co https://svn.freebsd.org/base/head src
cd /usr
rm -rf src
ln -s /FreeBSD/src src
Note that exporting /usr/obj via read-only NFS to your other boxes will allow you to build on your main server and install from your other
boxes. If you also want to do builds on some or all of the clients you can simply have /usr/obj be a local directory on those clients. You
should never export /usr/obj read-write, it will lead to all sorts of problems and issues down the line and presents a security problem as
well. It is far easier to do builds on the master server and then only do installs on the clients.
I usually maintain my ports tree via svn or portsnap. With some fancy softlinks you can make the ports tree available both on your master
server and on all of your other machines. Note that the ports tree exists only on the HEAD ports branch, so its always usable even on a
-STABLE box. This is what you do:
(THESE COMMANDS ON THE MASTER SERVER AND ON ALL CLIENTS)
cd /usr
rm -rf ports
ln -s /FreeBSD/ports ports
cd /usr/ports (this pushes into the softlink)
rm -rf distfiles (ON MASTER SERVER ONLY)
ln -s /usr/ports.distfiles distfiles (ON MASTER SERVER ONLY)
mkdir /usr/ports.distfiles
mkdir /usr/ports.workdir
Since /usr/ports is softlinked into what will be read-only on all of your clients, you have to tell the ports system to use a different work-
ing directory to hold ports builds. You want to add a line to your make.conf(5) file on the master server and on all your clients:
WRKDIRPREFIX=/usr/ports.workdir
You should try to make the directory you use for the ports working directory as well as the directory used to hold distfiles consistent
across all of your machines. If there is not enough room in /usr/ports.distfiles and /usr/ports.workdir I usually make those softlinks
(since this is on /usr these are per-machine) to where the distfiles and working space really are.
EXPORTING VIA NFS FROM THE MASTER SERVER
The master server needs to export /FreeBSD and /usr/obj via NFS so all the rest of your machines can get at them. I strongly recommend using
a read-only export for both security and safety. The environment I am describing in this manual page is designed primarily around read-only
NFS exports. Your exports file on the master server should contain the following lines:
/FreeBSD -ro -alldirs -maproot=root: -network YOURLAN -mask YOURLANMASK
/usr/obj -ro -alldirs -maproot=root: -network YOURLAN -mask YOURLANMASK
Of course, NFS server operations must also be configured on that machine. This is typically done via your /etc/rc.conf:
nfs_server_enable="YES"
nfs_server_flags="-u -t -n 4"
THE CLIENT ENVIRONMENT
All of your client machines can import the development/build environment directory simply by NFS mounting /FreeBSD and /usr/obj from the mas-
ter server. A typical /etc/fstab entry on your client machines will be something like this:
masterserver:/FreeBSD /FreeBSD nfs ro,bg 0 0
masterserver:/usr/obj /usr/obj nfs ro,bg 0 0
And, of course, you should configure the client for NFS client operations via /etc/rc.conf. In particular, this will turn on nfsiod(8) which
will improve client-side NFS performance:
nfs_client_enable="YES"
Each client should create softlinks for /usr/ports and /usr/src that point into the NFS-mounted environment. If a particular client is run-
ning -CURRENT, /usr/src should be a softlink to /FreeBSD/src. If it is running -STABLE, /usr/src should be a softlink to
/FreeBSD/FreeBSD-4.x/src. I do not usually create a /usr/src2 softlink on clients, that is used as a convenient shortcut when working on the
source code on the master server only and could create massive confusion (of the human variety) on a client.
(ON EACH CLIENT)
cd /usr
rm -rf ports src
ln -s /FreeBSD/ports ports
ln -s /FreeBSD/src src
Do not forget to create the working directories so you can build ports, as previously described. If these are not good locations, make them
softlinks to the correct location. Remember that /usr/ports/distfiles is exported by the master server and is therefore going to point to
the same place (typically /usr/ports.distfiles) on every machine.
mkdir /usr/ports.distfiles
mkdir /usr/ports.workdir
BUILDING KERNELS
Here is how you build a -STABLE kernel (on your main development box). If you want to create a custom kernel, copy GENERIC to KERNELNAME and
then edit it before configuring and building. The kernel configuration file lives in /usr/src/sys/i386/conf/KERNELNAME.
cd /usr/src
make buildkernel KERNCONF=KERNELNAME
WARNING! If you are familiar with the old config/cd/make method of building a -STABLE kernel, note that the config(8) method will put the
build environment in /usr/src/sys/i386/compile/KERNELNAME instead of in /usr/obj.
Building a -CURRENT kernel
cd /usr/src2 (on the master server)
make buildkernel KERNCONF=KERNELNAME
INSTALLING KERNELS
Installing a -STABLE kernel (typically done on a client, only do this on your main development server if you want to install a new kernel for
your main development server):
cd /usr/src
make installkernel KERNCONF=KERNELNAME
If you are using the older config/cd/make build mechanism for -STABLE, you would install using:
cd /usr/src/sys/i386/compile/KERNELNAME
make install
Installing a -CURRENT kernel (typically done only on a client)
(remember /usr/src is pointing to the client's specific environment)
cd /usr/src
make installkernel KERNCONF=KERNELNAME
BUILDING THE WORLD
This environment is designed such that you do all builds on the master server, and then install from each client. You can do builds on a
client only if /usr/obj is local to that client. Building the world is easy:
cd /usr/src
make buildworld
If you are on the master server you are running in a -STABLE environment, but that does not prevent you from building the -CURRENT world.
Just cd(1) into the appropriate source directory and you are set. Do not accidentally install it on your master server though!
cd /usr/src2
make buildworld
INSTALLING THE WORLD
You can build on your main development server and install on clients. The main development server must export /FreeBSD and /usr/obj via
read-only NFS to the clients.
NOTE!!! If /usr/obj is a softlink on the master server, it must also be the EXACT SAME softlink on each client. If /usr/obj is a directory
in /usr or a mount point on the master server, then it must be (interchangeably) a directory in /usr or a mount point on each client. This
is because the absolute paths are expected to be the same when building the world as when installing it, and you generally build it on your
main development box and install it from a client. If you do not set up /usr/obj properly you will not be able to build on machine and
install on another.
(ON THE CLIENT)
(remember /usr/src is pointing to the client's specific environment)
cd /usr/src
make installworld
WARNING! If builds work on the master server but installs do not work from the clients, for example you try to install and the client com-
plains that the install tried to write into the read-only /usr/obj, then it is likely that the make.conf(5) file on the client does not match
the one on the master server closely enough and the install is trying to install something that was not built.
DOING DEVELOPMENT ON A CLIENT (NOT JUST INSTALLING)
Developers often want to run buildkernel's or buildworld's on client boxes simply to life-test the box. You do this in the same manner that
you buildkernel and buildworld on your master server. All you have to do is make sure that /usr/obj is pointing to local storage. If you
followed my advise and made /usr/obj its own partition on the master server, then it is typically going to be an NFS mount on the client.
Simply unmounting /usr/obj will leave you with a /usr/obj that is a subdirectory in /usr which is typically local to the client. You can
then do builds to your heart's content!
MAINTAINING A LOCAL BRANCH
There is absolutely nothing preventing you from breaking out other versions of the source tree into /FreeBSD/XXX. In fact, my /FreeBSD par-
tition also contains OpenBSD, NetBSD, and various flavors of Linux. You may not necessarily be able to build non-FreeBSD operating systems
on your master server, but being able to collect and manage source distributions from a central server is a very useful thing to be able to
do and you can certainly export to machines which can build those other operating systems.
Many developers choose to maintain a local branch of FreeBSD to test patches or build a custom distribution. This can be done with svn or
another source code management system (git, mercurial, Perforce, BitKeeper) with its own repository.
UPDATING VIA SVN
By using a cron(8) job to maintain an updated svn repository, the source tree can be updated at any time as follows:
(on the main development server)
cd /usr
svn update src doc ports
It is that simple, and since you are exporting the whole lot to your clients, your clients have immediate visibility into the updated source.
This is a good time to also remind you that most of the svn(1) operations you do will be done as ``root''.
SEE ALSO
crontab(1), crontab(5), make.conf(5), build(7), firewall(7), release(7), tuning(7), diskless(8)
HISTORY
The development manual page was originally written by Matthew Dillon <dillon@FreeBSD.org> and first appeared in FreeBSD 5.0, December 2002.
It was since extensively modified by Eitan Adler <eadler@FreeBSD.org> to reflect the repository conversion from cvs(1) to svn(1).
BSD
March 7, 2013 BSD