EM_AUTOGRIP(1) User Contributed Perl Documentation EM_AUTOGRIP(1)
em_autogrip - create and maintain an Emdebian Grip repository
Syntax: em_autogrip -b PATH [OPTIONS] [COMMAND [PACKAGES ...]]
-b|--base-path PATH: path to the top level grip directory [required]
-p|--package PACKAGES ... : add binary package(s) to the repository
-s|--source PACKAGES ... : add source package(s) to the repository
-t|--testing: only work on testing instead of unstable
--noskipold: reprepro option for newly added packages
--missing: print a list of missing source packages
--build-depends: print a list of missing build dependencies
--britney: print the status of testing migrations
-?|-h|--help|--version: print this help message and exit
-n|--dry-run: check which packages would be processed
-m|--mirror MIRROR: use a different Debian mirror for setup
--filter-name STRING: alternative name for the filter repository
--grip-name STRING: alternative name for the grip repository
--add-new: if a source package is found to be missing,
or outdated in unstable, add it to the list.
The default is to update all the packages so far existing in the filter repository, in all architectures.
After adding binary packages, ensure that em_autogrip is run without any options so that any missing source packages and any other Emdebian
TDebs can be updated.
Although em_autogrip will setup the initial configuration files for the repository, it will not modify any existing files *except* the
pkglist filter that prevents the mirror adding unwanted packages.
The mirror option only has an effect if there is no repository already found at the specified directory.
In particular, em_autogrip will only handle unstable by default. Migrations to testing and stable, even the creation of testing and
stable, are not handled by em_autogrip. em_autogrip defaults to including packages into unstable and will only include packages directly
into testing in "--testing" mode when updating to versions of packages uploaded into Debian via testing-proposed-updates or when catching
up with a new (or stalled) repository. "--testing" mode requires a pre-configured testing configuration in "reprepro".
Public repositories should also use Secure-Apt by adding a value for SignWith: to each distribution in the Grip repository (there is no
point signing the filter repository as it should not be public and is merely a filtered copy of existing, officially signed, repositories).
em_autogrip also updates the locale repository, shared by Emdebian Grip and Emdebian Crush.
Note that em_autogrip will only update the *binary* package(s) specified, even when it includes the full source package. This is down to
how reprepro runs the filtered update - all binary packages expected to be listed in 'dpkg --get-selections' are included and even if a
source package includes another binary, it will not be downloaded in the reprepro update. As em_autogrip does not actually build any
packages from source, unless reprepro downloads the pre-built binary into the filter repository, that binary package will not be available
to em_autogrip. This means that the same source package in Debian may be listed as generating a *smaller* number of binary packages in
Secure Apt and reprepro
The secret key for the GnuPG key specified with SignWith: needs to be in the secret keyring of each user performing repository updates.
To verify the release in update rules, copy /etc/apt/trusted.gpg to ~/.gnupg/trustedkeys.gpg for all users who need to run updates. To add
keys to the list available for "gpgv" use:
C<gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --import keys.gpg>
Problems with the automatic gripping of packages:
1. Source packages need to complement binary packages.
2. Binary packages with the same name as the source package cause both
to be included
3. Some such binary packages cause unwanted dependencies to be added.
4. Some Architecture:all packages are dependencies of packages that only
exist on some architectures, which breaks the edos-debcheck.
An example of 3. is lsb. An example of 4. is acpi-support-base.
The source package lsb is needed to complement lsb-desktop but the lsb binary package is a meta-package for the entire lsb suite which
brings in all of Qt.
acpi-support-base is Architecture: all but depends on acpid which is Architecture: any [i386 amd64] - i.e. acpi-support-base should only
exist on i386 and amd64 but as it is Architecture: all, it gets added to arm, armel, mips, mipsel and powerpc as well - at which point it
has to be removed. There are ongoing discussions about such packages.
Signal:Noise ratio in output
One important point here - reprepro outputs a lot of messages and may include lots of statements about errors and checksum mismatches,
'skipping foo' and 'downgrading bar' from and to the same version. The problem is that the useful information is hidden within all the
noise, so not all reprepro STDERR (or STDOUT) output can be simply ignored. For now, just go by effects. If something is broken, look for
errors that relate specifically to that package but ignore "errors" where everything is fine. Something like that. More work in the
Emdebian::Grip module should isolate duplicate operations and unnecessary work, which in turn, should cut out most of the noise.
In --testing mode, em_autogrip checks for packages that have missing or outdated source packages in unstable and outputs a sample command
that can be run to fill the gap. If --add-new is used, that sample command will be run - it does mean that --add-new requires --testing and
that a second run of --testing without --add-new will be needed. This support is part of grip_cron.sh
In the absence of a quicker way to identify which real package Provides: a virtual dependency, apt-cache showpkg is used against the main
system cache. If this machine is not running Debian unstable, the list may be inaccurate or skip dependencies that are provided by packages
that are only available in unstable (or if running stable, packages which are only in unstable or testing).
If a package fails to build from source in Debian, "em_autogrip" will keep on trying to update it until the same version exists in the
filter repository for all supported architectures.
Equally, manual tinkering with packages in the Grip repository, e.g. adding modified versions for testing, will cause the original Debian
version to keep appearing in the "em_autogrip" updates and reprepro will ignore the built package as long as the modified version is
"em_autogrip" does not handle removals from the archive - these are manual within Debian too. Packages that only exist in stable or
oldstable will confuse "em_autogrip", especially if the old package name is 'Provided' by another package which already exists in Grip.
Adding lots of packages in one run
Sometimes, perhaps when setting up a new mirror, a full list of packages already exists on another site. Copying that pkglist into the new
site will clear that list as the filter repository on the new site is empty. To avoid this problem, create the pkglist you need, then run
the filter update run directly:
reprepro -b /PATH/filter -v update
Now run "em_autogrip" without specifying any packages.
em_autogrip -b /PATH/
Note that "reprepro" needs the path to the filter directory, "em_autogrip" needs the path to the directory above where it can find
./filter/, ./grip/ and ./locale/.
"em_autogrip" will then update the pkglist file with the final contents of the filter repository.
Ubuntu / non-Debian sources/suites
Emdebian Grip is still Debian, so although non-Debian repositories can be supported, the resulting Grip repository still requires a Debian-
like layout. In particular, an 'unstable' suite must exist, even if the codename of that suite is not called 'sid'. Equally, if the
repository is to support britney migrations, a suite called 'testing' must exist.
Remember, suites will change when a Debian stable release is made (i.e. testing points to something else after the release compared to
what it contained before the release). Codenames do not change - squeeze always contains squeeze, even once squeeze is released as stable.
The list of architectures supported by a particular Grip setup cannot be easily changed - a lot of repository updates are needed before new
architectures can be added to the array. Existing architectures can be dropped relatively easily. Sequence is unimportant.
@archlist = qw/i386 amd64 arm armel powerpc mips mipsel/;
recursive edos considered risky
Recursion is still risky so edos is left as a manual step. The problem appears to be that once the repository gets out of step with
Debian, an update must happen before edos can be resolved. Once the repository is up to date, edos can be run, apparently, without
problems. Testing continues to see if simply moving the function lower in the flow resolves the problems.
Copyright and Licence
Copyright (C) 2007-2010 Neil Williams <firstname.lastname@example.org>
This package is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
perl v5.12.3 2011-03-27 EM_AUTOGRIP(1)