.\" $Id: mpull.man,v 1.13 2012/04/02 17:25:01 ksb Exp $
.\" by KS Braunsdorf
.\" $Compile: Display%h
.\" $Display: groff -tbl -Tascii -man %f |${PAGER:-less}
.\" $Display(*): groff -tbl -T%s -man %f
.\" $Install: %b -mDeinstall %o %f && cp %f $DESTDIR/usr/local/man/man8/mpull.8
.\" $Deinstall: ${rm-rm} -f $DESTDIR/usr/local/man/[cm]a[nt][18]/mpull.8*
.TH MPULL 8 LOCAL
.SH NAME
mpull - locally update master source products via rsync and mmsrc
.SH SYNOPSIS
.ds PN "mpull
\fI\*(PN\fP [\fIrsync-opts\fP] \fImsrc-dir\fP [\fImmsrc-opts\fP] [\fIutility\fP]
.br
\fI\*(PN\fP \fB\-h\fP
.br
\fI\*(PN\fP \fB\-V\fP

.SH COMPLETE
\fI\*(PN\fP [\fIrsync-opts\fP] \fImsrc-dir\fP [\fB\-lsz\fP] [\fB\-B\fP\~\fImacros\fP] [\fB\-C\fP\~\fIconfigs\fP] [\fB\-D\fP\~\fIname[=value]\fP] [\fB\-d\fP\~\fIflags\fP] [\fB\-E\fP\~\fIcompares\fP] [\fB\-f\fP\~\fImakefile\fP] [\fB\-G\fP\~\fIguard\fP] [\fB\-I\fP\~\fIdirname\fP] [\fB\-j\fP\~\fIm4prep\fP] [\fB\-k\fP\~\fIkey\fP] [\fB\-m\fP\~\fIprereq\fP] [\fB\-o\fP\~\fIattributes\fP] [\fB\-U\fP\~\fIname\fP] [\fB\-u\fP\~\fIlogin\fP] [\fB\-X\fP\~\fIex-configs\fP] [\fB\-y\fP\~\fIyoke\fP] [\fB\-Y\fP\~\fItop\fP] [\fB\-Z\fP\~\fIzero-config\fP] [\fIutility\fP]

.SH DESCRIPTION

At very large scale it is sometimes useful to pull updates to machines
(virtual instances) rather than push them from a central service.
This program uses \fBrsync\fP(1) to fetch an ephemeral copy of the
current master source for a single directory, uses \fBmmsrc\fP(8) to
update a local platform cache, then executes an update \fIutility\fP.

.P
The default configuration file is \*(lqauto.cf\*(rq, since \fI\*(PN\fP
uses \fBmmsrc\fP to do the local update, but any other may be specified
in \fImmsrc-opts\fP to force a better choice (under \fB\-C\fP).
.P
All the \fBmmsrc\fP rules apply to the local update except one.
A local source cache is updated with the shadow sources, so
a \fB\-y\fP option is synthesized to set \fBINTO\fP.  The value
is based on the \fImsrc-dir\fP presented prefixed with the
local cache (see \fBMPULL_SRC\fP below).
.P
The actual compilation or installation of a tool is always done
from that local shadow copy of the master source.  So the current
working directory of \fI\*(PN\fP does not have to contain a
recipe file.
.P
The default \fIutility\fP is \fBmake\fP, because that's \fBmmsrc\fP's
default.  Use special \fIutility\fP \*(lq:\*(rq to just build the
shadow copy.
.P
Any leading words that start with a dash (\fB\-\fP) are taken to be
options to \fBrsync\fP.  The most common use for this is to exclude
any revision control subdirectories (viz. \*(lq--exclude=RCS\*(rq).
.P
When shadow copy must be configured on the master source host,
enable \fBmsrcmux\fP(7l) on that host then use \fBmuxcat\fP
as the client on the local host. See \fBmuxcat\fP(1l).

.SH OPTIONS
If the program is called as \fI\*(PN\fP then no options are forced.
See \fBmmsrc\fP for option descriptions.
.TP
.nf
\fB\-d\fP \fBX\fP
.fi
As in \fBmmsrc\fP we show the commands we are using to pull data
and launch commands.  Not every command is reflected in the output.
.TP
.nf
\fB\-h\fP
.fi
Output the standard help list, this is largely cloned from \fBmmsrc\fP.
.TP
.nf
\fB\-V\fP
.fi
Show only version information.
.TP
.nf
\fB\-P\fP\fIjobs\fP
.fi
Due to limits in the shell's option parser this option is excluded.
.TP
.nf
\fImmsrc-opts\fP
.fi
The other options available under \fBmmsrc\fP(8) are passed as given.
These are inspected for their values: \fB\-C\fP, \fB\-X\fP, \fB\-Z\fP,
and \fB\-D\fP.


.SH ENVIRONMENT

The environment variables \fBHOME\fP, \fBSHELL\fP, \fBTMPDIR\fP, and
\fBPATH\fP are consulted for their traditional purposes.
As well as \fBHXMD_LIB\fP, which is used as \fBhxmd\fP does to
set an explicit search list for command-line configuration files.
.P
These variable are installed in the environment so recursive calls
to \FI\*(PN\fP may use the same configuration.
.TP
.nf
\fBMPULL_CONFIG\fP
.fi
The list configuration files provided in a format \fBhxmd\fP accepts.
The default list is \*(lq-Cauto.cf\*(rq, but explicit \fB\-C\fP options
replace that guess.
.TP
.nf
\fBMPULL_HOST\fP
.fi
The name for this host in the above configuration files.
The value is copied from any \fB-DHOST=\fP options.  It defaults
to \*(lqlocalhost\*(rq, as in \fBmmsrc\fP.
.TP
.nf
\fBMPULL_FROM\fP
.fi
The location of the master source repository service.  This must provide
either an anonymous \fBrsync\fP service, or \fBrsync\fP over \fBssh\fP.
This is copied from a host attribute with the same name, if none is
present in the environment.  The format for \fBrsync\fP is:
.nf
	\fIhost\fP[\fB/\fP\fIport\fP]\fB::\fP[\fImodule\fP]
.fi
Where \fIport\fP defaults to 873 and both \fImodule\fP and
\fIhost\fP default to \*(lqmsrc\*(lq.
The port specification is only required when you can't start \fBrsync\fP on
its canonical privileged port.  In that case I usually pick 18873.
For example
.nf
	MPULL_FROM="ripley/18873::/opt/npcguild/msrc"
.fi
.sp
For \fBrsync\fP over \fBssh\fP used 1 colon (\fB:\fP) rather than 2,
and replace the module with the path to the root of the master source:
.nf
	\fIhost\fP[\fB/\fP\fIport\fP]\fB:\fP[\fIpath\fP]
.fi
For example
.nf
	MPULL_FROM="lv426:/opt/npcguild/msrc"
.fi
.TP
.nf
\fBMPULL_SRC\fP
.fi
The path to this hosts shadow copy of the master source.
This is copied from a host attribute with the same name, if none is
present in the environment.  Mortal logins usually  can't write in
\fB/usr/src\fP, so set this to someplace you can write.  For example:
.nf
	MPULL_SRC="$HOME/src"
.fi
.TP
.nf
\fBMPULL_RSYNC\fP
.fi
The name of the \fBrsync\fP program.  Defaults to
\*(lqrsync\*(rq.
.TP
.nf
\fBMPULL_RSOPTS\fP
.fi
Any extra \fBrsync\fP options required for the \fBrsync\fP command line.
The options we provide (see \fB\-d X\fP output) look like:
.nf
	\fIrsync\fP \fB\-arSH \-\-port=\fP\fPport\fP \fB$MPULL_FROM\fP/\fIPkg\fP/ \fItempdir\fP/\fIPkg\fP
.fi
The \fB$MPULL_RSOPTS\fP value is inserted after the \fIport\fP specification.
.P
Be sure to \fBexport\fP these so they are visible to \fI\*(PN\fP.


.SH EXAMPLES
.TP
.nf
\fI\*(PN\fP \-V
.fi
The standard version information.
.TP
.nf
\fI\*(PN\fP local/bin/glob \-Clocal.cf make install clean
.fi
Construct a platform cache for the \fBglob\fP program on this host from
the attributes in \fBlocal.cf\fP for \*(lqlocalhost\*(rq.
.TP
.nf
\fI\*(PN\fP local/bin/oue  \-DHOST=sulaco  mk \-mInstall \*.man
.fi
Install the manual page for \fBoue\fP as if this host were named
\*(lqsulaco\*(rq, which must be defined in \*(auto.cf\*(rq.
.TP
.nf
MPULL_SRC=$HOME/src \fI\*(PN\fP local/bin/oue  \-Csite.cf :
.fi
Force a shadow copy into your home directory under src/local/bin/oue,
and do nothing with it.  Note that the host \*(lqlocalhost\*(rq must
be defined in that configuration file.
.TP
.nf
MPULL_FROM=nostromo:/usr/msrc  \fI\*(PN\fP  local/bin/mk \-dX pwd
.fi
Use \fBrsync\fP over \fBssh\fP to grab \fBmk\fP from the cache on
nostromo, and watch the commands as they are run.
.TP
.nf
\fI\*(PN\fP \-vv \-\-exclude=RCS local/bin/glob \-dX ls \e; pwd
.fi
Force \fBrsync\fP to skip any RCS directories and show what it is
downloading to fetch the source to \fBglob\fP.
Report the actions taken by \fI\*(PN\fP and \fBmmsrc\fP to
\fBls\fP(1) the shadow local directory after it is updated.
Also run \fBpwd\fP to display the location of the shadow directory
(before it is removed).
.TP
.nf
MPULL_FROM=msrc::Pkgs \fI\*(PN\fP msrc_base \-dX ls
.fi
Deploy the package directory for msrc_base to the local source tree.
This really doesn't do what you want, because
the gather phase didn't happen from the level3 recipe (it was taken to be
a level2 recipe, which is it not).  So what ends up on the local machine
doesn't build.  A completely different \fBtcpmux\fP service would be
needed to pull package sources to a client.

.SH BUGS
It would be much better if \fI\*(PN\fP could parse \fBrsync\fP options on the
command-line before the \fImsrc-dir\fP parameter.  But the \fBrsync\fP option
parser is really hard to emulate in a script, since it allows
arbitrary aliases.  So we assume the options end with
the first word that doesn't start with a dash (\fB\-\fP).  Use
assignments (\fB\-\-exclude=\fP\fIparam\fP) for or quoted spaces to
force option specifications through.
.\" rsync option aliases are common at many sites, making it impossible (ksb)
.P
\fIMpull\fP can't pass \fB\-h\fP or \fB\-V\fP to rsync as the first option.
Which doesn't really limit the use of \fBrsync\fP at all.
.P
The rule \fBmmsrc\fP uses as a fall-back (the first host defined in
the first \fB\-C\fP configuration file) doesn't work for the 2 calls to
\fBhxmd\fP \fI\*(PN\fP uses to extract \fBMPULL_FROM\fP and
\fBMPULL_LOCALROOT\fP.  Work around by setting these in the environment to
prevent the calls to \fBhxmd\fP(8).
.P
It would be clever to allow a persistent local copy of the master
source tree: because \fI\*(PN\fP downloads the configuration management
meta-information with the current files every time.
This uses more network bandwidth than it would if it those files
(and subdirs like RCS or CVS) were persisted locally.
If you want to do that you don't need the script wrapper, just use \fBrsync\fP.
If you want to skip the meta information put in an exclude option in
\fB$MPULL_RSOPTS\fP and you are good to go.
.P
Ironically \fI\*(PN\fP cannot update \fBmmsrc\fP.
Because the source for \fBmmsrc\fP requires peer directories to
execute the master recipe, so it fails to install the shadow copy in
the local source cache.  The best way to update the base tools is to
install the level3 package \fBmsrc_base\fP.  Download the package
source then build it locally.
.P
All the run-time bugs of \fBmmsrc\fP.

.SH AUTHOR
KS Braunsdorf
.br
NonPlayer Character Guild
.br
mmsrc at no-SPAM here ksb dot-here npcguild.org

.SH "SEE ALSO"
.hlm 0
mmsrc(8l), msrcmux(7l), rsync(1), hxmd(8l), msrc(8l), muxcat(1l),
m4(1), environ(8), ssh(1), ksh(1)