You are in a maze of twisty little config files, all alike


I'm currently preparing to create a non-Zope bundle of CalCore and all it's
dependencies. Since these dependencies are mostly Zope 3 packages (interface
and schema) it seems natural to use Zope Corps tool for this: zpkg.



Unfortunately zpkg is a complex tool with cryptic documentation. I
have made packages with zpkg before, and it was hard. This time around, it
was equally hard, because I had forgotten everything. I'm sorry to say this,
but when it's hard to do something, and hard to remember how you did it,
this is a sign of a bad design. Configuring a product to use zpkg is far
from easy or logical, and there are many confusingly similar files, all
named SOMETHING.cfg.

Getting started


First you need to check it out:

  svn co svn://svn.zope.org/repos/main/zpkgtools/tags/zpkg-1.0.0
zpkgtools

or, for the more adventurous:

  svn co svn://svn.zope.org/repos/main/zpkgtools/trunk zpkgtools



The main application is in bin/zpkg. It takes many parameters, and after a
while I found that the best thing to do is to create a configuration file,
and call that with -C. I called mine BUILD.cfg. Zope corp likes to call them
the name of the package, such as Zope.cfg, but for me that is confusing, it
sounds way to similar to the zope.conf file.



The format of the configuration files the same format as the zope.conf
file, that is you use # for comments, and write keyword value
pairs, and create groups with <tag> </tag>.



The first keyword I needed was:

    collect-dependencies  yes

This tells zpkg to include the dependencies of the product that I'm
packaging and include them in the package. That's what the whole thing is
about, if you don't need that you don't need zpkg at all.



Also, I'd like to not have to write the name of the resource I'm packaging
every time I'm packaging, so I include the line

    default-collection calcore



However, this will create a package named calcore-x.x.x.tgz. This is
confusingly similar to the CalCore-x.x.x.tgz packages created by Nuxeos
packaging tool, which includes CalCore only and not dependencies, so I also
add

    release-name CalCore-bundle

to make the file be called CalCore-bundle-x.x.x.tgz instead.



Now I can run zpkg -C BUILD.cfg, and it will try to build the release.
However, it tries to package the calcore resource by default, and has
absolutely no idea how to do that, so you get an error.

Defining resources


Resources are defined in a resource map. This is a list of resource names,
and a location where you can find them. You can either put the list of
resources into the configuration file directly like this:

  <resources>

    calcore src/calcore

  </resources>

or you can put it in a separate file, lets call it RESOURCES.cfg, like
this:

    calcore src/calcore

and just include it in the main file with

    resource-map RESOURCES.cfg



The resource definition for calcore points to a subdirectory in the current
hierarchy. In most cases you have the product directly in the same directory
as the configuration file you are now creating, and then you would define it
up with a dot:

    calcore .

Including the dependencies


At this point, you can now actually make a package. It will however not
include the dependencies, which of course is the whole point of this
excercise. For this, we need to tell zpkg which dependencies the resource
has. This is done with a DEPENDENCIES.cfg file, which simply lists the
resources that this resource needs. The calcore resource was defined up as
being in src/calcore, so that's where the DEPENDENCIES.cfg needs to be. It
looks like this:

  zope.interface

  zope.schema

  zope.i18nmessageid

  iCalendar



Of course, zpkg has absolutely no idea where to find these resources, so we
have to extend the RESOURCES.cfg file:



  zope.interface
svn://svn.zope.org/repos/main/Zope3/tags/ZopeX3-3.0.0/src/zope/interface

  zope.schema
svn://svn.zope.org/repos/main/Zope3/tags/ZopeX3-3.0.0/src/zope/schema

  zope.testing
svn://svn.zope.org/repos/main/Zope3/tags/ZopeX3-3.0.0/src/zope/testing

  zope.i18nmessageid
svn://svn.zope.org/repos/main/Zope3/tags/ZopeX3-3.0.0/src/zope/i18nmessageid

  iCalendar
svn:http://codespeak.net/svn/iCalendar/tag/iCalendar-0.10



Note here that these resources are defined up as svn tags! zpkg will fetch
the files directly from the svn of each product. Very practical. You can
also have a * instead of the tag-name, in which case it will use some kind
of logic to figure out which tag to use (what logic I don't know). Also note
that I included zope.testing above. That's because several of the resources
above define up their own DEPENDENCIES.cfg, and of course, those
dependencies must be defined as well, if they are to be included.
zope.testing is really not needed unless you want to run the unittests, but
I include it anyway for good measure.



For some reason zpkg will complain that the resource "zope" is not defined
and skip it. That's a good thing, because it's would mean that all of Zope3
was included, which is quite silly. Why it wants to include it is beyond me,
none of the resources have it in their DEPENDENCIES.cfg file.

Adding files to the distribution root


I want to have some files in the distribution root. A README.txt that tells
you what the package is and how to install it should be in every package. I
also want the HISTORY file to be included (but named HISTORY.txt), the GPL
license, with the name COPYING.txt and the whole doc directory. To add them
to the root I need to create a PACKAGE.cfg in the resource directory, that
includes the follwing lines:

<load>

  HISTORY.txt
svn:http://svn.nuxeo.org/pub/CalCore/tags/*/HISTORY

  COPYING.txt
svn:http://svn.nuxeo.org/pub/CalCore/tags/*/COPYING.txt

  README.txt 
svn:http://svn.nuxeo.org/pub/CalCore/tags/*/README.txt

  doc        
svn:http://svn.nuxeo.org/pub/CalCore/tags/*/doc

</load>



<distribution>

  HISTORY.txt

  COPYING.txt

  README.txt

  doc

</distribution>



The first part adds the files into the resource directory. It isn't
possible in zpkg to include files that are above the resource root, so I
instead load them from svn. This can also be used to have the latest version
of the licensing in a separate package and thereby automatically include the
correct version every time you package, and other cool things.



The distribution tag also tells zpkg to include these files in the
distribution root. (They will end up both there and in the resource
directory).

Nice extras


The PUBLICATION.cfg is not needed, but it's a place to put in meta data
about the product, which is a good thing:

  Name: calcore

  Summary: Python calendaring

  Home-page: /

  Author: Martijn Faassen, Lennart Regebro, Nuxeo SARL.

  Author-email: [email protected]

  Licence: GPL 2

  Description: A python package for making personal and group
calendars.



Note that PUBLICATION.cfg uses an RFC-822 type format, that is, it uses
keyword-value pairs with a colon after the keyword, while all the other
config files does not have a colon.

Thats it!


There is a lot of cfg files involved and it can be very confusing, and I'm
in no way near understanding all the details. But at least, after some trial
and error, I succeeded in making a tgz that includes what I want.

(Post originally written by Lennart Regebro on the old Nuxeo blogs.)