Discussion:
[Zope3-dev] faulty releases and pypi access
Christian Theune
2007-09-26 09:16:44 UTC
Permalink
We have another case of faulty released eggs.

I reviewed the first that popped up for me, which was
zope.app.publication:

- someone uploaded a stable 3.4.1 egg, but this is just a snapshot
from the trunk, there is no tag for this release.

- the egg does not contain data files correctly. unpacking the egg
misses the zcml files.

- the 3.4.0 egg is invalid as well as the zcml files are missing too

I did:

- create a tag for the brown-bagged 3.4.1 release
- removed the uploaded files from the cheeseshop as I don't have newer
replacements immediately available.

The whole list of things that might be relevant here is:

- zope.securitypolicy
- zope.session, zope.app.session
- zope.app.authentication
- zope.app.i18n
- zope.i18nmessageid
- zope.app.applicationcontrol
- zope.app.appsetup

I'll go through those now and try to fix it up.

This is IMHO a good example why we shouldn't go for 'everyone can make a
release'.

Christian
--
gocept gmbh & co. kg - forsterstrasse 29 - 06112 halle/saale - germany
www.gocept.com - ***@gocept.com - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Dies ist ein digital signierter Nachrichtenteil
Url : http://mail.zope.org/pipermail/zope3-dev/attachments/20070926/01e89bf7/attachment.bin
Christian Theune
2007-09-26 09:49:57 UTC
Permalink
Hey,

here is an update.

The issue is that the eggs were released as ZIP files and for some
reason those don't work correctly with the data files.

I can reproduce the problem by creating the packages myself as ZIP files
(doesn't work) and then as tar files (does work).

My proposal for what to do (Roger, maybe you can do that?):

- Remove the broken files.

- Create tags for the wrong trunk releases

- Create a new release and tag in tar format.

I'll show what I mean by releasing a fix for zope.app.i18n.
Post by Christian Theune
We have another case of faulty released eggs.
- zope.securitypolicy
Actually this is zope.app.securitypolicy.
Post by Christian Theune
- zope.session, zope.app.session
- zope.app.authentication
- zope.app.i18n
- zope.i18nmessageid
- zope.app.applicationcontrol
- zope.app.appsetup
--
gocept gmbh & co. kg - forsterstrasse 29 - 06112 halle/saale - germany
www.gocept.com - ***@gocept.com - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Dies ist ein digital signierter Nachrichtenteil
Url : http://mail.zope.org/pipermail/zope3-dev/attachments/20070926/d927854d/attachment-0001.bin
Christian Theune
2007-09-26 10:02:45 UTC
Permalink
Post by Christian Theune
Hey,
here is an update.
The issue is that the eggs were released as ZIP files and for some
reason those don't work correctly with the data files.
I can reproduce the problem by creating the packages myself as ZIP files
(doesn't work) and then as tar files (does work).
- Remove the broken files.
- Create tags for the wrong trunk releases
- Create a new release and tag in tar format.
I'll show what I mean by releasing a fix for zope.app.i18n.
Hmm. While doing that I also noticed that we were at 3.4.0a1 yesterday
evening. The stable release was made from that without making a
maintenance branch and bumping the trunk to 3.5.

Christian
--
gocept gmbh & co. kg - forsterstrasse 29 - 06112 halle/saale - germany
www.gocept.com - ***@gocept.com - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Dies ist ein digital signierter Nachrichtenteil
Url : http://mail.zope.org/pipermail/zope3-dev/attachments/20070926/9f37ce49/attachment.bin
Baiju M
2007-09-26 10:09:34 UTC
Permalink
Post by Christian Theune
Post by Christian Theune
Hey,
here is an update.
The issue is that the eggs were released as ZIP files and for some
reason those don't work correctly with the data files.
I can reproduce the problem by creating the packages myself as ZIP files
(doesn't work) and then as tar files (does work).
- Remove the broken files.
- Create tags for the wrong trunk releases
- Create a new release and tag in tar format.
I'll show what I mean by releasing a fix for zope.app.i18n.
Hmm. While doing that I also noticed that we were at 3.4.0a1 yesterday
evening. The stable release was made from that without making a
maintenance branch and bumping the trunk to 3.5.
We decided not to bump minor release in trunk recently while making
these final release, is it ?

(But I have already bumped some of them earlier, but we can change it
unless a 3.5 release has come out)

Regards,
Baiju M
Christian Theune
2007-09-26 10:15:04 UTC
Permalink
Post by Baiju M
We decided not to bump minor release in trunk recently while making
these final release, is it ?
I tried looking for that but didn't find that decision, so I thought
we're doing maintenance branches.
--
gocept gmbh & co. kg - forsterstrasse 29 - 06112 halle/saale - germany
www.gocept.com - ***@gocept.com - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Dies ist ein digital signierter Nachrichtenteil
Url : http://mail.zope.org/pipermail/zope3-dev/attachments/20070926/d3e596f1/attachment.bin
Stephan Richter
2007-09-26 14:51:45 UTC
Permalink
Post by Christian Theune
Hmm. While doing that I also noticed that we were at 3.4.0a1 yesterday
evening. The stable release was made from that without making a
maintenance branch and bumping the trunk to 3.5.
There is conflicting information here. :-) Some people say we need branches,
others say we don't. And where is an agreed-upon document that you have to
list the next version in the setup.py file after the release? Because I
disagree with that, since you cannot know the next version.

Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
Jim Fulton
2007-09-26 14:54:28 UTC
Permalink
Post by Stephan Richter
Post by Christian Theune
Hmm. While doing that I also noticed that we were at 3.4.0a1
yesterday
evening. The stable release was made from that without making a
maintenance branch and bumping the trunk to 3.5.
There is conflicting information here. :-) Some people say we need branches,
others say we don't. And where is an agreed-upon document that you have to
list the next version in the setup.py file after the release?
Because I
disagree with that, since you cannot know the next version.
Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
_______________________________________________
Zope3-dev mailing list
Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com
--
Jim Fulton
Zope Corporation
Jim Fulton
2007-09-26 14:55:02 UTC
Permalink
Sorry, I decided not to reply and hit the wrong button in my mailer. :)
Post by Jim Fulton
Post by Stephan Richter
Post by Christian Theune
Hmm. While doing that I also noticed that we were at 3.4.0a1
yesterday
evening. The stable release was made from that without making a
maintenance branch and bumping the trunk to 3.5.
There is conflicting information here. :-) Some people say we need branches,
others say we don't. And where is an agreed-upon document that you have to
list the next version in the setup.py file after the release?
Because I
disagree with that, since you cannot know the next version.
Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
_______________________________________________
Zope3-dev mailing list
Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com
--
Jim Fulton
Zope Corporation
_______________________________________________
Zope3-dev mailing list
Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com
--
Jim Fulton
Zope Corporation
Marius Gedminas
2007-09-26 15:53:05 UTC
Permalink
Post by Jim Fulton
Sorry, I decided not to reply and hit the wrong button in my mailer. :)
You were just applying "explicit is better than implicit" to email
replies. :-)

Marius Gedminas
--
Just to be extra clear about this: yes, it is morally wrong for us to have
written RetchMail, and it is morally wrong for you to use it. But try it, it's
really fast!
-- http://open.nit.ca/wiki/index.php?page=RetchMail
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mail.zope.org/pipermail/zope3-dev/attachments/20070926/c53d793e/attachment.bin
Martijn Faassen
2007-09-26 15:21:12 UTC
Permalink
Post by Stephan Richter
Post by Christian Theune
Hmm. While doing that I also noticed that we were at 3.4.0a1 yesterday
evening. The stable release was made from that without making a
maintenance branch and bumping the trunk to 3.5.
There is conflicting information here. :-) Some people say we need branches,
others say we don't.
I think it'd be good to have a tag in any case. A tag can always be
converted into a branch later.
Post by Stephan Richter
And where is an agreed-upon document that you have to
list the next version in the setup.py file after the release? Because I
disagree with that, since you cannot know the next version.
I disagree with too, for the same reason.

What about using CHANGES.txt, which we should be maintaining anyway?

Before making a release, change CHANGES.txt, and convert the section:

unreleased
----------

Fixed bugs blah blah.

To:

3.5.1 (2007-09-26)
------------------

Fixed bugs blah blah.

That is, come up with a release number, a release date, update setup.py
with the release number, and tag it.

Then, after the release, add in a new section:

unreleased
----------

This way checking CHANGES.txt should tell you what's going on with
releases. If someone forgot to do the last step, you see immediately
something is wrong, as you want to add your change to the 'unreleased'
section but there's nothing there yet. You can then reconstruct what
happened from the cheeseshop or the tags, and put in a new 'unreleased'
section.

Regards,

Martijn
Stephan Richter
2007-09-26 15:33:54 UTC
Permalink
Post by Martijn Faassen
This way checking CHANGES.txt should tell you what's going on with
releases. If someone forgot to do the last step, you see immediately
something is wrong, as you want to add your change to the 'unreleased'
section but there's nothing there yet. You can then reconstruct what
happened from the cheeseshop or the tags, and put in a new 'unreleased'
section.
Good point; of course I agree. :-) A well-maintained CHANGES.txt file is worth
a lot.

Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
Philipp von Weitershausen
2007-09-26 15:36:45 UTC
Permalink
Post by Martijn Faassen
And where is an agreed-upon document that you have to list the next
version in the setup.py file after the release? Because I disagree
with that, since you cannot know the next version.
I disagree with too, for the same reason.
I'm not saying you should foresee the future, but I think you can very
well know the next *anticipated* version. Doing so has some benefits:

* We need *a* convention, otherwise people will either release an egg
with the same version twice (this happened to Jim with the ZODB
yesterday, due to the lack of a convention) or skip a release number
because they both increment after and before a release (happened to
Roger yesterday; it's harmless but confusing).

* It is a convention that setuptools encourages (third para in
http://peak.telecommunity.com/DevCenter/setuptools#managing-continuous-releases-using-subversion)

* On the trunk, you'll know what release line it is for. Is it 3.4.x? Or
3.5.x?

* Even more important, when the trunk was used for 3.4.x and now you
want to add a feature and want to bump it to 3.5.x, you explicitly see
this in setup.py. And then you know that you have to create a 3.4 branch
before you do this.
Post by Martijn Faassen
What about using CHANGES.txt, which we should be maintaining anyway?
[snip]
These are very good points. My guide [1] already recommends this practice.

[1]
http://svn.zope.org/*checkout*/Sandbox/philikon/foundation/maintaining-software.txt
--
http://worldcookery.com -- Professional Zope documentation and training
Martijn Faassen
2007-09-26 15:49:17 UTC
Permalink
Hey,
[snip]
Post by Philipp von Weitershausen
Post by Martijn Faassen
What about using CHANGES.txt, which we should be maintaining anyway?
[snip]
These are very good points. My guide [1] already recommends this practice.
[1]
http://svn.zope.org/*checkout*/Sandbox/philikon/foundation/maintaining-software.txt
I can't find my description of this practice in the guide. In
particular, the practice of using unreleased entries.

Note that if you already anticipate the version number, you can instead do this:

1.3.7 (unreleased)
---------------------------

And then change the unreleased to the date once you're releasing.

The method I describe above tries to make sure that some amount of
reconstruction can be done in case of mistakes.

Regards,

Martijn
Martijn Faassen
2007-09-26 15:53:21 UTC
Permalink
Post by Martijn Faassen
Hey,
[snip]
Post by Philipp von Weitershausen
Post by Martijn Faassen
What about using CHANGES.txt, which we should be maintaining anyway?
[snip]
These are very good points. My guide [1] already recommends this practice.
[1]
http://svn.zope.org/*checkout*/Sandbox/philikon/foundation/maintaining-software.txt
I can't find my description of this practice in the guide. In
particular, the practice of using unreleased entries.
Now I did. It was in a section that didn't mention CHANGES.txt (other
areas do), so I didn't find it. So forget about my complaint, I'm
sorry.

Regards,

Martijn
Philipp von Weitershausen
2007-09-26 15:57:20 UTC
Permalink
Post by Martijn Faassen
[snip]
Post by Philipp von Weitershausen
Post by Martijn Faassen
What about using CHANGES.txt, which we should be maintaining anyway?
[snip]
These are very good points. My guide [1] already recommends this practice.
[1]
http://svn.zope.org/*checkout*/Sandbox/philikon/foundation/
maintaining-software.txt
I can't find my description of this practice in the guide. In
particular, the practice of using unreleased entries.
It's way down at the bottom, under the "Releasing software" section.
Post by Martijn Faassen
1.3.7 (unreleased)
---------------------------
And then change the unreleased to the date once you're releasing.
That's exactly what the guide recommends.
Jim Fulton
2007-09-26 15:40:29 UTC
Permalink
Post by Martijn Faassen
Post by Stephan Richter
Post by Christian Theune
Hmm. While doing that I also noticed that we were at 3.4.0a1
yesterday
evening. The stable release was made from that without making a
maintenance branch and bumping the trunk to 3.5.
There is conflicting information here. :-) Some people say we need
branches, others say we don't.
I think it'd be good to have a tag in any case. A tag can always be
converted into a branch later.
Yup. I don't think there is disagreement about tags and we don't
really need agreement on branches.
Post by Martijn Faassen
Post by Stephan Richter
And where is an agreed-upon document that you have to list the
next version in the setup.py file after the release? Because I
disagree with that, since you cannot know the next version.
I disagree with too, for the same reason.
What about using CHANGES.txt, which we should be maintaining anyway?
I agree with a change log. CHANGES.txt is difficult to get included
in distributions. Having one requires a more complex setup.py. I'd
rather recommend including a change log in README.txt.
Post by Martijn Faassen
unreleased
----------
Fixed bugs blah blah.
3.5.1 (2007-09-26)
------------------
Fixed bugs blah blah.
That is, come up with a release number, a release date, update
setup.py with the release number, and tag it.
unreleased
----------
This way checking CHANGES.txt should tell you what's going on with
releases. If someone forgot to do the last step, you see
immediately something is wrong, as you want to add your change to
the 'unreleased' section but there's nothing there yet. You can
then reconstruct what happened from the cheeseshop or the tags, and
put in a new 'unreleased' section.
I agree except for the location of the change log.

Jim

P.S. ZODB's NEWS.txt/History.txt aren't workable for me and I plan to
switch ZODB to using a change log in it's readme. Unfortunately,
it's setup is so complex It will take me a largish chunk of time to
unwind this.

--
Jim Fulton
Zope Corporation
Stephan Richter
2007-09-26 15:42:53 UTC
Permalink
Post by Martijn Faassen
What about using CHANGES.txt, which we should be maintaining anyway?
I agree with a change log. ?CHANGES.txt is difficult to get included ?
in distributions. ?Having one requires a more complex setup.py. ?I'd ?
rather recommend including a change log in README.txt.
-1. I don't like that. We include CHANGES.txt via the long description; that's
good enough for me.

Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
Jim Fulton
2007-09-26 15:54:09 UTC
Permalink
Post by Stephan Richter
Post by Jim Fulton
Post by Martijn Faassen
What about using CHANGES.txt, which we should be maintaining anyway?
I agree with a change log. CHANGES.txt is difficult to get included
in distributions. Having one requires a more complex setup.py. I'd
rather recommend including a change log in README.txt.
-1. I don't like that. We include CHANGES.txt via the long
description; that's
good enough for me.
Do you think that the change log should be included in the distribution?

Jim

--
Jim Fulton
Zope Corporation
Stephan Richter
2007-09-26 15:58:52 UTC
Permalink
Post by Jim Fulton
Post by Martijn Faassen
What about using CHANGES.txt, which we should be maintaining anyway?
I agree with a change log. ?CHANGES.txt is difficult to get included
in distributions. ?Having one requires a more complex setup.py. ?I'd
rather recommend including a change log in README.txt.
-1. I don't like that. We include CHANGES.txt via the long ?
description; that's
good enough for me.
Do you think that the change log should be included in the distribution?
I am indifferent about it. I cannot see a strong argument for having it in the
release. Because people read the changes before deciding to download the
package. On the other hand, package managers for Linux distributions will
hate us, probably. ;-)

Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
Tres Seaver
2007-09-26 16:16:57 UTC
Permalink
Post by Jim Fulton
Post by Stephan Richter
Post by Jim Fulton
Post by Martijn Faassen
What about using CHANGES.txt, which we should be maintaining anyway?
I agree with a change log. CHANGES.txt is difficult to get included
in distributions. Having one requires a more complex setup.py. I'd
rather recommend including a change log in README.txt.
-1. I don't like that. We include CHANGES.txt via the long
description; that's
good enough for me.
Do you think that the change log should be included in the distribution?
+10. The cheeseshop metadata is not where I would expect to look for a
changelog; having it readable there is only helpful *before* I've
downloaded.

FWIW, I think that both README.txt and CHANGES.txt should be "package
data", so that they are discoverable after installing an egg. The
top-level README.txt should be boilerplate, and point to those files
(the setup.py process can then stitch them all todgether).


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 ***@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
Marius Gedminas
2007-09-26 16:42:47 UTC
Permalink
Post by Tres Seaver
Post by Jim Fulton
Post by Stephan Richter
Post by Jim Fulton
Post by Martijn Faassen
What about using CHANGES.txt, which we should be maintaining anyway?
I agree with a change log. CHANGES.txt is difficult to get included
in distributions. Having one requires a more complex setup.py. I'd
rather recommend including a change log in README.txt.
-1. I don't like that. We include CHANGES.txt via the long
description; that's
good enough for me.
Do you think that the change log should be included in the distribution?
+10. The cheeseshop metadata is not where I would expect to look for a
changelog; having it readable there is only helpful *before* I've
downloaded.
FWIW, I think that both README.txt and CHANGES.txt should be "package
data", so that they are discoverable after installing an egg. The
top-level README.txt should be boilerplate, and point to those files
(the setup.py process can then stitch them all todgether).
Example from Debian: all packages have a /usr/share/doc/$packagename
with the changelog and readme in there, and this is a Very Good Thing.

+1 for CHANGES.txt (or NEWS.txt) in a separate file from README.txt

+1 for the latest changelog entries visible on the cheeseshop page (see
an announcement, go to cheeseshop, see whether you want to upgrade or
not)

+1 for README.txt and CHANGES.txt available to you after you install an
egg.

Marius Gedminas
--
Despite all appearances, your boss is a thinking, feeling, human being.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mail.zope.org/pipermail/zope3-dev/attachments/20070926/6e491b81/attachment.bin
Martijn Faassen
2007-09-26 17:03:06 UTC
Permalink
Hey,

Marius Gedminas wrote:
[snip]
Post by Marius Gedminas
+1 for CHANGES.txt (or NEWS.txt) in a separate file from README.txt
+1 for the latest changelog entries visible on the cheeseshop page (see
an announcement, go to cheeseshop, see whether you want to upgrade or
not)
+1 for README.txt and CHANGES.txt available to you after you install an
egg.
+1 for those too.

What does one need to tell setup.py to make sure CHANGES.txt is
available? I understand it isn't by default, then? Hm, it does appear to
be there by default. I checked grok 0.10's tgz and it's there, and we
didn't do anything special.

I don't understand Jim's objections to a separate CHANGES.txt then. Is
it just because it's more work to include the changes on the cheeseshop
homepage if you separate out the text over multiple files?

Regards,

Martijn
Fred Drake
2007-09-26 18:18:27 UTC
Permalink
Post by Martijn Faassen
What does one need to tell setup.py to make sure CHANGES.txt is
available? I understand it isn't by default, then? Hm, it does appear to
be there by default. I checked grok 0.10's tgz and it's there, and we
didn't do anything special.
Do you mean available in the unpacked sdist, or as part of the
installation? I don't think any of README, README.txt, CHANGES,
CHANGES.txt (from the root of the distribution, not from inside the
package) are actually installed. If they are, I'd love to know where.


-Fred
--
Fred L. Drake, Jr. <fdrake at gmail.com>
"Chaos is the score upon which reality is written." --Henry Miller
Martijn Faassen
2007-09-26 18:25:27 UTC
Permalink
Hey,
Post by Fred Drake
Post by Martijn Faassen
What does one need to tell setup.py to make sure CHANGES.txt is
available? I understand it isn't by default, then? Hm, it does appear to
be there by default. I checked grok 0.10's tgz and it's there, and we
didn't do anything special.
Do you mean available in the unpacked sdist, or as part of the
installation? I don't think any of README, README.txt, CHANGES,
CHANGES.txt (from the root of the distribution, not from inside the
package) are actually installed. If they are, I'd love to know where.
I'm just talking about the sdist. Jim gave me the impression that this
was somehow difficult, but perhaps he meant something else.

Regards,

Martijn
Jim Fulton
2007-09-26 18:28:21 UTC
Permalink
Post by Fred Drake
Post by Martijn Faassen
What does one need to tell setup.py to make sure CHANGES.txt is
available? I understand it isn't by default, then? Hm, it does appear to
be there by default. I checked grok 0.10's tgz and it's there, and we
didn't do anything special.
Do you mean available in the unpacked sdist, or as part of the
installation? I don't think any of README, README.txt, CHANGES,
CHANGES.txt (from the root of the distribution, not from inside the
package) are actually installed. If they are, I'd love to know where.
By default READM(.txt) is installed in a source distribution.
Anything else in the root (aside from setup.py of course and source
files themselves) aren't without extra setup chants. (These chants
can be figured out with some effort. I never remember them, so, if I
want to do something like this, I have to figure them out again or
try to find an example with them.)

If we are going to have a change log, which we should, I would prefer
it to be included in source distributions. Including a file other
that README in the root requires extra effort that I don't want to
require -- writing setup.py files is hard enough as it is.

I'm not crazy about Tres' idea of putting these files in the Python
packages themselves, but it does have the advantage that it causes
the files to be included in eggs as well as source distributions.

Jim

--
Jim Fulton
Zope Corporation
Philipp von Weitershausen
2007-09-26 20:35:33 UTC
Permalink
Post by Fred Drake
Post by Martijn Faassen
What does one need to tell setup.py to make sure CHANGES.txt is
available? I understand it isn't by default, then? Hm, it does appear to
be there by default. I checked grok 0.10's tgz and it's there, and we
didn't do anything special.
Do you mean available in the unpacked sdist, or as part of the
installation? I don't think any of README, README.txt, CHANGES,
CHANGES.txt (from the root of the distribution, not from inside the
package) are actually installed. If they are, I'd love to know where.
By default READM(.txt) is installed in a source distribution. Anything
else in the root (aside from setup.py of course and source files
themselves) aren't without extra setup chants. (These chants can be
figured out with some effort. I never remember them, so, if I want to
do something like this, I have to figure them out again or try to find
an example with them.)
Actually, no package data is ever included unless you're either

* working from an svn checkout, in which case setuptools will use the
list of which files are in svn and which aren't as a hint of what to
include and what not

* creating a MANIFEST.in file to tell setuptools what to include explicitly.
--
http://worldcookery.com -- Professional Zope documentation and training
Jim Fulton
2007-09-26 20:46:35 UTC
Permalink
Post by Philipp von Weitershausen
Post by Jim Fulton
Post by Fred Drake
Post by Martijn Faassen
What does one need to tell setup.py to make sure CHANGES.txt is
available? I understand it isn't by default, then? Hm, it does appear to
be there by default. I checked grok 0.10's tgz and it's there, and we
didn't do anything special.
Do you mean available in the unpacked sdist, or as part of the
installation? I don't think any of README, README.txt, CHANGES,
CHANGES.txt (from the root of the distribution, not from inside the
package) are actually installed. If they are, I'd love to know where.
By default READM(.txt) is installed in a source distribution.
Anything else in the root (aside from setup.py of course and
source files themselves) aren't without extra setup chants. (These
chants can be figured out with some effort. I never remember
them, so, if I want to do something like this, I have to figure
them out again or try to find an example with them.)
I was wrong. Dang. I think I understand setuptools/distutils and
I'm proven wrong again. Sheesh.
Post by Philipp von Weitershausen
Actually, no package data is ever included unless you're either
* working from an svn checkout, in which case setuptools will use
the list of which files are in svn and which aren't as a hint of
what to include and what not
Certainly, I expect CHANGES.txt to be in svn. I've just doeble
checked that files from the root of the package that are checked into
svn *are* included in sdists.

I retract my objection to CHANGES.txt and offer my apologies.

Jim

--
Jim Fulton
Zope Corporation
Martijn Faassen
2007-09-26 20:58:50 UTC
Permalink
[snip]
Post by Philipp von Weitershausen
* working from an svn checkout, in which case setuptools will use the
list of which files are in svn and which aren't as a hint of what to
include and what not
Certainly, I expect CHANGES.txt to be in svn. I've just doeble checked
that files from the root of the package that are checked into svn *are*
included in sdists.
I retract my objection to CHANGES.txt and offer my apologies.
Excellent, we all agree then. :) I did go and check, which is why I was
so mystified by your statements that it wasn't there.

Regards,

Martijn
Tres Seaver
2007-09-27 01:29:26 UTC
Permalink
Post by Philipp von Weitershausen
Post by Fred Drake
Post by Martijn Faassen
What does one need to tell setup.py to make sure CHANGES.txt is
available? I understand it isn't by default, then? Hm, it does appear to
be there by default. I checked grok 0.10's tgz and it's there, and we
didn't do anything special.
Do you mean available in the unpacked sdist, or as part of the
installation? I don't think any of README, README.txt, CHANGES,
CHANGES.txt (from the root of the distribution, not from inside the
package) are actually installed. If they are, I'd love to know where.
By default READM(.txt) is installed in a source distribution. Anything
else in the root (aside from setup.py of course and source files
themselves) aren't without extra setup chants. (These chants can be
figured out with some effort. I never remember them, so, if I want to
do something like this, I have to figure them out again or try to find
an example with them.)
Actually, no package data is ever included unless you're either
* working from an svn checkout, in which case setuptools will use the
list of which files are in svn and which aren't as a hint of what to
include and what not
* creating a MANIFEST.in file to tell setuptools what to include explicitly.
or you write your own package finder. Frankly, anybody who is making
package releases from anything other than a versino-controlled checkout
is insane.

Further, anybody who finds the effort of creating a "fresh' checkout
bevore making a release too burdensome should consider themselves
self-selected out of the "release manager" pool.

I'm *not* kidding about that: taking shortcuts durng the release
process transfers pain / cost to everyone downstream.


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 ***@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
Philipp von Weitershausen
2007-09-27 09:48:25 UTC
Permalink
Post by Tres Seaver
Further, anybody who finds the effort of creating a "fresh' checkout
bevore making a release too burdensome should consider themselves
self-selected out of the "release manager" pool.
I'm *not* kidding about that: taking shortcuts durng the release
process transfers pain / cost to everyone downstream.
Amen, brother. Well said.
Martijn Faassen
2007-09-27 12:07:39 UTC
Permalink
Hey,

On 9/27/07, Tres Seaver <***@palladion.com> wrote:
[snip]
Post by Tres Seaver
Further, anybody who finds the effort of creating a "fresh' checkout
bevore making a release too burdensome should consider themselves
self-selected out of the "release manager" pool.
I'm *not* kidding about that: taking shortcuts durng the release
process transfers pain / cost to everyone downstream.
While I fully agree that releases should be done with care and
attention, and that doing bad releases creates pain/cost for
everybody, this line of argumentation can be used to back up any form
of complication to the release process. :)

Let's focus on the reasons for each step and keep the discussion at
that level, please? I think it's useful if people ask "is that really
necessary?" for steps in the release process. Just weigh the pros and
cons. I'd like to understand more about the trouble Philipp ran into
doing releases from the original checkout and then tagging.

Regards,

Martijn
Philipp von Weitershausen
2007-09-27 12:21:13 UTC
Permalink
Post by Martijn Faassen
[snip]
Post by Tres Seaver
Further, anybody who finds the effort of creating a "fresh' checkout
bevore making a release too burdensome should consider themselves
self-selected out of the "release manager" pool.
I'm *not* kidding about that: taking shortcuts durng the release
process transfers pain / cost to everyone downstream.
While I fully agree that releases should be done with care and
attention, and that doing bad releases creates pain/cost for
everybody, this line of argumentation can be used to back up any form
of complication to the release process. :)
Let's focus on the reasons for each step and keep the discussion at
that level, please? I think it's useful if people ask "is that really
necessary?" for steps in the release process. Just weigh the pros and
cons. I'd like to understand more about the trouble Philipp ran into
doing releases from the original checkout and then tagging.
I've summarized this in another thread [1], but I'll be happy to
elaborate here:

* People actually forget to make the tag. This happened to Roger the
other day. It happened to Jim before with zc.buildout. I'm not
mentioning their names to point fingers. I'm just pointing out that
people who've been with us for a long time forget. If the process
said they should actually check out the tag, they'll hardly forget.

* People create the wrong tags or tags in the wrong places. This
happened to Jim the other day with ZODB 3.7.1. Again, I'm not blaming
him. But I believe if he had to check out the tag to make the
release, he would've caught his mistake.

* People forget to clean up the 'build' directory. This happened to
me the other day. Let me elaborate: I have a trunk checkout of
zopeproject that I use to develop it. Whenever I wanted to make a
release, I'd just prepare it, tag it and generate the distributino
from the trunk checkout. The problem was that I had moved stuff
around, but the old stuff was still making it to the egg because it
still sat around in the 'build' direcdtory that distutils leaves. I
know it sucks that distutils doesn't clean up after itself, but
that's just the way it is. If you're forced to use a clean checkout,
this can't happen.

* People forget to check in important files. This happened to Baiju
the other day. He created a CHANGES.txt file and referred to it from
setup.py. The only problem is that he forgot to check it in. It can
happen, I'm not blaming him. The problem is that setuptools looks at
which files are in svn in order to create the archive manifest. So
CHANGES.txt wasn't in the tarball and the egg was completely
uninstallable (because setup.py couldn't be executed since
CHANGES.txt was missing).

These are four separate cases where I've actually witnessed myself or
other people mess up. We're forgetful, we can't do anything about
that. We can, however, force us to catch our mistakes. I believe that
if we made everybody create the tarballs from the tag, it would
improve the situation a lot.

I'm beginning to see a consensus that we should make it impossible to
create distributions from the trunk or a release branch.


[1] http://mail.zope.org/pipermail/zope3-dev/2007-September/023724.html
Martijn Faassen
2007-09-27 12:33:58 UTC
Permalink
[snip]
Post by Philipp von Weitershausen
Post by Martijn Faassen
Let's focus on the reasons for each step and keep the discussion at
that level, please? I think it's useful if people ask "is that really
necessary?" for steps in the release process. Just weigh the pros and
cons. I'd like to understand more about the trouble Philipp ran into
doing releases from the original checkout and then tagging.
I've summarized this in another thread [1], but I'll be happy to
[snip]
Post by Philipp von Weitershausen
These are four separate cases where I've actually witnessed myself or
other people mess up. We're forgetful, we can't do anything about that.
We can, however, force us to catch our mistakes. I believe that if we
made everybody create the tarballs from the tag, it would improve the
situation a lot.
I'm beginning to see a consensus that we should make it impossible to
create distributions from the trunk or a release branch.
Thanks, very clear. You convinced me. Even if distutils would start
cleaning up after itself the other issues still stand and won't be going
away.

Regards,

Martijn
Stephan Richter
2007-09-27 12:47:54 UTC
Permalink
These are four separate cases where I've actually witnessed myself or ?
other people mess up. We're forgetful, we can't do anything about ?
that. We can, however, force us to catch our mistakes. I believe that ?
if we made everybody create the tarballs from the tag, it would ?
improve the situation a lot.
Of course, an additional or other approach would be to implement a tool that
checks various things. I agree that the problems you listed are solvable with
doing the release from the tag, but there are cases that are not caught:

1. In your last case, if bajium would have used "svn switch --relocate" the
file would still be around and the release would work. I imagine that most
people would use "svn switch" because making another checkout is just a
package management mess.

2. My Trove classifier problem is not solved either using this approach.
Contrary to what Tres hinted on in his E-mail , if there are errors while
registering with PyPI, no new release is added. Also, "buildout setup .
egg_info" does not verify the Trove classifiers. (Note that I think the Trove
classifiers are only one example of something going wrong.)

3. A serious problem occurs, if you accidentally specify the wrong package
name and version in setup.py. Both happened to me yesterday, but I caught it
in time.

A fairly simple tool can find and report all the problems found and offer
assistance. I think it is worth investing in one, especially since it will
reduce my overhead since my manual checking now becomes automated.

Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
Martijn Faassen
2007-09-27 12:57:59 UTC
Permalink
Hey,
[snip]
Post by Stephan Richter
A fairly simple tool can find and report all the problems found and offer
assistance. I think it is worth investing in one, especially since it will
reduce my overhead since my manual checking now becomes automated.
Agreed a tool could be helpful in this case as well.

Regards,

Martijn
Philipp von Weitershausen
2007-09-27 13:03:17 UTC
Permalink
Post by Stephan Richter
Post by Philipp von Weitershausen
These are four separate cases where I've actually witnessed myself or
other people mess up. We're forgetful, we can't do anything about
that. We can, however, force us to catch our mistakes. I believe that
if we made everybody create the tarballs from the tag, it would
improve the situation a lot.
Of course, an additional or other approach would be to implement a tool that
checks various things. I agree that the problems you listed are solvable with
1. In your last case, if bajium would have used "svn switch --
relocate" the
file would still be around and the release would work. I imagine that most
people would use "svn switch" because making another checkout is just a
package management mess.
Why is making another checkout a package management mess? Go to /tmp
or ~/temp or whatever, get the checkout, do your release stuff and
delete it again. Is this so hard? Sorry, but I fail to see how this
is messy.

Also, regardless of what you imagine people do, if the process says
"get a new, fresh checkout" then this is what people should do. If
they use svn switch instead, then they're not following the process.
End of story.
Post by Stephan Richter
2. My Trove classifier problem is not solved either using this
approach.
Contrary to what Tres hinted on in his E-mail , if there are errors while
registering with PyPI, no new release is added. Also, "buildout setup .
egg_info" does not verify the Trove classifiers. (Note that I think the Trove
classifiers are only one example of something going wrong.)
buildout setup? I've never seen that. I can't find any documentation
on it in zc.buildout. How does it work? Does it call setup.py for
every development egg in buildout.cfg? Why not just call python
setup.py?

Anyway, I think the Trove classifiers are an edge case. I've never
seen anyone get those wrong before, to be honest. I *have* seen
people get other things wrong and I think these problems would all be
solved if we made it impossible to create distributions from a branch
or the trunk, but made people create them from tags instead.
Post by Stephan Richter
3. A serious problem occurs, if you accidentally specify the wrong package
name and version in setup.py. Both happened to me yesterday, but I caught it
in time.
I assume you mean the egg name, not the package name.

I'm not sure how a tool could help you here. I suppose it could use
some conventions (e.g. look at the packages inside the egg) or look
at the subversion URL.
Post by Stephan Richter
A fairly simple tool can find and report all the problems found and offer
assistance. I think it is worth investing in one, especially since it will
reduce my overhead since my manual checking now becomes automated.
I'm not arguing against such a tool. If you are willing to come up
with it, go for it. We should still have a proper *human* process
first. A tool can then help us do the tedious bits.
Baiju M
2007-09-27 13:15:07 UTC
Permalink
Post by Philipp von Weitershausen
Post by Stephan Richter
Post by Philipp von Weitershausen
These are four separate cases where I've actually witnessed
myself or other people mess up. We're forgetful, we can't do
anything about that. We can, however, force us to catch our
mistakes. I believe that if we made everybody create the tarballs
from the tag, it would improve the situation a lot.
Of course, an additional or other approach would be to implement a
tool that checks various things. I agree that the problems you
listed are solvable with doing the release from the tag, but there
1. In your last case, if bajium would have used "svn switch
--relocate" the file would still be around and the release would
work. I imagine that most people would use "svn switch" because
making another checkout is just a package management mess.
Why is making another checkout a package management mess? Go to /tmp
or ~/temp or whatever, get the checkout, do your release stuff and
delete it again. Is this so hard? Sorry, but I fail to see how this
is messy.
Also, regardless of what you imagine people do, if the process says
"get a new, fresh checkout" then this is what people should do. If
they use svn switch instead, then they're not following the process.
End of story.
Release from a fresh tag check out is always good.

<personal>
I have released eggs before and after my mistake. After my mistake,
I created check list for my convenience here:
http://wiki.zope.org/zope3/BaijuMuthukadan
An now I am making release from tag checkouts.
</personal>

Regards,
Baiju M
Raphael Ritz
2007-09-27 13:11:29 UTC
Permalink
Stephan Richter wrote:

[..]
Post by Stephan Richter
A fairly simple tool can find and report all the problems found and offer
assistance. I think it is worth investing in one, especially since it will
reduce my overhead since my manual checking now becomes automated.
FWIW:
If this tool were available right now so that people could start using
it today that would be great. But I don't see it yet :-(

But we do have a problem right now.

And what Philipp suggests doesn't require anything to be done other
than people starting to adopt the approach he is pushing.

I don't see this in conflict. Rather as complementing each other.

Raphael
Post by Stephan Richter
Regards,
Stephan
Tres Seaver
2007-09-27 01:29:46 UTC
Permalink
Post by Philipp von Weitershausen
Post by Fred Drake
Post by Martijn Faassen
What does one need to tell setup.py to make sure CHANGES.txt is
available? I understand it isn't by default, then? Hm, it does appear to
be there by default. I checked grok 0.10's tgz and it's there, and we
didn't do anything special.
Do you mean available in the unpacked sdist, or as part of the
installation? I don't think any of README, README.txt, CHANGES,
CHANGES.txt (from the root of the distribution, not from inside the
package) are actually installed. If they are, I'd love to know where.
By default READM(.txt) is installed in a source distribution. Anything
else in the root (aside from setup.py of course and source files
themselves) aren't without extra setup chants. (These chants can be
figured out with some effort. I never remember them, so, if I want to
do something like this, I have to figure them out again or try to find
an example with them.)
Actually, no package data is ever included unless you're either
* working from an svn checkout, in which case setuptools will use the
list of which files are in svn and which aren't as a hint of what to
include and what not
* creating a MANIFEST.in file to tell setuptools what to include explicitly.
or you write your own package finder. Frankly, anybody who is making
package releases from anything other than a versino-controlled checkout
is insane.

Further, anybody who finds the effort of creating a "fresh' checkout
bevore making a release too burdensome should consider themselves
self-selected out of the "release manager" pool.

I'm *not* kidding about that: taking shortcuts durng the release
process transfers pain / cost to everyone downstream.


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 ***@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
Tres Seaver
2007-09-27 01:26:19 UTC
Permalink
Post by Jim Fulton
Post by Fred Drake
Post by Martijn Faassen
What does one need to tell setup.py to make sure CHANGES.txt is
available? I understand it isn't by default, then? Hm, it does appear to
be there by default. I checked grok 0.10's tgz and it's there, and we
didn't do anything special.
Do you mean available in the unpacked sdist, or as part of the
installation? I don't think any of README, README.txt, CHANGES,
CHANGES.txt (from the root of the distribution, not from inside the
package) are actually installed. If they are, I'd love to know where.
By default READM(.txt) is installed in a source distribution.
Anything else in the root (aside from setup.py of course and source
files themselves) aren't without extra setup chants. (These chants
can be figured out with some effort. I never remember them, so, if I
want to do something like this, I have to figure them out again or
try to find an example with them.)
If we are going to have a change log, which we should, I would prefer
it to be included in source distributions.
I want them present in the *installed* egg, not just in the source
distribution: setuptools doesn't preserve source distributions after
creating / installing the '.egg' version.
Post by Jim Fulton
Including a file other
that README in the root requires extra effort that I don't want to
require -- writing setup.py files is hard enough as it is.
Put the "real" README.txt and CHANGES.txt in the actual package: the
version which is a peer of 'setup.py' should be a stub which points to
the "real" versions.
Post by Jim Fulton
I'm not crazy about Tres' idea of putting these files in the Python
packages themselves, but it does have the advantage that it causes
the files to be included in eggs as well as source distributions.
Not having those files available after installing is a complete
non-starter for me.


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 ***@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
Jim Fulton
2007-09-27 15:12:14 UTC
Permalink
On Sep 26, 2007, at 8:26 PM, Tres Seaver wrote:
...
Post by Tres Seaver
Post by Jim Fulton
If we are going to have a change log, which we should, I would prefer
it to be included in source distributions.
I want them present in the *installed* egg, not just in the source
distribution: setuptools doesn't preserve source distributions after
creating / installing the '.egg' version.
Agreed, but....
Post by Tres Seaver
Post by Jim Fulton
Including a file other
that README in the root requires extra effort that I don't want to
require -- writing setup.py files is hard enough as it is.
Put the "real" README.txt and CHANGES.txt in the actual package: the
version which is a peer of 'setup.py' should be a stub which points to
the "real" versions.
I can live with this, but I don't like it. It feels more complicated
to me.

Jim

--
Jim Fulton
Zope Corporation
Martijn Faassen
2007-09-27 17:48:00 UTC
Permalink
Hey,
[snip]
Post by Jim Fulton
Post by Tres Seaver
Post by Jim Fulton
Including a file other
that README in the root requires extra effort that I don't want to
require -- writing setup.py files is hard enough as it is.
Put the "real" README.txt and CHANGES.txt in the actual package: the
version which is a peer of 'setup.py' should be a stub which points to
the "real" versions.
I can live with this, but I don't like it. It feels more complicated
to me.
I don't like it either. I thought we resolved this though so I'm not
sure why we're discussing this. CHANGES.txt in the root dir it is,
right?

Regards,

Martijn
Tres Seaver
2007-09-27 17:53:43 UTC
Permalink
Post by Martijn Faassen
Hey,
[snip]
Post by Jim Fulton
Post by Tres Seaver
Post by Jim Fulton
Including a file other
that README in the root requires extra effort that I don't want to
require -- writing setup.py files is hard enough as it is.
Put the "real" README.txt and CHANGES.txt in the actual package: the
version which is a peer of 'setup.py' should be a stub which points to
the "real" versions.
I can live with this, but I don't like it. It feels more complicated
to me.
I don't like it either. I thought we resolved this though so I'm not
sure why we're discussing this. CHANGES.txt in the root dir it is,
right?
- -1. I argued for putting the CHANGES.txt and the "real" README.txt in
the *package* dir, making them available in the *installed egg*, not
just the source ditribution.


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 ***@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
Martijn Faassen
2007-09-27 17:59:47 UTC
Permalink
Hey,

On 9/27/07, Tres Seaver <***@palladion.com> wrote:
[snip]
Post by Tres Seaver
Post by Martijn Faassen
I don't like it either. I thought we resolved this though so I'm not
sure why we're discussing this. CHANGES.txt in the root dir it is,
right?
- -1. I argued for putting the CHANGES.txt and the "real" README.txt in
the *package* dir, making them available in the *installed egg*, not
just the source ditribution.
Okay, I hadn't read it that way as the word 'package' is ambiguous.
That seems completely against the way everybody else does this. Why is
having this available in the installed egg so important?

Regards,

Martijn

Tres Seaver
2007-09-27 01:30:18 UTC
Permalink
Post by Jim Fulton
Post by Fred Drake
Post by Martijn Faassen
What does one need to tell setup.py to make sure CHANGES.txt is
available? I understand it isn't by default, then? Hm, it does appear to
be there by default. I checked grok 0.10's tgz and it's there, and we
didn't do anything special.
Do you mean available in the unpacked sdist, or as part of the
installation? I don't think any of README, README.txt, CHANGES,
CHANGES.txt (from the root of the distribution, not from inside the
package) are actually installed. If they are, I'd love to know where.
By default READM(.txt) is installed in a source distribution.
Anything else in the root (aside from setup.py of course and source
files themselves) aren't without extra setup chants. (These chants
can be figured out with some effort. I never remember them, so, if I
want to do something like this, I have to figure them out again or
try to find an example with them.)
If we are going to have a change log, which we should, I would prefer
it to be included in source distributions.
I want them present in the *installed* egg, not just in the source
distribution: setuptools doesn't preserve source distributions after
creating / installing the '.egg' version.
Post by Jim Fulton
Including a file other
that README in the root requires extra effort that I don't want to
require -- writing setup.py files is hard enough as it is.
Put the "real" README.txt and CHANGES.txt in the actual package: the
version which is a peer of 'setup.py' should be a stub which points to
the "real" versions.
Post by Jim Fulton
I'm not crazy about Tres' idea of putting these files in the Python
packages themselves, but it does have the advantage that it causes
the files to be included in eggs as well as source distributions.
Not having those files available after installing is a complete
non-starter for me.


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 ***@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
Tres Seaver
2007-09-26 16:17:33 UTC
Permalink
Post by Jim Fulton
Post by Stephan Richter
Post by Jim Fulton
Post by Martijn Faassen
What about using CHANGES.txt, which we should be maintaining anyway?
I agree with a change log. CHANGES.txt is difficult to get included
in distributions. Having one requires a more complex setup.py. I'd
rather recommend including a change log in README.txt.
-1. I don't like that. We include CHANGES.txt via the long
description; that's
good enough for me.
Do you think that the change log should be included in the distribution?
+10. The cheeseshop metadata is not where I would expect to look for a
changelog; having it readable there is only helpful *before* I've
downloaded.

FWIW, I think that both README.txt and CHANGES.txt should be "package
data", so that they are discoverable after installing an egg. The
top-level README.txt should be boilerplate, and point to those files
(the setup.py process can then stitch them all todgether).


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 ***@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
Philipp von Weitershausen
2007-09-26 15:27:08 UTC
Permalink
Post by Stephan Richter
Post by Christian Theune
Hmm. While doing that I also noticed that we were at 3.4.0a1 yesterday
evening. The stable release was made from that without making a
maintenance branch and bumping the trunk to 3.5.
There is conflicting information here. :-) Some people say we need branches,
others say we don't. And where is an agreed-upon document that you have to
list the next version in the setup.py file after the release?
It's not agreed upon, it's just what I wrote down:

http://svn.zope.org/*checkout*/Sandbox/philikon/foundation/maintaining-software.txt

Back when I put it up for discussion, all I got was a huge discussion
about PEP8, which is just a small and rather unimportant paragraph.
Unfortunately it's somewhere towards the beginning of the file. I wonder
if anybody read any further and got to the actually important stuff. Sigh.
Post by Stephan Richter
Because I disagree with that, since you cannot know the next version.
You can always know the minimum version. If you just released 3.4.2, I
think it's sensible to point the next release to 3.4.3. If you later
decide that you really need a feature release, you can always bump to
3.5.0a1 (which would be the first release in the 3.5.x series).
--
http://worldcookery.com -- Professional Zope documentation and training
Benji York
2007-09-26 21:34:28 UTC
Permalink
Post by Philipp von Weitershausen
Post by Stephan Richter
Because I disagree with that, since you cannot know the next version.
You can always know the minimum version. If you just released 3.4.2, I
think it's sensible to point the next release to 3.4.3. If you later
decide that you really need a feature release, you can always bump to
3.5.0a1 (which would be the first release in the 3.5.x series).
Why not leave the version totally out of the setup.py in the trunk?
After branching for a release we can set the version (e.g., 1.2), make a
release, and tag the branch.

We could either leave the version on the branch at the "last" release or
continue the trend of mad bumping and have it at the "next" release
(which since this is a branch, we can actually predict).

I prefer the "last" version, but "next" would work too.
--
Benji York
Senior Software Engineer
Zope Corporation
Jim Fulton
2007-09-26 22:19:02 UTC
Permalink
Post by Benji York
Post by Philipp von Weitershausen
Post by Stephan Richter
Because I disagree with that, since you cannot know the next
version.
You can always know the minimum version. If you just released
3.4.2, I think it's sensible to point the next release to 3.4.3.
If you later decide that you really need a feature release, you
can always bump to 3.5.0a1 (which would be the first release in
the 3.5.x series).
Gary and Benji made me pay attention to this point. :)

I *really* don't like setting the version to the number for the next
release, since one often doesn't know what it is.
Post by Benji York
Why not leave the version totally out of the setup.py in the trunk?
Benji and Gary won me over to this point of view. :)
Post by Benji York
After branching for a release we can set the version (e.g., 1.2),
make a release, and tag the branch.
We should not require branches. I would only bother creating
maintenance branches when they are needed.

Z, B, G, and I propose the following:

- Leave the version # out of setup.py on the trunk and on branches.

When it is time to make a release, either from the trunk or from a
maint branch:

- Update changes.txt, adding a heading for the new # and date

- Create a tag

- check out or switch to the tag

- Set the version in setup.py on the tag. Check it in.

- Make the release from the tag.

(BTW, when creating maint branches after the fact, the branch should
be made by copying the trunk at the revision that the first release
branch for that tag was made, so as not to accidentally get a version
#.)

I'd prefer not to make an edict, so Benji and Gary will now persuade
you that this is the best way to go. ;)

Jim

--
Jim Fulton
Zope Corporation
Stephan Richter
2007-09-26 22:28:48 UTC
Permalink
Post by Jim Fulton
- Update changes.txt, adding a heading for the new # and date
- Create a tag
- check out or switch to the tag
- Set the version in setup.py on the tag. Check it in.
- Make the release from the tag.
Changing tags is not that good. I'd rather check in a aversion number
then. ;-)

Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
Jim Fulton
2007-09-26 22:34:48 UTC
Permalink
Post by Stephan Richter
Post by Jim Fulton
- Update changes.txt, adding a heading for the new # and date
- Create a tag
- check out or switch to the tag
- Set the version in setup.py on the tag. Check it in.
- Make the release from the tag.
Changing tags is not that good. I'd rather check in a aversion number
then. ;-)
This is exactly what we've been doing for Zope 3 releases -- for the
same reason. I think it is the one acceptable and reasonable change
to a tag. The benefit of forcing us to release from a tag is, imo,
significant.

Jim

--
Jim Fulton
Zope Corporation
Stephan Richter
2007-09-26 22:41:35 UTC
Permalink
Post by Stephan Richter
Post by Jim Fulton
- Update changes.txt, adding a heading for the new # and date
- Create a tag
- check out or switch to the tag
- Set the version in setup.py on the tag. Check it in.
- Make the release from the tag.
Changing tags is not that good. I'd rather check in a aversion number
then. ;-)
This is exactly what we've been doing for Zope 3 releases -- for the ?
same reason. ?I think it is the one acceptable and reasonable change ?
to a tag. ?The benefit of forcing us to release from a tag is, imo, ?
significant.
Here is a problem that I discussed with Marius earlier today.

I often do not know whether all my setup.py settings are correct until I try
to upload a release. I frequently get the "Development Status" classifier
wrong and I won't be told it is wrong until I try to register the release.

So I usually create the release first and upload it and after that create the
tag.

Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
Philipp von Weitershausen
2007-09-26 22:49:43 UTC
Permalink
Post by Stephan Richter
Post by Jim Fulton
Post by Stephan Richter
Post by Jim Fulton
- Update changes.txt, adding a heading for the new # and date
- Create a tag
- check out or switch to the tag
- Set the version in setup.py on the tag. Check it in.
- Make the release from the tag.
Changing tags is not that good. I'd rather check in a aversion number
then. ;-)
This is exactly what we've been doing for Zope 3 releases -- for the
same reason. I think it is the one acceptable and reasonable change
to a tag. The benefit of forcing us to release from a tag is, imo,
significant.
Here is a problem that I discussed with Marius earlier today.
I often do not know whether all my setup.py settings are correct until I try
to upload a release. I frequently get the "Development Status"
classifier
wrong and I won't be told it is wrong until I try to register the release.
A good way to catch errors before you mess with the CheeseShop is to
create the egg info and look at it:

$ python setup.py egg_info
$ less src/EGG.egg-info/PKG-INFO
Post by Stephan Richter
So I usually create the release first and upload it and after that create the
tag.
Well, you could just as well re-tag when you screwed up. Or just fix
minor things on the tag and merge them back to the branch.
Tres Seaver
2007-09-27 01:35:42 UTC
Permalink
Post by Stephan Richter
Post by Jim Fulton
Post by Stephan Richter
Post by Jim Fulton
- Update changes.txt, adding a heading for the new # and date
- Create a tag
- check out or switch to the tag
- Set the version in setup.py on the tag. Check it in.
- Make the release from the tag.
Changing tags is not that good. I'd rather check in a aversion number
then. ;-)
This is exactly what we've been doing for Zope 3 releases -- for the
same reason. I think it is the one acceptable and reasonable change
to a tag. The benefit of forcing us to release from a tag is, imo,
significant.
Here is a problem that I discussed with Marius earlier today.
I often do not know whether all my setup.py settings are correct until I try
to upload a release.
You should *never* be uploading a release that you don't already *know*
is correct.

< I frequently get the "Development Status" classifier
Post by Stephan Richter
wrong and I won't be told it is wrong until I try to register the release.
If you are using 'setup.py register upload' to do your checking, you
need a better tool: the consequences of carelessness are too
significant (bad releases punish everyone *else*).
Post by Stephan Richter
So I usually create the release first and upload it and after that create the
tag.
- -100. Get it right, check it in, *then* upload the distribution.


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 ***@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
Stephan Richter
2007-09-27 03:52:50 UTC
Permalink
Post by Stephan Richter
So I usually create the release first and upload it and after that create
the tag.
-100. ?Get it right, check it in, *then* upload the distribution.
We do not have the tools. There is simply no telling beforehand. Marius and I
worked on the ideas of a tool that he proposed earlier today.

Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
Philipp von Weitershausen
2007-09-27 10:17:08 UTC
Permalink
Post by Stephan Richter
Post by Tres Seaver
Post by Stephan Richter
So I usually create the release first and upload it and after that create
the tag.
-100. Get it right, check it in, *then* upload the distribution.
We do not have the tools. There is simply no telling beforehand. Marius and I
worked on the ideas of a tool that he proposed earlier today.
There is some telling beforehand:

* As I already said, you can generate all the package metadata with

python setup.py egg_info

and then inspect it in src/EGG.egg-info/PKG-INFO. This is
equivalent to checking
the PyPI page, it contains the same information. I frequently do
this, also to make
sure my setup.py actually executes before I tag (I often forget a
comma or so.)

* You can generate an egg or a tarball locally, without uploading it.
Then you can
take a look at the tarball or the egg. Unzip it if necessary. Call
python setup.py
egg_info inside the tarball, if necessary, to see if all the
CHANGES.txt, README.txt
etc. files are there as well, especially when you know you messed
this part up
before.

You could also easy_install the egg and/or tarball in a virtualenv
(so that your
global site-packages won't be affected).
Stephan Richter
2007-09-27 11:21:08 UTC
Permalink
Post by Philipp von Weitershausen
* As I already said, you can generate all the package metadata with
python setup.py egg_info
and then inspect it in src/EGG.egg-info/PKG-INFO. This is
equivalent to checking
the PyPI page, it contains the same information. I frequently do
this, also to make
sure my setup.py actually executes before I tag (I often forget a
comma or so.)
egg_info does not validate the trove classifiers, for example. I tried this
last night before writing this mail. If the the setup.py file has a syntax
error, I will know about it when running buildout.
Post by Philipp von Weitershausen
* You can generate an egg or a tarball locally, without uploading it.
Then you can
take a look at the tarball or the egg. Unzip it if necessary. Call
python setup.py
egg_info inside the tarball, if necessary, to see if all the
CHANGES.txt, README.txt
etc. files are there as well, especially when you know you messed
this part up
before.
Still does not solve my problem use case.
Post by Philipp von Weitershausen
You could also easy_install the egg and/or tarball in a virtualenv
(so that your
global site-packages won't be affected).
I still do not see how this solves the problem.

Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
Philipp von Weitershausen
2007-09-27 12:40:37 UTC
Permalink
Post by Stephan Richter
egg_info does not validate the trove classifiers, for example. I tried this
last night before writing this mail.
Well, to be honest, I wonder how you can mess up with the
classifiers. I just always copy them from http://pypi.python.org/pypi?
%3Aaction=list_classifiers. Anything else is just insane...

But, if you wish for such a tool, let your wish be my command. With
the attached verifyclassifiers.py script, you may do so using the
following command:

python setup.py --classifiers | python verifyclassifiers.py
Post by Stephan Richter
If the the setup.py file has a syntax error, I will know about it
when running buildout.
True, but running buildout can take a long time.
Post by Stephan Richter
Post by Philipp von Weitershausen
* You can generate an egg or a tarball locally, without uploading it.
Then you can take a look at the tarball or the egg. Unzip it if
necessary. Call
python setup.py egg_info inside the tarball, if necessary, to
see if all the
CHANGES.txt, README.txt etc. files are there as well, especially
when you know
you messed this part up before.
Still does not solve my problem use case.
Post by Philipp von Weitershausen
You could also easy_install the egg and/or tarball in a virtualenv
(so that your global site-packages won't be affected).
I still do not see how this solves the problem.
They're measures for making sure the distribution has the right
metadata and is easy_install'able. What other problem do we have?


-------------- next part --------------
A non-text attachment was scrubbed...
Name: verifyclassifiers.py
Type: text/x-python-script
Size: 365 bytes
Desc: not available
Url : http://mail.zope.org/pipermail/zope3-dev/attachments/20070927/3341f22f/verifyclassifiers-0001.bin
-------------- next part --------------
Tres Seaver
2007-09-27 01:40:08 UTC
Permalink
Post by Stephan Richter
Post by Jim Fulton
Post by Stephan Richter
Post by Jim Fulton
- Update changes.txt, adding a heading for the new # and date
- Create a tag
- check out or switch to the tag
- Set the version in setup.py on the tag. Check it in.
- Make the release from the tag.
Changing tags is not that good. I'd rather check in a aversion number
then. ;-)
This is exactly what we've been doing for Zope 3 releases -- for the
same reason. I think it is the one acceptable and reasonable change
to a tag. The benefit of forcing us to release from a tag is, imo,
significant.
Here is a problem that I discussed with Marius earlier today.
I often do not know whether all my setup.py settings are correct until I try
to upload a release.
You should *never* be uploading a release that you don't already *know*
is correct.

< I frequently get the "Development Status" classifier
Post by Stephan Richter
wrong and I won't be told it is wrong until I try to register the release.
If you are using 'setup.py register upload' to do your checking, you
need a better tool: the consequences of carelessness are too
significant (bad releases punish everyone *else*).
Post by Stephan Richter
So I usually create the release first and upload it and after that create the
tag.
- -100. Get it right, check it in, *then* upload the distribution.


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 ***@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
Philipp von Weitershausen
2007-09-26 22:40:44 UTC
Permalink
Post by Jim Fulton
Post by Benji York
Post by Philipp von Weitershausen
Post by Stephan Richter
Because I disagree with that, since you cannot know the next version.
You can always know the minimum version. If you just released
3.4.2, I think it's sensible to point the next release to 3.4.3.
If you later decide that you really need a feature release, you
can always bump to 3.5.0a1 (which would be the first release in
the 3.5.x series).
Gary and Benji made me pay attention to this point. :)
I *really* don't like setting the version to the number for the
next release, since one often doesn't know what it is.
Maybe. However, when you do the actual release then, you're still
going to have to find out which version number to use. On release
branches this is a trivial step, of course: You just look at the
latest tagged version and increment accordingly. On the trunk it
might be trickier. After 1.0, do we have 1.1? Or 2.0?

Maye this aspect isn't such a big deal, though.
Post by Jim Fulton
Post by Benji York
Why not leave the version totally out of the setup.py in the trunk?
Benji and Gary won me over to this point of view. :)
There's a *really* good reason for putting the upcoming version
number in setup.py, though: development eggs.

Let's say X depends on Y and I'm developing them simultaneously as
development eggs in one sandbox (e.g. buildout). I add a feature to Y
that I'd like to use in X. That means I'll have to change X's version
dependency regarding Y so that it now depends on Y>a.b where a.b is
the latest release that didn't have the feature I added.

Will Y with the setup.py stating version='unreleased' satisfy the
Y>a.b requirement that X (rightfully) has? If not, then I think we
have a problem. If yes, then I find that very confusing.

I know that if Y's setup.py stated version='a.b-dev', it will work.
This what my guide currently suggests (including taking it out just
for release tags).
Post by Jim Fulton
Post by Benji York
After branching for a release we can set the version (e.g., 1.2),
make a release, and tag the branch.
We should not require branches. I would only bother creating
maintenance branches when they are needed.
+1
Who's Z? :)
Post by Jim Fulton
- Leave the version # out of setup.py on the trunk and on branches.
When it is time to make a release, either from the trunk or from a
- Update changes.txt, adding a heading for the new # and date
- Create a tag
- check out or switch to the tag
- Set the version in setup.py on the tag. Check it in.
- Make the release from the tag.
I could live with that, even with the fact that you'd be modifying a
tag, as long as it's done in this exact order and the only
modificdations to a tag woudl be setting setup.py.

I still see the development egg case the best argument for putting
the next anticipated version number into setup.py. I think the
compromise of version number + 'dev' marker would work best.
Jim Fulton
2007-09-26 23:14:15 UTC
Permalink
Post by Philipp von Weitershausen
Post by Jim Fulton
Post by Benji York
Post by Philipp von Weitershausen
Post by Stephan Richter
Because I disagree with that, since you cannot know the next version.
You can always know the minimum version. If you just released
3.4.2, I think it's sensible to point the next release to 3.4.3.
If you later decide that you really need a feature release, you
can always bump to 3.5.0a1 (which would be the first release in
the 3.5.x series).
Gary and Benji made me pay attention to this point. :)
I *really* don't like setting the version to the number for the
next release, since one often doesn't know what it is.
Maybe. However, when you do the actual release then, you're still
going to have to find out which version number to use. On release
branches this is a trivial step, of course: You just look at the
latest tagged version and increment accordingly. On the trunk it
might be trickier. After 1.0, do we have 1.1? Or 2.0?
Maye this aspect isn't such a big deal, though.
You can always look at the previous tags or at CHANGES.txt.
Post by Philipp von Weitershausen
Post by Jim Fulton
Post by Benji York
Why not leave the version totally out of the setup.py in the trunk?
Benji and Gary won me over to this point of view. :)
There's a *really* good reason for putting the upcoming version
number in setup.py, though: development eggs.
Good point.
Post by Philipp von Weitershausen
Let's say X depends on Y and I'm developing them simultaneously as
development eggs in one sandbox (e.g. buildout). I add a feature to
Y that I'd like to use in X. That means I'll have to change X's
version dependency regarding Y so that it now depends on Y>a.b
where a.b is the latest release that didn't have the feature I added.
Will Y with the setup.py stating version='unreleased' satisfy the
Y>a.b requirement that X (rightfully) has? If not, then I think we
have a problem. If yes, then I find that very confusing.
Leaving aside for the moment the fact that "develop" eggs are used
for system egg installs :(,

I think if a buildout uses a develop egg, there should be a very
strong presumption that it it should be used. OTOH, if you have a
part that depends on a previous major version, then you don't really
want to use a develop egg for the later release. This is a bit of an
edge case though.
Post by Philipp von Weitershausen
I know that if Y's setup.py stated version='a.b-dev', it will work.
This what my guide currently suggests (including taking it out just
for release tags).
A problem with this is that someone else might make a release of Y,
either to fix a bug or to add a feature while your working on the new
feature in Y.
Post by Philipp von Weitershausen
Post by Jim Fulton
Post by Benji York
After branching for a release we can set the version (e.g., 1.2),
make a release, and tag the branch.
We should not require branches. I would only bother creating
maintenance branches when they are needed.
+1
Who's Z? :)
Post by Jim Fulton
- Leave the version # out of setup.py on the trunk and on branches.
When it is time to make a release, either from the trunk or from a
- Update changes.txt, adding a heading for the new # and date
- Create a tag
- check out or switch to the tag
- Set the version in setup.py on the tag. Check it in.
- Make the release from the tag.
I could live with that, even with the fact that you'd be modifying
a tag, as long as it's done in this exact order and the only
modificdations to a tag woudl be setting setup.py.
I still see the development egg case the best argument for putting
the next anticipated version number into setup.py. I think the
compromise of version number + 'dev' marker would work best.
I agree that the develop egg case is problematic. I'd like to think
about that.

If you ever actually make dev releases, then guessing wrong about the
next version could cause real problems. Consider the following case:

We've just released 1.1. We guess the next release is 1.2. We change
things and release, 1.2dev-r#####. Someone fixes a bug and releases
1.1.1. Now there's a dev release of 1.2 that is actually older than
the 1.1.1 release but that setuptools considers to be newer. I think
this is fairly problematic. Then again, I think that dev releases
are problematic in a lot of ways.

Jim

--
Jim Fulton
Zope Corporation
Philipp von Weitershausen
2007-09-26 23:26:11 UTC
Permalink
Post by Jim Fulton
Post by Philipp von Weitershausen
Let's say X depends on Y and I'm developing them simultaneously as
development eggs in one sandbox (e.g. buildout). I add a feature
to Y that I'd like to use in X. That means I'll have to change X's
version dependency regarding Y so that it now depends on Y>a.b
where a.b is the latest release that didn't have the feature I added.
Will Y with the setup.py stating version='unreleased' satisfy the
Y>a.b requirement that X (rightfully) has? If not, then I think we
have a problem. If yes, then I find that very confusing.
Leaving aside for the moment the fact that "develop" eggs are used
for system egg installs :(,
I think if a buildout uses a develop egg, there should be a very
strong presumption that it it should be used. OTOH, if you have a
part that depends on a previous major version, then you don't
really want to use a develop egg for the later release. This is a
bit of an edge case though.
It is as long as you ignore the fact that all major Linux distros
install develop eggs into site-packages.
Post by Jim Fulton
Post by Philipp von Weitershausen
I still see the development egg case the best argument for putting
the next anticipated version number into setup.py. I think the
compromise of version number + 'dev' marker would work best.
I agree that the develop egg case is problematic. I'd like to
think about that.
If you ever actually make dev releases, then guessing wrong about
the next version could cause real problems.
Oh, I'm not actually thinking about dev releases. I too think they're
very problematic. I just suggested using the 'dev' marker as a way to
tell branches or the trunk apart from tags. On the branches and
trunk, setup.py's version is the next anticipated release + 'dev',
while on tags it's just release version. That way people won't
accidentally make releases from branches or the trunk (because they'd
get an egg with a 'dev' in the version). They'll have to use a tag.
Jim Fulton
2007-09-26 23:32:00 UTC
Permalink
Post by Philipp von Weitershausen
Post by Jim Fulton
Post by Philipp von Weitershausen
Let's say X depends on Y and I'm developing them simultaneously
as development eggs in one sandbox (e.g. buildout). I add a
feature to Y that I'd like to use in X. That means I'll have to
change X's version dependency regarding Y so that it now depends
on Y>a.b where a.b is the latest release that didn't have the
feature I added.
Will Y with the setup.py stating version='unreleased' satisfy the
Y>a.b requirement that X (rightfully) has? If not, then I think
we have a problem. If yes, then I find that very confusing.
Leaving aside for the moment the fact that "develop" eggs are used
for system egg installs :(,
I think if a buildout uses a develop egg, there should be a very
strong presumption that it it should be used. OTOH, if you have a
part that depends on a previous major version, then you don't
really want to use a develop egg for the later release. This is a
bit of an edge case though.
It is as long as you ignore the fact that all major Linux distros
install develop eggs into site-packages.
I already left that aside. :)

Buildout should be able to detect this case and deal with it, which
is why I left it aside.
Post by Philipp von Weitershausen
Post by Jim Fulton
Post by Philipp von Weitershausen
I still see the development egg case the best argument for
putting the next anticipated version number into setup.py. I
think the compromise of version number + 'dev' marker would work
best.
I agree that the develop egg case is problematic. I'd like to
think about that.
If you ever actually make dev releases, then guessing wrong about
the next version could cause real problems.
Oh, I'm not actually thinking about dev releases. I too think
they're very problematic. I just suggested using the 'dev' marker
as a way to tell branches or the trunk apart from tags.
But people will make these releases and leaving the version out will
mean that these releases will be harmless.
Post by Philipp von Weitershausen
On the branches and trunk, setup.py's version is the next
anticipated release + 'dev', while on tags it's just release
version. That way people won't accidentally make releases from
branches or the trunk (because they'd get an egg with a 'dev' in
the version). They'll have to use a tag.
You think getting an egg with dev in the version will stop them?

Jim

--
Jim Fulton
Zope Corporation
Tres Seaver
2007-09-27 01:43:36 UTC
Permalink
Post by Philipp von Weitershausen
Post by Jim Fulton
Post by Benji York
Post by Philipp von Weitershausen
Post by Stephan Richter
Because I disagree with that, since you cannot know the next version.
You can always know the minimum version. If you just released
3.4.2, I think it's sensible to point the next release to 3.4.3.
If you later decide that you really need a feature release, you
can always bump to 3.5.0a1 (which would be the first release in
the 3.5.x series).
Gary and Benji made me pay attention to this point. :)
I *really* don't like setting the version to the number for the
next release, since one often doesn't know what it is.
Maybe. However, when you do the actual release then, you're still
going to have to find out which version number to use. On release
branches this is a trivial step, of course: You just look at the
latest tagged version and increment accordingly. On the trunk it
might be trickier. After 1.0, do we have 1.1? Or 2.0?
Maye this aspect isn't such a big deal, though.
Post by Jim Fulton
Post by Benji York
Why not leave the version totally out of the setup.py in the trunk?
Benji and Gary won me over to this point of view. :)
There's a *really* good reason for putting the upcoming version
number in setup.py, though: development eggs.
- -100. 'development eggs' which get released into the wild are an
abomination of desolation, and should be avoided at *any* cost. (I'm
assuming that you mean eggs / distributions which get uploaded to some
package repository, rather than being installed via 'setup.py develop').
Post by Philipp von Weitershausen
Let's say X depends on Y and I'm developing them simultaneously as
development eggs in one sandbox (e.g. buildout). I add a feature to Y
that I'd like to use in X. That means I'll have to change X's version
dependency regarding Y so that it now depends on Y>a.b where a.b is
the latest release that didn't have the feature I added.
Logically speaking, you *can't* release Y with the new requirement until
*after* a release of X is availabe with the required feature: while
they are both in development, hacking on Y's 'install_requires' is
pointless.

Once X is released with the new feature, *then* you can update the
develpoment egg for 'Y' to mandate that version of 'X': at that point,
you should be removing the development egg of 'X', in order to verify
that 'Y' works with an installable 'X'.
Post by Philipp von Weitershausen
Will Y with the setup.py stating version='unreleased' satisfy the
Y>a.b requirement that X (rightfully) has? If not, then I think we
have a problem. If yes, then I find that very confusing.
'Y' can't be changed simultaneously with 'X': that is the nature of
dependency management.
Post by Philipp von Weitershausen
I know that if Y's setup.py stated version='a.b-dev', it will work.
This what my guide currently suggests (including taking it out just
for release tags).
I would *never* check in a release of 'Y' which mandated a non-released
version of 'X'.
Post by Philipp von Weitershausen
Post by Jim Fulton
Post by Benji York
After branching for a release we can set the version (e.g., 1.2),
make a release, and tag the branch.
We should not require branches. I would only bother creating
maintenance branches when they are needed.
+1
Agreed. Creating a release branch in SVN from a tag is trivial, and
therefore shouldn't be required "ahead of time."
Post by Philipp von Weitershausen
Who's Z? :)
Post by Jim Fulton
- Leave the version # out of setup.py on the trunk and on branches.
When it is time to make a release, either from the trunk or from a
- Update changes.txt, adding a heading for the new # and date
- Create a tag
- check out or switch to the tag
- Set the version in setup.py on the tag. Check it in.
- Make the release from the tag.
I could live with that, even with the fact that you'd be modifying a
tag, as long as it's done in this exact order and the only
modificdations to a tag woudl be setting setup.py.
I'm fine with making "packaging only" changes to a tag.
Post by Philipp von Weitershausen
I still see the development egg case the best argument for putting
the next anticipated version number into setup.py. I think the
compromise of version number + 'dev' marker would work best.
I'd rather have no version number at all.


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 ***@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
Tres Seaver
2007-09-27 01:45:12 UTC
Permalink
Post by Philipp von Weitershausen
Post by Jim Fulton
Post by Benji York
Post by Philipp von Weitershausen
Post by Stephan Richter
Because I disagree with that, since you cannot know the next version.
You can always know the minimum version. If you just released
3.4.2, I think it's sensible to point the next release to 3.4.3.
If you later decide that you really need a feature release, you
can always bump to 3.5.0a1 (which would be the first release in
the 3.5.x series).
Gary and Benji made me pay attention to this point. :)
I *really* don't like setting the version to the number for the
next release, since one often doesn't know what it is.
Maybe. However, when you do the actual release then, you're still
going to have to find out which version number to use. On release
branches this is a trivial step, of course: You just look at the
latest tagged version and increment accordingly. On the trunk it
might be trickier. After 1.0, do we have 1.1? Or 2.0?
Maye this aspect isn't such a big deal, though.
Post by Jim Fulton
Post by Benji York
Why not leave the version totally out of the setup.py in the trunk?
Benji and Gary won me over to this point of view. :)
There's a *really* good reason for putting the upcoming version
number in setup.py, though: development eggs.
- -100. 'development eggs' which get released into the wild are an
abomination of desolation, and should be avoided at *any* cost. (I'm
assuming that you mean eggs / distributions which get uploaded to some
package repository, rather than being installed via 'setup.py develop').
Post by Philipp von Weitershausen
Let's say X depends on Y and I'm developing them simultaneously as
development eggs in one sandbox (e.g. buildout). I add a feature to Y
that I'd like to use in X. That means I'll have to change X's version
dependency regarding Y so that it now depends on Y>a.b where a.b is
the latest release that didn't have the feature I added.
Logically speaking, you *can't* release Y with the new requirement until
*after* a release of X is availabe with the required feature: while
they are both in development, hacking on Y's 'install_requires' is
pointless.

Once X is released with the new feature, *then* you can update the
develpoment egg for 'Y' to mandate that version of 'X': at that point,
you should be removing the development egg of 'X', in order to verify
that 'Y' works with an installable 'X'.
Post by Philipp von Weitershausen
Will Y with the setup.py stating version='unreleased' satisfy the
Y>a.b requirement that X (rightfully) has? If not, then I think we
have a problem. If yes, then I find that very confusing.
'Y' can't be changed simultaneously with 'X': that is the nature of
dependency management.
Post by Philipp von Weitershausen
I know that if Y's setup.py stated version='a.b-dev', it will work.
This what my guide currently suggests (including taking it out just
for release tags).
I would *never* check in a release of 'Y' which mandated a non-released
version of 'X'.
Post by Philipp von Weitershausen
Post by Jim Fulton
Post by Benji York
After branching for a release we can set the version (e.g., 1.2),
make a release, and tag the branch.
We should not require branches. I would only bother creating
maintenance branches when they are needed.
+1
Agreed. Creating a release branch in SVN from a tag is trivial, and
therefore shouldn't be required "ahead of time."
Post by Philipp von Weitershausen
Who's Z? :)
Post by Jim Fulton
- Leave the version # out of setup.py on the trunk and on branches.
When it is time to make a release, either from the trunk or from a
- Update changes.txt, adding a heading for the new # and date
- Create a tag
- check out or switch to the tag
- Set the version in setup.py on the tag. Check it in.
- Make the release from the tag.
I could live with that, even with the fact that you'd be modifying a
tag, as long as it's done in this exact order and the only
modificdations to a tag woudl be setting setup.py.
I'm fine with making "packaging only" changes to a tag.
Post by Philipp von Weitershausen
I still see the development egg case the best argument for putting
the next anticipated version number into setup.py. I think the
compromise of version number + 'dev' marker would work best.
I'd rather have no version number at all.


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 ***@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
Philipp von Weitershausen
2007-09-26 22:43:54 UTC
Permalink
Post by Benji York
Post by Philipp von Weitershausen
Post by Stephan Richter
Because I disagree with that, since you cannot know the next
version.
You can always know the minimum version. If you just released
3.4.2, I think it's sensible to point the next release to 3.4.3.
If you later decide that you really need a feature release, you
can always bump to 3.5.0a1 (which would be the first release in
the 3.5.x series).
Why not leave the version totally out of the setup.py in the trunk?
After branching for a release we can set the version (e.g., 1.2),
make a release, and tag the branch.
We could either leave the version on the branch at the "last"
release or continue the trend of mad bumping and have it at the
"next" release (which since this is a branch, we can actually
predict).
I prefer the "last" version, but "next" would work too.
setuptools suggests setting it to the "next" version. Apart from the
fact that it makes more sense to me, the biggest reason I can see is
the development egg use case. A development egg from the repository
should have a newer version than the latest released egg.
Tres Seaver
2007-09-27 01:50:01 UTC
Permalink
Post by Philipp von Weitershausen
Post by Benji York
Post by Philipp von Weitershausen
Post by Stephan Richter
Because I disagree with that, since you cannot know the next version.
You can always know the minimum version. If you just released
3.4.2, I think it's sensible to point the next release to 3.4.3.
If you later decide that you really need a feature release, you
can always bump to 3.5.0a1 (which would be the first release in
the 3.5.x series).
Why not leave the version totally out of the setup.py in the trunk?
After branching for a release we can set the version (e.g., 1.2),
make a release, and tag the branch.
We could either leave the version on the branch at the "last"
release or continue the trend of mad bumping and have it at the
"next" release (which since this is a branch, we can actually
predict).
I prefer the "last" version, but "next" would work too.
setuptools suggests setting it to the "next" version. Apart from the
fact that it makes more sense to me, the biggest reason I can see is
the development egg use case. A development egg from the repository
should have a newer version than the latest released egg.
If it were impossible to create an 'sdist' or 'bdist_egg' from such a
checkout, I would be OK with this. However, history shows that people
*will* make distributions from such "not-ready-for-prime-time"
checkouts, at which point the damage is orders of magnitude bigger than
any value derived from convenience in installing the 'develop' egg.


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 ***@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
Tres Seaver
2007-09-27 01:50:25 UTC
Permalink
Post by Philipp von Weitershausen
Post by Benji York
Post by Philipp von Weitershausen
Post by Stephan Richter
Because I disagree with that, since you cannot know the next version.
You can always know the minimum version. If you just released
3.4.2, I think it's sensible to point the next release to 3.4.3.
If you later decide that you really need a feature release, you
can always bump to 3.5.0a1 (which would be the first release in
the 3.5.x series).
Why not leave the version totally out of the setup.py in the trunk?
After branching for a release we can set the version (e.g., 1.2),
make a release, and tag the branch.
We could either leave the version on the branch at the "last"
release or continue the trend of mad bumping and have it at the
"next" release (which since this is a branch, we can actually
predict).
I prefer the "last" version, but "next" would work too.
setuptools suggests setting it to the "next" version. Apart from the
fact that it makes more sense to me, the biggest reason I can see is
the development egg use case. A development egg from the repository
should have a newer version than the latest released egg.
If it were impossible to create an 'sdist' or 'bdist_egg' from such a
checkout, I would be OK with this. However, history shows that people
*will* make distributions from such "not-ready-for-prime-time"
checkouts, at which point the damage is orders of magnitude bigger than
any value derived from convenience in installing the 'develop' egg.


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 ***@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
Tres Seaver
2007-09-27 01:30:57 UTC
Permalink
Post by Benji York
Post by Philipp von Weitershausen
Post by Stephan Richter
Because I disagree with that, since you cannot know the next version.
You can always know the minimum version. If you just released 3.4.2, I
think it's sensible to point the next release to 3.4.3. If you later
decide that you really need a feature release, you can always bump to
3.5.0a1 (which would be the first release in the 3.5.x series).
Why not leave the version totally out of the setup.py in the trunk?
After branching for a release we can set the version (e.g., 1.2), make a
release, and tag the branch.
We could either leave the version on the branch at the "last" release or
continue the trend of mad bumping and have it at the "next" release
(which since this is a branch, we can actually predict).
I prefer the "last" version, but "next" would work too.
I prefer neither, as I don't want to encourage people to make releases
without doing the bookkeeping required to get the versions right.


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 ***@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
Tres Seaver
2007-09-27 01:35:14 UTC
Permalink
Post by Benji York
Post by Philipp von Weitershausen
Post by Stephan Richter
Because I disagree with that, since you cannot know the next version.
You can always know the minimum version. If you just released 3.4.2, I
think it's sensible to point the next release to 3.4.3. If you later
decide that you really need a feature release, you can always bump to
3.5.0a1 (which would be the first release in the 3.5.x series).
Why not leave the version totally out of the setup.py in the trunk?
After branching for a release we can set the version (e.g., 1.2), make a
release, and tag the branch.
We could either leave the version on the branch at the "last" release or
continue the trend of mad bumping and have it at the "next" release
(which since this is a branch, we can actually predict).
I prefer the "last" version, but "next" would work too.
I prefer neither, as I don't want to encourage people to make releases
without doing the bookkeeping required to get the versions right.


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 ***@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
Tres Seaver
2007-09-26 15:50:19 UTC
Permalink
Post by Stephan Richter
Post by Christian Theune
Hmm. While doing that I also noticed that we were at 3.4.0a1 yesterday
evening. The stable release was made from that without making a
maintenance branch and bumping the trunk to 3.5.
There is conflicting information here. :-) Some people say we need branches,
others say we don't. And where is an agreed-upon document that you have to
list the next version in the setup.py file after the release? Because I
disagree with that, since you cannot know the next version.
+1, The trunk should *always* say 'unreleased' or 'trunk', except in
the five minutes before cutting a release tag (if not using a release
branch). Release branches should have '-branch' or something appended
to the version, except just before cutting a release tag.

Anything else makes it too easy to cut a premature / broken release;
such mistakes are going to damange our story if we don't get out ahead
of them.


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 ***@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
Tres Seaver
2007-09-26 16:01:45 UTC
Permalink
Post by Stephan Richter
Post by Christian Theune
Hmm. While doing that I also noticed that we were at 3.4.0a1 yesterday
evening. The stable release was made from that without making a
maintenance branch and bumping the trunk to 3.5.
There is conflicting information here. :-) Some people say we need branches,
others say we don't. And where is an agreed-upon document that you have to
list the next version in the setup.py file after the release? Because I
disagree with that, since you cannot know the next version.
+1, The trunk should *always* say 'unreleased' or 'trunk', except in
the five minutes before cutting a release tag (if not using a release
branch). Release branches should have '-branch' or something appended
to the version, except just before cutting a release tag.

Anything else makes it too easy to cut a premature / broken release;
such mistakes are going to damange our story if we don't get out ahead
of them.


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 ***@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
Stefan H. Holek
2007-09-26 10:53:21 UTC
Permalink
WinZip has the habit of ignoring files it deems "empty" and paths it
deems "too long". Best to avoid.

Stefan
Post by Christian Theune
The issue is that the eggs were released as ZIP files and for some
reason those don't work correctly with the data files.
--
It doesn't necessarily do it in chronological order, though.
--Douglas Adams
Christian Theune
2007-09-26 10:56:43 UTC
Permalink
Post by Stefan H. Holek
WinZip has the habit of ignoring files it deems "empty" and paths it
deems "too long". Best to avoid.
Nothing about winzip. The files are in there. I also couldn't find an
issue in the egg info files.
--
gocept gmbh & co. kg - forsterstrasse 29 - 06112 halle/saale - germany
www.gocept.com - ***@gocept.com - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Dies ist ein digital signierter Nachrichtenteil
Url : http://mail.zope.org/pipermail/zope3-dev/attachments/20070926/b2547c14/attachment.bin
Baiju M
2007-09-26 13:02:13 UTC
Permalink
Post by Christian Theune
Post by Stefan H. Holek
WinZip has the habit of ignoring files it deems "empty" and paths it
deems "too long". Best to avoid.
Nothing about winzip. The files are in there. I also couldn't find an
issue in the egg info files.
When I manually download and extract zope.app.appsetup zip file
I can see a 'schema' folder and a ZCML file under that.
But when I easy_install, that 'schema' directory is missing.

Regards,
Baiju M
Baiju M
2007-09-26 13:04:31 UTC
Permalink
Post by Baiju M
Post by Christian Theune
Post by Stefan H. Holek
WinZip has the habit of ignoring files it deems "empty" and paths
it deems "too long". Best to avoid.
Nothing about winzip. The files are in there. I also couldn't find an
issue in the egg info files.
When I manually download and extract zope.app.appsetup zip file
I can see a 'schema' folder and a ZCML file under that.
But when I easy_install, that 'schema' directory is missing.
Oops, that is not a ZCML file but a schema.xml file

--
Baiju M
Stephan Richter
2007-09-26 14:54:24 UTC
Permalink
WinZip has the habit of ignoring files it deems "empty" and paths it ?
deems "too long". Best to avoid.
The problem is that Roger has installed TAR and GZIP. They are also available
in the path. However, for some reason it uses WinZip. Now, he has tested the
eggs on his machine and they worked. Oh man, ...

Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
Jim Fulton
2007-09-26 14:55:47 UTC
Permalink
This would be a good issue to bring up on the distutils-sig list.

Jim
Post by Stephan Richter
Post by Stefan H. Holek
WinZip has the habit of ignoring files it deems "empty" and paths it
deems "too long". Best to avoid.
The problem is that Roger has installed TAR and GZIP. They are also available
in the path. However, for some reason it uses WinZip. Now, he has tested the
eggs on his machine and they worked. Oh man, ...
Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
_______________________________________________
Zope3-dev mailing list
Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com
--
Jim Fulton
Zope Corporation
Stephan Richter
2007-09-26 14:16:31 UTC
Permalink
Post by Christian Theune
The issue is that the eggs were released as ZIP files and for some
reason those don't work correctly with the data files.
I can reproduce the problem by creating the packages myself as ZIP files
(doesn't work) and then as tar files (does work).
Okay, thanks a lot for this research. So this is a problem that noone could
anticipate. Roger tried the eggs on his machine and they worked there.

So that's something we learned: We have to make tag.gz releases.

BTW, which OS so you use? Roger told me he did several releases with Windows
at Lovely Systems and all was fine there. But then everyone else at Lovely
has Macs.
Post by Christian Theune
- Remove the broken files.
- Create tags for the wrong trunk releases
- Create a new release and tag in tar format.
I'll show what I mean by releasing a fix for zope.app.i18n.
I will review all the released eggs with Roger.

Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
Christian Theune
2007-09-26 14:25:42 UTC
Permalink
Post by Stephan Richter
Post by Christian Theune
The issue is that the eggs were released as ZIP files and for some
reason those don't work correctly with the data files.
I can reproduce the problem by creating the packages myself as ZIP files
(doesn't work) and then as tar files (does work).
Okay, thanks a lot for this research. So this is a problem that noone could
anticipate. Roger tried the eggs on his machine and they worked there.
Right -- and those subtle things are hard to find if not anticipated.
Post by Stephan Richter
So that's something we learned: We have to make tag.gz releases.
Right. However this seems like a bug in setuptools. (Sidenote:
isn't .tar.gz the default of setuptools on windows too?)
Post by Stephan Richter
BTW, which OS so you use? Roger told me he did several releases with Windows
at Lovely Systems and all was fine there. But then everyone else at Lovely
has Macs.
I'm on Linux. We at gocept have a mixture between Linux and Mac OS X
systems.
Post by Stephan Richter
I will review all the released eggs with Roger.
K, thanks.

Christian
--
gocept gmbh & co. kg - forsterstrasse 29 - 06112 halle/saale - germany
www.gocept.com - ***@gocept.com - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Dies ist ein digital signierter Nachrichtenteil
Url : http://mail.zope.org/pipermail/zope3-dev/attachments/20070926/7d8713c8/attachment.bin
Philipp von Weitershausen
2007-09-26 14:27:31 UTC
Permalink
Post by Stephan Richter
Post by Christian Theune
The issue is that the eggs were released as ZIP files and for some
reason those don't work correctly with the data files.
I can reproduce the problem by creating the packages myself as ZIP files
(doesn't work) and then as tar files (does work).
Okay, thanks a lot for this research. So this is a problem that noone could
anticipate. Roger tried the eggs on his machine and they worked there.
Yes, this is really a weird setuptools bug. Apparently it can't properly
install package metadata from ZIP distributions on Linux and Mac.
Post by Stephan Richter
So that's something we learned: We have to make tag.gz releases.
Yes. You can do this via

python setup.py sdist --format=gztar
--
http://worldcookery.com -- Professional Zope documentation and training
Tres Seaver
2007-09-26 15:53:54 UTC
Permalink
Post by Stephan Richter
Post by Christian Theune
The issue is that the eggs were released as ZIP files and for some
reason those don't work correctly with the data files.
I can reproduce the problem by creating the packages myself as ZIP files
(doesn't work) and then as tar files (does work).
Okay, thanks a lot for this research. So this is a problem that noone could
anticipate. Roger tried the eggs on his machine and they worked there.
So that's something we learned: We have to make tag.gz releases.
Why would we zip / tar files up by hand, rather than using 'setup.py sdist'?
Post by Stephan Richter
BTW, which OS so you use? Roger told me he did several releases with Windows
at Lovely Systems and all was fine there. But then everyone else at Lovely
has Macs.
Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 ***@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
Stephan Richter
2007-09-26 16:00:17 UTC
Permalink
Post by Tres Seaver
Why would we zip / tar files up by hand, rather than using 'setup.py sdist'?
We use this, but on Windows it uses winzip by default. You have to specify
the --format option as Philipp pointed out earlier.

Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
Tres Seaver
2007-09-26 16:06:07 UTC
Permalink
Post by Stephan Richter
Post by Christian Theune
The issue is that the eggs were released as ZIP files and for some
reason those don't work correctly with the data files.
I can reproduce the problem by creating the packages myself as ZIP files
(doesn't work) and then as tar files (does work).
Okay, thanks a lot for this research. So this is a problem that noone could
anticipate. Roger tried the eggs on his machine and they worked there.
So that's something we learned: We have to make tag.gz releases.
Why would we zip / tar files up by hand, rather than using 'setup.py sdist'?
Post by Stephan Richter
BTW, which OS so you use? Roger told me he did several releases with Windows
at Lovely Systems and all was fine there. But then everyone else at Lovely
has Macs.
Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 ***@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
Gary Poster
2007-09-26 14:39:50 UTC
Permalink
Post by Christian Theune
- Remove the broken files.
I'm not sure if this is related, but I noticed yesterday at least of
couple of eggs that we are using had been removed, in this case from
zope.org. Please, whoever is doing this, stop. If a release is
brown-bagged as far as you are concerned for some reason, please do
not assume it is ok to delete for others.

I should make our own private copies of the eggs we use, to
completely isolate us from these sorts of things, but I have not
gotten around to it...nor am I thrilled at the prospect of that
overhead.

Gary
Fred Drake
2007-09-26 15:10:26 UTC
Permalink
Post by Gary Poster
I should make our own private copies of the eggs we use, to
completely isolate us from these sorts of things, but I have not
gotten around to it...nor am I thrilled at the prospect of that
overhead.
This is also a technical issue: As long as zc.buildout and setuptools
foolishly accept dependency links from an egg, it'll be painful to
detect accidental reliance on external repositories.


-Fred
--
Fred L. Drake, Jr. <fdrake at gmail.com>
"Chaos is the score upon which reality is written." --Henry Miller
Jim Fulton
2007-09-26 15:35:07 UTC
Permalink
Post by Fred Drake
Post by Gary Poster
I should make our own private copies of the eggs we use, to
completely isolate us from these sorts of things, but I have not
gotten around to it...nor am I thrilled at the prospect of that
overhead.
This is also a technical issue: As long as zc.buildout and setuptools
foolishly accept dependency links from an egg, it'll be painful to
detect accidental reliance on external repositories.
That's a good point. I wouldn't go so far as to say "foolishly", but
I would say that this is a policy that should be overrideable.
Checkins to buildout (with tests, of course) accepted.

Jim

--
Jim Fulton
Zope Corporation
Dieter Maurer
2007-09-26 20:32:46 UTC
Permalink
Post by Gary Poster
...
Post by Christian Theune
- Remove the broken files.
I'm not sure if this is related, but I noticed yesterday at least of
couple of eggs that we are using had been removed, in this case from
zope.org. Please, whoever is doing this, stop. If a release is
brown-bagged as far as you are concerned for some reason, please do
not assume it is ok to delete for others.
I do not agree.

When I have understood Christian correctly, then these distributions
were not working at all (they lacked ZCML files).

Distributions not working at all should be deleted as soon
as one recognizes they are not working at all -- to limit
the damage they may cause.

Of course, not having a tag in the SVN repository is
not a sufficient reason to remove otherwise fully working
distributions again.
--
Dieter
Gary Poster
2007-09-26 20:47:33 UTC
Permalink
Post by Dieter Maurer
Post by Gary Poster
...
Post by Christian Theune
- Remove the broken files.
I'm not sure if this is related, but I noticed yesterday at least of
couple of eggs that we are using had been removed, in this case from
zope.org. Please, whoever is doing this, stop. If a release is
brown-bagged as far as you are concerned for some reason, please do
not assume it is ok to delete for others.
I do not agree.
When I have understood Christian correctly, then these distributions
were not working at all (they lacked ZCML files).
Distributions not working at all should be deleted as soon
as one recognizes they are not working at all -- to limit
the damage they may cause.
Of course, not having a tag in the SVN repository is
not a sufficient reason to remove otherwise fully working
distributions again.
If you released some software that does an os.system('rm -rf .') or
something, then, OK, maybe you can convince me. So, yes, I have a
line past which maybe deletion is ok with me.

But I disagree with yours--code is still importable without ZCML
files and I may only be using a given package in that way and relying
on it in my buildouts, not having noticed the lack of zcml because it
is irrelevant to me.

In the vast majority of cases--for instance, in the case of someone
who apparently removed a zope.app.wsgi egg recently--it simply
shouldn't be done.

So, yes, you are right, I stated an extreme position and I could be
argued away from the edge. But the extremity of my position is a
simple, followable rule that I certainly prefer over the case you
describe.

Gary
Jim Fulton
2007-09-26 12:28:03 UTC
Permalink
Post by Christian Theune
This is IMHO a good example why we shouldn't go for 'everyone can make a
release'.
I had the same feeling yesterday. I kept silent because the
alternative is that only a few -- including me -- make a release. :)
This deserves more thought though.

Jim

--
Jim Fulton
Zope Corporation
Stephan Richter
2007-09-26 14:12:18 UTC
Permalink
Post by Christian Theune
- zope.securitypolicy
- zope.session, zope.app.session
- zope.app.authentication
- zope.app.i18n
- zope.i18nmessageid
- zope.app.applicationcontrol
- zope.app.appsetup
I'll go through those now and try to fix it up.
I'll review those with Roger and fix what needs to be fixed.
Post by Christian Theune
This is IMHO a good example why we shouldn't go for 'everyone can make a
release'.
I totally disagree. If we trust people with repository access, then we have to
trust them with release making. If you subscribe to the egg process, you have
to do frequent releases.

Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
Wichert Akkerman
2007-09-26 14:20:46 UTC
Permalink
Post by Stephan Richter
I totally disagree. If we trust people with repository access, then we have to
trust them with release making. If you subscribe to the egg process, you have
to do frequent releases.
Why would eggs require more frequent releases?

Wichert.
--
Wichert Akkerman <***@wiggy.net> It is simple to make things.
http://www.wiggy.net/ It is hard to make things simple.
Stephan Richter
2007-09-26 14:56:11 UTC
Permalink
Post by Wichert Akkerman
Post by Stephan Richter
I totally disagree. If we trust people with repository access, then we
have to trust them with release making. If you subscribe to the egg
process, you have to do frequent releases.
Why would eggs require more frequent releases?
Have you tried working with a pure egg-based project? It's unavoidable.

Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
Wichert Akkerman
2007-09-26 14:57:54 UTC
Permalink
Post by Stephan Richter
Post by Wichert Akkerman
Post by Stephan Richter
I totally disagree. If we trust people with repository access, then we
have to trust them with release making. If you subscribe to the egg
process, you have to do frequent releases.
Why would eggs require more frequent releases?
Have you tried working with a pure egg-based project? It's unavoidable.
I have worked on large projects where you had modules and you could only
use released versions. I don't see the problem.

Wichert.
--
Wichert Akkerman <***@wiggy.net> It is simple to make things.
http://www.wiggy.net/ It is hard to make things simple.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.zope.org/pipermail/zope3-dev/attachments/20070926/74a445e7/attachment.htm
Martijn Faassen
2007-09-26 14:46:00 UTC
Permalink
Post by Stephan Richter
Post by Christian Theune
- zope.securitypolicy
- zope.session, zope.app.session
- zope.app.authentication
- zope.app.i18n
- zope.i18nmessageid
- zope.app.applicationcontrol
- zope.app.appsetup
I'll go through those now and try to fix it up.
I'll review those with Roger and fix what needs to be fixed.
Post by Christian Theune
This is IMHO a good example why we shouldn't go for 'everyone can make a
release'.
I totally disagree. If we trust people with repository access, then we have to
trust them with release making. If you subscribe to the egg process, you have
to do frequent releases.
I think tagging things in svn is a minimal requirement, though. I
understood that some of this stuff wasn't tagged?

Regards,

Martijn
Stephan Richter
2007-09-26 14:58:03 UTC
Permalink
Post by Martijn Faassen
I think tagging things in svn is a minimal requirement, though. I
understood that some of this stuff wasn't tagged?
I agree. And Roger simply forgot. As Marius pointed out, as we do more
releases, problems like this will occur frequently. Making mistakes is human,
so let's develop a tool that does some basic checking.

Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
Philipp von Weitershausen
2007-09-26 15:13:34 UTC
Permalink
Post by Stephan Richter
Post by Martijn Faassen
I think tagging things in svn is a minimal requirement, though. I
understood that some of this stuff wasn't tagged?
I agree. And Roger simply forgot. As Marius pointed out, as we do more
releases, problems like this will occur frequently. Making mistakes is human,
so let's develop a tool that does some basic checking.
I'm not yet convinced that we will have more releases. We have more
releases now because we're trying to get to stable versions. After that,
the eggs are on their own and I think we'll make much less releases.

But even if we got more releases to do, I don't see the problem. With
more releases we'd get more practice and make less mistakes.

One of the things we should realize in this process is that -- even
though setuptools makes it so damn easy to register and uplaod stuff to
PyPI -- releases are actually important business. They can't just be
created in a jiffy. They require attention and concentration. And a
well-defined process.
--
http://worldcookery.com -- Professional Zope documentation and training
Jim Fulton
2007-09-26 15:36:07 UTC
Permalink
Well said.

Jim
Post by Philipp von Weitershausen
Post by Stephan Richter
Post by Martijn Faassen
I think tagging things in svn is a minimal requirement, though. I
understood that some of this stuff wasn't tagged?
I agree. And Roger simply forgot. As Marius pointed out, as we do
more releases, problems like this will occur frequently. Making
mistakes is human, so let's develop a tool that does some basic
checking.
I'm not yet convinced that we will have more releases. We have more
releases now because we're trying to get to stable versions. After
that, the eggs are on their own and I think we'll make much less
releases.
But even if we got more releases to do, I don't see the problem.
With more releases we'd get more practice and make less mistakes.
One of the things we should realize in this process is that -- even
though setuptools makes it so damn easy to register and uplaod
stuff to PyPI -- releases are actually important business. They
can't just be created in a jiffy. They require attention and
concentration. And a well-defined process.
--
http://worldcookery.com -- Professional Zope documentation and
training
_______________________________________________
Zope3-dev mailing list
Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com
--
Jim Fulton
Zope Corporation
Stephan Richter
2007-09-26 16:03:13 UTC
Permalink
Post by Philipp von Weitershausen
Post by Stephan Richter
Post by Martijn Faassen
I think tagging things in svn is a minimal requirement, though. I
understood that some of this stuff wasn't tagged?
I agree. And Roger simply forgot. As Marius pointed out, as we do more
releases, problems like this will occur frequently. Making mistakes is
human, so let's develop a tool that does some basic checking.
I'm not yet convinced that we will have more releases. We have more
releases now because we're trying to get to stable versions. After that,
the eggs are on their own and I think we'll make much less releases.
I would not hold my breath on this one, but I have unsupported hope you are
right. ;-)
Post by Philipp von Weitershausen
But even if we got more releases to do, I don't see the problem. With
more releases we'd get more practice and make less mistakes.
This is logically incorrect. It is the same as saying: More guns make us
safer.

The error rate would have to drop by the same factor the releases are
increased, which I do not think will be the case, especially since more
people are doing releasing now.,
Post by Philipp von Weitershausen
One of the things we should realize in this process is that -- even
though setuptools makes it so damn easy to register and uplaod stuff to
PyPI -- releases are actually important business. They can't just be
created in a jiffy. They require attention and concentration. And a
well-defined process.
I agree. But all the process in the world will not drop the error rate to
zero. I think that Marius' tool suggestion is very good and I also think we
should seriously look at how Linux distributions manage that process. Having
some staging mechanism would be good.

Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
Dieter Maurer
2007-09-26 20:29:04 UTC
Permalink
Post by Stephan Richter
...
I totally disagree. If we trust people with repository access, then we have to
trust them with release making. If you subscribe to the egg process, you have
to do frequent releases.
Maybe, if you fix dependancies to single versions ;-)
--
Dieter
Loading...