Discussion:
[Rock-dev] Rock Release 1408
Janosch Machowinski
2014-09-03 14:56:01 UTC
Permalink
Hey,
rock1408_rc1 has been out there for two weeks now.
Until now there have been zero bug reports, and also
zero feedback. If no one has objections, or discovers
any bugs until tomorrow, I would make rc1 a final release
then.
Greetings
Janosch
--
Dipl. Inf. Janosch Machowinski
SAR- & Sicherheitsrobotik

Universit?t Bremen
FB 3 - Mathematik und Informatik
AG Robotik
Robert-Hooke-Stra?e 1
28359 Bremen, Germany

Zentrale: +49 421 178 45-6611

Besuchsadresse der Nebengesch?ftstelle:
Robert-Hooke-Stra?e 5
28359 Bremen, Germany

Tel.: +49 421 178 45-6614
Empfang: +49 421 178 45-6600
Fax: +49 421 178 45-4150
E-Mail: jmachowinski at informatik.uni-bremen.de

Weitere Informationen: http://www.informatik.uni-bremen.de/robotik
Sylvain Joyeux
2014-09-03 15:01:42 UTC
Permalink
Sounds good to me. I'll update the stable branches as well as the
stable flavor once you've released it.

On Wed, Sep 3, 2014 at 11:56 AM, Janosch Machowinski
Post by Janosch Machowinski
Hey,
rock1408_rc1 has been out there for two weeks now.
Until now there have been zero bug reports, and also
zero feedback. If no one has objections, or discovers
any bugs until tomorrow, I would make rc1 a final release
then.
Greetings
Janosch
--
Dipl. Inf. Janosch Machowinski
SAR- & Sicherheitsrobotik
Universit?t Bremen
FB 3 - Mathematik und Informatik
AG Robotik
Robert-Hooke-Stra?e 1
28359 Bremen, Germany
Zentrale: +49 421 178 45-6611
Robert-Hooke-Stra?e 5
28359 Bremen, Germany
Tel.: +49 421 178 45-6614
Empfang: +49 421 178 45-6600
Fax: +49 421 178 45-4150
E-Mail: jmachowinski at informatik.uni-bremen.de
Weitere Informationen: http://www.informatik.uni-bremen.de/robotik
_______________________________________________
Rock-dev mailing list
Rock-dev at dfki.de
http://www.dfki.de/mailman/cgi-bin/listinfo/rock-dev
Martin Zenzes
2014-09-03 16:03:24 UTC
Permalink
How man people (besides maybe a small number of core-guys) are using the
branch on a actual system? I could imagine a number close to zero --
changing the branch from a "working" checkout alone takes time, testing
and bug-reporting as well...

Greetings
Post by Sylvain Joyeux
Sounds good to me. I'll update the stable branches as well as the
stable flavor once you've released it.
On Wed, Sep 3, 2014 at 11:56 AM, Janosch Machowinski
Post by Janosch Machowinski
Hey,
rock1408_rc1 has been out there for two weeks now.
Until now there have been zero bug reports, and also
zero feedback. If no one has objections, or discovers
any bugs until tomorrow, I would make rc1 a final release
then.
Greetings
Janosch
--
Dipl. Inf. Janosch Machowinski
SAR- & Sicherheitsrobotik
Universit?t Bremen
FB 3 - Mathematik und Informatik
AG Robotik
Robert-Hooke-Stra?e 1
28359 Bremen, Germany
Zentrale: +49 421 178 45-6611
Robert-Hooke-Stra?e 5
28359 Bremen, Germany
Tel.: +49 421 178 45-6614
Empfang: +49 421 178 45-6600
Fax: +49 421 178 45-4150
E-Mail: jmachowinski at informatik.uni-bremen.de
Weitere Informationen: http://www.informatik.uni-bremen.de/robotik
_______________________________________________
Rock-dev mailing list
Rock-dev at dfki.de
http://www.dfki.de/mailman/cgi-bin/listinfo/rock-dev
_______________________________________________
Rock-dev mailing list
Rock-dev at dfki.de
http://www.dfki.de/mailman/cgi-bin/listinfo/rock-dev
--
M.Sc. Martin Zenzes
Space Robotics

Hauptgesch?ftsstelle Standort Bremen:
DFKI GmbH
Robotics Innovation Center
Robert-Hooke-Stra?e 5
28359 Bremen, Germany

Phone: +49 (0) 421 178 45 - 6658
Fax: +49 (0) 421 178 45 - 4150
E-Mail: martin.zenzes at dfki.de

Weitere Informationen: http://www.dfki.de/robotik
-----------------------------------------------------------------------
Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH
Firmensitz: Trippstadter Stra?e 122, D-67663 Kaiserslautern
Geschaeftsfuehrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster
(Vorsitzender) Dr. Walter Olthoff
Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes
Amtsgericht Kaiserslautern, HRB 2313
Sitz der Gesellschaft: Kaiserslautern (HRB 2313)
USt-Id.Nr.: DE 148646973
Steuernummer: 19/673/0060/3
-----------------------------------------------------------------------
Sylvain Joyeux
2014-09-03 16:42:36 UTC
Permalink
Post by Martin Zenzes
How man people (besides maybe a small number of core-guys) are using the
branch on a actual system? I could imagine a number close to zero --
changing the branch from a "working" checkout alone takes time, testing
and bug-reporting as well...
Given that, doing this, the 'stable' would become 'the latest
release', it would definitely become the development branch for the
projects I manage. And it would have close to zero extra work since it
would be "the latest release".

I am really wary of the whole release thing. Would there be a document
explaining how it solves the current problems without adding new ones,
maybe I would not. It, *in addition*, adds new problems (the most
problematic being the complete inability to mix the release code with
in-devel code). It has been dropped as the solution to all the
problems, but nobody wrote down *how* it solves them and what are its
downsides.

But, at least, *I know how to work around the limitations of the
stable flavor*. To finish, I don't force anybody to use it.

Sylvain
Janosch Machowinski
2014-09-03 17:26:55 UTC
Permalink
The main issue that is solve by this, is that you don't get
updated, because stable/next/master moved on. Now you
need to change the release explicitly.

Also I tried to freeze all external dependencies. The only
ones not being frozen are the ruby ones. Is there a way
to specify, which versions of the gems get installed ?

I don't see why you can't mix devel with release code.
You just set the override for the branch to whatever
you want, done.

Greetings
Janosch
Post by Sylvain Joyeux
Post by Martin Zenzes
How man people (besides maybe a small number of core-guys) are using the
branch on a actual system? I could imagine a number close to zero --
changing the branch from a "working" checkout alone takes time, testing
and bug-reporting as well...
Given that, doing this, the 'stable' would become 'the latest
release', it would definitely become the development branch for the
projects I manage. And it would have close to zero extra work since it
would be "the latest release".
I am really wary of the whole release thing. Would there be a document
explaining how it solves the current problems without adding new ones,
maybe I would not. It, *in addition*, adds new problems (the most
problematic being the complete inability to mix the release code with
in-devel code). It has been dropped as the solution to all the
problems, but nobody wrote down *how* it solves them and what are its
downsides.
But, at least, *I know how to work around the limitations of the
stable flavor*. To finish, I don't force anybody to use it.
Sylvain
_______________________________________________
Rock-dev mailing list
Rock-dev at dfki.de
http://www.dfki.de/mailman/cgi-bin/listinfo/rock-dev
Sylvain Joyeux
2014-09-04 00:52:39 UTC
Permalink
Post by Janosch Machowinski
I tested the branch for Asguard and Coyote.
By testing I mean running the robot and
navigate in the sand field.
That is very impressive. However, it is not what I am discussing. I
*know* that you want releases to be rock-solid (pun not intended), and
I am very grateful that you took so much time testing it. But the
question is about how the releases are going to work, not how
well-tested they are. Ideally, previous Rock release should have been
as well-tested (but were not).

In other words, the issue I have is the way the releases are
implemented, not the fact that we have one.
Post by Janosch Machowinski
The main issue that is solve by this, is that you don't get
updated, because stable/next/master moved on. Now you
need to change the release explicitly.
What puzzles me is that you seem to think that the flavor system goes
against having a proper release (at least, that's how I explain how
much you changed the package sets and the name of the wiki page). The
two thing needed to change a stable flavor into a release is:
- adding a new flavor that inherits stable with the right name. Then
you get the branching right (i.e. autoproj tracks the release name
instead of 'stable')
- adding the necessary overrides, which is something a little bit of
scripting could do very easily (all the infrastructure is there
because of "autoproj snapshot")

What you do get is:
- a way for people to tell "package X needs to go in the release"
(they move it to the stable flavor)
- very little changes between the dev and released system
(package-set-wise), thus avoiding build issues, forgotten packages
that got deleted, this kind of stuff.
Post by Janosch Machowinski
Also I tried to freeze all external dependencies. The only
ones not being frozen are the ruby ones. Is there a way
to specify, which versions of the gems get installed ?
Yes, but that's going to be a pain. Enabling it is part of the whole
snapshotting work.
Post by Janosch Machowinski
I don't see why you can't mix devel with release code.
You just set the override for the branch to whatever
you want, done.
This fails if:
- the master branch needs an osdeps that was not declared at release time
- you want to try a package that did not exist at release time
- the master branch depends on a package that did not exist at release time
- the way the package got built changed between the release and now
- the repository URL changed between the release and now
- there are differences in the way the package is patched
- probably other that I don't remember

I've not implemented support for all these in the flavor system just
for fun ... They were all concrete problems that people encountered
(the most problematic one being gui/vizkit that changed from
cmake_package to ruby_package).

I also believe that synchronizing the release of rock-core and the
rest makes no sense, but see

http://rock.opendfki.de/wiki/WikiStart/OngoingWork/RockFlavorsOrReleases

Sylvain
Janosch Machowinski
2014-09-04 07:20:58 UTC
Permalink
Post by Sylvain Joyeux
Post by Janosch Machowinski
I don't see why you can't mix devel with release code.
You just set the override for the branch to whatever
you want, done.
The idea is now a bit different, we (as discussed internally) want
the release to be as reporoducable as possible. Meaning, if I check
it out 4 years later it should behave as on day one. Including all the
bugs it had. Also release are only supported on the Distros, they
explicitly support.
Post by Sylvain Joyeux
- you want to try a package that did not exist at release time
Package should be then on a 'additional packages' package set. If you
want non release packages, you import an extra package set.
Post by Sylvain Joyeux
- the master branch needs an osdeps that was not declared at release time
- the master branch depends on a package that did not exist at release time
Agreed, these use cases are rather rough as you need to create a custom
package
set / add it to your project package set.
Post by Sylvain Joyeux
- the way the package got built changed between the release and now
This is something we do not want. If the package was build before without
option X, we want it to stay this way. As enabling option X might change
the behavior of the package.
Post by Sylvain Joyeux
- the repository URL changed between the release and now
The version we are interested in, for the release, should hopefully
still be at
the 'old' url. Actually there were thoughts to mirror external repositories,
to be sure that we can rebuild anything years later.
Post by Sylvain Joyeux
- there are differences in the way the package is patched
Same as above. Might change runtime behavior.
Post by Sylvain Joyeux
- probably other that I don't remember
I've not implemented support for all these in the flavor system just
for fun ... They were all concrete problems that people encountered
(the most problematic one being gui/vizkit that changed from
cmake_package to ruby_package).
I remember this. The problem we wanted to solve back then was
to make it easy to mix the branches, and have only one set of
osdeps and package definitons. As said above, this does not
match the use case of the release, as the target is reproducibility.
All the osdeps need to be frozen to a specific commit / version,
which is the opposite of how we wanted it in the master/next/stable
system.
Greetings
Janosch
--
Dipl. Inf. Janosch Machowinski
SAR- & Sicherheitsrobotik

Universit?t Bremen
FB 3 - Mathematik und Informatik
AG Robotik
Robert-Hooke-Stra?e 1
28359 Bremen, Germany

Zentrale: +49 421 178 45-6611

Besuchsadresse der Nebengesch?ftstelle:
Robert-Hooke-Stra?e 5
28359 Bremen, Germany

Tel.: +49 421 178 45-6614
Empfang: +49 421 178 45-6600
Fax: +49 421 178 45-4150
E-Mail: jmachowinski at informatik.uni-bremen.de

Weitere Informationen: http://www.informatik.uni-bremen.de/robotik
Sylvain Joyeux
2014-09-05 14:02:02 UTC
Permalink
Post by Janosch Machowinski
Post by Sylvain Joyeux
- you want to try a package that did not exist at release time
Package should be then on a 'additional packages' package set. If you
want non release packages, you import an extra package set.
So ... Now you want a 'stable' package set and a 'master' package set
? What about packages whose property change between stable and master
(long list already posted ?). We need to duplicate ?

Why change so much when the flavor system already provides a good base
to migrate to a "release frozen in time" system ? You clearly
completely misunderstand how the flavor system works. The stable
flavor *already* provides all the guarantees you are talking about
*until the next stable is pushed*, while still allowing to easily mix
master and stable. Branching the package set "just before the next
release" would then provide release-guarantees for the years to come.

In other words, the heavy patching you've done in the release branches
is really only making the whole release process more complex, and
reducing the usability of Rock for its developers.
Post by Janosch Machowinski
Post by Sylvain Joyeux
- the master branch needs an osdeps that was not declared at release time
- the master branch depends on a package that did not exist at release time
Agreed, these use cases are rather rough as you need to create a custom
package set / add it to your project package set.
Post by Sylvain Joyeux
- the way the package got built changed between the release and now
This is something we do not want. If the package was build before without
option X, we want it to stay this way. As enabling option X might change
the behavior of the package.
What I am talking about is: what if the stable branch of a package and
the master branch of a package need different build options ? This
happened many times before.
Post by Janosch Machowinski
Post by Sylvain Joyeux
- the repository URL changed between the release and now
The version we are interested in, for the release, should hopefully
still be at the 'old' url. Actually there were thoughts to mirror external repositories,
to be sure that we can rebuild anything years later.
Again, I am talking about using the devel version of a released
package while being on a release.
Post by Janosch Machowinski
Post by Sylvain Joyeux
- there are differences in the way the package is patched
Same as above. Might change runtime behavior.
And same as above, I am talking about using the devel version of a
released package while being on a release. Of course it changes
runtime behaviour, it is what the user *wants*.

There is no "pure user" in Rock (or in ROS for that matter). All
"users" also develop packages, which might or might not be part of the
latest Rock release. Any way to "release" rock will have to cater to
these.
Post by Janosch Machowinski
Post by Sylvain Joyeux
I've not implemented support for all these in the flavor system just
for fun ... They were all concrete problems that people encountered
(the most problematic one being gui/vizkit that changed from
cmake_package to ruby_package).
I remember this. The problem we wanted to solve back then was
to make it easy to mix the branches, and have only one set of
osdeps and package definitons. As said above, this does not
match the use case of the release, as the target is reproducibility.
Of course it does if you branch just before *the next release*.
Post by Janosch Machowinski
All the osdeps need to be frozen to a specific commit / version,
which is the opposite of how we wanted it in the master/next/stable
system.
No, it is not. 'stable' was always meant to be fixed until the next
stable release. What we did not bother with is providing a way to go
back to older releases.

Sylvain
Janosch Machowinski
2014-09-05 15:51:58 UTC
Permalink
Post by Sylvain Joyeux
Post by Janosch Machowinski
Post by Sylvain Joyeux
- you want to try a package that did not exist at release time
Package should be then on a 'additional packages' package set. If you
want non release packages, you import an extra package set.
So ... Now you want a 'stable' package set and a 'master' package set
? What about packages whose property change between stable and master
(long list already posted ?). We need to duplicate ?
Why change so much when the flavor system already provides a good base
to migrate to a "release frozen in time" system ? You clearly
completely misunderstand how the flavor system works. The stable
flavor *already* provides all the guarantees you are talking about
*until the next stable is pushed*, while still allowing to easily mix
master and stable. Branching the package set "just before the next
release" would then provide release-guarantees for the years to come.
In other words, the heavy patching you've done in the release branches
is really only making the whole release process more complex, and
reducing the usability of Rock for its developers.
I seriously don't see your point. We started all packages of in projects
and after they got some maturity we moved them over to the global
package set. Nothing changed here.
Post by Sylvain Joyeux
What I am talking about is: what if the stable branch of a package and
the master branch of a package need different build options ? This
happened many times before.
Then this should be handled in you local overrides, were you also
change the branch to master/whatever
Post by Sylvain Joyeux
Post by Janosch Machowinski
Post by Sylvain Joyeux
- the repository URL changed between the release and now
The version we are interested in, for the release, should hopefully
still be at the 'old' url. Actually there were thoughts to mirror external repositories,
to be sure that we can rebuild anything years later.
Again, I am talking about using the devel version of a released
package while being on a release.
In this case you need to overwrite the url to your fork repo anyway.
You should only need to change to a devel version, if you are
actually working on it.
Post by Sylvain Joyeux
Post by Janosch Machowinski
Post by Sylvain Joyeux
- there are differences in the way the package is patched
Same as above. Might change runtime behavior.
And same as above, I am talking about using the devel version of a
released package while being on a release. Of course it changes
runtime behaviour, it is what the user *wants*.
Definitely not. I don't want my stable packages to be patched
because something in devel changed. Patches are defined in
the source.yml and apply to all version.
Post by Sylvain Joyeux
There is no "pure user" in Rock (or in ROS for that matter). All
"users" also develop packages, which might or might not be part of the
latest Rock release. Any way to "release" rock will have to cater to
these.
For 90% of the packages I use, on 'my' robots, I don't do any
development. I just stay on the release version. The only ones
I actively develop are the project / robot specific packages. These
ones are in my own package set. So yes, we are gladly at a point,
were a user does not need to touch every package.
Post by Sylvain Joyeux
Post by Janosch Machowinski
All the osdeps need to be frozen to a specific commit / version,
which is the opposite of how we wanted it in the master/next/stable
system.
No, it is not. 'stable' was always meant to be fixed until the next
stable release. What we did not bother with is providing a way to go
back to older releases.
Perhaps it was meant as this, but it was not working this way. If you
changed a osdep before, it applied to all three flavors. Same for the
source.yml. If someone changed the git commit for external/gmapping,
this would have applied to all of them and by chance broke them all.
Greetings
Janosch
--
Dipl. Inf. Janosch Machowinski
SAR- & Sicherheitsrobotik

Universit?t Bremen
FB 3 - Mathematik und Informatik
AG Robotik
Robert-Hooke-Stra?e 1
28359 Bremen, Germany

Zentrale: +49 421 178 45-6611

Besuchsadresse der Nebengesch?ftstelle:
Robert-Hooke-Stra?e 5
28359 Bremen, Germany

Tel.: +49 421 178 45-6614
Empfang: +49 421 178 45-6600
Fax: +49 421 178 45-4150
E-Mail: jmachowinski at informatik.uni-bremen.de

Weitere Informationen: http://www.informatik.uni-bremen.de/robotik
Sylvain Joyeux
2014-09-06 12:10:32 UTC
Permalink
Post by Janosch Machowinski
I seriously don't see your point. We started all packages of in projects
and after they got some maturity we moved them over to the global
package set. Nothing changed here.
That's definitely not how all packages ended up in Rock, and whether
it is a good way to do it is arguable. Some packages stay "in devel"
in private package sets for more than a year, which leads to others to
start the same developments because these packages are not available
(and they can't even know that they exist). And it obviously hinders
collaboration, as the private package sets are not available to
everyone.

What you basically say in the rest is: using devel version of packages
will require the user to look into the devel branch of the package set
and copy/paste all the necessary bits. A.k.a. a real pain. You claim
it is worth it. It very well might be, but that's a decision that
should be made willingly, not under the hood "just because you decided
so".

That brings me back to the start of the discussion: I *will* maintain
the stable flavor by synchronizing it with the release. If anything,
this discussion really demonstrates that you've given close to zero
thoughts about the consequences of the change. And are definitely not
ready to inform our user how this change will dramatically modify
their workflow.
Post by Janosch Machowinski
Definitely not. I don't want my stable packages to be patched
because something in devel changed. Patches are defined in
the source.yml and apply to all version.
Patches don't have to be defined in source.yml and can be made
version-specific. And there's nothing that says that we can't improve
autoproj on the way to make it nicer for releases.
Post by Janosch Machowinski
Post by Sylvain Joyeux
There is no "pure user" in Rock (or in ROS for that matter). All
"users" also develop packages, which might or might not be part of the
latest Rock release. Any way to "release" rock will have to cater to
these.
For 90% of the packages I use, on 'my' robots, I don't do any
development. I just stay on the release version.
The only ones
I actively develop are the project / robot specific packages. These
ones are in my own package set. So yes, we are gladly at a point,
were a user does not need to touch every package.
We don't live in the same world at all, it seems. Let's just think
about all the patches that ended up in trajectory_follower, in the
corridor servoing, in envire, in vizkit and even a few bugfixes to
orogen and typelib (to name only a few) during the preparation of
spacebot last year. They are all packages that were released at the
time.
Post by Janosch Machowinski
Perhaps it was meant as this, but it was not working this way. If you
changed a osdep before, it applied to all three flavors. Same for the
source.yml. If someone changed the git commit for external/gmapping,
this would have applied to all of them and by chance broke them all.
That's exactly what I was saying. It is *meant to be that way*.
Scrapping everything (and, in the way, dropping some currently heavily
used functionality) because a few aspects don't work "just the way you
want" instead of looking first into incrementally evolving the current
system is a very obvious symptom of NME.

Then there's the question of how the release should be done / managed.
But did you actually bother reading the wiki page ?

That was my point at the beginning of this thread: you want to release
yesterday, and *of course* it has to be done in a new way *right now*.
Can't release the way everyone knows how to deal with (using
snapshotting to ensure repeatability), and wait for a "100% frozen"
release until we figure out how to do it best. So let's break
everything, because keeping the current way while test-driving a
release system would be more confusing to users. Users that know how
to work within the flavor system will be less confused by a completely
new one. That's obvious.

Sylvain

Janosch Machowinski
2014-09-03 17:13:49 UTC
Permalink
I tested the branch for Asguard and Coyote.
By testing I mean running the robot and
navigate in the sand field.

For spacebot is is building at least, but as
part of spacebot is switched to orogen_loaders
nothing is working for me there.

The branch is known to compile on the 'usual suspects'
Ubuntu 12.04 / 14.04 and Debian stable.
Greetings
Janosch
Post by Martin Zenzes
How man people (besides maybe a small number of core-guys) are using the
branch on a actual system? I could imagine a number close to zero --
changing the branch from a "working" checkout alone takes time, testing
and bug-reporting as well...
Greetings
Post by Sylvain Joyeux
Sounds good to me. I'll update the stable branches as well as the
stable flavor once you've released it.
On Wed, Sep 3, 2014 at 11:56 AM, Janosch Machowinski
Post by Janosch Machowinski
Hey,
rock1408_rc1 has been out there for two weeks now.
Until now there have been zero bug reports, and also
zero feedback. If no one has objections, or discovers
any bugs until tomorrow, I would make rc1 a final release
then.
Greetings
Janosch
--
Dipl. Inf. Janosch Machowinski
SAR- & Sicherheitsrobotik
Universit?t Bremen
FB 3 - Mathematik und Informatik
AG Robotik
Robert-Hooke-Stra?e 1
28359 Bremen, Germany
Zentrale: +49 421 178 45-6611
Robert-Hooke-Stra?e 5
28359 Bremen, Germany
Tel.: +49 421 178 45-6614
Empfang: +49 421 178 45-6600
Fax: +49 421 178 45-4150
E-Mail: jmachowinski at informatik.uni-bremen.de
Weitere Informationen: http://www.informatik.uni-bremen.de/robotik
_______________________________________________
Rock-dev mailing list
Rock-dev at dfki.de
http://www.dfki.de/mailman/cgi-bin/listinfo/rock-dev
_______________________________________________
Rock-dev mailing list
Rock-dev at dfki.de
http://www.dfki.de/mailman/cgi-bin/listinfo/rock-dev
Loading...