[Rock-dev] typelib: merging toolchain-2.7 into master / OS 10.9

Discussion:

Alexander Duda

2014-05-20 09:49:03 UTC

Hi,

is there a particular reason why toolchain-2.7 is not merged into
master? I am asking because the current master is still using dyncall
0.6 which is not compatible with OS X 10.9.

Furthermore there are a number of issues with the current build on OSX
but I think they also apply for other Os:
* rpath is not turned on for all targets therefore typelib_ruby cannot
find some libs
* rpath seperator is ";" instead of the used ":"
* typelib_ruby is build as module. But at the same time its header are
used by orocos.rb. This results in undefined references which works for
gcc but not for clang. Therefore, either we should install a typlib_ruby
lib or use header only.

I have a merged and fixed version on my machine. Shall I create a pull
request?

Greets Alex
--
Dipl.-Ing. Alexander Duda
Unterwasserrobotik
Robotics Innovation Center

Hauptgesch?ftsstelle Standort Bremen:
DFKI GmbH
Robotics Innovation Center
Robert-Hooke-Stra?e 1
28359 Bremen, Germany

Tel.: +49 421 178 45-6620
Zentrale: +49 421 178 45-0
Fax: +49 421 178 45-4150 (Faxe bitte namentlich kennzeichnen)
E-Mail: Alexander.Duda at dfki.de

Weitere Informationen: http://www.dfki.de/robotik
-----------------------------------------------------------------------
Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH
Firmensitz: Trippstadter Stra?e 122, D-67663 Kaiserslautern
Geschaeftsfuehrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster
(Vorsitzender) Dr. Walter Olthoff
Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes
Amtsgericht Kaiserslautern, HRB 2313
Sitz der Gesellschaft: Kaiserslautern (HRB 2313)
USt-Id.Nr.: DE 148646973
Steuernummer: 19/673/0060/3

Sylvain Joyeux

2014-05-20 13:17:02 UTC

Permalink

Post by Alexander Duda
Hi,
is there a particular reason why toolchain-2.7 is not merged into
master?

Because the orocos toolchain developers develop on the side because that's
easier for them, and believe that it is my job to merge the stuff back into
master proper. We therefore end up with a fork of typelib, which we wanted
to avoid in the first place [/rant]

Post by Alexander Duda
I am asking because the current master is still using dyncall
0.6 which is not compatible with OS X 10.9.

dyncall is being unused for a pretty long time in typelib ... I think that
if it is indeed problematic, we should probably just rip out the support
for method calls from typelib altogether. (These days, ffi would be a much
better choice)

??? Don't understand this. You are allowed to link to a module. Moreover,
the whole point of having dynamically loadable modules is that undefined
references should be allowed. We should definitely not install two versions
of the same shared object.

In other words, I think there *is* a way to setup typelib_ruby so that
clang does not complain. What specific error do you get ?

Post by Alexander Duda
I have a merged and fixed version on my machine. Shall I create a pull
request?

Yes, on github.

Sylvain
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.dfki.de/pipermail/rock-dev/attachments/20140520/de8979c5/attachment.htm

Alexander Duda

2014-05-20 13:52:21 UTC

Permalink

Post by Alexander Duda
Hi,
is there a particular reason why toolchain-2.7 is not merged into
master?
Because the orocos toolchain developers develop on the side because that's easier for them, and believe that it is my job to merge the stuff back into master proper. We therefore end up with a fork of typelib, which we wanted to avoid in the first place [/rant]
I am asking because the current master is still using dyncall
0.6 which is not compatible with OS X 10.9.
dyncall is being unused for a pretty long time in typelib ... I think that if it is indeed problematic, we should probably just rip out the support for method calls from typelib altogether. (These days, ffi would be a much better choice)

Ok, I wil have a look.

Post by Alexander Duda
Furthermore there are a number of issues with the current build on OSX
* rpath is not turned on for all targets therefore typelib_ruby cannot
find some libs
* rpath seperator is ";" instead of the used ":"
* typelib_ruby is build as module. But at the same time its header are
used by orocos.rb. This results in undefined references which works for
gcc but not for clang. Therefore, either we should install a typlib_ruby
lib or use header only.
??? Don't understand this. You are allowed to link to a module. Moreover, the whole point of having dynamically loadable modules is that undefined references should be allowed. We should definitely not install two versions of the same shared object.

According to http://stackoverflow.com/questions/7619607/how-portable-is-linking-executables-against-loadable-modules this is not supported on OS X. In general the question is if this is good practice or not?

The error message is for tools/orocos.rb:
Undefined symbols for architecture x86_64:
"typelib_get(unsigned long)", referenced from:
local_output_port_write(unsigned long, unsigned long, unsigned long) in ruby_task_context.cc.o
local_input_port_read(unsigned long, unsigned long, unsigned long, unsigned long) in ruby_task_context.cc.o
property_do_read(unsigned long, unsigned long, unsigned long, unsigned long) in datahandling.cc.o
property_do_write(unsigned long, unsigned long, unsigned long, unsigned long) in datahandling.cc.o
attribute_do_read(unsigned long, unsigned long, unsigned long, unsigned long) in datahandling.cc.o
attribute_do_write(unsigned long, unsigned long, unsigned long, unsigned long) in datahandling.cc.o
operation_call(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) in operations.cc.o

I am not sure if we should tune clang to not complain about undefined references. According to "http://www.akkadia.org/drepper/dsohowto.pdf?
this is should be avoided: "It is therefore highly recommended to never depend on undefined symbols.?

The options I see are:
* tune clang
* header only typelib_ruby interface
* installing typelib_ruby interface as system library

You would go for tune clang?

Greets Alex

Post by Alexander Duda
In other words, I think there *is* a way to setup typelib_ruby so that clang does not complain. What specific error do you get ?
I have a merged and fixed version on my machine. Shall I create a pull
request?
Yes, on github.
Sylvain

--
Dipl.-Ing. Alexander Duda
Unterwasserrobotik
Robotics Innovation Center

Hauptgesch?ftsstelle Standort Bremen:
DFKI GmbH
Robotics Innovation Center
Robert-Hooke-Stra?e 1
28359 Bremen, Germany

Tel.: +49 421 178 45-6620
Zentrale: +49 421 178 45-0
Fax: +49 421 178 45-4150 (Faxe bitte namentlich kennzeichnen)
E-Mail: Alexander.Duda at dfki.de

Weitere Informationen: http://www.dfki.de/robotik
-----------------------------------------------------------------------
Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH
Firmensitz: Trippstadter Stra?e 122, D-67663 Kaiserslautern
Geschaeftsfuehrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster
(Vorsitzender) Dr. Walter Olthoff
Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes
Amtsgericht Kaiserslautern, HRB 2313
Sitz der Gesellschaft: Kaiserslautern (HRB 2313)
USt-Id.Nr.: DE 148646973
Steuernummer: 19/673/0060/3

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.dfki.de/pipermail/rock-dev/attachments/20140520/0a3e7a03/attachment.htm

Sylvain Joyeux

2014-05-20 13:55:47 UTC

Permalink

No, I would go with linking the orocos.rb module against typelib_ruby,
which is perfectly legal.

I do believe that there is already a pkg-config file that references
typelib_ruby.

Post by Sylvain Joyeux

Post by Alexander Duda
Hi,
is there a particular reason why toolchain-2.7 is not merged into
master?

Post by Alexander Duda
I am asking because the current master is still using dyncall
0.6 which is not compatible with OS X 10.9.

??? Don't understand this. You are allowed to link to a module. Moreover,
the whole point of having dynamically loadable modules is that undefined
references should be allowed. We should definitely not install two versions
of the same shared object.
According to
http://stackoverflow.com/questions/7619607/how-portable-is-linking-executables-against-loadable-modulesthis is not supported on OS X. In general the question is if this is good
practice or not?
local_output_port_write(unsigned long, unsigned long, unsigned long)
in ruby_task_context.cc.o
local_input_port_read(unsigned long, unsigned long, unsigned long,
unsigned long) in ruby_task_context.cc.o
property_do_read(unsigned long, unsigned long, unsigned long,
unsigned long) in datahandling.cc.o
property_do_write(unsigned long, unsigned long, unsigned long,
unsigned long) in datahandling.cc.o
attribute_do_read(unsigned long, unsigned long, unsigned long,
unsigned long) in datahandling.cc.o
attribute_do_write(unsigned long, unsigned long, unsigned long,
unsigned long) in datahandling.cc.o
operation_call(unsigned long, unsigned long, unsigned long, unsigned
long, unsigned long, unsigned long) in operations.cc.o
I am not sure if we should tune clang to not complain about undefined
references. According to "http://www.akkadia.org/drepper/dsohowto.pdf?
this is should be avoided: "It is therefore highly recommended to never
depend on undefined symbols.?
* tune clang
* header only typelib_ruby interface
* installing typelib_ruby interface as system library
You would go for tune clang?
Greets Alex
In other words, I think there *is* a way to setup typelib_ruby so that
clang does not complain. What specific error do you get ?

Post by Alexander Duda
I have a merged and fixed version on my machine. Shall I create a pull
request?

Yes, on github.
Sylvain
--
Dipl.-Ing. Alexander Duda
Unterwasserrobotik
Robotics Innovation Center
DFKI GmbH
Robotics Innovation Center
Robert-Hooke-Stra?e 1
28359 Bremen, Germany
Tel.: +49 421 178 45-6620
Zentrale: +49 421 178 45-0
Fax: +49 421 178 45-4150 (Faxe bitte namentlich kennzeichnen)
E-Mail: Alexander.Duda at dfki.de
Weitere Informationen: http://www.dfki.de/robotik
-----------------------------------------------------------------------
Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH
Firmensitz: Trippstadter Stra?e 122, D-67663 Kaiserslautern
Geschaeftsfuehrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster
(Vorsitzender) Dr. Walter Olthoff
Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes
Amtsgericht Kaiserslautern, HRB 2313
Sitz der Gesellschaft: Kaiserslautern (HRB 2313)
USt-Id.Nr.: DE 148646973
Steuernummer: 19/673/0060/3

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.dfki.de/pipermail/rock-dev/attachments/20140520/f8c5b896/attachment-0001.htm

Alexander Duda

2014-05-20 19:41:36 UTC

Permalink

No, I would go with linking the orocos.rb module against typelib_ruby, which is perfectly legal.

This is only legal on unix based systems because they do not distinct between modules and shared libraries. OS X is using Mach-O as binary format and here it is not allowed to directly link against modules.

In the case of orocos.rb the only function in question is typelib_get which can easily be implemented as inline function or we could simply copy it to orocos.rb.

/* Returns the Value object wrapped into +value+ */
159 Value typelib_get(VALUE value)
160 {
161 void* object = 0;
162 Data_Get_Struct(value, void, object);
163 return *reinterpret_cast<Value*>(object);
164 }

Any objections if we copy the function to oroocs.rb and remove all includes to typelib_ruby?

Greets Alex

I do believe that there is already a pkg-config file that references typelib_ruby.

Ok, I wil have a look.

According to http://stackoverflow.com/questions/7619607/how-portable-is-linking-executables-against-loadable-modules this is not supported on OS X. In general the question is if this is good practice or not?
local_output_port_write(unsigned long, unsigned long, unsigned long) in ruby_task_context.cc.o
local_input_port_read(unsigned long, unsigned long, unsigned long, unsigned long) in ruby_task_context.cc.o
property_do_read(unsigned long, unsigned long, unsigned long, unsigned long) in datahandling.cc.o
property_do_write(unsigned long, unsigned long, unsigned long, unsigned long) in datahandling.cc.o
attribute_do_read(unsigned long, unsigned long, unsigned long, unsigned long) in datahandling.cc.o
attribute_do_write(unsigned long, unsigned long, unsigned long, unsigned long) in datahandling.cc.o
operation_call(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) in operations.cc.o
I am not sure if we should tune clang to not complain about undefined references. According to "http://www.akkadia.org/drepper/dsohowto.pdf?
this is should be avoided: "It is therefore highly recommended to never depend on undefined symbols.?
* tune clang
* header only typelib_ruby interface
* installing typelib_ruby interface as system library
You would go for tune clang?
Greets Alex

--
Dipl.-Ing. Alexander Duda
Unterwasserrobotik
Robotics Innovation Center
DFKI GmbH
Robotics Innovation Center
Robert-Hooke-Stra?e 1
28359 Bremen, Germany
Tel.: +49 421 178 45-6620
Zentrale: +49 421 178 45-0
Fax: +49 421 178 45-4150 (Faxe bitte namentlich kennzeichnen)
E-Mail: Alexander.Duda at dfki.de
Weitere Informationen: http://www.dfki.de/robotik
-----------------------------------------------------------------------
Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH
Firmensitz: Trippstadter Stra?e 122, D-67663 Kaiserslautern
Geschaeftsfuehrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster
(Vorsitzender) Dr. Walter Olthoff
Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes
Amtsgericht Kaiserslautern, HRB 2313
Sitz der Gesellschaft: Kaiserslautern (HRB 2313)
USt-Id.Nr.: DE 148646973
Steuernummer: 19/673/0060/3

Sylvain Joyeux

2014-05-20 20:01:20 UTC

Permalink

Yes, because changing how typelib wraps its C++ objects on the Ruby side
would break orocos.rb.

If it indeed solves your problem, I would make the method inline.

Post by Sylvain Joyeux
No, I would go with linking the orocos.rb module against typelib_ruby,
which is perfectly legal.
This is only legal on unix based systems because they do not distinct
between modules and shared libraries. OS X is using Mach-O as binary format
and here it is not allowed to directly link against modules.
In the case of orocos.rb the only function in question is typelib_get
which can easily be implemented as inline function or we could simply copy
it to orocos.rb.
/* Returns the Value object wrapped into +value+ */
159 Value typelib_get(VALUE value)
160 {
161 void* object = 0;
162 Data_Get_Struct(value, void, object);
163 return *reinterpret_cast<Value*>(object);
164 }
Any objections if we copy the function to oroocs.rb and remove all
includes to typelib_ruby?
Greets Alex
I do believe that there is already a pkg-config file that references typelib_ruby.

Post by Sylvain Joyeux

Post by Alexander Duda
Hi,
is there a particular reason why toolchain-2.7 is not merged into
master?

Because the orocos toolchain developers develop on the side because
that's easier for them, and believe that it is my job to merge the stuff
back into master proper. We therefore end up with a fork of typelib, which
we wanted to avoid in the first place [/rant]

Post by Alexander Duda
I am asking because the current master is still using dyncall
0.6 which is not compatible with OS X 10.9.

dyncall is being unused for a pretty long time in typelib ... I think
that if it is indeed problematic, we should probably just rip out the
support for method calls from typelib altogether. (These days, ffi would be
a much better choice)
Ok, I wil have a look.

??? Don't understand this. You are allowed to link to a module. Moreover,
the whole point of having dynamically loadable modules is that undefined
references should be allowed. We should definitely not install two versions
of the same shared object.
According to
http://stackoverflow.com/questions/7619607/how-portable-is-linking-executables-against-loadable-modulesthis is not supported on OS X. In general the question is if this is good
practice or not?
local_output_port_write(unsigned long, unsigned long, unsigned
long) in ruby_task_context.cc.o
local_input_port_read(unsigned long, unsigned long, unsigned long,
unsigned long) in ruby_task_context.cc.o
property_do_read(unsigned long, unsigned long, unsigned long,
unsigned long) in datahandling.cc.o
property_do_write(unsigned long, unsigned long, unsigned long,
unsigned long) in datahandling.cc.o
attribute_do_read(unsigned long, unsigned long, unsigned long,
unsigned long) in datahandling.cc.o
attribute_do_write(unsigned long, unsigned long, unsigned long,
unsigned long) in datahandling.cc.o
operation_call(unsigned long, unsigned long, unsigned long,
unsigned long, unsigned long, unsigned long) in operations.cc.o
I am not sure if we should tune clang to not complain about undefined
references. According to "http://www.akkadia.org/drepper/dsohowto.pdf?
this is should be avoided: "It is therefore highly recommended to never
depend on undefined symbols.?
* tune clang
* header only typelib_ruby interface
* installing typelib_ruby interface as system library
You would go for tune clang?
Greets Alex
In other words, I think there *is* a way to setup typelib_ruby so that
clang does not complain. What specific error do you get ?

Post by Alexander Duda
I have a merged and fixed version on my machine. Shall I create a pull
request?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.dfki.de/pipermail/rock-dev/attachments/20140520/7192d3b0/attachment.htm

Peter Soetens

2014-05-26 19:31:57 UTC

Permalink

Post by Sylvain Joyeux

Post by Alexander Duda
Hi,
is there a particular reason why toolchain-2.7 is not merged into
master?

The real reason for sticking to toolchain-2.7 is because we've been
using Ubuntu 12.04
the last 2 years and that master branches did no longer work on 12.04,
due to a Ruby
versioning shift to 1.9.x (and now 2.x?). I know this doesn't apply to
typelib (C++),
but it halted our merging of master of orogen and related packages.

We did try to do the merge.

Post by Sylvain Joyeux

Post by Alexander Duda
I am asking because the current master is still using dyncall
0.6 which is not compatible with OS X 10.9.

dyncall is being unused for a pretty long time in typelib ... I think that
if it is indeed problematic, we should probably just rip out the support for
method calls from typelib altogether. (These days, ffi would be a much
better choice)
Ok, I wil have a look.

If dyncall goes, I'm in favour as well because it indeed goes wrong on Mac OS-X.

We're very much in favour of helping to port to OS-X. We did quite
some work to that end
the past months for orocos-toolchain.

Peter