opendds-main Mailing List for OpenDDS

To all:

Based on the feedback from the community, we are closing the mailing
lists.  Please use GitHub Issues <https://github.com/OpenDDS/OpenDDS/issues> to
report bugs and GitHub Discussions
<https://github.com/OpenDDS/OpenDDS/discussions> for questions.  We will
also use GitHub Discussions for release announcements.

Thank you.

Justin Wilson

On Fri, Jul 28, 2023 at 3:36 PM Justin Wilson <wi...@un...>
wrote:

> To all:
>
> OpenDDS has been hosted on GitHub <https://github.com/OpenDDS/OpenDDS>
> since 2015.  Since then, we have seen a steady decline in the activity of
> the SourceForge mailing lists and a steady increase in the use of GitHub
> features like Issues <https://github.com/OpenDDS/OpenDDS/issues> and
> Discussions <https://github.com/OpenDDS/OpenDDS/discussions>.  The
> OpenDDS Foundation intends to close the SourceForge mailing lists to unify
> the community around a single forum for the benefit of all users.  However,
> before taking any action, we are soliciting feedback from mailing list
> users (via a mailing list post) to determine if this will be a positive
> change for the OpenDDS user community.
>
> Thanks.
>
> Justin Wilson
>

Just please make sure I am informed about where discussions take place.
________________________________
From: Justin Wilson <wi...@un...>
Sent: Friday, July 28, 2023 1:36 PM
To: General Discussion concerning OpenDDS <ope...@li...>; ope...@li... <ope...@li...>; ope...@li... <ope...@li...>
Subject: [opendds-main] Mailing Lists

To all:

OpenDDS has been hosted on GitHub<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_OpenDDS_OpenDDS&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=AQA5VdTC9I8e4SCS5u7nYLx1LGpLg9l16mu-_36xV3SLUYeaKJBpGz0YUvEsYjzQ&m=NQDaJwKqmZhPgerkwa4mNCWaa2PiSqIZu0SoiKjLWFkXLsgfdTtSl1yC1K8rHchc&s=rn179NYX-3dXvTej7mSMJnT47yj3FQwf5HVn43CRgMc&e=> since 2015.  Since then, we have seen a steady decline in the activity of the SourceForge mailing lists and a steady increase in the use of GitHub features like Issues<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_OpenDDS_OpenDDS_issues&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=AQA5VdTC9I8e4SCS5u7nYLx1LGpLg9l16mu-_36xV3SLUYeaKJBpGz0YUvEsYjzQ&m=NQDaJwKqmZhPgerkwa4mNCWaa2PiSqIZu0SoiKjLWFkXLsgfdTtSl1yC1K8rHchc&s=8QzIiKa_ORcSXCMTYuJI6W8u1ottTGcvgsgKYeLBp3g&e=> and Discussions<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_OpenDDS_OpenDDS_discussions&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=AQA5VdTC9I8e4SCS5u7nYLx1LGpLg9l16mu-_36xV3SLUYeaKJBpGz0YUvEsYjzQ&m=NQDaJwKqmZhPgerkwa4mNCWaa2PiSqIZu0SoiKjLWFkXLsgfdTtSl1yC1K8rHchc&s=aUN8TbCiD2O_uwwvTYapZeWX3fRzyurFAPfUS1qfgtU&e=>.  The OpenDDS Foundation intends to close the SourceForge mailing lists to unify the community around a single forum for the benefit of all users.  However, before taking any action, we are soliciting feedback from mailing list users (via a mailing list post) to determine if this will be a positive change for the OpenDDS user community.

Thanks.

Justin Wilson

To all:

OpenDDS has been hosted on GitHub <https://github.com/OpenDDS/OpenDDS>
since 2015.  Since then, we have seen a steady decline in the activity of
the SourceForge mailing lists and a steady increase in the use of GitHub
features like Issues <https://github.com/OpenDDS/OpenDDS/issues> and
Discussions <https://github.com/OpenDDS/OpenDDS/discussions>.  The OpenDDS
Foundation intends to close the SourceForge mailing lists to unify the
community around a single forum for the benefit of all users.  However,
before taking any action, we are soliciting feedback from mailing list
users (via a mailing list post) to determine if this will be a positive
change for the OpenDDS user community.

Thanks.

Justin Wilson

Simon:

You mentioned that you are using best-effort reliability.
The publisher is sending out a stream of instance registration, sample,
dispose, and unregister messages.
Since this message stream is unreliable, the disposes and unregisters may
never arrive at the subscriber, and there is no retry.
Thus, the subscriber is left in a state where it thinks the instances are
alive because it has no reason to believe they are dead.
Presumably, the subscriber has taken samples from these instances in the
past but there will be no new samples in the future.
That answers the second question as to why take_next_sample never gets a
sample from those handles.
Alternatively, it could be the case that only the register message is
received.

One way of dealing with this would be to use resource limits.
However, what is really needed is an additional member in the reader data
lifecycle qos that applies to instances that are alive but have not
received a sample recently.
(This seems related to the deadline qos and lifespan qos but the semantics
of those never change the instance state except in the case of shared
ownership.)
This would be a useful extension,that is, the user could set a period in
the reader that causes alive instances to be purged if a sample is not
received in the period.

I believe release_instance is the correct method to use.

Justin

On Fri, Jul 14, 2023 at 6:08 AM Simon Rogers (C2I) <
Sim...@ul...> wrote:

> Hi
>
>
>
> We’re testing high rates of pub/sub using a best effort topic and rtps_udp
> transport on Windows, OpenDDS 3.22.
>
> We are on purpose overloading (thousands of instances
> created/updated/destroyed in short period) our subscriber. We see the same
> issue if we overload the subscriber or introduce packet loss.
>
>
>
> After running our high rate test and then disposing our instances we see
> that DataReaderImpl has instance handles left behind (we call
> DataReaderImpl::get_instance_handles to see these).
>
> This number of instance handles in the DataReader grows while the
> overloading/packet loss is occurring, and they never seem to get cleaned up.
>
> We can remove them by calling DataReaderImpl-> release_instance(handle)
> but it’s unclear if we should be doing this.
>
> take_next_sample never gives us instances with handles that match these
> leaked ones.
>
>
>
> In summary the questions are:
>
>    - Why does the DataReader have instance handles which don’t get
>    cleaned up
>    - Why does take_next_sample never get a sample with these handles (and
>    we never get an on_sample_rejected/on_sample_lost)
>
>
>
> Any thought/ideas would be welcomed.
>
> Thanks
>
> Simon
>
>
>
>
>
>
> _______________________________________________
> opendds-main mailing list
> ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opendds-main
>

Hi

We're testing high rates of pub/sub using a best effort topic and rtps_udp transport on Windows, OpenDDS 3.22.
We are on purpose overloading (thousands of instances created/updated/destroyed in short period) our subscriber. We see the same issue if we overload the subscriber or introduce packet loss.

After running our high rate test and then disposing our instances we see that DataReaderImpl has instance handles left behind (we call DataReaderImpl::get_instance_handles to see these).
This number of instance handles in the DataReader grows while the overloading/packet loss is occurring, and they never seem to get cleaned up.
We can remove them by calling DataReaderImpl-> release_instance(handle) but it's unclear if we should be doing this.
take_next_sample never gives us instances with handles that match these leaked ones.

In summary the questions are:

  *   Why does the DataReader have instance handles which don't get cleaned up
  *   Why does take_next_sample never get a sample with these handles (and we never get an on_sample_rejected/on_sample_lost)

Any thought/ideas would be welcomed.
Thanks
Simon

Hi Simon,

Participants that have UseRtpsRelay=1 and not RtpsRelayOnly=1 should still
be doing local discovery.  If this isn't what you're observing, a packet
capture and log file would be helpful.

Thanks,
Adam Mitz
Principal Software Engineer and Partner
Object Computing, Inc.

On Mon, May 15, 2023 at 10:43 AM Simon Rogers (C2I) <
Sim...@ul...> wrote:

> I’m testing the RTPSRelay to allow communication between two sets of
> participants on two LANs, separated by routers which don’t allow Multicast.
>
> If I enable use of the relay by setting UseRtpsRelay=1 etc. in the
> participant ini files, participants on a LAN won’t discover peers unless
> the relay is running. Is there a way to use the relay for remote/cloud
> participant discovery, AND allow multicast SPDP/SEDP for local participants?
>
>
>
> Thanks
>
> Simon
>
>
>
>
> _______________________________________________
> opendds-main mailing list
> ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opendds-main
>

I'm testing the RTPSRelay to allow communication between two sets of participants on two LANs, separated by routers which don't allow Multicast.
If I enable use of the relay by setting UseRtpsRelay=1 etc. in the participant ini files, participants on a LAN won't discover peers unless the relay is running. Is there a way to use the relay for remote/cloud participant discovery, AND allow multicast SPDP/SEDP for local participants?

Thanks
Simon

Simon:

There is no assumption of one repeater per participant.
The multicast repeater should work for the situation you describe.
For a sanity check, you could set up two multicast repeaters on the same
host running on two different ports and serving two different multicast
groups.
Each one would need to know about the other's unicast IP:port.
Then, have process A join one multicast group and process B join the other.

Justin

On Mon, Apr 24, 2023 at 3:32 PM Simon Rogers (C2I) <
Sim...@ul...> wrote:

> Or are you using the Multicast Repeater to serve multiple hosts (we did
> not consider this in our original design)?
>
> This. I have two LANs which each support multicast, full of
> hosts/participants. I was hoping to use one repeater in each LAN to allow
> discovery to happen (without adding heaps of end point addresses/ports to
> my ini files).
>
> I’ve seen the ‘Cloud’ configuration tutorial which shows use of the
> Repeater. Is the assumption that there’s only one client participant per
> repeater?
>
>
>
> Thanks
>
> Simon
>
>
>
> *From:* Justin Wilson <wi...@un...>
> *Sent:* 24 April 2023 15:22
> *To:* General Discussion concerning OpenDDS <
> ope...@li...>
> *Subject:* Re: [opendds-main] Sedp::remote_knows_about_local_i - could
> not find local publication
>
>
>
> Simon:
>
>
>
> Could you expand on the Multicast Repeater failure?
>
> Assuming one Windows participant and one Linux participant, then there
> should be three links to check:
>
> 1.       Linux participant to local repeater.
>
> 2.       Linux repeater to Windows repeater.
>
> 3.       Windows participant to local repeater.
>
> Which links or links show an issue?
>
>
>
> Or are you using the Multicast Repeater to serve multiple hosts (we did
> not consider this in our original design)?
>
> Assuming you have a network that supports multicast and another network
> that does not, the setup would involve one multicast repeater in the
> multicast network and another repeater for each host in multicast-less
> network.
>
> There are no mechanisms to prevent infinite loops and/or echos.
>
> Such a situation would arise if you had two repeaters in the same
> multicast network configured to talk to each other.
>
>
>
> Another idea is to use the RtpsRelay in a discovery-only mode, i.e., don't
> configure the client participants to use the data port.
>
> Unless you use partitions, you should turn on the option to allow empty
> partitions.
>
> We have plans to make this "even better" by adding an SPDP forwarding mode
> to the RtpsRelay but it hasn't been done yet.
>
> Be aware that doing full-mesh discovery through the RtpsRelay can lead to
> performance problems due to fan out.
>
>
>
> Justin
>
>
>
>
>
> On Mon, Apr 24, 2023 at 7:14 AM Simon Rogers (C2I) <
> Sim...@ul...> wrote:
>
> Justin
>
> Thank you for your reply. Surprisingly, PeriodicDirectedSpdp=1 seems to
> have helped, although we’ve only been testing for a couple of days. I’ll
> follow your advice and increase the debugging.
>
>
>
> Incidentally, I would have preferred to use the Multicast Repeater rather
> than specifying lists of endpoints in our ini files, but in initial testing
> we couldn’t get Windows participants to communicate via the Repeater
> running in a Linux VM (the networks don’t support Multicast). We were also
> concerned about the possibility of message loops when using the Repeater.
> Have steps been taken in the Multicast Repeater to prevent infinite
> loops/echoes back and forth? Especially with more than two networks linked
> by repeaters.
>
>
>
> Thanks
>
> Simon
>
> _______________________________________________
> opendds-main mailing list
> ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opendds-main
>

Or are you using the Multicast Repeater to serve multiple hosts (we did not consider this in our original design)?
This. I have two LANs which each support multicast, full of hosts/participants. I was hoping to use one repeater in each LAN to allow discovery to happen (without adding heaps of end point addresses/ports to my ini files).
I’ve seen the ‘Cloud’ configuration tutorial which shows use of the Repeater. Is the assumption that there’s only one client participant per repeater?

Thanks
Simon

From: Justin Wilson <wi...@un...>
Sent: 24 April 2023 15:22
To: General Discussion concerning OpenDDS <ope...@li...>
Subject: Re: [opendds-main] Sedp::remote_knows_about_local_i - could not find local publication

Simon:

Could you expand on the Multicast Repeater failure?
Assuming one Windows participant and one Linux participant, then there should be three links to check:
1.       Linux participant to local repeater.
2.       Linux repeater to Windows repeater.
3.       Windows participant to local repeater.
Which links or links show an issue?

Or are you using the Multicast Repeater to serve multiple hosts (we did not consider this in our original design)?
Assuming you have a network that supports multicast and another network that does not, the setup would involve one multicast repeater in the multicast network and another repeater for each host in multicast-less network.
There are no mechanisms to prevent infinite loops and/or echos.
Such a situation would arise if you had two repeaters in the same multicast network configured to talk to each other.

Another idea is to use the RtpsRelay in a discovery-only mode, i.e., don't configure the client participants to use the data port.
Unless you use partitions, you should turn on the option to allow empty partitions.
We have plans to make this "even better" by adding an SPDP forwarding mode to the RtpsRelay but it hasn't been done yet.
Be aware that doing full-mesh discovery through the RtpsRelay can lead to performance problems due to fan out.

Justin


On Mon, Apr 24, 2023 at 7:14 AM Simon Rogers (C2I) <Sim...@ul...<mailto:Sim...@ul...>> wrote:
Justin
Thank you for your reply. Surprisingly, PeriodicDirectedSpdp=1 seems to have helped, although we’ve only been testing for a couple of days. I’ll follow your advice and increase the debugging.

Incidentally, I would have preferred to use the Multicast Repeater rather than specifying lists of endpoints in our ini files, but in initial testing we couldn’t get Windows participants to communicate via the Repeater running in a Linux VM (the networks don’t support Multicast). We were also concerned about the possibility of message loops when using the Repeater. Have steps been taken in the Multicast Repeater to prevent infinite loops/echoes back and forth? Especially with more than two networks linked by repeaters.

Thanks
Simon

Simon:

Could you expand on the Multicast Repeater failure?
Assuming one Windows participant and one Linux participant, then there
should be three links to check:

   1. Linux participant to local repeater.
   2. Linux repeater to Windows repeater.
   3. Windows participant to local repeater.

Which links or links show an issue?

Or are you using the Multicast Repeater to serve multiple hosts (we did not
consider this in our original design)?
Assuming you have a network that supports multicast and another network
that does not, the setup would involve one multicast repeater in the
multicast network and another repeater for each host in multicast-less
network.
There are no mechanisms to prevent infinite loops and/or echos.
Such a situation would arise if you had two repeaters in the same multicast
network configured to talk to each other.

Another idea is to use the RtpsRelay in a discovery-only mode, i.e., don't
configure the client participants to use the data port.
Unless you use partitions, you should turn on the option to allow empty
partitions.
We have plans to make this "even better" by adding an SPDP forwarding mode
to the RtpsRelay but it hasn't been done yet.
Be aware that doing full-mesh discovery through the RtpsRelay can lead to
performance problems due to fan out.

Justin


On Mon, Apr 24, 2023 at 7:14 AM Simon Rogers (C2I) <
Sim...@ul...> wrote:

> Justin
>
> Thank you for your reply. Surprisingly, PeriodicDirectedSpdp=1 seems to
> have helped, although we’ve only been testing for a couple of days. I’ll
> follow your advice and increase the debugging.
>
>
>
> Incidentally, I would have preferred to use the Multicast Repeater rather
> than specifying lists of endpoints in our ini files, but in initial testing
> we couldn’t get Windows participants to communicate via the Repeater
> running in a Linux VM (the networks don’t support Multicast). We were also
> concerned about the possibility of message loops when using the Repeater.
> Have steps been taken in the Multicast Repeater to prevent infinite
> loops/echoes back and forth? Especially with more than two networks linked
> by repeaters.
>
>
>
> Thanks
>
> Simon
>
>
>
> *From:* Justin Wilson <wi...@un...>
> *Sent:* 21 April 2023 18:39
> *To:* General Discussion concerning OpenDDS <
> ope...@li...>
> *Subject:* Re: [opendds-main] Sedp::remote_knows_about_local_i - could
> not find local publication
>
>
>
> Simon:
>
>
>
> PeriodicDirectedSpdp=1 probably won't help here unless SPDP messages are
> overrunning a buffer on the send or receive side.
>
> If that is in fact the case, then it may make sense to cycle through the
> SpdpSendAddrs much like PeriodicDirectedSpdp.
>
> Please file an issue that is what you find.
>
>
>
> However, if SPDP is the issue, I would expect to see participants being
> removed.
>
> Your logs would contain "participant %C exceeded lease duration, removing".
>
>
>
> The error you mentioned isn't supposed to happen.
>
> If it's on the subscriber side, then something got corrupted because it
> shouldn't be complaining about a missing local writer.
>
> If it's on the publisher side, then it is strange that it can't find a
> local writer and that the association record was created in the first place.
>
> That is, removal of the data writer should have cleaned up a pending
> association record.
>
>
>
> I'll give some background on the Association Record logic in the hopes
> that it will help you uncover the issue.
>
> The Association Record logic was added to delay associations until both
> sides are ready.
>
> Suppose you are on the writer side and received a remote subscription that
> matches a local writer.
>
> Presumably, a match can be made immediately on the writer side.
>
> However, what if the reader side has not even received the publication?
>
> Since the reader side isn't ready, any heartbeat or data from the writer
> might be rejected.
>
> Since SEDP publication endpoints are reliable, the writer side can check
> that the other side has acknowledged it and then proceed to create the
> association.
>
> Security and XTypes makes this situation even more complex.
>
>
>
> If you turn up the logging high enough, you should see the association
> records being processed.
>
> This and other discovery-related information may help you determine if the
> "disconnect" is happening from discovery or if there is another issue
> blocking the receipt of data.
>
>
>
> Justin
>
>
>
>
>
>
>
> On Fri, Apr 21, 2023 at 5:32 AM Simon Rogers (C2I) <
> Sim...@ul...> wrote:
>
> Hi
>
> We have a system running on Windows and Linux using OpenDDS 3.22.
>
> We don’t have a Multicast capable network so are using Unicast SPDP by
> specifying a list of addresses using SpdpSendAddrs=…, and using unicast
> SEDP (SedpMulticast=0) and transport (use_multicast=0), with transport type
> rtps_udp.
>
>
>
> After a period of hours some of our subscribers seem to stop receiving
> data from publishers. Making our subscriber application destroy and then
> recreate its data-reader gives us errors:
>
> Sedp::remote_knows_about_local_i – could not find local publication
> ####.####.####.####(####)
>
>
>
> Would there be any options which might help in this configuration where we
> have no Multicast – I was wondering about PeriodicDirectedSpdp=1?
>
>
>
> Thanks
>
> Simon
>
> _______________________________________________
> opendds-main mailing list
> ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opendds-main
>
> _______________________________________________
> opendds-main mailing list
> ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opendds-main
>

Justin
Thank you for your reply. Surprisingly, PeriodicDirectedSpdp=1 seems to have helped, although we’ve only been testing for a couple of days. I’ll follow your advice and increase the debugging.

Incidentally, I would have preferred to use the Multicast Repeater rather than specifying lists of endpoints in our ini files, but in initial testing we couldn’t get Windows participants to communicate via the Repeater running in a Linux VM (the networks don’t support Multicast). We were also concerned about the possibility of message loops when using the Repeater. Have steps been taken in the Multicast Repeater to prevent infinite loops/echoes back and forth? Especially with more than two networks linked by repeaters.

Thanks
Simon

From: Justin Wilson <wi...@un...>
Sent: 21 April 2023 18:39
To: General Discussion concerning OpenDDS <ope...@li...>
Subject: Re: [opendds-main] Sedp::remote_knows_about_local_i - could not find local publication

Simon:

PeriodicDirectedSpdp=1 probably won't help here unless SPDP messages are overrunning a buffer on the send or receive side.
If that is in fact the case, then it may make sense to cycle through the SpdpSendAddrs much like PeriodicDirectedSpdp.
Please file an issue that is what you find.

However, if SPDP is the issue, I would expect to see participants being removed.
Your logs would contain "participant %C exceeded lease duration, removing".

The error you mentioned isn't supposed to happen.
If it's on the subscriber side, then something got corrupted because it shouldn't be complaining about a missing local writer.
If it's on the publisher side, then it is strange that it can't find a local writer and that the association record was created in the first place.
That is, removal of the data writer should have cleaned up a pending association record.

I'll give some background on the Association Record logic in the hopes that it will help you uncover the issue.
The Association Record logic was added to delay associations until both sides are ready.
Suppose you are on the writer side and received a remote subscription that matches a local writer.
Presumably, a match can be made immediately on the writer side.
However, what if the reader side has not even received the publication?
Since the reader side isn't ready, any heartbeat or data from the writer might be rejected.
Since SEDP publication endpoints are reliable, the writer side can check that the other side has acknowledged it and then proceed to create the association.
Security and XTypes makes this situation even more complex.

If you turn up the logging high enough, you should see the association records being processed.
This and other discovery-related information may help you determine if the "disconnect" is happening from discovery or if there is another issue blocking the receipt of data.

Justin



On Fri, Apr 21, 2023 at 5:32 AM Simon Rogers (C2I) <Sim...@ul...<mailto:Sim...@ul...>> wrote:
Hi
We have a system running on Windows and Linux using OpenDDS 3.22.
We don’t have a Multicast capable network so are using Unicast SPDP by specifying a list of addresses using SpdpSendAddrs=…, and using unicast SEDP (SedpMulticast=0) and transport (use_multicast=0), with transport type rtps_udp.

After a period of hours some of our subscribers seem to stop receiving data from publishers. Making our subscriber application destroy and then recreate its data-reader gives us errors:
Sedp::remote_knows_about_local_i – could not find local publication ####.####.####.####(####)

Would there be any options which might help in this configuration where we have no Multicast – I was wondering about PeriodicDirectedSpdp=1?

Thanks
Simon
_______________________________________________
opendds-main mailing list
ope...@li...<mailto:ope...@li...>
https://lists.sourceforge.net/lists/listinfo/opendds-main

Simon:

PeriodicDirectedSpdp=1 probably won't help here unless SPDP messages are
overrunning a buffer on the send or receive side.
If that is in fact the case, then it may make sense to cycle through the
SpdpSendAddrs much like PeriodicDirectedSpdp.
Please file an issue that is what you find.

However, if SPDP is the issue, I would expect to see participants being
removed.
Your logs would contain "participant %C exceeded lease duration, removing".

The error you mentioned isn't supposed to happen.
If it's on the subscriber side, then something got corrupted because it
shouldn't be complaining about a missing local writer.
If it's on the publisher side, then it is strange that it can't find a
local writer and that the association record was created in the first place.
That is, removal of the data writer should have cleaned up a pending
association record.

I'll give some background on the Association Record logic in the hopes that
it will help you uncover the issue.
The Association Record logic was added to delay associations until both
sides are ready.
Suppose you are on the writer side and received a remote subscription that
matches a local writer.
Presumably, a match can be made immediately on the writer side.
However, what if the reader side has not even received the publication?
Since the reader side isn't ready, any heartbeat or data from the writer
might be rejected.
Since SEDP publication endpoints are reliable, the writer side can check
that the other side has acknowledged it and then proceed to create the
association.
Security and XTypes makes this situation even more complex.

If you turn up the logging high enough, you should see the association
records being processed.
This and other discovery-related information may help you determine if the
"disconnect" is happening from discovery or if there is another issue
blocking the receipt of data.

Justin



On Fri, Apr 21, 2023 at 5:32 AM Simon Rogers (C2I) <
Sim...@ul...> wrote:

> Hi
>
> We have a system running on Windows and Linux using OpenDDS 3.22.
>
> We don’t have a Multicast capable network so are using Unicast SPDP by
> specifying a list of addresses using SpdpSendAddrs=…, and using unicast
> SEDP (SedpMulticast=0) and transport (use_multicast=0), with transport type
> rtps_udp.
>
>
>
> After a period of hours some of our subscribers seem to stop receiving
> data from publishers. Making our subscriber application destroy and then
> recreate its data-reader gives us errors:
>
> Sedp::remote_knows_about_local_i – could not find local publication
> ####.####.####.####(####)
>
>
>
> Would there be any options which might help in this configuration where we
> have no Multicast – I was wondering about PeriodicDirectedSpdp=1?
>
>
>
> Thanks
>
> Simon
> _______________________________________________
> opendds-main mailing list
> ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opendds-main
>

Hi
We have a system running on Windows and Linux using OpenDDS 3.22.
We don't have a Multicast capable network so are using Unicast SPDP by specifying a list of addresses using SpdpSendAddrs=..., and using unicast SEDP (SedpMulticast=0) and transport (use_multicast=0), with transport type rtps_udp.

After a period of hours some of our subscribers seem to stop receiving data from publishers. Making our subscriber application destroy and then recreate its data-reader gives us errors:
Sedp::remote_knows_about_local_i - could not find local publication ####.####.####.####(####)

Would there be any options which might help in this configuration where we have no Multicast - I was wondering about PeriodicDirectedSpdp=1?

Thanks
Simon

OpenDDS version 3.24.0 is now available for download at
https://github.com/OpenDDS/OpenDDS/releases/tag/DDS-3.24

The OpenDDS Developer's Guide for this version is available at
https://opendds.readthedocs.io/en/dds-3.24

Updates in this version:
Additions:

   - The OpenDDS Developer's Guide is now available at
   https://opendds.readthedocs.io/ (#4051
   <https://github.com/OpenDDS/OpenDDS/pull/4051>, #4094
   <https://github.com/OpenDDS/OpenDDS/pull/4094>, #4092
   <https://github.com/OpenDDS/OpenDDS/pull/4092>, #4100
   <https://github.com/OpenDDS/OpenDDS/pull/4100>, #4101
   <https://github.com/OpenDDS/OpenDDS/pull/4101>, #4095
   <https://github.com/OpenDDS/OpenDDS/pull/4095>, #4103
   <https://github.com/OpenDDS/OpenDDS/pull/4103>, #4102
   <https://github.com/OpenDDS/OpenDDS/pull/4102>, #4104
   <https://github.com/OpenDDS/OpenDDS/pull/4104>, #4105
   <https://github.com/OpenDDS/OpenDDS/pull/4105>)
      - The Sphinx/reStructuredText source for this new format is now
      located in the repo at docs/devguide
   - DOCGroup ACE6/TAO2 is now the default ACE/TAO for OpenDDS, OCI ACE/TAO
   is no longer supported (#4069
   <https://github.com/OpenDDS/OpenDDS/pull/4069>)
   - Dynamic content subscription (#3988
   <https://github.com/OpenDDS/OpenDDS/pull/3988>)
      - This allows DynamicDataReaders to use QueryCondition and
      ContentFilteredTopic and allows DynamicDataWriters to do filtering on
      behalf of matched DataReaders that use ContentFilteredTopic.
   - DynamicData:
      - Can now read and write enum members as strings (#4022
      <https://github.com/OpenDDS/OpenDDS/pull/4022>)
      - get_int64_value and get_uint64_value can now cast from different
      types (#4078 <https://github.com/OpenDDS/OpenDDS/pull/4078>)
      - DynamicDataImpl now uses lazy initialization to reduce memory usage
      (#4024 <https://github.com/OpenDDS/OpenDDS/pull/4024>)
   - Added aliases for IDL types from XTypes spec such as DDS::UInt32 (#3994
   <https://github.com/OpenDDS/OpenDDS/pull/3994>)
      - See DdsDcpsCore.idl for all of them.
   - Added PublicationMatchedStatus Current Count To RtpsRelay Statistics (
   #4006 <https://github.com/OpenDDS/OpenDDS/pull/4006>)
   - Allow reassembly of overlapping fragment ranges in RTPS (#4035
   <https://github.com/OpenDDS/OpenDDS/pull/4035>, #4047
   <https://github.com/OpenDDS/OpenDDS/pull/4047>)
   - Can now cross-compile on macOS (#4048
   <https://github.com/OpenDDS/OpenDDS/pull/4048>)
   - Added hardening features to RtpsRelay (#4045
   <https://github.com/OpenDDS/OpenDDS/pull/4045>)
      - These are configured with the new options -MaxAddrSetSize and
      -RejectedAddressDuration.
   - Expanded support for using C++ keywords in IDL (#4073
   <https://github.com/OpenDDS/OpenDDS/pull/4073>)
   - Improved support for anonymous types in unions branches (#4078
   <https://github.com/OpenDDS/OpenDDS/pull/4078>)
   - IDL file and generated TypeSupport.idl can now be in different
   directories (#4077 <https://github.com/OpenDDS/OpenDDS/pull/4077>)

Fixes:

   - Fixed rtps_relay_address_change deadlocks (#3989
   <https://github.com/OpenDDS/OpenDDS/pull/3989>)
   - Fixed RtpsUdpTransport data race from relay_stun_mutex_ (#3990
   <https://github.com/OpenDDS/OpenDDS/pull/3990>)
   - Fixed invalid socket handles in RtpsUdpTransport (#4002
   <https://github.com/OpenDDS/OpenDDS/pull/4002>)
   - Fixed index increment in GuidPartitionTable::prepare_relay_partitions (
   #4005 <https://github.com/OpenDDS/OpenDDS/pull/4005>)
   - Fixed a bug in content filtering with enum comparisons on serialized
   samples (#4038 <https://github.com/OpenDDS/OpenDDS/pull/4038>)
   - Fixed transport config and transport instance derived from template
   conflicting (#4058 <https://github.com/OpenDDS/OpenDDS/pull/4058>)
   - Improved reliability of the shared memory transport (#4028
   <https://github.com/OpenDDS/OpenDDS/pull/4028>)
   - Secure writers and readers in same participant can now associate (#4041
   <https://github.com/OpenDDS/OpenDDS/pull/4041>)
   - Fixed issue with using -o in tao_idl/opendds_idl options in
   OPENDDS_TARGET_SOURCES and those directories are now automatically
   included (#4071 <https://github.com/OpenDDS/OpenDDS/pull/4071>)
   - XTypes (#4078 <https://github.com/OpenDDS/OpenDDS/pull/4078>):
      - TypeObjects struct and union members used to be sorted by member
      ID, but they are now sorted by declaration order as the XTypes spec calls
      for.
      By default member IDs increment starting at 0, and in that case the
      TypeObjects will be the same.
      If @autoid(hash), --default-autoid hash, or @id(ID) are being used
      then the order could be different.
      This could cause some reader/writer matching incompatibility with
      older versions of OpenDDS:
         - Topics with final and appendable structs will no longer match.
         - If DISALLOW_TYPE_COERCION QoS is being used, then all topics
         where the order differ will not longer match.
         Note that this is true for any time the type hash changes.
         - Pass the --old-typeobject-member-order option to opendds_idl to
         use the non-standard order.
      - The size of XCDR2 member parameters in mutable structs and unions
      is now correctly interpreted when the "length code" is 5, 6, or 7.
         - This is an optimization that OpenDDS doesn't serialize samples
         with, so this could only be an issue when dealing with
samples from other
         DDS implementations.
      - DynamicDataImpl (DynamicData made by DynamicDataFactory that can be
      passed to DynamicDataWriter):
         - get_member_id_at_index now returns ids for members that haven't
         been initialized yet.
         - Fixed incorrect serialization of keyed unions for instance
         registration, disposal, and unregistration samples.
         - Fixed errors from serializing some cases of arrays and sequences.

Notes:

   - Release files will only be uploaded to GitHub from now on
   - OpenDDS::DCPS::RepoId has been removed, if needed use
   OpenDDS::DCPS::GUID_t instead (#3972
   <https://github.com/OpenDDS/OpenDDS/pull/3972>)

Hi,

After cross compiling OpenDDS on Ubuntu, I copied the minimal directories to the Pi, per the documentation. But I'm having an issue getting CMake to work on RPi, in order to build my app there. I get:

CMake Error at /home/pi/Documents/OpenDDS/OpenDDS-3.16/build/target/cmake/init.cmake:26 (message):
  Missing required dependencies
  OPENDDS_IDL;ACE_LIBRARY;ACE_GPERF;TAO_LIBRARY;TAO_IDL
Call Stack (most recent call first):
  /home/pi/Documents/OpenDDS/OpenDDS-3.16/build/target/cmake/OpenDDSConfig.cmake:366 (_OPENDDS_RETURN_ERR)
  CMakeLists.txt:14 (find_package)


CMake Error at CMakeLists.txt:51 (OPENDDS_TARGET_SOURCES):
  Unknown CMake command "OPENDDS_TARGET_SOURCES".


-- Configuring incomplete, errors occurred!

Here is the CMakeLists.txt file:

project(pfMessenger C CXX)
cmake_minimum_required(VERSION 3.7.2)

set(ENV{PKG_CONFIG_PATH} "~/Documents/DIP/dist/lib/pkgconfig")

find_package(PkgConfig REQUIRED)

set(opendds_libs
  OpenDDS::Dcps # Core OpenDDS Library
  OpenDDS::InfoRepoDiscovery OpenDDS::Tcp # For run_test.pl
  OpenDDS::Rtps OpenDDS::Rtps_Udp # For run_test.pl --rtps
)

find_package(OpenDDS REQUIRED)

# pf_SubService
add_executable ( pf_SubService
  App_Subprob_Service.cpp
  pf_Command.cpp
  pf_Commands.cpp
  pf_Commands_Writer.cpp
  pf_Constructs_Courier.cpp
  pf_Costs.cpp
  pf_Costs_Writer.cpp
  pf_DataCollection_Module.cpp
  pf_Domain.cpp
  pf_FilteredTopic.cpp
  pf_Graph.cpp
  pf_GraphMap.cpp
  pf_Headlight_Module.cpp
  pf_OverallProblem.cpp
  pf_Participant.cpp
  pf_PhysicalNode_Module.cpp
  pf_PhysicalNodes_Module.cpp
  pf_Prices.cpp
  pf_Prices_Listener.cpp
  pf_Prices_Reader.cpp
  pf_Publisher.cpp
  pf_Rmp_Module.cpp
  pf_SubprobService_Solver_Manager.cpp
  pf_SubprobService_Subprob_Listener.cpp
  pf_SubprobService_Subprob_Reader.cpp
  pf_ServiceId_Manager.cpp
  pf_Subprob_Module.cpp
  pf_Subscriber
  pf_Topic.cpp
  pf_Utilize_Prices.cpp
  pf_Utilize_Subprob.cpp
)

OPENDDS_TARGET_SOURCES(pf_SubService Messenger.idl)
target_link_libraries(pf_SubService ${opendds_libs})
target_link_libraries(pf_SubService ~/Documents/DIP/build/SYMPHONY/5.6.19/src/OsiSym/.libs/libOsiSym.so)
target_include_directories(pf_SubService PUBLIC
  "~/Documents/DIP/dist/include/coin/")

Thanks! Stan

Disregard!  Somehow I wasn't looking at the most recent documentation. Following the new doc I was able to solve this issue. Thanks!
________________________________
From: Stan Stringfellow <sta...@pl...>
Sent: Sunday, April 9, 2023 2:55 AM
To: General Discussion concerning OpenDDS <ope...@li...>
Cc: Stan Stringfellow <sta...@pl...>
Subject: Issue cross-compiling for Raspberry Pi

Hi,

When I try to cross-compile OpenDDS for RPi, the make fails. It was working before.

I notice during the configure stage it skips some things, including:

Skipping opendds_idl (opendds_idl.mpc); it avoids cross_compile.

Later in the make stage, it says:

make[2]:
/home/stanstringfellow/Documents/OpenDDS/Pi/OpenDDS-
3.16/build/host/bin/opendds_idl: Command not found

(I'm still using 3.16 because I don't want to change anything right now. But I tried it with 3.23.1 and had the same issue.)

The full output is attached.

Thanks! Stan

Hi,

When I try to cross-compile OpenDDS for RPi, the make fails. It was working before.

I notice during the configure stage it skips some things, including:

Skipping opendds_idl (opendds_idl.mpc); it avoids cross_compile.

Later in the make stage, it says:

make[2]:
/home/stanstringfellow/Documents/OpenDDS/Pi/OpenDDS-
3.16/build/host/bin/opendds_idl: Command not found

(I'm still using 3.16 because I don't want to change anything right now. But I tried it with 3.23.1 and had the same issue.)

The full output is attached.

Thanks! Stan

Simon:

It is possible but you would have had to configure your build.  The mpc
feature is built_in_topics and the C++ macro is DDS_HAS_MINIMUM_BIT.

There is the possibility that you are getting samples indicating
unregister/dispose in which case the writer may be gone, i.e., no longer in
the built-in topics.
(I view this as a bug, gap, annoyance, etc. with the DDS spec.)

Presumably, if you turn up logging, you will be able to get more
information as to why the publication is not there.

Alternatively, you could install an Observer (see
https://objectcomputing.com/resources/publications/mnb/2021/06/09/logging-samples-json-opendds-apps)
so you can watch all of the traffic about your samples and the built-in
topics.

Justin



On Mon, Mar 27, 2023 at 10:32 AM Simon Rogers (C2I) <
Sim...@ul...> wrote:

> Justin
>
> Thanks for your response. I’ve tried calling get_matched_publication_data
> on both the data reader and the narrowed data reader, passing in the
> publication_handle. In all cases I get uninitialized data out in the
> PublicationBuiltinTopicData.
>
> Is it possible that when running using RTPS discovery, the built in topics
> are disabled? I’m confident my data reader is matched with a data writer
> and is receiving data.
>
>
>
> *From:* Justin Wilson via opendds-main <ope...@li...>
>
> *Sent:* 27 March 2023 14:38
> *To:* General Discussion concerning OpenDDS <
> ope...@li...>
> *Cc:* Justin Wilson <wi...@ob...>
> *Subject:* Re: [opendds-main] How to get publisher info,
> get_matched_publication_data
>
>
>
> Simon:
>
>
>
> Try get_matched_publication_data using the publication_handle in the
> SampleInfo.
>
>
>
> Justin
>
>
>
> On Mon, Mar 27, 2023 at 5:45 AM Simon Rogers (C2I) <
> Sim...@ul...> wrote:
>
> I am trying to get PublicationBuiltinTopicData for the publisher (such as
> participant key) when I receive some data in my subscriber.
>
>
>
> I get the DDS::SampleInfo for the received data, and call
> get_matched_publication_data on my DataReader, passing in the
> publication_handle from the sample info.
>
>
>
> This results in: DataReaderImpl::get_handle_instance: lookup for 0x12
> failed.
>
>
>
> I am using RTPS discovery rather than InfoRepo.
>
>
>
> How should I go about finding the publisher participant key or similar id
> when I receive a sample in my subscriber?
>
>
>
> Thanks
>
> Simon
>
>
>
> _______________________________________________
> opendds-main mailing list
> ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opendds-main
>
>
>
>
> --
>
> Justin Wilson
>
> Principal Software Engineer
>
>
>
> objectcomputing.com
>
> YOUR OUTCOMES ENGINEERED™
> _______________________________________________
> opendds-main mailing list
> ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opendds-main
>


-- 
Justin Wilson
Principal Software Engineer

objectcomputing.com
YOUR OUTCOMES ENGINEERED™

Justin
Thanks for your response. I’ve tried calling get_matched_publication_data on both the data reader and the narrowed data reader, passing in the publication_handle. In all cases I get uninitialized data out in the PublicationBuiltinTopicData.
Is it possible that when running using RTPS discovery, the built in topics are disabled? I’m confident my data reader is matched with a data writer and is receiving data.

From: Justin Wilson via opendds-main <ope...@li...>
Sent: 27 March 2023 14:38
To: General Discussion concerning OpenDDS <ope...@li...>
Cc: Justin Wilson <wi...@ob...>
Subject: Re: [opendds-main] How to get publisher info, get_matched_publication_data

Simon:

Try get_matched_publication_data using the publication_handle in the SampleInfo.

Justin

On Mon, Mar 27, 2023 at 5:45 AM Simon Rogers (C2I) <Sim...@ul...<mailto:Sim...@ul...>> wrote:
I am trying to get PublicationBuiltinTopicData for the publisher (such as participant key) when I receive some data in my subscriber.

I get the DDS::SampleInfo for the received data, and call get_matched_publication_data on my DataReader, passing in the publication_handle from the sample info.

This results in: DataReaderImpl::get_handle_instance: lookup for 0x12 failed.

I am using RTPS discovery rather than InfoRepo.

How should I go about finding the publisher participant key or similar id when I receive a sample in my subscriber?

Thanks
Simon

_______________________________________________
opendds-main mailing list
ope...@li...<mailto:ope...@li...>
https://lists.sourceforge.net/lists/listinfo/opendds-main


--
Justin Wilson
Principal Software Engineer


objectcomputing.com<https://objectcomputing.com/>
YOUR OUTCOMES ENGINEERED™

Simon:

Try get_matched_publication_data using the publication_handle in the
SampleInfo.

Justin

On Mon, Mar 27, 2023 at 5:45 AM Simon Rogers (C2I) <
Sim...@ul...> wrote:

> I am trying to get PublicationBuiltinTopicData for the publisher (such as
> participant key) when I receive some data in my subscriber.
>
>
>
> I get the DDS::SampleInfo for the received data, and call
> get_matched_publication_data on my DataReader, passing in the
> publication_handle from the sample info.
>
>
>
> This results in: DataReaderImpl::get_handle_instance: lookup for 0x12
> failed.
>
>
>
> I am using RTPS discovery rather than InfoRepo.
>
>
>
> How should I go about finding the publisher participant key or similar id
> when I receive a sample in my subscriber?
>
>
>
> Thanks
>
> Simon
>
>
> _______________________________________________
> opendds-main mailing list
> ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opendds-main
>


-- 
Justin Wilson
Principal Software Engineer

objectcomputing.com
YOUR OUTCOMES ENGINEERED™

I am trying to get PublicationBuiltinTopicData for the publisher (such as participant key) when I receive some data in my subscriber.

I get the DDS::SampleInfo for the received data, and call get_matched_publication_data on my DataReader, passing in the publication_handle from the sample info.

This results in: DataReaderImpl::get_handle_instance: lookup for 0x12 failed.

I am using RTPS discovery rather than InfoRepo.

How should I go about finding the publisher participant key or similar id when I receive a sample in my subscriber?

Thanks
Simon

Hi Simon,

On 3/21/2023 8:25 AM, Simon Rogers (C2I) wrote:
>
> Thanks for your reply Adam.
>
> This is the ideal solution for us – but how do I determine the remote Spdp port to send to, 4567 in your example? That is, how do I control the Spdp port number a participant uses?
>
You can control it through configuration (see below) -- or, if controlling it isn't that essential you can observe which port is in use with debug logging.  At DCPSDebugLevel 4 or higher you'll see Spdp::SpdpTransport::open_unicast_socket() - opened unicast socket on port %d


To influence this port number:

* The RTPS spec defines an expression / procedure for determining the port.  OpenDDS follows this and lets users control the parameters PB (port base), DG (domain gain), D1, and PG (participant gain) -- see Table 7-5 in the Developer's Guide.

OR

* Set the port directly in RtpsConfiguration::spdp_local_address().  Based on what I'm seeing in the code, the port number is configurable in code but not when using the .ini file -- which may be a bug.  Note that you can load a configuration file and then modify the configuration in code before creating a Domain Participant.

Thanks,
Adam Mitz
Principal Software Engineer and Partner
Object Computing, Inc.


> I thought I could specify a local port for a participant using ‘SpdpLocalAddress’ – but this option only takes an address.
>
>  
>
> On 3/21/2023 6:54 AM, Simon Rogers (C2I) wrote:
>
>     I have two groups of 30 RTPS participants, separated by a network which won’t forward Multicast packets, so discovery fails between the groups.
>
>      
>
>     I wondered if I can configure SPDP to use unicast. The Reference manual suggests I can by using ‘SpdpSendAddrs=[host:port],[host:port]...’ to provide a list of SPDP destinations. It says ‘This can be a combination of Unicast and Multicast addresses.’
>
>      
>
>     But I can’t see which options would allow me to specify a unicast SPDP configuration. Can I do this?
>
> This same option should be used to send SPDP messages to a unicast address:
>
> SpdpSendAddrs=10.1.2.3:4567
>
>  
>
>
>
> _______________________________________________
> opendds-main mailing list
> ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opendds-main

Thanks for your reply Adam.
This is the ideal solution for us – but how do I determine the remote Spdp port to send to, 4567 in your example? That is, how do I control the Spdp port number a participant uses?
I thought I could specify a local port for a participant using ‘SpdpLocalAddress’ – but this option only takes an address.

On 3/21/2023 6:54 AM, Simon Rogers (C2I) wrote:
I have two groups of 30 RTPS participants, separated by a network which won’t forward Multicast packets, so discovery fails between the groups.

I wondered if I can configure SPDP to use unicast. The Reference manual suggests I can by using ‘SpdpSendAddrs=[host:port],[host:port]...’ to provide a list of SPDP destinations. It says ‘This can be a combination of Unicast and Multicast addresses.’

But I can’t see which options would allow me to specify a unicast SPDP configuration. Can I do this?

This same option should be used to send SPDP messages to a unicast address:

SpdpSendAddrs=10.1.2.3:4567

Hi Simon,

On 3/21/2023 6:54 AM, Simon Rogers (C2I) wrote:
>
> I have two groups of 30 RTPS participants, separated by a network which won’t forward Multicast packets, so discovery fails between the groups.
>
>  
>
> I wondered if I can configure SPDP to use unicast. The Reference manual suggests I can by using ‘SpdpSendAddrs=[host:port],[host:port]...’ to provide a list of SPDP destinations. It says ‘This can be a combination of Unicast and Multicast addresses.’
>
>  
>
> But I can’t see which options would allow me to specify a unicast SPDP configuration. Can I do this?
>
This same option should be used to send SPDP messages to a unicast address:

SpdpSendAddrs=10.1.2.3:4567


> Alternatively I understand that the RTPSRelay might be an option. But from what I have seen all participant traffic flows through the replay – I would still like my local 30 participants to communicate directly with each other. Or have I got that wrong? Where would the relay go in this setup?
>
The RtpsRelay is an option.  Peers that can exchange messages without the relay's assistance should continue to do.

Another tool we've developed for these kinds of scenarios is the multicast repeater https://github.com/OpenDDS/cloud-multicast-repeater

Thanks,
Adam Mitz
Principal Software Engineer and Partner
Object Computing, Inc.

I have two groups of 30 RTPS participants, separated by a network which won't forward Multicast packets, so discovery fails between the groups.

I wondered if I can configure SPDP to use unicast. The Reference manual suggests I can by using 'SpdpSendAddrs=[host:port],[host:port]...' to provide a list of SPDP destinations. It says 'This can be a combination of Unicast and Multicast addresses.'

But I can't see which options would allow me to specify a unicast SPDP configuration. Can I do this?

Alternatively I understand that the RTPSRelay might be an option. But from what I have seen all participant traffic flows through the replay - I would still like my local 30 participants to communicate directly with each other. Or have I got that wrong? Where would the relay go in this setup?

Thanks for your help
Simon