dataspace-sabul Mailing List for DataSpace

hello sir/mem,
i am trying to install UDT(udt1.2.sdk.tar)  in ns-2.34 in fedora8 by using
following commands as per the help/readme file.


*make -e os=LINUX arch=IA32      --in the dir of ns-2.34/UDT*

for to install the UDT module to  NS-2 ,i am following steps given in
index.html

*Step 1*. make a new dir named udt (optional) under ns directory, e.g.,
./ns-allinone-2.26/ns-2.26.

*Step 2*. updates the *Makefile* in the same directory as step 1.

here,i am updating makefile in ns-2.34 ,is it true?or the same dir means udt
dir?

then i am follow step3 to step6 as given in index.html help file.

*But* ,after that if i recompile using make command in ns2.34 then i got
following error.

**** missing separator (did you mean TAB instead of 8 spaces?).stop.*

after that if i am do modification of TAB than i got following error.

**** commands commence before first target. stop.*

**

please help me to solve problem.

Darshana

QkFOTkVEIEZJTEVOQU1FIEFMRVJUCgpZb3VyIG1lc3NhZ2UgdG86IGFjZHNo
YXJlQGFjZHN5c3RlbXMuY29tCndhcyBibG9ja2VkIGJ5IG91ciBTcGFtIEZp
cmV3YWxsLiBUaGUgZW1haWwgeW91IHNlbnQgd2l0aCB0aGUgZm9sbG93aW5n
IHN1YmplY3QgaGFzIE5PVCBCRUVOIERFTElWRVJFRDoKClN1YmplY3Q6IFJl
OiByZWFkIGl0IGltbWVkaWF0ZWx5CgpBbiBhdHRhY2htZW50IGluIHRoYXQg
bWFpbCB3YXMgb2YgYSBmaWxlIHR5cGUgdGhhdCB0aGUgU3BhbSBGaXJld2Fs
bCBpcyBzZXQgdG8gYmxvY2suCgoK

Greetings.

Just let you know that the UDT version 2.0 has been released and is
available for download at a new sourceforge project:

sourceforge.net/projects/udt

Thanks,

Yunhong Gu, PhD Candidate

Laboratory for Advanced Computing
University of Illinois at Chicago
SEO 700, M/C 249, 851 S Morgan St
Chicago, IL 60607-7045
T (312) 996-0305
F (312) 355-0373

Dear Open Source developer

I am doing a research project on "Fun and Software Development" in which I kindly invite you to participate.
You will find the online survey under http://fasd.ethz.ch/qsf/. The questionnaire consists of 53 questions and you will need about 15 minutes to complete it.

With the FASD project (Fun and Software Development) we want to define the motivational significance of fun when software developers decide to engage in Open Source projects. What is special about our research project is that a similar survey is planned with software developers in commercial firms. This procedure allows the immediate comparison between the involved individuals and the conditions of production of these two development models. Thus we hope to obtain substantial new insights to the phenomenon of Open Source Development.


With many thanks for your participation,
Benno Luthiger


PS:
The results of the survey will be published under http://www.isu.unizh.ch/fuehrung/blprojects/FASD/.
We have set up the mailing list fa...@we... for this study. Please see http://fasd.ethz.ch/qsf/mailinglist_en.html for registration to this mailing list.

_______________________________________________________________________

Benno Luthiger
Swiss Federal Institute of Technology Zurich
8092 Zurich

Mail: benno.luthiger(at)id.ethz.ch
_______________________________________________________________________

Hi, Steven,

On Tue, 24 Feb 2004, Steven M. Carter wrote:
> That would be odd:
> 
> [scarter@unet1 scarter]$ ls -l
> total 36885892
> -rw-rw-r--  1 scarter scarter 10485760000 Feb 19 15:47 10g_foo
> -rw-rw-r--  1 scarter scarter  5242880000 Feb 22 22:14 5g_datafile
> -rw-rw-r--  1 scarter scarter  2147483647 Feb 24 18:03 5g_foo
> -rw-rw-r--  1 scarter scarter 13239889920 Feb 19 15:42 datafile
> 
> 5g_foo is the file that was aborted.  There is a 10g and 13g file in that 
> same directory.  The OS had no problem with those.
> 
> Tsunami transfered a 10g file with no problems :)

Sorry, I checked the "sendfile" code and found that I misused the type of 
"streamsize", which I thought can be used for 64-bit file size (and 
somehow I remember that I did use this program to transfer large files). 
Anyway, I corrected it and upload a new file to the CVS of sourceforge, 
and you can check out the most recent source code there. Or, you can 
simply modify the "sendfile.cpp" file by yourself:

in line 71, change 
streamsize size = ifs.tellg();
to
unsigned long long size = ifs.tellg();
or
streampos size = ifs.tellg();


> It now uses 9000 byte packets (I see 9014 on the wire as opposed to 1514),
> but the transfer rate goes from ~830Mb/s to ~395Mb/s.

I don't understand this either. However, my experience on using 9000byte 
MTU is very little. I read your testing report on both UDT and other 
protocols, including iperf, it seems 830Mbps is normal for the link you 
are testing, so I think you can continue to use 1500byte setting. 

From my experiences, 9000MTU only has significant effect on 10GigE links. 
On 1GigE links and below, 1500byte MTU works fine. On the other hand, even 
if you set 9000 UDT_MTU option on a 1500 MTU link, the performance should 
not degrade that much. It can be related to hardware/OS though. 

> 
> > For the file corruption error, it may be caused by a bug in UDT that we 
> > already fixed. You can find a new version of UDT release (v1.2) at
> > sourceforge.net/projects/dataspace.
> 
> I need to do a test from an IA32 machine to another IA32 machine to rule 
> that out.
> 

Please let me know if the file corruption error is still there.

Thanks,
Gu

On Tue, 24 Feb 2004, Yunhong Gu wrote:
> 
> Hi, Steven,
> 
> Thanks for the information, it is detailed enough for me to trace the 
> error.
> 
> For the "File size limit exceeded" error, it seems that the OS you are 
> using on unet1 does not support file size exceeding 2^31. UDT support 
> 64-bit file size and we have tested it to tranfser a 100GB file. (At 
> least the new version works fine with this, see below.)

That would be odd:

[scarter@unet1 scarter]$ ls -l
total 36885892
-rw-rw-r--  1 scarter scarter 10485760000 Feb 19 15:47 10g_foo
-rw-rw-r--  1 scarter scarter  5242880000 Feb 22 22:14 5g_datafile
-rw-rw-r--  1 scarter scarter  2147483647 Feb 24 18:03 5g_foo
-rw-rw-r--  1 scarter scarter 13239889920 Feb 19 15:42 datafile

5g_foo is the file that was aborted.  There is a 10g and 13g file in that 
same directory.  The OS had no problem with those.

Tsunami transfered a 10g file with no problems :)

> For the MTU error, it needs to be set at both side, and UDT will choose a 
> smaller value at the connection set up phase; so the appclient side also 
> needs to set UDT_MTU as 9000, otherwise 1500, the smaller value, will 
> still be used.

It now uses 9000 byte packets (I see 9014 on the wire as opposed to 1514),
but the transfer rate goes from ~830Mb/s to ~395Mb/s.

> For the file corruption error, it may be caused by a bug in UDT that we 
> already fixed. You can find a new version of UDT release (v1.2) at
> sourceforge.net/projects/dataspace.

I need to do a test from an IA32 machine to another IA32 machine to rule 
that out.

> The project of lambdaftp from which you download UDT is obsoleted, please 
> go to the dataspace project webpage to look for new release of UDT in 
> future.

My notes were wrong on that.  I am using version 1.2.

Thanks,

Steven.

-- 
sc...@or...
Oak Ridge National Laboratory

Hi, Steven,

Thanks for the information, it is detailed enough for me to trace the 
error.

For the "File size limit exceeded" error, it seems that the OS you are 
using on unet1 does not support file size exceeding 2^31. UDT support 
64-bit file size and we have tested it to tranfser a 100GB file. (At 
least the new version works fine with this, see below.)

For the MTU error, it needs to be set at both side, and UDT will choose a 
smaller value at the connection set up phase; so the appclient side also 
needs to set UDT_MTU as 9000, otherwise 1500, the smaller value, will 
still be used.

For the file corruption error, it may be caused by a bug in UDT that we 
already fixed. You can find a new version of UDT release (v1.2) at
sourceforge.net/projects/dataspace.

The project of lambdaftp from which you download UDT is obsoleted, please 
go to the dataspace project webpage to look for new release of UDT in 
future.

Please let me know if you have any further questions.

Thanks,
Gu



On Tue, 24 Feb 2004, Steven M. Carter wrote:

> On Mon, 23 Feb 2004, Yunhong Gu wrote:
> 
> > 
> > Hi, Steven,
> > 
> > I never tried UDT on an Opteron machine to test, but it should NOT matter 
> > that UDT is running on Xeon or Opteron, or any other CPU. There are 
> > assembly code in UDT for Intel CPUs; on other CPU it simply uses the 
> > system calls provided by OS. And since you have use it successeully for 
> > memory-memory transfer, it seems the problem is related to file 
> > operations.
> > 
> > Can you let me know more details of the problem? Does the file transfer 
> > always fail (no matter what the file size is)? Does it work fine on other 
> > machines (e.g, between two Xeon machines) with the same OS?
> 
> The memory to memory tests seems to work when unet2 (Opteron) sends to
> unet1 (Xeon), but not the other way.  The same is true with small file
> transfers.  Large file transfers die with "File size limit exceeded".
> 
> Also, setting UDT_MTU in appserver.cpp doesn't seem to work.  It still 
> uses an MTU of 1500.
> 
> My notes are at:
> http://www.ccs.ornl.gov/~scarter/net100/udpblaster/udt.html
> 
> Thanks,
> 
> Steven.
> 
> 

-- 
Yunhong Gu, PhD Candidate

Laboratory for Advanced Computing
University of Illinois at Chicago
SEO 700, M/C 249, 851 S Morgan St
Chicago, IL 60607-7045
T (312) 996-0305
F (312) 355-0373

On Mon, 23 Feb 2004, Yunhong Gu wrote:

> 
> Hi, Steven,
> 
> I never tried UDT on an Opteron machine to test, but it should NOT matter 
> that UDT is running on Xeon or Opteron, or any other CPU. There are 
> assembly code in UDT for Intel CPUs; on other CPU it simply uses the 
> system calls provided by OS. And since you have use it successeully for 
> memory-memory transfer, it seems the problem is related to file 
> operations.
> 
> Can you let me know more details of the problem? Does the file transfer 
> always fail (no matter what the file size is)? Does it work fine on other 
> machines (e.g, between two Xeon machines) with the same OS?

The memory to memory tests seems to work when unet2 (Opteron) sends to
unet1 (Xeon), but not the other way.  The same is true with small file
transfers.  Large file transfers die with "File size limit exceeded".

Also, setting UDT_MTU in appserver.cpp doesn't seem to work.  It still 
uses an MTU of 1500.

My notes are at:
http://www.ccs.ornl.gov/~scarter/net100/udpblaster/udt.html

Thanks,

Steven.

-- 
sc...@or...
Oak Ridge National Laboratory

Hi, Steven,

I never tried UDT on an Opteron machine to test, but it should NOT matter 
that UDT is running on Xeon or Opteron, or any other CPU. There are 
assembly code in UDT for Intel CPUs; on other CPU it simply uses the 
system calls provided by OS. And since you have use it successeully for 
memory-memory transfer, it seems the problem is related to file 
operations.

Can you let me know more details of the problem? Does the file transfer 
always fail (no matter what the file size is)? Does it work fine on other 
machines (e.g, between two Xeon machines) with the same OS?

Thanks,
Gu

-- 
Yunhong Gu, PhD Candidate

Laboratory for Advanced Computing
University of Illinois at Chicago
SEO 700, M/C 249, 851 S Morgan St
Chicago, IL 60607-7045
T (312) 996-0305
F (312) 355-0373

I am trying to run a test between a dual Xeon and a dual Opteron.  UDT 
seems to compile on both, and the memory to memory test runs ok, but the 
file transfer fails.  I wanted to make sure the I am not having a problem 
because one side is an Opteron before I do too much debugging.

Thanks,

Steven.

-- 
sc...@or...
Oak Ridge National Laboratory

Good afternoon,

	I am attempting to run DataSpace 3 under Solaris 8. I had to make
several changes to get DataSpace to compile. These changes are described
in one of the bug reports submitted to the project site. Dataspace was
compiled with mysql support.

	I am running into problem after installing the software, namely, I
cannot import tables into a mysql database. The steps I take are to:
	1) Delete any existing information from the "DATA" subdirectory
	2) Run "DataCenter -create &"
	3) Run "dstp &"
	4) At this point both the services appear to be running and the
initial datasources created.
	5) I then try running importASCII to import the simple kicks
dataset as done in the documentation. I deviate at the end though, and try
to import into a mysql table instead of a flat file. I then get the
following output:


	accepted soap
	Connection 0 being served on socket 6
	Finished with code 0
	Going away...
	accepted soap
	Adding DSTP Source: c1
	Attempting Connection to Database:test
	mysql_real_connect failed
	result of add source to 127.0.0.1:18002 = 0


Afterwards, something appears to have died (soap?) subsequent attempts to
use importASCII won't even get this far. The servers must be restarted.

Is this a problem that someone has experienced before or perhaps suggest
what the problem likely is? Perhaps I've overlooked something?

Thanks,
Robert Kane

Hello,

The type "ofstream" seems to be depricated and is not recognised anymore
by GCC V. 3.2.3:

c++ -DHAVE_CONFIG_H -I. -I. -I..     -g -O2 -c sabul.cpp
In file included from sabul.cpp:51:
sabul.h:174: 'ofstream' is used as a type, but is not defined as a type.
sabul.cpp: In constructor `CSabul::CSabul()':
sabul.cpp:83: `m_Log' undeclared (first use this function)
sabul.cpp:83: (Each undeclared identifier is reported only once for each 
   function it appears in.

With GCC V. 2.95.4 there are no problems. On the other hand in
"lftp/ftp.cc" and "lftpd/ftpd.cc" still "and" and "or" logical operators
are used which are not recognised by GCC V. 2.95.4, but may be you
replaced them already in your CVS content.

#define and &&
#define or  ||

was the trick to compile those files.

    Kind regards,
    Hans

Hi Gu,

Thanks for your reply. I will have a look at UDT, however a good
functioning of SABUL is also important for us because it seems to
function quite stable.

    Regards,
    Hans

On Tue, Jun 24, 2003 at 10:07:27AM -0500, Yunhong Gu wrote:
> 
> 
> Hi, Hans,
> 
> If SABUL is used independently, such as serves as testing or 
> performance evaluation purpose in your 
> project, you may like to donwload UDT, which is the next generation 
> SABUL. Currently it is only available in CVS (module name: UDT). UDT can 
> allow you to control all these including the timely receiving control you 
> metioned yestoday.
> 
> The usage is similar, except that the class name is changed from 
> CSabul****** to CUDT, which provides transfer function in both directions.
> 
> If you already integrate SABUL into another software system, I would like 
> to modify this output problem.
> 
> Thanks,
> Gu
> 
> 
> 
> On Tue, 24 Jun 2003, Hans Blom wrote:
> 
> > Hello,
> > 
> > The "CSabulSender" and "CSabulRecver" classes are always dropping
> > "sabul.log" classes in the current working directory, also when the
> > "SBL_VERBOSE" flag has been selected to log to screen. By the way: this
> > flag seems to have no effect in the "CSabulRecver" class and a value
> > "true" should be selected to print to the screen in contradiction with
> > the information in "manual.doc".
> > 
> > Would it be possible to allow a larger user control via the API for the
> > log facility, for instance to supply a log filename, or no logging at
> > all? At our hosts we want to put a SABUL sender and receiver executable
> > in the executable path for the test users and we don't want to get
> > "sabul.log" files in all kind of user directories or to have unexpected
> > error messages because the current directory appears to be not user
> > writable.
> > 
> >     Thanks and best regards,
> >     Hans
> > 
> > 
> > -------------------------------------------------------
> > This SF.Net email is sponsored by: INetU
> > Attention Web Developers & Consultants: Become An INetU Hosting Partner.
> > Refer Dedicated Servers. We Manage Them. You Get 10% Monthly Commission!
> > INetU Dedicated Managed Hosting http://www.inetu.net/partner/index.php
> > _______________________________________________
> > Dataspace-sabul mailing list
> > Dat...@li...
> > https://lists.sourceforge.net/lists/listinfo/dataspace-sabul
> > 
> > 
> 

Hi, Hans,

If SABUL is used independently, such as serves as testing or 
performance evaluation purpose in your 
project, you may like to donwload UDT, which is the next generation 
SABUL. Currently it is only available in CVS (module name: UDT). UDT can 
allow you to control all these including the timely receiving control you 
metioned yestoday.

The usage is similar, except that the class name is changed from 
CSabul****** to CUDT, which provides transfer function in both directions.

If you already integrate SABUL into another software system, I would like 
to modify this output problem.

Thanks,
Gu



On Tue, 24 Jun 2003, Hans Blom wrote:

> Hello,
> 
> The "CSabulSender" and "CSabulRecver" classes are always dropping
> "sabul.log" classes in the current working directory, also when the
> "SBL_VERBOSE" flag has been selected to log to screen. By the way: this
> flag seems to have no effect in the "CSabulRecver" class and a value
> "true" should be selected to print to the screen in contradiction with
> the information in "manual.doc".
> 
> Would it be possible to allow a larger user control via the API for the
> log facility, for instance to supply a log filename, or no logging at
> all? At our hosts we want to put a SABUL sender and receiver executable
> in the executable path for the test users and we don't want to get
> "sabul.log" files in all kind of user directories or to have unexpected
> error messages because the current directory appears to be not user
> writable.
> 
>     Thanks and best regards,
>     Hans
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: INetU
> Attention Web Developers & Consultants: Become An INetU Hosting Partner.
> Refer Dedicated Servers. We Manage Them. You Get 10% Monthly Commission!
> INetU Dedicated Managed Hosting http://www.inetu.net/partner/index.php
> _______________________________________________
> Dataspace-sabul mailing list
> Dat...@li...
> https://lists.sourceforge.net/lists/listinfo/dataspace-sabul
> 
> 

Hello,

The "CSabulSender" and "CSabulRecver" classes are always dropping
"sabul.log" classes in the current working directory, also when the
"SBL_VERBOSE" flag has been selected to log to screen. By the way: this
flag seems to have no effect in the "CSabulRecver" class and a value
"true" should be selected to print to the screen in contradiction with
the information in "manual.doc".

Would it be possible to allow a larger user control via the API for the
log facility, for instance to supply a log filename, or no logging at
all? At our hosts we want to put a SABUL sender and receiver executable
in the executable path for the test users and we don't want to get
"sabul.log" files in all kind of user directories or to have unexpected
error messages because the current directory appears to be not user
writable.

    Thanks and best regards,
    Hans

Thanks,

I'll add it soon.

Gu


On Mon, 23 Jun 2003, Hans Blom wrote:

> Hello,
> 
> For our tests it would be convenient to send also data for a fixed time
> duration instead of a fixed size. In that case at the receiver site the
> data size to receive is unknown. To be able to handle this situation at
> a high level it would be convenient if the "CSabulRecver::recv" member
> function would throw execptions at the corresponding signals, for
> instance at "SIGPIPE" which occurs after reading from a closed pipe. As
> far as I know (please correct me when I am wrong) no such signals are
> yet thrown.
> 
>     Kind regards,
>     Hans
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: INetU
> Attention Web Developers & Consultants: Become An INetU Hosting Partner.
> Refer Dedicated Servers. We Manage Them. You Get 10% Monthly Commission!
> INetU Dedicated Managed Hosting http://www.inetu.net/partner/index.php
> _______________________________________________
> Dataspace-sabul mailing list
> Dat...@li...
> https://lists.sourceforge.net/lists/listinfo/dataspace-sabul
> 
> 

Hello,

For our tests it would be convenient to send also data for a fixed time
duration instead of a fixed size. In that case at the receiver site the
data size to receive is unknown. To be able to handle this situation at
a high level it would be convenient if the "CSabulRecver::recv" member
function would throw execptions at the corresponding signals, for
instance at "SIGPIPE" which occurs after reading from a closed pipe. As
far as I know (please correct me when I am wrong) no such signals are
yet thrown.

    Kind regards,
    Hans

This is a discussion forum for SABUL, UDT and any issues for high 
performance data transfer protocols.

Gu