jxcl-devel Mailing List for The JXCL Project

On Mon, 7 Mar 2005, De Winter, Jack wrote:

> Hi, I am having a nasty time getting the Ucoverage tool up and running.
> I have eliminated all of the options except the noted ones, and get the
> following output. Is there something simple I am forgetting to setup
> that isn't covered in the notes?

UCovered is intended to be used with Ant, JUnit, a suite of unit tests,
and some software under test.  The ucovered/example/3x6 subdirectory
contains a complete test setup, one in which three test classes (TestA,
TestB, and TestC) drive six classes under test (Front1, Front2, Front3,
Middle1, Middle2, and Back).  The object of the game is to determine
to what degree the test classes exercise the classes under test.  In
this case there are three test classes and six classes under test.

What I would do is first verify that example/3x6 works in your
environment.  Then I would modify example/3x6/build.xml so that it
uses your test classes under Ant and JUnit to exercise the software
you want you measure coverage over.

> Cheers,
> Jack
>
> C:\projects\CalcService>%JAVADIR%\java  org.jxcl.ucovered.textui.TestRunner amazon.SurchargeCalculation.business.tests.TestSupportObjects checkCoverage="true" checkIncludes="amazon.SurchargeCalculation.business" checkExcludes="amazon.SurchargeCalculation.business" filtertrace="true" haltOnError="true" haltOnFailure="true"  propfile="myPropFile" showOutput="true" todir="output
> NO TALK OPTIONS SET
> Textui.handleArgs: setting up JXCL with classpath:
> .;\scripts\classpath\junit.jar;\scripts\classpath\jxcl-0.7.jar;\scripts\classpath\ucovered-0.7.jar;\scripts\classpath\ant-1.5.4.jar;\scripts\classpath\bcel-5.1.jar
> TestRunner.runWithIt - test amazon.SurchargeCalculation.business.tests.TestSupportObjects
>
> ===========================
>   UCovered COVERAGE REPORT
> * the registry is empty *
> ===========================
>
>
> C:\projects\CalcService>%JAVADIR%\java  junit.textui.TestRunner amazon.SurchargeCalculation.business.tests.TestSupportObjects
> ................
> Time: 0.06
>
> OK (16 tests)
>
>
> C:\projects\CalcService>
> --------
> Jack De Winter - Software Development Engineer
> dew...@am...
> Ordering - POSSE - Taxation
> (206) 266-7265

--
Jim Dixon  jd...@di...   tel +44 117 982 0786  mobile +44 797 373 7881
http://jxcl.sourceforge.net                       Java unit test coverage
http://xlattice.sourceforge.net         p2p communications infrastructure

This is the first release of UCovered, JXCL's Java test coverage
tool, as a separate component.  There are only minor changes in
the source code.  A few minor bugs have been corrected, exposing
one more (the Ant task can only be run with fork="false").  The
current release of JXCL, jxcl-0.7, must be installed before running
UCovered.  All other necessary jars (dependencies) are included with
jxcl-0.7.
--
Jim Dixon  jd...@di...   tel +44 117 982 0786  mobile +44 797 373 7881
http://jxcl.sourceforge.net                       Java unit test coverage
http://xlattice.sourceforge.net         p2p communications infrastructure

This release is the first step in integrating XLattice's XML and
project management facilities into JXCL.  There are no
substantial changes in JXCL's source code.  The release does
not include the UCovered Java coverage test tool.  It does
include jars for all dependencies (Ant, JUnit, XLattice, etc).
A separate UCovered release will appear within 24 hours.

--
Jim Dixon  jd...@di...   tel +44 117 982 0786  mobile +44 797 373 7881
http://jxcl.sourceforge.net                       Java unit test coverage
http://xlattice.sourceforge.net         p2p communications infrastructure

On Mon, 28 Jun 2004, Stefan Wagner wrote:

> Can someone please explain to me which kinds of coverage are supported by jxcl
> and what is their exact definition? Is branch, decision, or even conditon
> coverage possible?

The UCovered package within JXCL currently does statement coverage.
That is, it outputs a report showing what percentage of lines of code
in the software under test have been exercised by the unit tests.

JXCL and UCovered are currenntly being reorganized with particular
emphasis on XML reports.  I expect that this will take a month or so.

--
Jim Dixon  jd...@di...   tel +44 117 982 0786  mobile +44 797 373 7881
http://jxcl.sourceforge.net                       Java unit test coverage
http://xlattice.sourceforge.net         p2p communications infrastructure

=2D----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello!

Can someone please explain to me which kinds of coverage are supported by j=
xcl=20
and what is their exact definition? Is branch, decision, or even conditon=20
coverage possible?

Thanks,
Stefan

=2D --=20
Stefan Wagner, Software & Systems Engineering
Technische Universitaet Muenchen, Institut fuer Informatik
phone : +49 89 289-17840     fax: +49 89 289-17307
www   : http://www4.in.tum.de/~wagnerst
=2D----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)

iD8DBQFA4BPbQgueae8+2RkRAi2WAJ9gu5sQcC4V62xOrZABAsYjeWpSQACfRHIa
tJ4faCQOdyCShspb99vZ74A=3D
=3DwYmD
=2D----END PGP SIGNATURE-----

Dear Open Source developer

I am doing a research project on "Fun and Software Development" in which I kindly invite you to participate.
You will find the online survey under http://fasd.ethz.ch/qsf/. The questionnaire consists of 53 questions and you will need about 15 minutes to complete it.

With the FASD project (Fun and Software Development) we want to define the motivational significance of fun when software developers decide to engage in Open Source projects. What is special about our research project is that a similar survey is planned with software developers in commercial firms. This procedure allows the immediate comparison between the involved individuals and the conditions of production of these two development models. Thus we hope to obtain substantial new insights to the phenomenon of Open Source Development.


With many thanks for your participation,
Benno Luthiger


PS:
The results of the survey will be published under http://www.isu.unizh.ch/fuehrung/blprojects/FASD/.
We have set up the mailing list fa...@we... for this study. Please see http://fasd.ethz.ch/qsf/mailinglist_en.html for registration to this mailing list.

_______________________________________________________________________

Benno Luthiger
Swiss Federal Institute of Technology Zurich
8092 Zurich

Mail: benno.luthiger(at)id.ethz.ch
_______________________________________________________________________

A new release has been posted to the Sourceforge download area.  This
release renames the JXunit module to UCovered, changes the documentation
accordingly, and makes some minor bug fixes.

CVS is currently *lagging* the tarballs, although this should be corrected
shortly.

The next release will be a beta release and should incorporate
* fixes for all known code bugs
* a corrected Maven build file
* improved documentation
* improved report facilities

This should be available by the end of the month.

--
Jim Dixon  jd...@di...   tel +44 117 982 0786  mobile +44 797 373 7881
http://jxcl.sourceforge.net                       Java unit test coverage
http://xlattice.sourceforge.net         p2p communications infrastructure

On Sat, 17 Apr 2004, Ben Walding wrote:

> Ah, I now see a note at the bottom of the Quilt page talking about the
> splits etc.
>
> I recently posted a couple of bugs into the quilt issue tracker about
> problems compiling on a win32 platform - the / vs. \ and file:// issues
> were a big problem.  I should be able to repatch jxcl to fix those
> issues as well (it looks to be pretty much the same issues).

I think that you will find that these particular problems were fixed in
the most recent JXCL release.  There is still one remaining bug which
should be fixed in the next.

> I'm planning on writing a unit testing plugin / coverage plugin for
> eclipse based on jxcl / quilt (http://undercover.codehaus.org/ when I
> get going)

Sounds good.  Please keep us posted.

--
Jim Dixon  jd...@di...   tel +44 117 982 0786  mobile +44 797 373 7881
http://jxcl.sourceforge.net                       Java unit test coverage
http://xlattice.sourceforge.net         p2p communications infrastructure

On Fri, 16 Apr 2004, David Bernard wrote:

> Thanks, so I wait few days. IHMO publish online some reports would be
> great for interesting/potential users, like this they could see the
> result of the "already existing docs".

A good idea.

The schedule is too tight to get this into the next alpha release, but
I will try to work some sort of sample reports into the beta release,
which may be out in May.

> Quoting Jim Dixon <jd...@di...>:
>
> > On Fri, 16 Apr 2004, David Bernard wrote:
> >
> > > Hi,
> > >
> > > 1)I want to use/eval a test coverage : which one, Quit-0.6-a5 or
> > > JXunit (in JXCL) ?
> >
> > JXCL is a fork of the Quilt project; the current version of JXCL is
> > essentially Quilt-0.7.
> >
> > > 2)Have you some sample report (xml, html) ? Could you send me or
> > > publish online ?
> >
> > If you download the binary distribution and run the examples, it will
> > produce several sample reports.
> >
> > Note that a new release of JXCL is due out within the next few days.

--
Jim Dixon  jd...@di...   tel +44 117 982 0786  mobile +44 797 373 7881
http://jxcl.sourceforge.net                       Java unit test coverage
http://xlattice.sourceforge.net         p2p communications infrastructure

On Sat, 17 Apr 2004, Ben Walding wrote:

> I've just started trying to use Quilt, but it seems like JXCL has taken
> it's place?

Yes.  I rewrote Quilt, producing Quilt v0.6.  This was then used as the
basis of JXCL v0.7

> If that's the case, then a note on the quilt page would be a really good
> idea!

But politically a bit difficult ;-)

> Cheers,
>
> Ben

--
Jim Dixon  jd...@di...   tel +44 117 982 0786  mobile +44 797 373 7881
http://jxcl.sourceforge.net                       Java unit test coverage
http://xlattice.sourceforge.net         p2p communications infrastructure

On Fri, 16 Apr 2004, David Bernard wrote:

> Hi,
>
> 1)I want to use/eval a test coverage : which one, Quit-0.6-a5 or
> JXunit (in JXCL) ?

JXCL is a fork of the Quilt project; the current version of JXCL is
essentially Quilt-0.7.

> 2)Have you some sample report (xml, html) ? Could you send me or
> publish online ?

If you download the binary distribution and run the examples, it will
produce several sample reports.

Note that a new release of JXCL is due out within the next few days.

> Thanks for your help.
>
>
>
> --------------------------------------------------------------
> David "Dwayne" Bernard             Freelance Developer (Java)
>                                    mailto:dw...@ja...
>       \|/                          http://dwayne.java-fan.com/
> --o0O @.@ O0o-------------------------------------------------

--
Jim Dixon  jd...@di...   tel +44 117 982 0786  mobile +44 797 373 7881
http://jxcl.sourceforge.net                       Java unit test coverage
http://xlattice.sourceforge.net         p2p communications infrastructure

A new release has been posted to the Sourceforge download area.  This
incorporates some bugfixes and Dale King's patches for Windows.

There will be a follow-up release by Friday 16 April renaming the
jxunit module to ucovered.  This should be followed a week later by
a beta release.

--
Jim Dixon  jd...@di...   tel +44 117 982 0786  mobile +44 797 373 7881
http://jxcl.sourceforge.net                       Java unit test coverage
http://xlattice.sourceforge.net         p2p communications infrastructure

The version of JXCL in CVS has been updated to reflect patches supplied
by Dale King (for which much thanks).  Also, the documentation and the
CVS code have been modified in accordance with changes in Ant with the
release of Ant 1.6.

After some further changes, another alpha release in JXCL, jxcl-0.7a3,
should be available at Sourceforge no later than Friday 8 April 2004.

--
Jim Dixon  jd...@di...   tel +44 117 982 0786  mobile +44 797 373 7881
http://jxcl.sourceforge.net                       Java unit test coverage
http://xlattice.sourceforge.net         p2p communications infrastructure

----- Original Message -----
From: "Jim Dixon" <jd...@di...>
To: "King Dale" <dal...@th...>
Cc: <jxc...@li...>
Sent: Tuesday, March 02, 2004 3:29 PM
Subject: RE: [Jxcl-devel] RE: So what's the status of JXCL


> On Tue, 2 Mar 2004, King Dale wrote:
>
> > I've been studying the code and think I have a grasp. I think we can fix
the
> > try-finally problems. It appears one problem is that the Eclipse
compiler
> > generates something entirely different than the Sun compiler. I think
what
> > javac generates has also changed from one version to another. Looking at
> > this old article from 1997 it describes the code that Eclipse generates:
> >
> > http://www.javaworld.com/javaworld/jw-02-1997/jw-02-hood.html
>
> More to read over coffee.  However, at a glance it's an inaccurate
> description of how the compiler generates code.

I think it is more of an outdated description of how things were done long
ago. Note that the article is from 1997.

> > So something has changed in javac land. What I see from javac duplicates
the
> > code from the finally clause in multiple places if it is small and uses
a
> > subroutine if it is longer.
>
> Got an example?  I test this using samples where the finally clause is one
> statement.  It always uses the subroutine.

Examples below.

> >                        Eclipse seems to always use the subroutine. When
> > javac uses the jsr, it duplicates the jsr in all paths that exit the try
> > block gracefully then does a goto to the code after the try. Eclipse
moves
> > the jsr to the end and has all those paths do a goto to the jsr which
then
> > falls through to the code after the try.
>
> That's different from what I see.

Which version are you using. The impression I got from the bug reports is
that they changed this for 1.4 or later.

> > I'll try to do a bug search in the bug database to see if I can learn
more.
> > I'm guessing that they did this to make life easier for HotSpot to
figure
> > things out, which makes like more complicated for those trying to figure
out
> > control flow with the output of other compilers.

Looking at the bug database there are complaints about the changes to the
generated code for finally and talk about the inlining of finally code,
which is what I see.

I've attached 3 files

Test.java is a version of your Finally class that adds another version of
runTest that makes the finally bock larger.
javap_javac_1.4.2_03.txt is the output of javap -verbose for the class
compiled with JDK1.4.2_03-b02.
javap_eclipse_3.0M7.txt is the javap -vebose output when compiled with
version 3.0M7 of Eclipse.

Note that javap screws up some line endings.

Notice the fun part that javac actually has duplicated the finally code in
multiple places so has multiple instances in the line number table.

I'm beginning to think annotation at the source level is the way to go. It
looks like there are too many ambiguities in the bytecode and it will
probably only get worse in 1.5.
--
  Dale King

On Mon, 23 Feb 2004, Dale King wrote:

> The POM for the Maven build of JXCL is missing this
>
> dependency:
>
>     <dependency>
>       <groupId>ant</groupId>
>       <artifactId>ant-optional</artifactId>
>       <version>1.5.3-1</version>
>       <url>http://jakarta.apache.org/ant/</url>
>     </dependency>

Corrected in CVS.

--
Jim Dixon  jd...@di...   tel +44 117 982 0786  mobile +44 797 373 7881
"Be liberal in what you accept,                                Jon Postel
and conservative in what you send."                               RFC 793
http://jxcl.sourceforge.net                       Java unit test coverage
http://xlattice.sourceforge.net         p2p communications infrastructure

On Tue, 2 Mar 2004, King Dale wrote:

> I've been studying the code and think I have a grasp. I think we can fix the
> try-finally problems. It appears one problem is that the Eclipse compiler
> generates something entirely different than the Sun compiler. I think what
> javac generates has also changed from one version to another. Looking at
> this old article from 1997 it describes the code that Eclipse generates:
>
> http://www.javaworld.com/javaworld/jw-02-1997/jw-02-hood.html

More to read over coffee.  However, at a glance it's an inaccurate
description of how the compiler generates code.

> So something has changed in javac land. What I see from javac duplicates the
> code from the finally clause in multiple places if it is small and uses a
> subroutine if it is longer.

Got an example?  I test this using samples where the finally clause is one
statement.  It always uses the subroutine.

>                        Eclipse seems to always use the subroutine. When
> javac uses the jsr, it duplicates the jsr in all paths that exit the try
> block gracefully then does a goto to the code after the try. Eclipse moves
> the jsr to the end and has all those paths do a goto to the jsr which then
> falls through to the code after the try.

That's different from what I see.

> I'll try to do a bug search in the bug database to see if I can learn more.
> I'm guessing that they did this to make life easier for HotSpot to figure
> things out, which makes like more complicated for those trying to figure out
> control flow with the output of other compilers.

--
Jim Dixon  jd...@di...   tel +44 117 982 0786  mobile +44 797 373 7881
"Be liberal in what you accept,                                Jon Postel
and conservative in what you send."                               RFC 793
http://jxcl.sourceforge.net                       Java unit test coverage
http://xlattice.sourceforge.net         p2p communications infrastructure

On Sat, 28 Feb 2004, King Dale wrote:

> > > I have some ideas about JXCL.
> > >
> > > One is that there are tools like Jcoverage that instrument the
> > > code separately from testing it and JXCL which instruments it
> > > on the fly. Both have advantages and disadvantages. But
> > > there is no reason why JXCL couldn't do both. In both cases
> > > you have to read a class, instrument it, and produce data in
> > > class file format. That could be called from either a
> > > classloader or from a tool that wrote it to disk.
> >
> > As you describe it, this is a fairly minor change.  All that you
> > have to do is write the byte array containing the instrumented code
> > to disk.
>
> Sure its easy, but you aren't exposing that functionality.

Well, of course I am -- it's a public interface.  However, I think that
you are talking about something else, like perhaps a command line option
that means "oh, and write the instrumented code into a separate directory
tree".

> > > I also want to be able to work in what I think is an absolute
> > > must for any coverage tool, which is the ability to declare
> > > certain code that isn't supposed to be executed so that it is
> > > not included in the total for calculating percentage. But it is
> > > absolutely vital that it also report it as an error if it is
> > > executed. I think it may be possible to implement that in JXCL
> > > even though it is only working from byte code.
> >
> > Can you define exactly what you mean here?  Are you talking about
> > marking certain ranges of source code?  Or would you be content with
> > excluding code at the method level?
>
> I mean marking ranges of source code, not at the method level! I actually
> tried to make the case to Joshua Bloch that the new metadata facility in 1.5
> should include annotating within a method for this very kind of thing, but
> was turned down.
>
> As I understand your code, you compute a control flow graph of the method so

It's not exactly a control flow graph...

> basically you could determine where the end of a block is. If we could
> insert a harmless statement into the code that could be easily identified by
> the instrumentation code then we should be able to mark code as unreachable.

Cheapest thing is a boolean, which is also intuitive.  Call it perhaps
org.jxcl.Counting.z$$zCount.  Then I think that it's easy for JXCL to pick
up
  z$$zCount = false;
to mean turn off counting and
  z$$zCount = true;
to turn it back on.  The compiler, of course, has to be persuaded not to
optimize the statements away or move them around.

> The first idea is simply a static method call on a certain class. So I could
> have:
>
> 	if( param == null ) // Should never happen
> 	{
> 		JXCL.unreachableCode();
> 		throw new NullPointerException();
> 	}
> 	doSomething();
>
> The method does not even have to do anything. It is just a signal to the
> instrumentation code, which simply looks for an invokestatic on that method.
> The tough part is that you have to figure out where the end of the block is
> because none of the rest of the code should be reachable from after that
> point until the end of the block. I think that should be possible from your
> control flow analysis.

What exactly do you mean by "the end of the block"?  If you mean the end
of the method, no problem.  But if you mean it literally (I mean, "as in
C"), as for example
	{ // beginning of the block
		// statements
	} // end of the block
it will probably be difficult, because you have no guarantee as to what
the compiler will generate from this sort of input.

> I also want to make this configurable because you might have different types
> of testing like unit testing, integration testing, system testing, etc.
> Whether this statement is reachable may vary based on that. For instance, in
> unit testing you might be able to pass a null param, but in integration and
> functional testing that should never happen. So it is reachable in unit
> testing but not integration testing.

JXCL allows you to write ANYTHING into the bytecode, or drop anything from
the bytecode.  So the thing to focus on is elegance.

> Under most coverage tools that will actually decrease your code coverage
> totals when in reality it is a good thing that you never pass a bad
> parameter.
>
> So I am thinking of having an overload of that method that takes a parameter
> to say when it is unreachable. Either an int which is a bit mask or a
> string. The assigning of bits or string tokens would be up to the user and
> not dictated by JXCL. So for this scenario we might have:
>
> 	JXCL.unreachableCode( TESTING.Integration | TESTING.System );
>
> or
>
> 	JXCL.unreachableCode( "integration, system" );
>
> There are probably several other ways to do it, but this is what I am
> contemplating. It just has to be something that the instrumentation code can
> easily identify and that is configurable to allow different testing
> scenarios.

I suspect that this would probably better be handled using assignments,
perhaps something like
	org.jxcl.TestRegime.unreachableDuring = TestRegime.SYSTEM;

However, I think that there should be a better way of expressing the basic
idea here.

> Another way I just thought of that also eliminates any dependencies on JXCL
> in the regular code would be allow the user to specify a method name that is
> used during instrumentation to flag unreachable code. The type of testing is
> then controlled by the class name upon which it is invoked. So the above
> could be done as:
>
> 	IntegrationTesting.unreachableCode();
> 	SystemTesting.unreachableCode();
>
> The definition of the IntegrationTesting and SystemTesting are not dictated
> by JXCL. It is just looking for a static method call of a given name with no
> parameters.

Yes.  This is more or less the approach taken by JUnit.

> I think this new idea might be better because it removes the need to figure
> out what the parameter is to the method call. You just have to look for
> certain invoke static calls. I like the idea of not having any dependence on

(Something missing in your text here.)

> All the instrumentation code has to do is include data in about what lines
> are marked unreachable in its output. The work of looking for executions of
> these lines is simply part of the reporting code.
>
> This kind of feature has not been implemented by anybody that I know of and
> to me seem like a basic necessity in code coverage. Without it you will
> never be able to reach 100% coverage. But remember that the reporting has to
> signal an error if one of these lines is executed. This prevents someone

This is a separate behaviour, but easily handled.

> from marking a whole bunch of code as unreachable to inflate the coverage
> numbers. The report should also indicate the percent of code marked
> unreachable so someone can't mark most of the code unreachable and not
> exercise it.

In the current version of the code, line numbering is erratic.  For
example, the reported line numbers sometimes go backwards.  But I believe
this can easily be fixed (famous last words).

> > > But then again I have never actually seen JXCL in action. I've
> > > had to work to even get it to build and run on Windows.
> >
> > The examples don't work?
>
> I haven't gotten them to work. I am trying from Windows, which is obviously
> something you had not tested (see my patch). The issue may be the Ant
> classpath bug you talked about.
>
> You report that you were geting several test failures with Maven. After my

In the current version of the code there is exactly one failure during
the Ant build, in TestFinally.

> patch I only got the ones to do with finally clauses. I started looking at
> those. TestFinally fails because the code generated by the compiler is not
> how the code thinks it is. You expect a try finally to have 3 null handlers
> when it only has two and you get an array bounds exception. Perhaps the code

Don't remember, to be honest, but I think that I get a different error.

> generated for finally has changed from what you expected.

I need to review the code.  As I recall, the vertex has a complex
connector with lessee ...
  edge to first vertex in try block
  one edge for each catch handler
  one edge for the finally handler

I should be looking at this over coffee in the morning.
--
Jim Dixon  jd...@di...   tel +44 117 982 0786  mobile +44 797 373 7881
"Be liberal in what you accept,                                Jon Postel
and conservative in what you send."                               RFC 793
http://jxcl.sourceforge.net                       Java unit test coverage
http://xlattice.sourceforge.net         p2p communications infrastructure

> -----Original Message-----
> From: Jim Dixon [mailto:jd...@di...]
> 
> On Sat, 28 Feb 2004, King Dale wrote:
> 
> > > > You'll see that I submitted a patch that fixes the class path
> > > > handling to work on Windoze.
> > >
> > > ? I haven't seen any patch.  Where did you send it?
> >
> > I uploaded it as a patch on Sourceforge:
> >
> > http://sourceforge.net/tracker/?group_id=92728&atid=601791
> 
> No files were attached.  There was no patch.
> 
> I would be very interested in seeing your proposed changes.  Can you
> attach the patch to a reply to this email?

Oops, apparently, I forgot to actually click the little box that said attach
the file. I've deleted it off my machine here at work. I'll upload it
tonight at home.

I'm interested to hear your thoughts on the ability to mark code as
unreachable.

I've been studying the code and think I have a grasp. I think we can fix the
try-finally problems. It appears one problem is that the Eclipse compiler
generates something entirely different than the Sun compiler. I think what
javac generates has also changed from one version to another. Looking at
this old article from 1997 it describes the code that Eclipse generates:

http://www.javaworld.com/javaworld/jw-02-1997/jw-02-hood.html

So something has changed in javac land. What I see from javac duplicates the
code from the finally clause in multiple places if it is small and uses a
subroutine if it is longer. Eclipse seems to always use the subroutine. When
javac uses the jsr, it duplicates the jsr in all paths that exit the try
block gracefully then does a goto to the code after the try. Eclipse moves
the jsr to the end and has all those paths do a goto to the jsr which then
falls through to the code after the try.

I'll try to do a bug search in the bug database to see if I can learn more.
I'm guessing that they did this to make life easier for HotSpot to figure
things out, which makes like more complicated for those trying to figure out
control flow with the output of other compilers.
-- 
 Dale King

On Sat, 28 Feb 2004, King Dale wrote:

> > > You'll see that I submitted a patch that fixes the class path
> > > handling to work on Windoze.
> >
> > ? I haven't seen any patch.  Where did you send it?
>
> I uploaded it as a patch on Sourceforge:
>
> http://sourceforge.net/tracker/?group_id=92728&atid=601791

No files were attached.  There was no patch.

I would be very interested in seeing your proposed changes.  Can you
attach the patch to a reply to this email?

--
Jim Dixon  jd...@di...   tel +44 117 982 0786  mobile +44 797 373 7881
http://jxcl.sourceforge.net                       Java unit test coverage
http://xlattice.sourceforge.net         p2p communications infrastructure

> -----Original Message-----
> From: Jim Dixon
> 
> On Sat, 28 Feb 2004, Dale King wrote:
> 
> > Just pinging to find out what the status of JXCL. Doesn't
> > seem to be any activity recently. I'm interested in seeing a
> > good OS coverage tool. Clover sucks. It tells me most of my
> > class was not executed when it was. JCoverage/GPL is OK,
> > but not great. I think JXCL can be better.
> >
> > You'll see that I submitted a patch that fixes the class path
> > handling to work on Windoze.
> 
> ? I haven't seen any patch.  Where did you send it?

I uploaded it as a patch on Sourceforge:

http://sourceforge.net/tracker/?group_id=92728&atid=601791

> > I have some ideas about JXCL.
> >
> > One is that there are tools like Jcoverage that instrument the
> > code separately from testing it and JXCL which instruments it
> > on the fly. Both have advantages and disadvantages. But
> > there is no reason why JXCL couldn't do both. In both cases
> > you have to read a class, instrument it, and produce data in
> > class file format. That could be called from either a
> > classloader or from a tool that wrote it to disk.
> 
> As you describe it, this is a fairly minor change.  All that you
> have to do is write the byte array containing the instrumented code
> to disk.

Sure its easy, but you aren't exposing that functionality.

> > I also want to be able to work in what I think is an absolute
> > must for any coverage tool, which is the ability to declare
> > certain code that isn't supposed to be executed so that it is
> > not included in the total for calculating percentage. But it is
> > absolutely vital that it also report it as an error if it is
> > executed. I think it may be possible to implement that in JXCL
> > even though it is only working from byte code.
> 
> Can you define exactly what you mean here?  Are you talking about
> marking certain ranges of source code?  Or would you be content with
> excluding code at the method level?

I mean marking ranges of source code, not at the method level! I actually
tried to make the case to Joshua Bloch that the new metadata facility in 1.5
should include annotating within a method for this very kind of thing, but
was turned down.

As I understand your code, you compute a control flow graph of the method so
basically you could determine where the end of a block is. If we could
insert a harmless statement into the code that could be easily identified by
the instrumentation code then we should be able to mark code as unreachable.
The first idea is simply a static method call on a certain class. So I could
have:

	if( param == null ) // Should never happen
	{
		JXCL.unreachableCode();
		throw new NullPointerException();
	}
	doSomething();

The method does not even have to do anything. It is just a signal to the
instrumentation code, which simply looks for an invokestatic on that method.
The tough part is that you have to figure out where the end of the block is
because none of the rest of the code should be reachable from after that
point until the end of the block. I think that should be possible from your
control flow analysis.

I also want to make this configurable because you might have different types
of testing like unit testing, integration testing, system testing, etc.
Whether this statement is reachable may vary based on that. For instance, in
unit testing you might be able to pass a null param, but in integration and
functional testing that should never happen. So it is reachable in unit
testing but not integration testing.

Under most coverage tools that will actually decrease your code coverage
totals when in reality it is a good thing that you never pass a bad
parameter.

So I am thinking of having an overload of that method that takes a parameter
to say when it is unreachable. Either an int which is a bit mask or a
string. The assigning of bits or string tokens would be up to the user and
not dictated by JXCL. So for this scenario we might have:

	JXCL.unreachableCode( TESTING.Integration | TESTING.System );

or

	JXCL.unreachableCode( "integration, system" );

There are probably several other ways to do it, but this is what I am
contemplating. It just has to be something that the instrumentation code can
easily identify and that is configurable to allow different testing
scenarios.

Another way I just thought of that also eliminates any dependencies on JXCL
in the regular code would be allow the user to specify a method name that is
used during instrumentation to flag unreachable code. The type of testing is
then controlled by the class name upon which it is invoked. So the above
could be done as:

	IntegrationTesting.unreachableCode();
	SystemTesting.unreachableCode();

The definition of the IntegrationTesting and SystemTesting are not dictated
by JXCL. It is just looking for a static method call of a given name with no
parameters.

I think this new idea might be better because it removes the need to figure
out what the parameter is to the method call. You just have to look for
certain invoke static calls. I like the idea of not having any dependence on

All the instrumentation code has to do is include data in about what lines
are marked unreachable in its output. The work of looking for executions of
these lines is simply part of the reporting code.

This kind of feature has not been implemented by anybody that I know of and
to me seem like a basic necessity in code coverage. Without it you will
never be able to reach 100% coverage. But remember that the reporting has to
signal an error if one of these lines is executed. This prevents someone
from marking a whole bunch of code as unreachable to inflate the coverage
numbers. The report should also indicate the percent of code marked
unreachable so someone can't mark most of the code unreachable and not
exercise it.

> > But then again I have never actually seen JXCL in action. I've
> > had to work to even get it to build and run on Windows.
> 
> The examples don't work?

I haven't gotten them to work. I am trying from Windows, which is obviously
something you had not tested (see my patch). The issue may be the Ant
classpath bug you talked about.

You report that you were geting several test failures with Maven. After my
patch I only got the ones to do with finally clauses. I started looking at
those. TestFinally fails because the code generated by the compiler is not
how the code thinks it is. You expect a try finally to have 3 null handlers
when it only has two and you get an array bounds exception. Perhaps the code
generated for finally has changed from what you expected.

-- 
 Dale King

On Sat, 28 Feb 2004, Dale King wrote:

> Just pinging to find out what the status of JXCL. Doesn't
> seem to be any activity recently. I'm interested in seeing a
> good OS coverage tool. Clover sucks. It tells me most of my
> class was not executed when it was. JCoverage/GPL is OK,
> but not great. I think JXCL can be better.
>
> You'll see that I submitted a patch that fixes the class path
> handling to work on Windoze.

? I haven't seen any patch.  Where did you send it?

> I have some ideas about JXCL.
>
> One is that there are tools like Jcoverage that instrument the
> code separately from testing it and JXCL which instruments it
> on the fly. Both have advantages and disadvantages. But
> there is no reason why JXCL couldn't do both. In both cases
> you have to read a class, instrument it, and produce data in
> class file format. That could be called from either a
> classloader or from a tool that wrote it to disk.

As you describe it, this is a fairly minor change.  All that you
have to do is write the byte array containing the instrumented code
to disk.

> I also want to be able to work in what I think is an absolute
> must for any coverage tool, which is the ability to declare
> certain code that isn't supposed to be executed so that it is
> not included in the total for calculating percentage. But it is
> absolutely vital that it also report it as an error if it is
> executed. I think it may be possible to implement that in JXCL
> even though it is only working from byte code.

Can you define exactly what you mean here?  Are you talking about
marking certain ranges of source code?  Or would you be content with
excluding code at the method level?

> But then again I have never actually seen JXCL in action. I've
> had to work to even get it to build and run on Windows.

The examples don't work?

--
Jim Dixon  jd...@di...   tel +44 117 982 0786  mobile +44 797 373 7881
"Be liberal in what you accept,                                Jon Postel
and conservative in what you send."                               RFC 793
http://jxcl.sourceforge.net                       Java unit test coverage
http://xlattice.sourceforge.net         p2p communications infrastructure

On Wed, 7 Jan 2004, John Mishanski wrote:

>     I am planning on using jxcl/jxunit during the
> development process on a project that i started this past
> month.  I see this project being very helpful to my work,
> and I applaud all of your hard work.
>
>     I was hoping that you could help me get up and running
> efficiently with the following situation.  I'm using the
> jxunit task as below:
>
> <jxunit dir="." fork="true" haltonerror="false"
> checkCoverage="true" checkIncludes="com.algebrahelp.calc."
> checkExcludes="com.algebrahelp.calc.test"
> includeAntRuntime="yes">
> 			<classpath>
> 				<pathelement location="${build}" />
> 				<pathelement location="${java.class.path}" />
> 			</classpath>
> 			<formatter type="xml"> </formatter>
> 			<sysproperty key="test-data" file="${test-data}" />
> 			<batchtest todir="${jxcl-reports}">
> 				<fileset dir="${build}">
> 					<include name="${unit-test-pattern}" />
> 					<exclude name="**/*$*.class"/>
> 				</fileset>
> 			</batchtest>
> 		</jxunit>
>
> When i use junit i only include the classpath necessary for
> my project, however it seems that jxunit is requiring that i
> include a classpath including
> org/jxcl/jxunit/textui/TestRunner and other classes which
> are not related to the classes I'm testing.  Is it possible
> to avoid including all of these classes by hand for a more
> junit-task-like behavior?  If so, could you point me to the
> appropriate documentation?

From example/3x6/build.xml:

	<fileset dir="../../${libdir}">
          <include name="*.jar"/>
        </fileset>

This is the usual approach: you put all your jars in a lib directory,
and then include all of them as above.  Many people use $ANT_HOME/lib
for this purpose.  I generally use $HOME/lib for personal stuff,
and $HOME/$PROJECT/lib and $HOME/$PROJECT/target/lib for projects like
JXCL.

I would think that setting
	includeAntRuntime="yes"
would do what you want, but it doesn't appear to.  I will look into
this.

> Thanks,
>
> John Mishanski
>
> algebrahelp.com
>

--
Jim Dixon  jd...@di...   tel +44 117 982 0786  mobile +44 797 373 7881
http://jxcl.sourceforge.net                       Java unit test coverage
http://xlattice.sourceforge.net         p2p communications infrastructure

--
Jim Dixon  jd...@di...   tel +44 117 982 0786  mobile +44 797 373 7881