reduce-algebra-developers — Discussion of development, administration and support for Reduce

[Reduce-algebra-developers] redlog question

[Kostas O. <ko...@re...>]

According to my reading of the Redlog manual, sec. 3.7, the following 
should work, and not ask the question:

Reduce (Free CSL version), 20-Oct-15 ...

1: load redlog;

2: rlset ibalp;

{}

3: II := a -> b;

II := a -> b

4: rlsimpl II;

Declare -> (bimpl) predicate ?  (Y or N)

?

Am I doing something wrong, or is the documentation perhaps out of date?

                                 Kostas

Re: [Reduce-algebra-developers] expansion of gamma function

[Rainer S. <rai...@gm...>]

Hello,

the failure of the taylor package to expand gamma and psi comes from interaction 
with rules for psi and polygamma. When I remove these rules, the expansion 
works.

In particular, these rules need to be removed:

   psi(~z)  =>  infinity
      when repart z = floor repart z and impart z = 0 and z < 1,

   polygamma(~n,~x)  =>  infinity
      when numberp x and impart x = 0 and x = floor x and x < 1,

   psi(~x+~n) => psi(x+n-1) + 1/(x+n-1) when numberp n and n >= 1,

   polygamma(~m,~x+~n) => polygamma(m,x+n-1)+(-1)^m*factorial(m)
        /(x+n-1)^(m+1) when numberp n and fixp m and n >= 1,

The first two I will remove from the system: the symbol infinity isn't special, 
hence introduing it at random points in a computation leads to nonsense.

The third and fourth rule are part of the ruleset psi_rules, so they can easily 
be removed and added later:

Reduce (Free PSL version), 10-Oct-2015 ...

1: clearrules psi_rules;

2: taylor(psi(1+x),x,0,5);

            2                      4
          pi                2    pi    3            4                 6
psi(1) + -----*x - zeta(3)*x  + -----*x  - zeta(5)*x  + (1 term) + O(x )
           6                     90

3: taylor(gamma(1+x),x,0,5);

                        2     2                3            2
                6*psi(1)  + pi    2    2*psi(1)  + psi(1)*pi  - 4*zeta(3)   3
1 + psi(1)*x + -----------------*x  + ------------------------------------*x
                      12                               12

             4            2   2                            4
    20*psi(1)  + 20*psi(1) *pi  - 160*psi(1)*zeta(3) + 3*pi    4
 + ----------------------------------------------------------*x  + (1 term)
                              480

      6
 + O(x )


I'll check how removing and adding back of rules can be done automatically in 
the taylor code.

Note that gamma and friends still need to be treated specially for expansion 
around nonnegative integers. I'm looking at it as well.

   Rainer

Re: [Reduce-algebra-developers] expansion of gamma function

[Andrey G. G. <A.G...@in...>]

On Thu, 15 Oct 2015, Andrey G. Grozin wrote:
> It does not help if f(0) is not integer.
If f(0) is half-integer, it's also easy: first use

gamma(x) = pi^(1/2)*2^(1-2*x)*gamma(2*x)/gamma(x+1/2)

Both gamma functions in the rhs can be expanded by the described 
algorithm.

Andrey

Re: [Reduce-algebra-developers] expansion of gamma function

[Alan B. <ala...@gm...>]

On 15/10/15 04:58, Andrey G. Grozin wrote:
> On Wed, 14 Oct 2015, Rainer Schöpf wrote:
>> However, it means that gamma, psi and friends must be treated specially.
With the current set of rules for differentiating gamma, psi and 
polygamma, tps will be unable to build a suitable recurrence relation to 
successfully expand these functions as power series. With the current 
state of my knowledge of the gamma function I can't say whether it is 
feasible to recast the differentiation rules in a more suitable form for 
tps.

Rainer is probably better placed to judge whether taylor can be adapted 
to handle the expansion of these functions by adding special rules for 
gamma & friends.

> Yes. But other systems (maxima, mathematica, ...) can expand gamma, 
> while reduce cannot :-(
>
Does anyone in the Reduce community know what the algorithms these 
systems use?

tps can find the series for gamma(1+x) about x=0 as described in my 
earlier post (This works if one uses an up-to-date version of Reduce!!):

     s := pssum(n=2, (-1)^n*zeta(n)/n, x, 0, n);
     s1 := ps(s-euler_gamma*x,x,0);
     s2 := ps(exp(s1), x, 0);

Perhaps not ideal, but it works!
>> In addition, the above formula doesn't help for the expandsion of 
>> gamma(f(x))
>> where f(x) is some non-trivial, but well known function, like 
>> gamma(x^2-x^3) .
> For any f(x) such that f(0) is integer it does. First reduce 
> gamma(f(x)) to gamma(f(x)+n) where n is integer and f(0)+n=1, then use 
> the above formula. In particular, for your example x^2-x^3 this works 
> fine.
>
> It does not help if f(0) is not integer.
>
> Andrey
>

Using pscompose will help in some cases:

s3:=ps(sin x, x, 0);
pscompose(s2, s3);

yields the series for gamma(1+sin x), but the series s3 must have an 
order greater than 0 (in order that the power series composition 
algorithm works).  So this doesn't help if f(0) is non-integral.

Alan

Re: [Reduce-algebra-developers] expansion of gamma function

[Andrey G. G. <A.G...@in...>]

On Wed, 14 Oct 2015, Rainer Schöpf wrote:
> I'll see that rules for psi(1) and psi(1/2) are added.
Also for all integer and half-integer arguments, please: they reduce to 
psi(1) and psi(1/2).

Andrey

Re: [Reduce-algebra-developers] expansion of gamma function

[Andrey G. G. <A.G...@in...>]

On Wed, 14 Oct 2015, Rainer Schöpf wrote:
> However, it means that gamma, psi and friends must be treated specially.
Yes. But other systems (maxima, mathematica, ...) can expand gamma, while 
reduce cannot :-(

> In addition, the above formula doesn't help for the expandsion of gamma(f(x))
> where f(x) is some non-trivial, but well known function, like gamma(x^2-x^3) .
For any f(x) such that f(0) is integer it does. First reduce gamma(f(x)) 
to gamma(f(x)+n) where n is integer and f(0)+n=1, then use the above 
formula. In particular, for your example x^2-x^3 this works fine.

It does not help if f(0) is not integer.

Andrey

Re: [Reduce-algebra-developers] expansion of gamma function

[Rainer S. <rai...@gm...>]

On Sat, 10 Oct 2015 at 20:38 +0600, Andrey G. Grozin wrote:

 > Can't
 > 
 > log(gamma(1+x)) = - euler_gamma*x + sum((-1)^n*zeta(n)/n*x^n,n,2,infinity)
 > 
 > be used by taylor and/or tps, to calculate the series for gamma(1+x) as the
 > exponent of the known series?

Yes, you're right. Thanks for pointing it out.

However, it means that gamma, psi and friends must be treated specially. I had 
hoped to avoid that.

In addition, the above formula doesn't help for the expandsion of gamma(f(x)) 
where f(x) is some non-trivial, but well known function, like gamma(x^2-x^3) .

 > By the way, why, after loading specfn and sfgamma, psi(1) is not simpilfied
 > to -euler_gamma?

Purely historical. I'll see that rules for psi(1) and psi(1/2) are added.

  Rainer

Re: [Reduce-algebra-developers] expansion of gamma function

[Rainer S. <rai...@gm...>]

Hi Alan,

 > Redfront 3.2, built 23-Nov-2014 ...
 > Reduce (Free PSL version), 23-Nov-2014 ...

It is a bit old, isn't it:    ^^^^^^^^^^^

 > [...]
 > 
 > However there seems to be a problem with manipulating this series (probably a
 > bug in tps connected with simplification of pssum).
 > 
 > It ought to be possible to do
 > 
 > 6: ws^2;
 > ^C***** Interrupt in assoc*

It works in my version of 10-Oct-2015.

Can you please try?

 Rainer

Re: [Reduce-algebra-developers] expansion of gamma function

[Alan B. <ala...@gm...>]

On 10/10/15 15:38, Andrey G. Grozin wrote:
>
> Can't
>
> log(gamma(1+x)) = - euler_gamma*x + 
> sum((-1)^n*zeta(n)/n*x^n,n,2,infinity)
>
> be used by taylor and/or tps, to calculate the series for gamma(1+x) 
> as the exponent of the known series?
>
> By the way, why, after loading specfn and sfgamma, psi(1) is not 
> simpilfied to -euler_gamma?
>
> Andrey
>
In tps at least this should be possible (RTFM!).

Redfront 3.2, built 23-Nov-2014 ...
Reduce (Free PSL version), 23-Nov-2014 ...

1: load_package specfn, sfgamma, tps;

2: r:=pssum(n=2, (-1)^n*zeta(n)/n,x,0,n);

         2                         4 6
       pi    2    zeta(3)   3    pi    4    zeta(5)   5 pi     6      7
r := -----*x  - ---------*x  + -----*x  - ---------*x  + ------*x  + O(x )
       12            3           360           5 5670

3: ps(r-euler_gamma*x,x,0);
                       2 4                         6
                     pi    2    zeta(3)   3    pi    4 zeta(5)   5    
pi     6
  - euler_gamma*x + -----*x  - ---------*x  + -----*x  - ---------*x  + 
------*x
                     12            3           360 5           5670

       7
  + O(x )

4: psexplim 12;

6

5: ws 3;

                       2 4                         6
                     pi    2    zeta(3)   3    pi    4 zeta(5)   5    
pi     6
  - euler_gamma*x + -----*x  - ---------*x  + -----*x  - ---------*x  + 
------*x
                     12            3           360 5           5670

                       8                           10
     zeta(7)   7     pi     8    zeta(9)   9     pi 10    zeta(11)   11
  - ---------*x  + -------*x  - ---------*x  + --------*x - ----------*x
        7           75600           9 935550            11

            12
      691*pi      12      13ps(exp(ws 3
  + ------------*x   + O(x  )
     7662154500


However there seems to be a problem with manipulating this series 
(probably a bug in tps connected with simplification of pssum).

It ought to be possible to do

6: ws^2;
^C***** Interrupt in assoc*

7: ps(exp(ws 3),x,0);

^C
***** Interrupt in mksq

but these seem to hang forever whereas simpler cases are fine; eg

8: ps(sin x,x,0);

      1   3     1    5     1     7      1      9 1       11      13
x - ---*x  + -----*x  - ------*x  + --------*x  - ----------*x   + O(x  )
      6        120        5040        362880        39916800

9: ws^2;

  2    1   4    2    6     1    8      2     10      2 12      13
x  - ---*x  + ----*x  - -----*x  + -------*x   - --------*x   + O(x  )
       3        45        315        14175         467775

10: ps(exp(ws 8),x,0);

          1   2    1   4    1    5     1    6    1    7 31    8     1     9
1 + x + ---*x  - ---*x  - ----*x  - -----*x  + ----*x  + ------*x  + 
------*x
          2        8        15        240        90 5760        5670

      2951     10     1     11      181      12      13
  - ---------*x   - ------*x   + ----------*x   + O(x  )
     3628800         3150         14515200


I will investigate the cause of the problem over the next week or so.

Alan

[Reduce-algebra-developers] Fun with hash tables

[Arthur N. <ac...@ca...>]

This is a message to record some of the fun I have been having recently, 
in case anybody else ends up interested.

I want to be able to display mathematical formulae nicely. To that end I 
need access to character metrics. In my most recent investigation I am 
expecting you use seven fonts
   Computer Modern Unicode Typewriter Text  (fixed pitch)
   "Odokai" [AR PL New Kai]                 (a range of CJK characters)
   STIX in Regular, Bold, Italic and BoldItalic (a serif face)
   STIX-Math                                (mathematical symbol support)
Between these I am faced with just over 33000 characters. I want tables 
that give character sizes, kern and ligature information and various extra 
stuff relating to the typesetting of mathematics. I would like the tables 
to be compact but fast to access. I am prepared to take the view that I 
will only use these particular fonts - so a user is not entitles to bring 
in another without needing to do substantial revision and rebuilding of 
everything.

By far the biggest table is one one giving character sizes and providing 
indirections into ligature and kern tables. To support fast access I make 
this a hash table. If I hashed each character individually then I would 
have 33000 entries in my table each using space to hold the hash key. For 
the data concerned most characters are arrange in blocks, so I use one 
hash table entry for every run of four characters. This wastes some space 
where the fonts have isolated characters in them, but saves repetition of 
hash keys. I end up with just over 10000 lines active in the hash table, 
so on average each line has three of the four positions in it ii use. It 
happens that using this arrangement also gives me a neat number of bits to 
be used for each hash table entry - 40 bytes which I store as five 64-bit 
words.

When establishing a hash table there are typically messy compromises to 
make. One would like a hash function that is cheap and simple to compute 
to save time there, but something too simple may compromise the efficiency 
of the table. One would like as small a table as possible to save space, 
but that will tend to lead to the need for a large number of probes to 
retrieve information. I use a variant of cuckoo hashing, and the main 
point of this note is to record what I view as the amazing way in which 
this lets me meet all my objectives simultaneously.

I introduce three separate hash functions and for any character I insist 
that it end up at one of the locations that these propose for it. Of 
course in unlucky cases two or even all of the functions will yield the 
same value and then the character concerned does not have as much 
flexibility about where it goes. Accepting that as a fact of life, I go 
further. Characters in the typewriter and the mathematical font whose 
codes are in the range U+0000 to U+007f (ie the basic latin alphabet in 
the two fonts I expect to be most heavily used) are treated so that only a 
single hash value is used. Then in the typewriter font I make it such that 
all characters at codepoints up to U+0600 (this covers a serious range of 
European alphabets) and the main range from the Maths font that cover 
European letters and mathematical sumbols only have two distinct hash 
values. Characters expected to arise less commonly - bold, italic, CJK and 
a whole raft of Unicode oddities - are allowed the full three 
possibilities.

The one, two or three locations in the hash table that each character is 
allowed to reside can be thought of as a bipartite graph, and standard 
algorithms (eg Hopcroft-Karp) can seek a maximum matching. If the one that 
is found manages to allocate a location for every single character one has
a hash table where the most important latin characters can be found in one 
probe, essentially all symbols used in mathematics in two (and hance in an 
average of around 1.5), while every character can be looked up in at worst 
three probes (and so in an average of around 2). An unsuccessful search 
for a character not present in any of the fonts only needs three probes. I 
feel that this allows one to believe that performance is good.

The remaining challenge is to ensure that the hash functions used are 
cheap to compute and the table is acceptably compact. I will start with 
the first of these, assuming that the hash table is of size N. The three
hash functions I use are
    h1(k) = k mod N
    h2(k) = k mod N1 + O1  [for some constants N1 and O1]
    h3(k) = (h1(k) + h2(k)) mod N
and if I arrange that N1+O1<=N I can be confident that h2(k) yields a 
value that will lie within table bounds. Especially given that faily often 
only the first one or two of these will need to be computed, and in the 
light of the fact that modern machines compute remainders reasonably fast 
these seem good and simple to implement and not unduly expensive. Observe 
that there is a lot of flexibility in the choice of the constants N1 and 
O1 as well as in choosing the table size N.

Because the font data is static and not expected to change at all often I 
can afford a potentially expensive search to pick N, N1 and O1 so as to 
find a small table. Existing known results about cuckoo hashing indicate 
that for variants like this with three probes one should expect that with 
good has functions it will usually be possible to have a table occupancy 
of around 91%. Here one can worry that the blocky allocation of codes in 
the fonts and the really naive hash functions might make things worst than 
that.

In its most direct form I start with a small table size N and try running 
the maximum matching process for all plausible values of N1 and O1, 
gradually increasing N until I am first able to create a complete table.
When doing this I judge that a tiny value of the secondary modulus N1 is 
not at all liable to be useful since it would not lead to data being 
spread out, so I scan for N1 from 2N/3 up to N-1. I then try all legal 
values of the offset O1 that will avoid h2(k) ever ending up too large. 
This is quite a painfully large search space. To speed it up my code has 
two tricks, both of which introduce risks. The first is that I create 8 
sub-processes and each investigate part of the search space in parallel. I 
avoid heavy duty synchronisation and an effect is that hypothetically one 
part of the search could run ahead and find a packing in a slightly larger 
size hash table before another had complete analysis of a smaller table 
size. Even though that might happen it would probably at worst lead to 
issue of a hash table just one larger than perfection. My other speed-up 
is to use binary search between a tiny table size that is clearly 
impossible to use and a large one where it is easy to fit everything in. 
This obviously speeds things up very usefully, but the binary search 
process really relies on having a monotonic function to investigate if it 
is to guarantee to find the smallest feasible table. Again I have not 
found this to be a big problem in practise. My code is such that a 
complile time option can be used to disable use of parallel search, and an 
alternative entrypoint into the code performs the original naive search 
and so can be used to check solutions found using the optimisations.

With this strategy - and let me remind you that it allows the most 
important characters to be looked up in one probe and almost all the 
relevant rest in only 2 - I find that to store the 10019 keys for my 
characters I end up with a table size of 10057. This is a 99.62% 
occupancy! My metrics table still ends up consuming around 400 Kbytes, but 
I am amazed by how little waste there is.

I have other hash tables to record information about variant and extended 
characters and to help position accents. So for instance a left 
parenthesis need to be associated with four of five characters 
representing gradually larger versions of itself, and then with individual 
characters that can be places adjacent to build up truly huge versions. 
A typical result here is that I have 141 characters for which "extended 
forms" are available, and the hash table set up much as described above 
ends up being of size 143. Again cuckoo hashing with an exhaustive search 
through a range of trivial hash functions has allowed an almost perfect 
hashing scheme to be created.

On a reasonably fast desktop machine running a complete sequential search 
for a good hash table for the 10000 entries I am working with takes of the 
order of a day. The binary search reduces this to a bit under an hour and 
for greater convenience I then seed in and merely verify the best solution 
- and the code that creates the tables then runs in seconds.

A further refinement could be to use not just a simple maximum matching 
algorithm, but one based on weighted graphs so that matchings that 
correspond to placing kets in their first choics position was 
systematically preferred. At present my expectation is that at the high 
table occupancies I am working with the fact that everything fits is 
essentially a fluke based on the exact particular hash parameters in use, 
and that this would not improve things much, but if one backed off to 
lower hash occupancy regimes it might be useful.

My code for all this is in a file "charmetrics.c" and when compiled with a 
"CREATE" flag it can be run to create "charmetrics.h" which contains the 
tables for use with C or C++ and also "charmetrics.red" so that metrics 
could be used directly from Reduce. All these in the csl/cslbase directort 
of the Reduce source tree at sourceforge.

Now would anybody like to join me in sorting out how to lay out 
mathematical formula for the screen using this information?

                        Arthur

Re: [Reduce-algebra-developers] expansion of gamma function

[Andrey G. G. <A.G...@in...>]

On Sat, 10 Oct 2015, Rainer Schöpf wrote:
> 5: ps(gamma(1+x),x,0);
>
> ***** Recursion too deep in ps!:unknown!-crule
<skipped>

> Something similar happens with the taylor package: It tries to compute the
> taylor series be differentiation, but
>
> df(gamma(x+1),x);
>
> gives
>
> gamma(x + 1)*(psi(x)*x + 1)
> -----------------------------
>              x
>
> which leads to computation of psi(0) and therefore to an error.

Can't

log(gamma(1+x)) = - euler_gamma*x + sum((-1)^n*zeta(n)/n*x^n,n,2,infinity)

be used by taylor and/or tps, to calculate the series for gamma(1+x) as 
the exponent of the known series?

By the way, why, after loading specfn and sfgamma, psi(1) is not 
simpilfied to -euler_gamma?

Andrey

[Reduce-algebra-developers] a patch to libreduce which allows one to run qreduce on both 64- and 32-bit systems

[Andrey G. G. <A.G...@in...>]

The attached patch autodetects if we are on a 64-bit system on on a 32-bit 
one, and tunes RedPy.c accordingly.

Unfortunately, it still only works with python-2.x. To use it with 
python-3.x, one has to modify RedPy.c substantially (or, perhaps, to 
re-write it in cython). I'll make such an attempt later.

Andrey

Re: [Reduce-algebra-developers] expansion of gamma function

[Rainer S. <rai...@gm...>]

On Fri, 9 Oct 2015 at 23:58 +0600, Andrey G. Grozin wrote:

 > grozin@dns ~/reduce-3235 $ bin/rfpsl
 > Redfront 3.2, built 09-Oct-2015 ...
 > Loading image file: 
 > /home/grozin/reduce-3235/pslbuild/x86_64-unknown-gentoo2.2/red/reduce.img
 > 
 > Reduce (Free PSL version),  9-Oct-2015 ...
 > 
 > 1: load_package specfn;
 > 
 > 2: load_package tps;
 > 
 > 3: psexplim 5$
 > 
 > 4: ps(gamma(1+x),x,0);
 > ***** `polygamma_aux' is an undefined function
 > 
 > 5:
 > 
 > What can polygamma_aux be?

polygamma_aux is an internal function in the package sfgamma, used for computing 
the value of the polygamma function for integer argument.

sfgamma it should be loaded automatically, and I will checkin a correction for 
that. If you load this package by hand, you see the effect that Alan described:

4: load_package sfgamma;

5: ps(gamma(1+x),x,0);

***** Recursion too deep in ps!:unknown!-crule 

 > I seem to remember that in elder days this expansion worked (though I 
 > may be wrong).

Something similar happens with the taylor package: It tries to compute the 
taylor series be differentiation, but 

df(gamma(x+1),x);

gives

 gamma(x + 1)*(psi(x)*x + 1)
-----------------------------
              x

which leads to computation of psi(0) and therefore to an error.


  Rainer

Re: [Reduce-algebra-developers] expansion of gamma function

[Alan B. <ala...@gm...>]

I note that the taylor package also fails on this example.

the tps package basically differentiates repeatedly the function to be 
expanded and tries to build a recurrence relation for the terms of the 
series and also tries to work out the value of the lowest term in the 
series.

It looks as if this process generates a call to an undefined function 
namely polygamma_aux probably when it is attempting to evaluate the 
function at x=0.

  Without looking at the code it appears that repeated differentiation 
of gamma will generate an infinite sequence of polygammas with their 
first argument increasing by 1 at each stage and it is hard to see how 
this will process will close.

I will try to look at the problem in the next week or so.

Alan Barnes

On 09/10/2015 18:32, gr...@ge... wrote:
> grozin@dns ~/reduce-3235 $ bin/rfpsl
> Redfront 3.2, built 09-Oct-2015 ...
> Loading image file:
> /home/grozin/reduce-3235/pslbuild/x86_64-unknown-gentoo2.2/red/reduce.img
>
> Reduce (Free PSL version),  9-Oct-2015 ...
>
> 1: load_package specfn;
>
> 2: load_package tps;
>
> 3: psexplim 5$
>
> 4: ps(gamma(1+x),x,0);
> ***** `polygamma_aux' is an undefined function
>
> 5:
>
> What can polygamma_aux be?
> I seem to remember that in elder days this expansion worked (though maybe
> I'm wrong).
>
> Andrey
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Reduce-algebra-developers mailing list
> Red...@li...
> https://lists.sourceforge.net/lists/listinfo/reduce-algebra-developers

[Reduce-algebra-developers] expansion of gamma function

[Andrey G. G. <A.G...@in...>]

grozin@dns ~/reduce-3235 $ bin/rfpsl
Redfront 3.2, built 09-Oct-2015 ...
Loading image file: 
/home/grozin/reduce-3235/pslbuild/x86_64-unknown-gentoo2.2/red/reduce.img

Reduce (Free PSL version),  9-Oct-2015 ...

1: load_package specfn;

2: load_package tps;

3: psexplim 5$

4: ps(gamma(1+x),x,0);
***** `polygamma_aux' is an undefined function

5:

What can polygamma_aux be?
I seem to remember that in elder days this expansion worked (though I 
may be wrong).

Andrey

[Reduce-algebra-developers] expansion of gamma function

[<gr...@ge...>]

grozin@dns ~/reduce-3235 $ bin/rfpsl
Redfront 3.2, built 09-Oct-2015 ...
Loading image file: 
/home/grozin/reduce-3235/pslbuild/x86_64-unknown-gentoo2.2/red/reduce.img

Reduce (Free PSL version),  9-Oct-2015 ...

1: load_package specfn;

2: load_package tps;

3: psexplim 5$

4: ps(gamma(1+x),x,0);
***** `polygamma_aux' is an undefined function

5:

What can polygamma_aux be?
I seem to remember that in elder days this expansion worked (though maybe 
I'm wrong).

Andrey

Re: [Reduce-algebra-developers] reduce on 64-bit gentoo

[Arthur N. <ac...@ca...>]

A memory violation should never happen, and once one has the rest of the 
build can be messed up arbitrarily badly. This is where it becomes truly 
bad that you are experiencing this on an operating system variant that I 
can not reproduce, since that makes it truly hard for me to discover what 
is going on. If you reckon you are a good C-code level debugger of 
significant size programs then at this stage you go
   ./configure --with-csl --enable-debug
and run the Reduce build under gdb passing an option "-x" to CSL/Reduce - 
which option tells it not to try to catch exceptions but to leave them to 
be passed to gdb. That lets you discover just where in the many many 
thousands of lines of code the calamity arises... but it can then be 
because of the delayed effect of corruption introduced earlier.

If we are going to try to dive down to that level I suggest stopping 
sharing the detailed exchanges with the whole group.

Can you make a gentoo virtual machine that matches your world sufficiently 
that it displays the bad behaviour, and find a web-host or file-share site 
that supports big enough files that you can get it to me so I can run it 
in virtualbox and get EXACTLY the experience you are?

          Arthur

On Thu, 8 Oct 2015, Andrey G. Grozin wrote:

> Dear Arthur,
>
> csl-sanity-check.sh does not show anything wrong. But 
> cslbuild/i686-unknown-gentoo2.2/csl/buildlogs/reduce.log shows:
>
> <skipped>
> +++ fl2bf compiled, 43 + 40 bytes
> fl2bfnil6nilnil
> Memory access violation detected
>
> ... continuing after error
>
> +++ Error unset variable: flag
>
> ... continuing after error
>

Re: [Reduce-algebra-developers] reduce on 64-bit gentoo

[Andrey G. G. <A.G...@in...>]

Dear Arthur,

csl-sanity-check.sh does not show anything wrong. But 
cslbuild/i686-unknown-gentoo2.2/csl/buildlogs/reduce.log shows:

<skipped>
+++ fl2bf compiled, 43 + 40 bytes
fl2bfnil6nilnil
Memory access violation detected

... continuing after error

+++ Error unset variable: flag

... continuing after error

+++ Error undefined function (1 arg): !~comma

Evaluating: (!~comma (quote lose))
... continuing after error

+++ Error unset variable: !;

... continuing after error

+++ Error unset variable: inline

... continuing after error

+++ Error unset variable: procedure

... continuing after error

+++ Error unset variable: int2id

... continuing after error

+++ Error unset variable: x

... continuing after error

+++ Error unset variable: !;

<many errormessages skipped>

+++ Error unset variable: end

... continuing after error

+++ Error unset variable: !;

... continuing after error
<end of file here>

On my 64-bit box, reduce.log ends with

Recompilation complete

The system is about to do a restart...
Reduce (Free CSL version), 09-Oct-15 ...
Fast-loading 
"/home/grozin/reduce-3235/cslbuild/x86_64-unknown-gentoo2.2/csl/reduce.img(user)"
Fast-loading 
"/home/grozin/reduce-3235/cslbuild/x86_64-unknown-gentoo2.2/csl/reduce.img(cslcompat)"
Fast-loading 
"/home/grozin/reduce-3235/cslbuild/x86_64-unknown-gentoo2.2/csl/reduce.img(smacros)"

+++ Creating a package: tmprint


nil


"**** **** REDUCE FULLY REBUILD **** ****"

So, yes, during generation of reduce.img on my 32-bit box something went 
wrong. In reduce-src-2014-11-30 on the same 32-bit box, reduce.log also 
ends with

Recompilation complete

The system is about to do a restart...
Reduce (Free CSL version), 07-Oct-15 ...
Fast-loading 
"/home/grozin/reduce-src-2014-11-30/cslbuild/i686-unknown-gentoo2.2/csl/reduce.img(user)"
Fast-loading 
"/home/grozin/reduce-src-2014-11-30/cslbuild/i686-unknown-gentoo2.2/csl/reduce.img(cslcompat)"
Fast-loading 
"/home/grozin/reduce-src-2014-11-30/cslbuild/i686-unknown-gentoo2.2/csl/reduce.img(smacros)"

+++ Creating a package: tmprint


nil


"**** **** REDUCE FULLY REBUILD **** ****"

Some change between 2014-11-30 and svn-3235 produced this failure. I 
suppose I can do bisection, though this would take quite some time. To 
what svn revision does reduce-src-2014-11-30 correspond?

Andrey

Re: [Reduce-algebra-developers] reduce on 64-bit gentoo

[Arthur N. <ac...@ca...>]

Yes, reduce.img should be quite a few megabytes large. If it is in a 
damaged state that tend to mean that a previous attempt to build it 
crashed (so you could look at the log frrom that). But if you just delete 
all of cslbuild, run configure again and go "make" you get a fully freshly 
built version - and if that is still small and broken it usually means you 
did not have some necesary development library or tool installed on the 
machine you were building on. Somewhere in all the output from the build 
(I tend to collect it by going
    script build.log
    make
    exit)
should be messages from which one can deduce what. I also HOPE that 
scripts/csl-sanity-check.sh can HELP get a system up to the necessary 
state.
  Arthur


On Thu, 8 Oct 2015, Andrey G. Grozin wrote:

> On Thu, 8 Oct 2015, Arthur Norman wrote:
> <skip>
>> So reduce.img(InitialImage) refers to the start-up heap image as stored 
>> as a component of the file reduce.img.
> Thank you for the explanation.
>
> I see that on my 32-bit gentoo box lsb-release was already installed (it's 
> a dependency of some other package installed there). And what I get is
>
> grozin@elrond ~/reduce-3235 $ bin/redcsl -w
> +++ Image file 
> "/home/grozin/reduce-3235/scripts/../cslbuild/i686-unknown-gentoo2.2/csl/reduce.img(InitialImage)" 
> can not be read
>
> grozin@elrond ~/reduce-3235 $ file 
> /home/grozin/reduce-3235/scripts/../cslbuild/i686-unknown-gentoo2.2/csl/reduce.img
> /home/grozin/reduce-3235/scripts/../cslbuild/i686-unknown-gentoo2.2/csl/reduce.img: 
> data
>
> So, /home/grozin/reduce-3235/cslbuild/i686-unknown-gentoo2.2/csl/reduce 
> trys to read 
> /home/grozin/reduce-3235/cslbuild/i686-unknown-gentoo2.2/csl/reduce.img(InitialImage) 
> but fails; the file 
> /home/grozin/reduce-3235/cslbuild/i686-unknown-gentoo2.2/csl/reduce.img 
> exists. Maybe, something is wrong with its internal steructure? It is 
> quite small:
>
> -rw-r--r-- 1 grozin grozin 38072 Oct  7 16:07 
> /home/grozin/reduce-3235/cslbuild/i686-unknown-gentoo2.2/csl/reduce.img
>
> On the 64-bit box it is much larger:
>
> -rw-r--r-- 1 grozin grozin  5282160 Oct  9 00:12 reduce.img
>
> Andrey
>

Re: [Reduce-algebra-developers] reduce on 64-bit gentoo

[Andrey G. G. <A.G...@in...>]

On Thu, 8 Oct 2015, Arthur Norman wrote:
<skip>
> So reduce.img(InitialImage) refers to the start-up heap image as stored as a 
> component of the file reduce.img.
Thank you for the explanation.

I see that on my 32-bit gentoo box lsb-release was already installed (it's 
a dependency of some other package installed there). And what I get is

grozin@elrond ~/reduce-3235 $ bin/redcsl -w
+++ Image file 
"/home/grozin/reduce-3235/scripts/../cslbuild/i686-unknown-gentoo2.2/csl/reduce.img(InitialImage)" 
can not be read

grozin@elrond ~/reduce-3235 $ file 
/home/grozin/reduce-3235/scripts/../cslbuild/i686-unknown-gentoo2.2/csl/reduce.img
/home/grozin/reduce-3235/scripts/../cslbuild/i686-unknown-gentoo2.2/csl/reduce.img: 
data

So, /home/grozin/reduce-3235/cslbuild/i686-unknown-gentoo2.2/csl/reduce 
trys to read 
/home/grozin/reduce-3235/cslbuild/i686-unknown-gentoo2.2/csl/reduce.img(InitialImage) 
but fails; the file 
/home/grozin/reduce-3235/cslbuild/i686-unknown-gentoo2.2/csl/reduce.img 
exists. Maybe, something is wrong with its internal steructure? It is 
quite small:

-rw-r--r-- 1 grozin grozin 38072 Oct  7 16:07 
/home/grozin/reduce-3235/cslbuild/i686-unknown-gentoo2.2/csl/reduce.img

On the 64-bit box it is much larger:

-rw-r--r-- 1 grozin grozin  5282160 Oct  9 00:12 reduce.img

Andrey

Re: [Reduce-algebra-developers] reduce on 64-bit gentoo

[Arthur N. <ac...@ca...>]

The file "reduce.img" in the CSL version is a single file that is 
structured as with a mini-file system within it. One sub-part of it is 
known as "InitialImage" and is the main heap image loaded at the start of 
a run. Other units within that one file are the "fasl" (for fast-loadable) 
files representing dynamically loadable units of the Code that makes up 
Reduce. When you go "load_package xxxx" in general it accesses and loads 
several of these - a top level one called (sort of) xxxx.fasl and then 
others that are sub-components of the package concerned.

CSL does it this way so that all those resources are in a single file, and 
you only pay the operating system overhead of going to the system's 
file-directory search once to find "reduce.img", while other systems will 
open and close files for every single fast-loadable file used. It also 
measn that there is just one (big) file there rather than many many 
smaller ones, and my view when I set that up was that if you had many 
files then perhaps a few would get lost or corrupted and you might not 
notice for ages....

So reduce.img(InitialImage) refers to the start-up heap image as stored as 
a component of the file reduce.img.

This has been the situation since around the initial release of CSL.

         Arthur

On Thu, 8 Oct 2015, Andrey G. Grozin wrote:

> After installing lsb-release and recompiling reduce on my 64-bit gentoo 
> box, everything works. So, when packaging reduce for gentoo, I'll have to 
> include lsb-release in the list of build-time dependences.
>
> What remains unsolved is the behavious on my 32-bit gentoo box. From where 
> has "(InitialImage)" appeared after the correct reduce.img file path? This 
> problem does not appear in the 2014-11-30 snapshot; it is a result of some 
> later change.
>
> Many thanks for your help,
> Andrey
>
>

Re: [Reduce-algebra-developers] reduce on 64-bit gentoo

[Andrey G. G. <A.G...@in...>]

After installing lsb-release and recompiling reduce on my 64-bit gentoo 
box, everything works. So, when packaging reduce for gentoo, I'll have to 
include lsb-release in the list of build-time dependences.

What remains unsolved is the behavious on my 32-bit gentoo box. From where 
has "(InitialImage)" appeared after the correct reduce.img file path? This 
problem does not appear in the 2014-11-30 snapshot; it is a result of some 
later change.

Many thanks for your help,
Andrey

Re: [Reduce-algebra-developers] reduce on 64-bit gentoo

[Arthur N. <ac...@ca...>]

Thanks - on my attempt "lsb_release -r" returns "n/a". Of course this is 
partly an effect that Gentoo is not a platform where everybody will match 
- surely part of the whole concept is that each user will have built and 
set up a system JUST for themselves and for choice optimised for exactly 
the CPU model and other hardware that they happen to have.

Anyway, with lsb_release installed how does a Reduce comfigure & build 
work for you. I rather hope it now builds in a directory with "gentoo2.2" 
in its name....

        Arthur

On Thu, 8 Oct 2015, Andrey G. Grozin wrote:

> After installing lsb-release,
>
> grozin@dns ~ $ lsb_release -d
> Description:    Gentoo Base System release 2.2
> grozin@dns ~ $ lsb_release -r
> Release:        2.2
>
> Andrey
>

Re: [Reduce-algebra-developers] reduce on 64-bit gentoo

[Andrey G. G. <A.G...@in...>]

After installing lsb-release,

grozin@dns ~ $ lsb_release -d
Description:    Gentoo Base System release 2.2
grozin@dns ~ $ lsb_release -r
Release:        2.2

Andrey

Re: [Reduce-algebra-developers] reduce on 64-bit gentoo

[Andrey G. G. <A.G...@in...>]

On Wed, 7 Oct 2015, Rainer Schöpf wrote:
> ./config.sub x86_64-pc-linux-gnu
x86_64-pc-linux-gnu

> ./config.sub x86_64-unknown-linux-gnu
x86_64-unknown-linux-gnu

> ./configure --with-psl
configure: Absolute path to source directory = /home/grozin/reduce-3235
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
configure: with_crlibm = yes
in-place build attempt = yes
configure: host=x86_64-pc-linux-gnu args= '--with-psl'
configure: Will build in the x86_64-pc-linux-gnu subdirectory
configure: +++ Will build in 
/home/grozin/reduce-3235/pslbuild/x86_64-pc-linux-gnu
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for cygpath... no
configure: Build platform specified as x86_64-pc-linux-gnu
configure: Will build this PSL using the AMD64_ext initial binaries
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile

Andrey