GCCMVS Files

GCC and PDPCLIB on MVS 3.8 and above
------------------------------------

If you want to code in C on MVS (3.8, XA,
OS/390, z/OS or above), there is now another
option available, and it is both free and clear 
of copyright conditions.

There are two separate parts to this, GCC and
PDPCLIB.  They are different products, written
by different people, and with different
licencing restrictions.

GCC itself is subject to GPL and thus you will 
have to read the licencing restrictions (COPYING)
regarding that.  However, any code of yours that
you get GCC to compile, is still yours.  And any 
public domain code that you get GCC to compile,
is still public domain.

PDPCLIB itself is public domain, so you can use
it, modify it, sell it, with no strings attached.
Thus, if you use only public domain code, plus
your own copyright code, you can in combination
produce your own copyright executables, with no
legal restrictions other than those you put on
yourself.  ie you can sell it, use it in a
commercial environment etc etc, with no strings
attached.

You can actually potentially mix and match
these two things.  E.g. GCC could be used
with a different C runtime library other than
PDPCLIB.  And PDPCLIB could be used with a
C compiler other than GCC (indeed, it was
originally written for use with C/370, to
produce stand-alone executables, which
aren't able to be produced natively).  The 
only combination that works "out of the box"
currently though, is GCC combined with PDPCLIB.
This combination is referred to from now on 
as "GCCMVS".

Now there are two ways you can obtain GCCMVS.
One is to get an existing user of GCCMVS to
dump the applicable libraries, ie:

PDPCLIB.INCLUDE
PDPCLIB.NCALIB
PDPCLIB.MACLIB
GCC.PROCLIB
GCC.LINKLIB
GCC.JCL

onto tape, then reload onto your system.  The
above libraries are the minimum you need.  If
you want the source code and documentation as
well, then get:

PDPCLIB.SOURCE
PDPCLIB.DOC
GCC.SOURCE
GCC.INCLUDE
GCC.DOC


If you have the source code and executables,
you can actually use GCCMVS to recompile
itself, so that you know for sure that you
have source code that matches the executables.

It is usual to do this in 3 stages, see below
for details.


The second way of obtaining GCCMVS is purely
from source code.  GCCMVS is simply a patch
of around 167k (zipped) on top of standard GCC 3.2.3.
Standard GCC 3.2.3 can be obtained from
ftp.gnu.org/gnu/gcc/gcc-3.2.3/gcc-3.2.3.tar.bz2
File is dated 2003-04-23 00:00 and is 
20,662,887 bytes in size.

However, you would probably be better off with the smaller:
ftp.gnu.org/gnu/gcc/gcc-3.2.3/gcc-core-3.2.3.tar.bz2
File is dated 2003-04-23 00:00 and is 
10,324,429 bytes in size.
Note that if you use this core file, there are 6
files in the patch that won't get applied, but they
aren't important.


The official place to obtain the patch from is:
http://gccmvs.sourceforge.net

GCCCMS (GCC for CMS) is part of the same product,
and you can read gcccms.txt for further information.


Note that at the moment there is a separate, but
similar, product - GCC for USS on OS/390, z/OS,
maintained by David Pitts, which is available from here:
http://www.cozx.com/~dpitts/gcc.html
A lot of code is shared between the two projects, but
not all.

Also note that if you are using MUSIC/SP, there is a
separately maintained project, maintained by Dave Edwards,
available here:
http://webpages.mcgill.ca/staff/group3/dedwar1/web/gccmu10.zip


If you obtain GCCMVS in source code form, then
you need the following tools:

1. patch
2. a c compiler (by default, gcc for windows with
pdpclib for windows is used (which also requires a
"make" which supports spaces), but you can use c 
compilers other than gcc, and you can use standard
gcc without pdpclib for windows too).
3. Miscellaneous decompression tools such as bzip2
and tar.

So do the following (note that all these commands
and the rest of the process is contained in cygwin.bat
as an EXAMPLE of what you MIGHT want to do if you had
a standard cygwin install and wanted to be up and
running as quickly as possible):

bzip2 -d gcc-3.2.3.tar.bz2
tar xvf gcc-3.2.3.tar
ren gcc-3.2.3 gcc
cd gcc
patch -p 1 -t <patch.txt (whatever the patch file is called)

Note that this file that you are reading right now is
called gccmvs.txt and after you've applied the patch,
you can switch to reading the gccmvs.txt that just got
produced. Just in case the one you are reading now is
out of date.

For your information, some of the files in
the gccmvs patch were originally obtained by doing:
./configure --target=i370-ibm-mvspdp --enable-languages=c
but you should NOT do that, otherwise you will
overwrite various working source files with
non-working ones.

Next you need to obtain pdpclib.  The official
place to get pdpclib from is:

http://pdos.sourceforge.net

However, it is also available from the same
place you got GCCMVS from, ie:
http://gccmvs.sourceforge.net


You should place pdpclib such that the directory
structure looks like this:
gcc/include
gcc/gcc
gcc/libiberty
pdos/pdpclib

ie "pdos" should be at the same level as "gcc".
"pdos" just needs to be an empty directory besides
"pdpclib".  You can extract the pdpclib files
into the "pdpclib" directory.

Then, assuming you are doing this from a DOS
prompt, change to the gcc/gcc directory and go:

gencomp (this step is actually optional, as the
source files it generates are actually already
part of the patch, so you're probably better off
omitting this step, and it is not available on
Unix or Windows non-gcc).

compile

which will create an executable called gccmvs.exe
on Windows, or gccmvs on Unix.

First problem - by default, "compile.bat" is set up
to compile using gcc/pdpclib-win32. You can easily
change this to use some other compiler, but if you
want to use the default, it requires you to actually
build pdpclib-win32 according to the instructions
included in pdpclib.txt in the pdpclib directory.
The advantage of using pdpclib-win32 is that it means
that the C runtime library you use for the PC is the
same/similar to that which is being used on the
mainframe. e.g. printf is missing the same things on
both, so if you use something unimplemented, you will
find out the problem on the PC before you get to the
mainframe.

Then run:

compmvs

(unless you want to use unmodified IFOX on MVS 3.8,
in which case go "compmvs -DIFOX")

which will generate S/370 assembler code.

Next run:

zipall (see note below)

which will create several zip files suitable for
uploading to the mainframe.  Specifically,

PDPCLIB.INCLUDE
PDPCLIB.SOURCE
PDPCLIB.DOC
GCC.PROCLIB
GCC.SOURCE
GCC.INCLUDE
GCC.DOC
GCC.S (for temporary use only)
GCC.JCL

Example JCL to allocate these datasets can
be found in stage0a.jcl.  Note that this will
need to be manually transferred to a
temporary dataset prior to being run.

Note regarding running zipall - for complete
integrity on a 31-bit system, it would be good
if you editted zipall.bat and removed all those
"rem" statements down the bottom. This will
prevent you from uploading the generated source,
and force gencomp to generate them.

A thing to note about all JCL is that you need
to define two prefixes, one for GCC datasets
and one for PDPCLIB datasets.  You probably
want to catalogue those prefixes.  Instructions
for doing so can be found here:
http://www.timpinkawa.net/hercules/gccmvs.html
The prefixes are used in proc statements using 
GCCPREF and PDPPREF.  Thus they all need to be
changed, as well as JOBCARD corrections.  One 
other thing is that some temporary files are 
assumed to have been transferred with a userid
of "HERC02".  You should change any occurrence
of HERC02 to something appropriate for your use.

In fact, the following changes will need to
be made to all JCL, and also to gccmvs.prc,
for operation in a 31-bit environment.  It is
recommended you use an edit macro to do all
this:

c all IFOX00 ASMA90
c all 'DM2CMP,' 'ST2CMP,'
c all 'DM3CMP,' 'ST3CMP,'
c all 'SYSGO' '*SYSGO'
c all AMODGEN MODGEN
c all 9216K 20M

(this last one can be used to change the whole
jobcard instead)

One more thing you will need to change if you
want to do full integrity on a 31-bit system is
to change gencomp.jcl to replace occurrences
of "BISON" and "SED" to whatever you have called
your installation libraries for those two products.

Now is your big chance to screw things up.
You need to transfer these files with the
same ASCII to EBCDIC translation that GCCMVS
expects.  Easiest to stuff up are the following:

| x'4f'
^ x'5f'
[ x'ad'
] x'bd'

If you actually try transferring this file you
are reading now to the mainframe, and do a
search on "screw" and put "hex on", you can
see how successful you are likely to be.  There
is a tool called "mvsunzip" supplied as part of
pdpclib which you can use on the mainframe.

Because of the chicken and egg situation, you'll 
need to manually upload the pdpclib asm files in 
order to create the mvsunzip executable if you want
to make use of this.  The JCL, stage0b.jcl is
based on the assumption that you are using that.
Also note that stage0b.jcl will need to be transferred
to a temporary dataset of your own creation
prior to being run.

Now run the JCL from GCC.JCL(STAGE1A) and
GCC.JCL(STAGE1B).  This will create an 
executable called "STAGE1", which is a working
gccmvs, but compiled via this convoluted process.

Note that a common substitute for this step
is to use an existing C compiler to produce
the stage1 executable.

Now for complete integrity on a 31-bit system,
you should run GCC.JCL(GENCOMP), which will
regenerate all the generated source files.
This assumes that you have bison and sed
available. If you don't have them, you can
compile them using the stage1 executable,
and you can temporarily rename stage1 to gcc.
If you already have old copies of bison, sed
and gcc, there is no need to recompile them
now with stage1. That step happens later.

Next run GCC.JCL(STAGE2A-C).  Making use of
the just-compiled STAGE1, it will compile from
source code the entire gccmvs suite, this
time to create an executable called STAGE2.
Although if you are running on MVS 3.8, due
to memory requirements, many of the modules
are simply reassembled instead of being
recompiled. Sorry! You can probably get more
modules to compile if you trim your system to
create more region available, but you'll never
get them all.

Next run GCC.JCL(STAGE3A-C).  Making use of the
just-compiled STAGE2 (which at this stage is
comprised entirely of mainframe-based gccmvs
code), it will create an executable called
"XGCC". Note that if you would like to transfer
GCC from a 31-bit system to a 24-bit system, you
can modify the stage3 JCL to put the generated
assembler into GCC.S (delete and recreate it
first), rather than having the assembler go into
temporary datasets. You can then transfer the
assembler to the 24-bit system and run the
stage1 jobs to create an executable that is
purely mainframe-generated.

Note that the executables STAGE2 and XGCC
should be identical.  If they're not, there's
a compiler bug.  You should thus compare them.
We don't supply a utility to compare two executables,
so if you don't already have such a tool, you'll
have to just compare file sizes and hope for the
best.

Now rename XGCC to GCC in GCC.LINKLIB. Then
run GCC.JCL(STAGE4). Unless you are using MVS 3.8
in which case run STAGE4Z instead.

Copy GCC.PROCLIB(GCCMVS) into the installation
libraries and modify it for your site's
requirements.

Congratulations, you now have GCCMVS fully
installed.

You can now compile your own C programs by
running the example jcl found in GCC.JCL(EXAMPLE).

But for complete integrity on a 31-bit system,
you should now go and recompile bison and sed,
then rename your GCC.SOURCE and GCC.INCLUDE
to ".OLD" datasets, repopulate them (from the
zip files), then run GENCOMP (to repopulate
the generated source files), and then confirm
that the source and include haven't changed
with your new compiler by comparing with the
old datasets.


Credits:

Everyone who worked on GCC and in particular
  the MVS portions, machine definition etc prior
  to David Pitts getting hold of it.
David Pitts for doing the port to OS/390 USS,
  without which the further work could not have
  been done.
Paul Edwards for doing the port to native (ie
  non-USS) MVS 3.8, MVS/XA, OS/390, Z/OS as
  well as getting GCC to be able to compile 
  itself on the mainframe (previously it was 
  only compilable by IBM's C compiler).
Phil Roberts for testing & debugging the native
  MVS port created by Paul Edwards.
Dave Wade for fixing the last compiler bug that
  was preventing GCC from compiling itself on the
  mainframe, and also for doing the VM/CMS port.


Version:

See version.c for the version number.


Support:

PDPCLIB is separately maintained by Paul Edwards,
and some sort of support may be available there.
Certainly submission of *public domain* (ie NOT
GPL) bug fixes/enhancements are welcome.

For GCC support, try going to H390-MVS@yahoogroups.com
to see if anyone there can help.


Technical notes about executables created in
this manner plus other issues/known bugs:

1. A "hello, world" program is approximately
39k in size.

2. The memory requirements (not including the
space for the executable) for a "hello, world"
program are something like 200k, assuming the default
stack size of 128k is being used in mvsstart.asm (and
I think the rest is dependent on the 3 standard files
block size, but I'd have to investigate how mvssupa.asm
currently works).  When running the program in batch,
the REGION= needs to be set to a minimum of about 213K.
Obviously you should try to be more generous
than this for any real application!

3. PDPCLIB is designed to conform to the ANSI
C89 standard (aka ISO/IEC 9899:1990), and there
are no extensions (like "open", "chmod" etc
found on other operating systems or compilers).
In fact, even in the standard functions you will
find that some of them haven't been implemented
fully.

4. What PDPCLIB on MVS is primarily designed for,
is fast processing of binary, fixed block datasets,
and also for text processing.  Or if you want to 
write a utility to traverse MVS control blocks, you 
can use it for that sort of thing.  When operating
on binary, fixed-length records, the path from
fwrite to the "PUT" macro is very short indeed.

5. The files that PDPCLIB operates on do not
have default DCB information (LRECL etc).  This
has the advantage that the programs will operate
on datasets of any LRECL, but does have the 
downside that you need to specify all the DCB
information in your JCL.

6. The executables that are produced can be run
as both TSO command processor or in batch.  The
parameter parsing will handle either method of
invocation.

7. This problem has now been rectified.

8. PDPCLIB makes no attempt to operate on VSAM
datasets, or anything else outside of bog-standard
sequential files.  It will also ignore any attempt
to open a file in update mode, or any attempt to
do random access. Also, RECFM=U is not being handled
properly, and needs to be written properly, after
figuring out what to actually do (ie compare to other
C compilers).

9. The compiler requires even static functions to
be unique within the first 8 characters within a
source file.  The compiler should be changed to
drop that requirement, by generating internal
labels such as @@F1234.

10. The compiler generates excessive instructions,
reloading the base register at every single label.
The smarts are already in the code generator to
avoid this, it's just that they're not quite smart
enough, so needed to be disabled! Search for do_spec_1
in i370.c to see the problem. An exception has been
made for functions that fit in a single page.

11. If you want to use unmodified IFOX on MVS 3.8, 
a lot of flags have been disabled in order to
not exceed the limit (400) of how many external
variables IFOX can handle.  This restriction can
be lifted on MVS/XA and above, and indeed, can be
gotten around on MVS 3.8 too.  On the other hand,
you are unlikely to miss the disabled flags too.
To get around the restriction on MVS 3.8, you can 
either use an assembler other than IFOX, or get:
http://www.prycroft6.com.au/vs2mods/download/asmxfesd.zip

12. As things currently stand, you can't actually
recompile gccmvs with gccmvs on MVS 3.8 because
some of the source files need more memory. It was
initially thought that this might have been a
problem because of PDPCLIB not providing its own
heap management, but when that was added, it showed
no improvement, so that change was backed out.
You'll just have to live with that restriction.
Fixing the excessive base register reloads in gcc
could get more modules to compile, and you can also
try hard to free up more memory than the default,
but you'll never get all of them. Another thing you
can do is compile gccmvs with the -O0 but using a
-O2 compiler. That allows a lot of modules to compile.
VM/370 has sufficient memory available to compile
everything -O0 if we use a -O2 compiled compiler.
The results aren't definite, but it appears that
the biggest module requires 11 meg for data plus
3 meg for the executable to compile unoptimized,
and an unknown amount (but greater than 13 meg of
data) in order to compile optimized. You will never
get a 14 meg region size in MVS 3.8.

13. This problem has now been rectified.

14. This problem has now been rectified.

15. Note that the source and include files are generally
stored in VB files, except for the bits with assembler in
it, which need to be FB80.

16. This problem has now been rectified.

17. This problem has now been rectified.

18. Compiler bugs bug1.c and bug2.c

19. When importing a new release it is necessary
to allow c-parse.c to be imported. This is set to 
ignore in .cvsignore. We have a local modification
to c-parse.c that allows it to handle both ASCII
and EBCDIC. But there is actually no need to use
this modified c-parse.c, as a version that works
on EBCDIC can be generated using bison on an
EBCDIC host, and that is in fact what the "gencomp"
JCL does.

20. When you need to refresh the generated files,
just run configure and make and it will build the
generated files. You can find a list of the 
generated files in genfiles.txt. Do a diff on the
files before copying them in as some generated
files we want to keep our own version. The command
to run configure is:
./configure --target=i370-ibm-mvspdp --enable-languages=c

21. This problem has now been rectified.

22. Due to a bug in the optimizer (-O2), there is a hack
in both emit-rtl.c and stdlib.c to allow us to use
optimization on everything. This hack should be removed
from both places when the bug (documented in bug16.c)
has been fixed. Just search for "hack".

23. There is a code generation problem which I didn't know
how to fix properly, so I instead hacked the de-tab routines
to change the code on the way out. Very ugly! See "hack to
end all hacks" in i370.c. Also see bug3.c for an example
program that demonstrates the bug.

24. There is a code generation problem with __builtin_alloca()
as demonstrated by bug4.c. It is currently being gotten around
by disabling the builtin, but should be fixed properly.

25. There is a code generation anomaly with __builtin_strlen()
as demonstrated by bug5.c. It is generating worse code than
the non-builtin version. It doesn't seem to generate any
inline code either.

26. Because of the way pdpclib implements i/o, the most
recent record remains buffered until the file is closed.
That means when you get an abend, your last printf will
not be printed, even if it had a '\n' in it. To get
around this problem when debugging an abend, end your
printfs with two '\n'.

27. The assembler code generated by gccmvs when run on the
PC is slightly different (even when the same parameters
are used for code generation) from that when run on the
mainframe. But functionally equivalent. This non-deterministic
nature of the compiler is disconcerting. It seems to not
always allocate registers consistently.

28. bug7.c and bug8.c in code generation.

29. This problem has now been rectified.

30. bug10.c when optimization on. Generating bad assembler.
This bug is being worked around via a hack in i370.c. Search
for "second hack". It needs to be removed.

31. This problem has now been rectified.

31. bug12.c - ansi option still allows variable arrays.

32. builtins have all been switched off in the compiler
by default (ie -fno-builtin), as they are generating 
bad code. Instead, we rely on the header file to 
selectively activate working builtins.

33. nostdinc is on by default so that you have to
specify where you get your include files from.

34. We should look at making "configure" and "make"
work for the mvspdp target, so that people on Unix
don't need to use our custom-made scripts.

35. PDPCLIB should be changed to read data a block
at a time rather than a record at a time. This will
allow us to efficiently process datasets with an
LRECL of 1, which fits the C paradigm more accurately.

36. This problem has now been rectified.

37. The AMODE/RMODE are being set in mvssupa.asm
differently to the way all other assembler files are
having their AMODE/RMODE set. They should be made
consistent, one way or the other.

38. This problem has now been rectified.

39. bug14.c internal compiler error unoptimized. Found when
trying to compile diffutils.

40. bug15.c internal compiler error with optimization on.
Found when trying to compile diffutils.

41. bug16.c internal compiler error with optimization on.
Found when trying to compile emit-rtl (in gcc itself).

42. We should have a utility to compare two executables so
that we can see that the stage2 and stage3 executables are
identical. A utility to do this can be written as follows:
BLDL each to find the length.
If lengths are equal, GETMAIN the space for it twice.
Load the first load module in the first GETMAINed space.
Copy it to the second GETMAINed space with MVCL.
Load the second load module to the first GETMAINed space.
Compare the two GETMAINed spaces with CLCL.

43. Note that due to compiler bugs, if you are having trouble
with some code which is being compiled unoptimized (-O0, the
default), try switching on optimization (-O2) and the bug may
disappear. Conversely, some bugs only occur on -O2, so switching
to -O0 will make them disappear. If you've got a bug that is
failing on both, you'll need to hack the code, or figure out
how to fix the compiler. You can report the bug, so that it
can be documented, but there probably won't be anyone available
with the skills to fix it. We really need someone who is willing
to polish this product off.

44. Because MVS 3.8 has a limitation of 15 aliases in the linker
(and presumably elsewhere), the same module is linked multiple
times, to ensure all aliases can be created. This opens up the
possibility of some object code being out of date if someone
links one of the modules manually without doing the other required
links as per STAGE4Z. The alternative is to add more than 15
aliases via other means, but no-one has explained why there is
a 15 alias limit in the first place, so we don't know the
ramifications of doing that.

45. If you are using a compiler that gives you an error when you
compile vasprintf.c, then add a define in the compile command to
define VALIST_NOT_PTR, and that should get around the problem.
See stdcompw.bat for an example. The gcc code is basically not 100%
C90-compliant as it depends on the implementation of va_list.

46. On 31-bit VM, GCC is unable to compile itself, because of an 
unknown bug which is apparently corrupting the free memory 
chains and causing an abend. It would be good to track this
bug down, as if there is a wild pointer in GCC, it could cause
problems in 24-bit VM and 24/31-bit MVS too, we have no idea.