for the implementation of operations we can reuse the ones
from lib/lib-arm64.c, risc-v long double is also float128.
Also implement ggoto, and PDIV, and use t0 in load/store as
temporary register if necessary, not one given by get_reg
(the latter can destroy assignments of long double parameters
in function calls that are already set up).
This let's us compile tcc.c and tcctest.c, though both
don't yet work.
A more automatic approach to code suppression (aka. nocode_wanted)
The simple rules are:
- Clear 'nocode_wanted' at (im/explicit) label IF it was used
- Set 'nocode_wanted' after unconditional jumps
Also in order to test this then I did add the "function might
return no value" warning, and then to make that work again I
did add the __attribute__((noreturn)).
Also moved the look ahead label check into the type parser
to gain a little speed.
new glibc really can't avoid it anymore, so let's provide it.
I've tried doing it only on systems that possibly are glibc
based. (For others it would be harmless as it simply wouldn't
be picked up from libtcc1.a)
win32/Makefile ("for cygwin") removed
- On cygwin, the normal ./configure && make can be used with either
cygwin's "GCC for Win32 Toolchain"
./configure --cross-prefix=i686-w64-mingw32-
or with an existing tcc:
./configure --cc=<old-tccdir>/tcc.exe
tcctest.c:
- exclude test_high_clobbers() on _WIN64 (does not work)
tests2/95_bitfield.c:
- use 'signed char' for ARM (where default 'char' is unsigned)
tests:
- remove -I "expr" diff option to allow tests with
busybox-diff.
libtcc.c, tcc.c:
- removed -iwithprefix option. It is supposed to be
combined with -iprefix which we don't have either.
tccgen.c:
- fix assignments and return of 'void', as in
void f() {
void *p, *q;
*p = *q:
return *p;
}
This appears to be allowed but should do nothing.
tcc.h, libtcc.c, tccpp.c:
- Revert "Introduce VIP sysinclude paths which are always searched first"
This reverts commit 1d5e386b0a.
The patch was giving tcc's system includes priority over -I which
is not how it should be.
tccelf.c:
- add DT_TEXTREL tag only if text relocations are actually
used (which is likely not the case on x86_64)
- prepare_dynamic_rel(): avoid relocation of unresolved
(weak) symbols
tccrun.c:
- for HAVE_SELINUX, use two mappings to the same (real) file.
(it was so once except the RX mapping wasn't used at all).
tccpe.c:
- fix relocation constant used for x86_64 (by Andrei E. Warentin)
- #ifndef _WIN32 do "chmod 755 ..." to get runnable exes on cygwin.
tccasm.c:
- keep forward asm labels static, otherwise they will endup
in dynsym eventually.
configure, Makefile:
- mingw32: respect ./configure options --bindir --docdir --libdir
- allow overriding tcc when building libtcc1.a and libtcc.def with
make XTCC=<tcc program to use>
- use $(wildcard ...) for install to allow installing just
a cross compiler for example
make cross-arm
make install
- use name <target>-libtcc1.a
build-tcc.bat:
- add options: -clean, -b bindir
tccgen.c:
doubles need to be aligned, on ARM. The section_reserve()
in init_putv does not do that.
-D ONE_SOURCE: is now the default and not longer needed. Also,
tcc.h now sets the default native target. These both make
compiling tcc simple as "gcc tcc.c -o tcc -ldl" again.
arm-asm.c:
enable pseudo asm also for inline asm
tests/tests2/Makefile:
disable bitfield tests except on windows and x86_64
and don't generate-always
tcc.c:
fix a loop with -dt on errors
configure:
print compiler version (as recognized)
tccpp.c:
actually define symbols for tcc -dt
clear static variables (needed for -dt or libtcc usage)
96_nodata_wanted.c:
use __label__ instead of asm
lib/files:
use native symbols (__i386__ etc.) instead of TCC_TARGET_...
- configure
* use aarch64 instead of arm64
- Makefile
* rename the custom include file to "config-extra.mak"
* Also avoid "rm -r /*" if $(tccdir) is empty
- pp/Makefile
* fix .expect generation with gcc
- tcc.h
* cleanup #defines for _MSC_VER
- tccgen.c:
* fix const-propagation for &,|
* fix anonymous named struct (ms-extension) and enable
-fms-extension by default
- i386-gen.c
* clear VT_DEFSIGN
- x86_64-gen.c/win64:
* fix passing structs in registers
* fix alloca (need to keep "func_scratch" below each alloca area on stack)
(This allows to compile a working gnu-make on win64)
- tccpp.c
* alternative approach to 37999a4fbf
This is to avoid some slowdown with ## token pasting.
* get_tok_str() : return <eof> for TOK_EOF
* -funsigned-char: apply to "string" literals as well
- tccpe/tools.c: -impdef: support both 32 and 64 bit dlls anyway
- configure:
- add --config-uClibc,-musl switch and suggest to use
it if uClibc/musl is detected
- make warning options magic clang compatible
- simplify (use $confvars instead of individual options)
- Revert "Remove some unused-parameter lint"
7443db0d5f
rather use -Wno-unused-parameter (or just not -Wextra)
- #ifdef functions that are unused on some targets
- tccgen.c: use PTR_SIZE==8 instead of (X86_64 || ARM64)
- tccpe.c: fix some warnings
- integrate dummy arm-asm better
* tccgen: re-allow long double constants for x87 cross
sizeof (long double) may be 12 or 16 depending on host platform
(i386/x86_64 on unix/windows).
Except that it's 8 if the host is on windows and not gcc
was used to compile tcc.
* win64: fix builtin_va_start after VT_REF removal
See also a8b83ce43a
* tcctest.c: remove outdated limitation for ll-bitfield test
It always worked, there is no reason why it should not work
in future.
* libtcc1.c: exclude long double conversion on ARM
* Makefile: remove CFLAGS from link recipes
* lib/Makefile: use target DEFINES as passed from main Makefile
* lib/armflush.c lib/va_list.c: factor out from libtcc1.c
* arm-gen.c: disable "depreciated" warnings for now
- configure/Makefiles: minor adjustments
- build-tcc.bat: add -static to gcc options
(avoids libgcc_s*.dll dependency with some mingw versions)
- tccpe.c/tcctools.c: eliminate MAX_PATH
(not available for cross compilers)
- tccasm.c: use uint64_t/strtoull in unary()
(unsigned long sometimes is only uint32_t, as always on windows)
- tccgen.c: Revert (f077d16c) "tccgen: gen_cast: cast FLOAT to DOUBLE"
Was a rather experimental, tentative commit, not really necessary
and somewhat ugly too.
- cleanup recent osx support:
- Makefile/libtcc.c: cleanup copy&paste code
- tccpp.c: restore deleted function
supports building cross compilers on the fly without need
for configure --enable-cross
$ make cross # all compilers
$ make cross-TARGET # only TARGET-compiler & its libtcc1.a
with TARGET one from
i386 x86_64 i386-win32 x86_64-win32 arm arm64 arm-wince c67
Type 'make help' for more information
since configure supports only native configuration
a file 'cross-tcc.mak' needs to be created manually.
It is included in the Makefile if present.
# ----------------------------------------------------
# Example config-cross.mak:
#
# windows -> i386-linux cross-compiler
# (it expects the linux files in <prefix>/i386-linux)
ROOT-i386 = {B}/i386-linux
CRT-i386 = $(ROOT-i386)/usr/lib
LIB-i386 = $(ROOT-i386)/lib:$(ROOT-i386)/usr/lib
INC-i386 = {B}/lib/include:$(ROOT-i386)/usr/include
DEF-i386 += -D__linux__
# ----------------------------------------------------
Also:
- use libtcc1-<target>.a instead of directories
- add dummy arm assembler
- remove include dependencies from armeabi.c/lib-arm64.c
- tccelf/ld_add_file: add SYSROOT (when defined) to absolute
filenames coming from ld-scripts
Forgot about it. It allows to compile several
sources (and other .o's) to one single .o file;
tcc -r -o all.o f1.c f2.c f3.S o4.o ...
Also:
- option -fold-struct-init-code removed, no effect anymore
- (tcc_)set_environment() moved to tcc.c
- win32/lib/(win)crt1 minor fix & add dependency
- debug line output for asm (tcc -c -g xxx.S) enabled
- configure/Makefiles: x86-64 -> x86_64 changes
- README: cleanup
usage:
tcc -ar [rcsv] lib files...
tcc -impdef lib.dll [-v] [-o lib.def]
also:
- support more files with -c: tcc -c f1.c f2.c ...
- fix a bug which caused tcc f1.c f2.S to produce no asm
- allow tcc -ar @listfile too
- change prototype: _void_ tcc_set_options(...)
- apply -Wl,-whole-archive when a librariy is given
as libxxx.a also (not just for -lxxx)
- lib/Makefile: add (win)crt1_w.o
- crt1.c/_runtmain: return to tcc & only use for UNICODE
(because it might be not 100% reliable with for example
wildcards (tcc *.c -run ...)
- tccrun.c/tccpe.c: load -run startup_code only if called
from tcc_run(). Otherwise main may not be defined. See
libtcc_test.c
- tests2/Makefile: pass extra options in FLAGS to allow
overriding TCC
Also:
- tccpe.c: support weak attribute. (I first tried to solve
the problem above by using it but then didn't)
- tccgen.c/tcc.h: allow function declaration after use:
int f() { return g(); }
int g() { return 1; }
may be a warning but not an error
see also 76cb1144ef
- tccgen.c: redundant code related to inline functions removed
(functions used anywhere have sym->c set automatically)
- tccgen.c: make 32bit llop non-equal test portable
(probably not on C67)
- dynarray_add: change prototype to possibly avoid aliasing
problems or at least warnings
- lib/alloca*.S: ".section .note.GNU-stack,"",%progbits" removed
(has no effect)
- tccpe: set SizeOfCode field (for correct upx decompression)
- libtcc.c: fixed alternative -run invocation
tcc "-run -lxxx ..." file.c
(meant to load the library after file).
Also supported now:
tcc files ... options ... -run @ arguments ...
There seems nothing wrong. With
int t1 = 176401255;
float f = 0.25;
int t2 = t1 * f; // 176401255 * 0.25 = 44100313.75
according to the arithmetic conversion rules, the number
176401255 needs to be converted to float, and the compiler
can choose either the nearest higher or nearest lower
representable number "in an implementation-defined manner".
Which may be 176401248 or 176401264. So as result both
44100312 and 44100313 are correct.
This reverts commit 664c19ad5e.
tests/Makefile: fix out-of-tree build issues
Also:
- win64: align(16) MEM_DEBUG user memory
on win64 the struct jmp_buf in the TCCState structure which we
allocate by tcc_malloc needs alignment 16 because the msvcrt
setjmp uses MMX instructions.
- libtcc_test.c: win32/64 need __attribute__((dllimport)) for
extern data objects
- tcctest.c: exclude stuff that gcc does not compile
except for relocation_test() the other issues are mostly ASM
related. We should probably check GCC versions but I have
no idea which mingw/gcc versions support what and which don't.
- lib/Makefile: use tcc to compile libtcc1.a (except on arm
which needs arm-asm
libtcc1 is the compiler support library and therefore needs
to function in a freestanding environment. In particular
it can't just use fprintf or stderr, which it was on x86-64
(but only when compiled by GCC). The tight integration between
libtcc1 and tcc itself makes it impossible to ever reach that
case so the abort() there is enough. abort() is strictly speaking
also not available in a freestanding environment, but it often is
nevertheless.
Support ./configure && make under msys2 (a new msys fork)
on win32 and win64.
Get rid of CONFIG_WIN64 make-var. (On windows, WIN32 in
general is used for both 32 and 64 bit platforms)
Also:
- cleanup win32/build-tcc.bat
- adjust win32/(doc/)tcc-win32.tx
On 2016-08-11 09:24 +0100, Balazs Kezes wrote:
> I think it's just that that copy_params() never restores the spilled
> registers. Maybe it needs some extra code at the end to see if any
> parameters have been spilled to stack and then restore them?
I've spent some time on this and I've found an alternative solution.
Although I'm not entirely sure about it but I've attached a patch
nevertheless.
And while poking at that I've found another problem affecting the
unsigned long long division on arm and I've attached a patch for that
too.
More details in the patches themselves. Please review and consider them
for merging! Thank you!
--
Balazs
[PATCH 1/2] Fix slow unsigned long long division on ARM
The macro AEABI_UXDIVMOD expands to this bit:
#define AEABI_UXDIVMOD(name,type, rettype, typemacro) \
...
while (num >= den) { \
...
while ((q << 1) * den <= num && q * den <= typemacro ## _MAX / 2) \
q <<= 1; \
...
With the current ULONG_MAX version the inner loop goes only until 4
billion so the outer loop will progress very slowly if num is large.
With ULLONG_MAX the inner loop works as expected. The current version is
probably a result of a typo.
The following bash snippet demonstrates the bug:
$ uname -a
Linux eper 4.4.16-2-ARCH #1 Wed Aug 10 20:03:13 MDT 2016 armv6l GNU/Linux
$ cat div.c
int printf(const char *, ...);
int main(void) {
unsigned long long num, denom;
num = 12345678901234567ULL;
denom = 7;
printf("%lld\n", num / denom);
return 0;
}
$ time tcc -run div.c
1763668414462081
real 0m16.291s
user 0m15.860s
sys 0m0.020s
[PATCH 2/2] Fix long long dereference during argument passing on ARMv6
For some reason the code spills the register to the stack. copy_params
in arm-gen.c doesn't expect this so bad code is generated. It's not
entirely clear why the saving part is necessary. It was added in commit
59c35638 with the comment "fixed long long code gen bug" with no further
clarification. Given that tcctest.c passes without this, maybe it's no
longer needed? Let's remove it.
Also add a new testcase just for this. After I've managed to make the
tests compile on a raspberry pi, I get the following diff without this
patch:
--- test.ref 2016-08-22 22:12:43.380000000 +0100
+++ test.out3 2016-08-22 22:12:49.990000000 +0100
@@ -499,7 +499,7 @@
2
1 0 1 0
4886718345
-shift: 9 9 9312
+shift: 291 291 291
shiftc: 36 36 2328
shiftc: 0 0 9998683865088
manyarg_test:
More discussion on this thread:
https://lists.nongnu.org/archive/html/tinycc-devel/2016-08/msg00004.html
Except
- that libtcc1.a is now installed in subdirs i386/ etc.
- the support for arm and arm64
- some of the "Darwin" fixes
- tests are mosly unchanged
Also
- removed the "legacy links for cross compilers" (was total mess)
- removed "out-of-tree" build support (was broken anyway)
- from win32/include/winapi: various .h
The winapi header set cannot be complete no matter what. So
lets have just the minimal set necessary to compile the examples.
- remove CMake support (hard to keep up to date)
- some other files
Also, drop useless changes in win32/lib/(win)crt1.c
mkdir build; cd build
../configure && make
../../lib/libtcc1.c:31: error: include file 'stddef.h' not found
Author: Avi Halachmi
Date: Sat Nov 14 18:40:36 2015 +0200
When building from the root tcc dir, $TOP and $top_srcdir
are the same, but with a custom build dir, we need top_srcdir
* Documentation is now in "docs".
* Source code is now in "src".
* Misc. fixes here and there so that everything still works.
I think I got everything in this commit, but I only tested this
on Linux (Make) and Windows (CMake), so I might've messed
something up on other platforms...
* define targetos=Windows when --enable-tcc32-mingw, --enable-cygwin, ...
* use TARGETOS insteed HOST_OS when selecting PROGS
* use "$(tccdir)" insteed $(tccdir) on install (spaces in path)
* install tcc.exe too
* produce bcheck.o when cross-compiling too (lib/Makefile)
* force bcheck.o linking by compiling inside tcc_set_output_type()
a dummy program with local array. Otherwise bcheck.o may be not linked.
* replace %xz format specifier with %p in bcheck (don't supported on
Windows)
* call a __bound_init when __bound_ptr_add, __bound_ptr_indir,
__bound_new_region, __bound_delete_region called.
This is because a __bound_init inside ".init" section is not called
on Windows for unknown reason.
* print on stderr a message when an illegal pointer is returned:
there is no segmentation violation on Windows for a program
compiled with "tcc -b"
* remove "C:" subdir on clean if $HOST_OS = "Linux"
* default CFLAGS="-Wall -g -O0" insteed CFLAGS="-Wall -g -O2"
to speed up compilation and more precise debugging.
- care about __attribute__ redefinition in the system headers
- an invalid pointer must be returned when (addr >= e->size),
and not (addr > e->size)
A test program:
#include <stdio.h>
#include <stdlib.h>
int main ()
{
int v[10];
fprintf(stderr, "&v[0] = %p\n", &v[0]);
fprintf(stderr, "&v[10] = %p\n", &v[10]);
exit(1);
return 0;
}
// tcc -b test.c
The output before a patch:
&v[0] = 0xbf929d8c
&v[10] = 0xbf929db4
The output after a patch:
&v[0] = 0xbff6e33c
&v[10] = 0xfffffffe
On Linux 32: sizeof(long)=32 == sizeof(void *)=32
on Linux 64: sizeof(long)=64 == sizeof(void *)=64
on Windows 64: sizeof(long)=32 != sizeof(void *)=64
Some structs are passed in registers. When they need more than
one the implementation of __va_arg on x86-64 didn't correctly account
for this. This fixes only the cases where the structs consist of
integer types, as there the register save area is consecutive.
Fixes some tests from 73_arm64.c, but still leaves those failing
that use floating point in the large-but-regpassed structs.
__clear_cache is defined in lib-arm64.c with a single call to
__arm64_clear_cache, which is the real built-in function and is
turned into inline assembler by gen_clear_cache in arm64-gen.c
A test program:
/* result of the new version inroduced in 4ad186c5ef: t2a = 44100312 */
#include<stdio.h>
int main() {
int t1 = 176401255;
float f = 0.25f;
int t2a = (int)(t1 * f); // must be 44100313
int t2b = (int)(t1 * (float)0.25f);
printf("t2a=%d t2b=%d \n",t2a,t2b);
return 0;
}
produce a
i386-win-mingw32-tcc
i386-win-tcc
x86_64-win-mingw32-tcc
x86_64-win-tcc
arm-win-mingw32ce-tcc
arm-win-tcc
instead of the
i386-w64-mingw32-tcc
i386-win32-tcc
x86_64-w64-mingw32-tcc
x86_64-win32-tcc
arm-wince-mingw32ce-tcc
arm-win32-tcc
Replacing a *-win32 directory names with a *-win names
because this names are based on the names of the tcc
x86_64-win32-tcc, i386-win32-tcc
Prevent libtcc1.a to be compiled with -fstack-protector-strong, so that
linking with tcc doesn't fail because symbol '__stack_chk_fail_local' is
not present in libtcc1.a. This is useful only if the CFLAGS passed from
the main Makefile contain this flag.
Not able to generate ELF files on NetBSD yet (lacks the note and crt1.o
is actually named crt0.o on NetBSD), but -run works with these extra
defines:
-D__lint__ -D"__symbolrename(x)=asm(#x)" -D__NetBSD__
The -D__lint__ is an ugly hack, TCC should be able to emulate GCC just
fine, but it seems TCC doesn't support __builtin_va_list yet?
typedef __builtin_va_list __va_list;
/usr/include/sys/ansi.h:72: error: ';' expected (got "__va_list")
This provides a simple implementation of alloca for ARM (and enables
the associated testcase). As tcc for ARM doesn't contain an assembler,
we'll have to resort using gcc for compiling it.
For program manipulating argv or arge as pointer with construct such as:
(while *argv++) {
do_something_with_argv;
}
it is necessary to have argv and arge inside a region. This patch create
regions argv and arge) if main is declared with those parameters.
Prior to this commit runtime library was compiled according to the host
because of the macro used to detec what architecture to choose. This
commit fixes this by using the TARGET_* macro instead.
- tccgen: error out for cast to void, as in
void foo(void) { return 1; }
This avoids an assertion failure in x86_64-gen.c, also.
also fix tests2/03_struct.c accordingly
- Error: "memory full" - be more specific
- Makefiles: remove circular dependencies, lookup tcctest.c from VPATH
- tcc.h: cleanup lib, include, crt and libgcc search paths"
avoid duplication or trailing slashes with no CONFIG_MULTIARCHDIR
(as from 9382d6f1a0)
- tcc.h: remove ";{B}" from PE search path
in ce5e12c2f9 James Lyon wrote:
"... I'm not sure this is the right way to fix this problem."
And the answer is: No, please. (copying libtcc1.a for tests instead)
- win32/build_tcc.bat: do not move away a versioned file
Variants __fixsfdi/__fixxfdi are not needed for now because
the value is converted to double always.
Also:
- remove __tcc_fpinit for unix as it seems redundant by the
__setfpucw call in the startup code
- avoid reference to s->runtime_main in cross compilers
- configure: fix --with-libgcc help
- tcctok.h: cleanup
- fix computation of absolute value (clearing the sign bit does not
since integers are encoded in 2's complement)
- test sign of integer in a more conventional way (binary and with the
high bit does not work for long long due to a bug in gtst)
- spacing in include
- Use runtime function for conversion
- Also initialize fp with tcc -run on windows
This fixes a bug where
double x = 1.0;
double y = 1.0000000000000001;
double z = x < y ? 0 : sqrt (x*x - y*y);
caused a bad sqrt because rounding precision for the x < y comparison
was different to the one used within the sqrt function.
This also fixes a bug where
printf("%d, %d", (int)pow(10, 2), (int)pow(10, 2));
would print
100, 99
Unrelated:
win32: document relative include & lib lookup
win32: normalize_slashes: do not mirror silly gcc behavior
This reverts part of commit 8a81f9e103
winapi: add missing WINAPI decl. for some functions
long double arguments require 16-byte alignment on the stack, which
requires adjustment when the the stack offset is not an evven number of
8-byte words.
I removed the XMM6/7 registers from the register list because they are not used
on Win64 however they are necessary for parameter passing on x86-64. I have now
restored them but not marked them with RC_FLOAT so they will not be used except
for parameter passing.
Modified tcctest.c so that it uses 'double' in place of 'long double'
with MinGW since this is what TCC does, and what Visual C++ does. Added
an option -norunsrc to tcc to allow argv[0] to be set independently of
the compiled source when using tcc -run, which allows tests that rely on
the value of argv[0] to work in out-of-tree builds.
Also added Makefile rules to automatically update out-of-tree build
Makefiles when in-tree Makefiles have changed.
On some architectures, ARM for instance, the data and instruction caches
are not coherent with each other. This is a problem for the -run feature
since instructions are written in memory, and are thus written in the
data cache first and then later flushed to the main memory. If the
instructions are executed before they are pushed out of the cache, then
the processor will fetch the old content from the memory and not the
newly generated code. The solution is to flush from the data cache all
the data in the memory region containing the instructions and to
invalidate the same region in the instruction cache.
- except for CONFIG_SYSROOT and CONFIG_TCCDIR
Strictly neccessary it is only for CONFIG_MULTIARCHDIR
because otherwise if it's in config.h it is impossible to
leave it undefined.
But it is also nicer not to use these definitions for
cross-compilers.
- Also:
lib/Makefile : include ../Makefile for CFLAGS
lib/libtcc1.c : fix an issue compiling tcc with tcc on x64
tests/Makefile:
- print-search-dirs when 'hello' fails
- split off hello-run
win32/include/_mingw.h:
- fix for compatibility with mingw headers
(While our headers in win32 are from mingw-64 and don't have
the problem)
tiny_libmaker:
- don't use "dangerous" mktemp
Should fix some warnings wrt. access out of array bounds.
tccelf.c: fix "static function unused" warning
x86_64-gen.c: fix "ctype.ref uninitialzed" warning and cleanup
tcc-win32.txt: remove obsolete limitation notes.
Also:
- fix "make tcc_p" (profiling version)
- remove old gcc flags:
-mpreferred-stack-boundary=2 -march=i386 -falign-functions=0
- remove test "hello" for Darwin (cannot compile to file)
* configure (fn_dirname): New.
Use it to ensure the creation of proper symlinks to Makefiles.
(config.mak): Define top_builddir and top_srcdir.
(CPPFLAGS): Be sure to find the headers.
* Makefile, lib/Makefile, tests/Makefile, tests2/Makefile: Adjust
to set VPATH properly.
Fix confusion between top_builddir and top_srcdir.
We were calling get_page() with t2 index which is not correct, since
get_page() operate on t1 indices. The bug is here from day-1, from
60f781c4 (first version of bounds checker) and show as a crash in
__bound_delete_region() at program exit:
$ ./tcc -B. -DTCC_TARGET_I386 -DCONFIG_MULTIARCHDIR=\"i386-linux-gnu\" -b -run -DONE_SOURCE \
./tcc.c -B. -DTCC_TARGET_I386 -DCONFIG_MULTIARCHDIR=\"i386-linux-gnu\" -run -DONE_SOURCE \
./tcc.c -B. -run tests/tcctest.c
(lot's of correct output from tcctest)
Runtime error: dereferencing invalid pointer
at 0xa7c21cc4 __bound_delete_region()
by (nil) ???
Segmentation fault
The fix is simple - last page should be get through t1_end, like it is
done in __bound_new_region().
After this patch, tcc is being able to compile itself with -b, then
compile itself again and run tcctest with correct output. Tests follow.
At startup __bound_init() wants to mark malloc zone as invalid memory,
so that any access to memory on heap, not allocated through malloc be
invalid. Other pages are initialized as empty regions, access to which
is not treated as invalid by bounds-checking.
The problem is code incorrectly assumed that heap goes right after bss,
and that is not correct for two cases:
1) if we are running from `tcc -b -run`, program text data and bss
will be already in malloced memory, possibly in mmaped region
insead of heap, and marking memory as invalid from _end
will not cover heap and probably wrongly mark correct regions.
2) if address space randomization is turned on, again heap does not
start from _end, and we'll mark as invalid something else instead
of malloc area.
For example with the following diagnostic patch ...
diff --git a/tcc.c b/tcc.c
index 5dd5725..31c46e8 100644
--- a/tcc.c
+++ b/tcc.c
@@ -479,6 +479,8 @@ static int parse_args(TCCState *s, int argc, char **argv)
return optind;
}
+extern int _etext, _edata, _end;
+
int main(int argc, char **argv)
{
int i;
@@ -487,6 +489,18 @@ int main(int argc, char **argv)
int64_t start_time = 0;
const char *default_file = NULL;
+ void *brk;
+
+ brk = sbrk(0);
+
+ fprintf(stderr, "\n>>> TCC\n\n");
+ fprintf(stderr, "etext:\t%10p\n", &_etext);
+ fprintf(stderr, "edata:\t%10p\n", &_edata);
+ fprintf(stderr, "end:\t%10p\n", &_end);
+ fprintf(stderr, "brk:\t%10p\n", brk);
+ fprintf(stderr, "stack:\t%10p\n", &brk);
+
+ fprintf(stderr, "&errno: %p\n", &errno);
s = tcc_new();
output_type = TCC_OUTPUT_EXE;
diff --git a/tccrun.c b/tccrun.c
index 531f46a..25ed30a 100644
--- a/tccrun.c
+++ b/tccrun.c
@@ -91,6 +91,8 @@ LIBTCCAPI int tcc_run(TCCState *s1, int argc, char **argv)
int (*prog_main)(int, char **);
int ret;
+ fprintf(stderr, "\n\ntcc_run() ...\n\n");
+
if (tcc_relocate(s1, TCC_RELOCATE_AUTO) < 0)
return -1;
diff --git a/lib/bcheck.c b/lib/bcheck.c
index ea5b233..8b26a5f 100644
--- a/lib/bcheck.c
+++ b/lib/bcheck.c
@@ -296,6 +326,8 @@ static void mark_invalid(unsigned long addr, unsigned long size)
start = addr;
end = addr + size;
+ fprintf(stderr, "mark_invalid %10p - %10p\n", (void *)addr, (void *)end);
+
t2_start = (start + BOUND_T3_SIZE - 1) >> BOUND_T3_BITS;
if (end != 0)
t2_end = end >> BOUND_T3_BITS;
... Look how memory is laid out for `tcc -b -run ...`:
$ ./tcc -B. -b -DTCC_TARGET_I386 -DCONFIG_MULTIARCHDIR=\"i386-linux-gnu\" -run \
-DONE_SOURCE ./tcc.c -B. -c x.c
>>> TCC
etext: 0x8065477
edata: 0x8070220
end: 0x807a95c
brk: 0x807b000
stack: 0xaffff0f0
&errno: 0xa7e25688
tcc_run() ...
mark_invalid 0xfff80000 - (nil)
mark_invalid 0xa7c31d98 - 0xafc31d98
>>> TCC
etext: 0xa7c22767
edata: 0xa7c2759c
end: 0xa7c31d98
brk: 0x8211000
stack: 0xafffeff0
&errno: 0xa7e25688
Runtime error: dereferencing invalid pointer
./tccpp.c:1953: at 0xa7beebdf parse_number() (included from ./libtcc.c, ./tcc.c)
./tccpp.c:3003: by 0xa7bf0708 next() (included from ./libtcc.c, ./tcc.c)
./tccgen.c:4465: by 0xa7bfe348 block() (included from ./libtcc.c, ./tcc.c)
./tccgen.c:4440: by 0xa7bfe212 block() (included from ./libtcc.c, ./tcc.c)
./tccgen.c:5529: by 0xa7c01929 gen_function() (included from ./libtcc.c, ./tcc.c)
./tccgen.c:5767: by 0xa7c02602 decl0() (included from ./libtcc.c, ./tcc.c)
The second mark_invalid goes right after in-memory-compiled program's
_end, and oops, that's not where malloc zone is (starts from brk), and oops
again, mark_invalid covers e.g. errno. Then compiled tcc is crasshing by
bcheck on errno access:
1776 static void parse_number(const char *p)
1777 {
1778 int b, t, shift, frac_bits, s, exp_val, ch;
...
1951 *q = '\0';
1952 t = toup(ch);
1953 errno = 0;
The solution here is to use sbrk(0) as approximation for the program
break start instead of &_end:
- if we are a separately compiled program, __bound_init() runs early,
and sbrk(0) should be equal or very near to start_brk (in case other
constructors malloc something), or
- if we are running from under `tcc -b -run`, sbrk(0) will return
start of heap portion which is under this program control, and not
mark as invalid earlier allocated memory.
With this patch `tcc -b -run tcc.c ...` succeeds compiling above
small-test program (diagnostic patch is still applied too):
$ ./tcc -B. -b -DTCC_TARGET_I386 -DCONFIG_MULTIARCHDIR=\"i386-linux-gnu\" -run \
-DONE_SOURCE ./tcc.c -B. -c x.c
>>> TCC
etext: 0x8065477
edata: 0x8070220
end: 0x807a95c
brk: 0x807b000
stack: 0xaffff0f0
&errno: 0xa7e25688
tcc_run() ...
mark_invalid 0xfff80000 - (nil)
mark_invalid 0x8211000 - 0x10211000
>>> TCC
etext: 0xa7c22777
edata: 0xa7c275ac
end: 0xa7c31da8
brk: 0x8211000
stack: 0xafffeff0
&errno: 0xa7e25688
(completes ok)
but running `tcc -b -run tcc.c -run tests/tcctest.c` sigsegv's - that's
the plot for the next patch.
On i386 and gcc-4.7 I found that __bound_local_new was miscompiled -
look:
#ifdef __i386__
/* return the frame pointer of the caller */
#define GET_CALLER_FP(fp)\
{\
unsigned long *fp1;\
__asm__ __volatile__ ("movl %%ebp,%0" :"=g" (fp1));\
fp = fp1[0];\
}
#endif
/* called when entering a function to add all the local regions */
void FASTCALL __bound_local_new(void *p1)
{
unsigned long addr, size, fp, *p = p1;
GET_CALLER_FP(fp);
for(;;) {
addr = p[0];
if (addr == 0)
break;
addr += fp;
size = p[1];
p += 2;
__bound_new_region((void *)addr, size);
}
}
__bound_local_new:
.LFB40:
.cfi_startproc
pushl %esi
.cfi_def_cfa_offset 8
.cfi_offset 6, -8
pushl %ebx
.cfi_def_cfa_offset 12
.cfi_offset 3, -12
subl $8, %esp // NOTE prologue does not touch %ebp
.cfi_def_cfa_offset 20
#APP
# 235 "lib/bcheck.c" 1
movl %ebp,%edx // %ebp -> fp1
# 0 "" 2
#NO_APP
movl (%edx), %esi // fp1[0] -> fp
movl (%eax), %edx
movl %eax, %ebx
testl %edx, %edx
je .L167
.p2align 2,,3
.L173:
movl 4(%ebx), %eax
addl $8, %ebx
movl %eax, 4(%esp)
addl %esi, %edx
movl %edx, (%esp)
call __bound_new_region
movl (%ebx), %edx
testl %edx, %edx
jne .L173
.L167:
addl $8, %esp
.cfi_def_cfa_offset 12
popl %ebx
.cfi_restore 3
.cfi_def_cfa_offset 8
popl %esi
.cfi_restore 6
.cfi_def_cfa_offset 4
ret
here GET_CALLER_FP() assumed that its using function setups it's stack
frame, i.e. first save, then set %ebp to stack frame start, and then it
has to do perform two lookups: 1) to get current stack frame through
%ebp, and 2) get caller stack frame through (%ebp).
And here is the problem: gcc decided not to setup %ebp for
__bound_local_new and in such case GET_CALLER_FP actually becomes
GET_CALLER_CALLER_FP and oops, wrong regions are registered in bcheck
tables...
The solution is to stop using hand written assembly and rely on gcc's
__builtin_frame_address(1) to get callers frame stack(*). I think for the
builtin gcc should generate correct code, independent of whether it
decides or not to omit frame pointer in using function - it knows it.
(*) judging by gcc history, __builtin_frame_address was there almost
from the beginning - at least it is present in 1992 as seen from the
following commit:
http://gcc.gnu.org/git/?p=gcc.git;a=commit;h=be07f7bdbac76d87d3006c89855491504d5d6202
so we can rely on it being supported by all versions of gcc.
In my environment the assembly of __bound_local_new changes as follows:
diff --git a/bcheck0.s b/bcheck1.s
index 4c02a5f..ef68918 100644
--- a/bcheck0.s
+++ b/bcheck1.s
@@ -1409,20 +1409,17 @@ __bound_init:
__bound_local_new:
.LFB40:
.cfi_startproc
- pushl %esi
+ pushl %ebp // NOTE prologue saves %ebp ...
.cfi_def_cfa_offset 8
- .cfi_offset 6, -8
+ .cfi_offset 5, -8
+ movl %esp, %ebp // ... and reset it to local stack frame
+ .cfi_def_cfa_register 5
+ pushl %esi
pushl %ebx
- .cfi_def_cfa_offset 12
- .cfi_offset 3, -12
subl $8, %esp
- .cfi_def_cfa_offset 20
-#APP
-# 235 "lib/bcheck.c" 1
- movl %ebp,%edx
-# 0 "" 2
-#NO_APP
- movl (%edx), %esi
+ .cfi_offset 6, -12
+ .cfi_offset 3, -16
+ movl 0(%ebp), %esi // stkframe -> stkframe.parent -> fp
movl (%eax), %edx
movl %eax, %ebx
testl %edx, %edx
@@ -1440,13 +1437,13 @@ __bound_local_new:
jne .L173
.L167:
addl $8, %esp
- .cfi_def_cfa_offset 12
popl %ebx
.cfi_restore 3
- .cfi_def_cfa_offset 8
popl %esi
.cfi_restore 6
- .cfi_def_cfa_offset 4
+ popl %ebp
+ .cfi_restore 5
+ .cfi_def_cfa 4, 4
ret
.cfi_endproc
i.e. now it compiles correctly.
Though I do not have x86_64 to test, my guess is that
__builtin_frame_address(1) should work there too. If not - please revert
only x86_64 part of the patch. Thanks.
Cc: Michael Matz <matz@suse.de>
On i386 and gcc-4.7 I found that libc_malloc was miscompiled - look:
static void *libc_malloc(size_t size)
{
void *ptr;
restore_malloc_hooks(); // __malloc_hook = saved_malloc_hook
ptr = malloc(size);
install_malloc_hooks(); // saved_malloc_hook = __malloc_hook, __malloc_hook = __bound_malloc
return ptr;
}
.type libc_malloc, @function
libc_malloc:
.LFB56:
.cfi_startproc
pushl %edx
.cfi_def_cfa_offset 8
movl %eax, (%esp)
call malloc
movl $__bound_malloc, __malloc_hook
movl $__bound_free, __free_hook
movl $__bound_realloc, __realloc_hook
movl $__bound_memalign, __memalign_hook
popl %ecx
.cfi_def_cfa_offset 4
ret
Here gcc inlined both restore_malloc_hooks() and install_malloc_hooks()
and decided that
saved_malloc_hook -> __malloc_hook -> saved_malloc_hook
stores are not needed and could be ommitted. Only it did not know
__molloc_hook affects malloc()...
So add compiler barrier to both install and restore hooks functions and
be done with it - the code is now ok:
diff --git a/bcheck0.s b/bcheck1.s
index 5f50293..4c02a5f 100644
--- a/bcheck0.s
+++ b/bcheck1.s
@@ -42,8 +42,24 @@ libc_malloc:
.cfi_startproc
pushl %edx
.cfi_def_cfa_offset 8
+ movl saved_malloc_hook, %edx
+ movl %edx, __malloc_hook
+ movl saved_free_hook, %edx
+ movl %edx, __free_hook
+ movl saved_realloc_hook, %edx
+ movl %edx, __realloc_hook
+ movl saved_memalign_hook, %edx
+ movl %edx, __memalign_hook
movl %eax, (%esp)
call malloc
+ movl __malloc_hook, %edx
+ movl %edx, saved_malloc_hook
+ movl __free_hook, %edx
+ movl %edx, saved_free_hook
+ movl __realloc_hook, %edx
+ movl %edx, saved_realloc_hook
+ movl __memalign_hook, %edx
+ movl %edx, saved_memalign_hook
movl $__bound_malloc, __malloc_hook
movl $__bound_free, __free_hook
movl $__bound_realloc, __realloc_hook
For barrier I use
__asm__ __volatile__ ("": : : "memory")
which is used as compiler barrier by Linux kernel, and mentioned in gcc
docs and in wikipedia [1].
Without this patch any program compiled with tcc -b crashes in startup
because of infinite recursion in libc_malloc.
[1] http://en.wikipedia.org/wiki/Memory_ordering#Compiler_memory_barrier
Other tests still have issues, currently with weak linking.
One of the primary stumbling blocks on OSX is the lack of support for
mach-o binaries. Therefore all tcc usage on OSX has to be limited to elf
binaries, presumably produced by tcc itself.
Therefore I had to enable building of tiny_libmaker for OSX. Then changed
the make to use tcc and tiny_libmaker to compile the tcclib1.
In order to compile the tests, specifically the parts that use weak linking,
I have had to define MACOSX_DEPLOYMENT_TARGET to 10.2, which seems like a
hack, but extensive searching seems to indicate that this is the only way
to make apple gcc allow weak linking. Using any other value, bigger or smaller
breaks weak linking.
Also added _ANSI_SOURCE define required by some OSX headers, and some cosmetic
gitignore changes. I believe these changes should not impact other platforms.
to the appropriate CFLAGS. In addition, memory hooks are very different
on OSX, so build of bcheck.c had to be disabled for now.
Change of the CFLAGS does affect builds on other platforms, and this needs
to be tested.
- add __builtin_va_arg_types to check how arguments were passed
- move most code of stdarg into libtcc1.c
- remove __builtin_malloc and __builtin_free
- add a test case based on the bug report
(http://www.mail-archive.com/tinycc-devel@nongnu.org/msg03036.html)
(Because GNU's alloca.h unconditionally #undef's alloca)
Also, remove gcc specific sections in headers. and
instead change tests such that gcc does not use them.
