- uses new `TinyAlloc`-ators for small `TokenSym`, `CString` and
`TokenString` instances
- conditional `TAL_DEBUG` for mem leaks and double frees detection
- on `TAL_DEBUG` collects allocation origin (file + line)
- conditional `TAL_INFO` for allocators stats (in release mode too)
- chain a new allocator twice current capacity on buffer exhaustion
parse_print_line_comment() and parse_print_comment() are
combined and made more simply:
* don't worry about speed with -E option
* don't handle straya in comments
Do we need to handle strays in regular
parse_line_comment() and
parse_comment() ?
A problem was in TOK_ASMDIR_text:
- sprintf(sname, ".%s", get_tok_str(tok1, NULL));
+ sprintf(sname, "%s", get_tok_str(tok1, NULL));
When tok1 is '.text', then sname is '..text'
From: Vlad Vissoultchev
Date: Mon, 11 Apr 2016 01:26:32 +0300
Subject: Fix pragma once guard when compiling multiple source files
When compiling multiple source files directly to executable cached
include files guard was incorrectly checked for TOK_once in ifndef_macro
member.
If two source files included the same header guarded by pragma once, then
the second one erroneously skipped it as `cached_includes` is not cleared
on second `tcc_compile`
When tccboot kernels compiles with
'Identifiers can start and/or', this kernel don't start.
It is hard to find what is wrong.
PS: there was no test for identifiers in *.S with '.'
There was already support for -dD option but in contrast -dM dumps only `#define` directives w/o actual preprocessor output.
The original -dD output differs from gcc output by additional comment in front of `#define`s so this quirk is left for -dM as well.
made like in pcc
(pcc.ludd.ltu.se/ftp/pub/pcc-docs/pcc-utf8-ver3.pdf)
We treat all chars with high bit set as alphabetic.
This allow code like
#include <stdio.h>
int Lefèvre=2;
int main() {
printf("Lefèvre=%d\n",Lefèvre);
return 0;
}
modified version of the old one which don't allow '.'
in #define Identifiers. This allow correctly preprocess
the following code in *.S
#define SRC(y...) \
9999: y; \
.section __ex_table, "a"; \
.long 9999b, 6001f ; \
// .previous
SRC(1: movw (%esi), %bx)
6001:
A test included.
remove non-existent or duplicate directories from include paths
if -fnormalize-inc-dirs is specified. This will help
to compile current coreutils package
- Identifiers can start and/or contain '.' in PARSE_FLAG_ASM_FILE
- Move all GAS directives under TOK_ASMDIR prefix
This patches breaks compilation of the tccboot (linux 2.4.26
kernel). A test.S which fails with this patches:
#define SRC(y...) \
9999: y; \
.section __ex_table, "a"; \
.long 9999b, 6001f<---->; \
.previous
SRC(1:<>movw (%esi), %bx<------>)
// 029-test.S:7: error: macro 'SRC' used with too many args
A CString used to be copied into a token string, which is an int array.
On a 64-bit architecture the pointers were misaligned, so ASan gave
lots of warnings. On a 64-bit architecture that required memory
accesses to be correctly aligned it would not work at all.
The CString is now included in CValue instead.
tcc_normalize_inc_dirs: normally no problem to be absolutly
gcc compatible as long as it can be done the tiny way.
This reverts to the state before recent related commits and
reimplements a (small) part of it to fix the reported problem.
Also: Revert "parsing "..." sequence"
c3975cf27c
&& p[1] == '.'
is not a reliable way to lookahead
- avoid memory allocation by using its (int) token number
- avoid additional function parameter by using Attribute
Also: fix some strange looking error messages
include dirs are prepared as in gcc
- for each duplicate path keep just the first one
- remove each include_path that exists in sysinclude_paths
include_next streamlined by introducing inc_path_index
in the BufferedFile
This version looks rigth. Comparing to the original
algorithm:
1) Loop breaking. We remember a start point after wich
we can try next path. Do not search include stack after
this.
2) But compare next file patch with the start point.
Skip if it the same. Remove "./" before comparing.
PS: a problems with compaling a coreutils-8.24.51-8802e
remain. There are errors messages like:
src/chgrp
src/chown-core.c:42: multiple definition of `make_timespec'
src/chgrp.c:42: first defined here
A problem is in the lib/config.h
#define _GL_INLINE_ extern inline // gcc
#define _GL_INLINE_ inline // tcc
A long description from the lib/config.h
* suppress extern inline with HP-UX cc, as it appears to be broken
* suppress extern inline with Sun C in standards-conformance mode
* suppress extern inline on configurations that mistakenly use
'static inline' to implement functions or macros in standard
C headers like <ctype.h>.
GCC and Clang are excluded from this list. Why not tcc?
don't give an error and simply ingnore directive
if we detect a loop of the #include_next.
With this aproach coreutils-8.24.51-8802e
compiles, but with errors:
lib/libcoreutils.a: error: 'xnmalloc' defined twice
lib/libcoreutils.a: error: 'xnrealloc' defined twice
don't panic with
error: '.' expected
if there is only two '.' chars. Return tok='.' in such case.
An asm code to test:
jz do_move0 # .. then we have a normal low
# .. or else we have a high
* Documentation is now in "docs".
* Source code is now in "src".
* Misc. fixes here and there so that everything still works.
I think I got everything in this commit, but I only tested this
on Linux (Make) and Windows (CMake), so I might've messed
something up on other platforms...
functionality was broken some time ago and was removed
by the "tccpp: fix issues, add tests"
fix: LINE_MACRO_OUTPUT_FORMAT_NONE in pp_line()
means: output '\n' and not "don't output at all"
In the linux kernel sources:
#ifdef __STDC__
#define SYMBOL_NAME_LABEL(X) X##:
#else
#define SYMBOL_NAME_LABEL(X) X/**/:
#endif
tcc is a STDC compiler and must handle 'X##:' case.
* fix some macro expansion issues
* add some pp tests in tests/pp
* improved tcc -E output for better diff'ability
* remove -dD feature (quirky code, exotic feature,
didn't work well)
Based partially on ideas / researches from PipCet
Some issues remain with VA_ARGS macros (if used in a
rather tricky way).
Also, to keep it simple, the pp doesn't automtically
add any extra spaces to separate tokens which otherwise
would form wrong tokens if re-read from tcc -E output
(such as '+' '=') GCC does that, other compilers don't.
* cleanups
- #line 01 "file" / # 01 "file" processing
- #pragma comment(lib,"foo")
- tcc -E: forward some pragmas to output (pack, comment(lib))
- fix macro parameter list parsing mess from
a3fc543459a715d7143d
(some coffee might help, next time ;)
- introduce TOK_PPSTR - to have character constants as
written in the file (similar to TOK_PPNUM)
- allow '\' appear in macros
- new functions begin/end_macro to:
- fix switching macro levels during expansion
- allow unget_tok to unget more than one tok
- slight speedup by using bitflags in isidnum_table
Also:
- x86_64.c : fix decl after statements
- i386-gen,c : fix a vstack leak with VLA on windows
- configure/Makefile : build on windows (MSYS) was broken
- tcc_warning: fflush stderr to keep output order (win32)
This requires moving TOK_PLCHLDR handling, but the new logic should make
things easier even if (when?) GNU comma handling is removed.
(Somewhat confusingly, GCC no longer supports GNU commas. See
http://gcc.gnu.org/onlinedocs/cpp/Variadic-Macros.html for a description
of past and current GCC behaviour.)
Fix handling of escape characters, spaces, and line feeds in macros or
macro arguments that might yet be subject to # stringification.
Should this be an -f option? I think memory usage increases only very
slightly (in particular, while line feeds, stray \s, and spaces are
preserved, comments are not), so it's probably not worth it to make it
one.
Note that macro_subst now checks for stray \s which are still left in
the input stream after macro substitution, if desired.
This patch depends on the previous patch, so if you revert that, please
revert this patch, too.
See http://lists.nongnu.org/archive/html/tinycc-devel/2015-05/msg00002.html
This adds a PARSE_FLAG_ACCEPT_STRAYS parse flag to accept stray
backslashes in the source code, and uses it for pure preprocessing.
For absolutely correct behaviour of # stringification, we need to use
this flag when parsing macro definitions and in macro arguments, as
well; this patch does not yet do so. The test case for that is something
like
#define STRINGIFY2(x) #x
#define STRINGIFY(x) STRINGIFY2(x)
STRINGIFY(\n)
which should produce "\n", not a parse error or "\\n".
See http://lists.nongnu.org/archive/html/tinycc-devel/2015-05/msg00002.html
Perhaps a better fix would be to ensure tok is set to TOK_EOF rather
than 0 at the end of a macro stream.
This partially fixes test2 of the examples given in:
http://lists.nongnu.org/archive/html/tinycc-devel/2015-05/msg00002.html
It's still failing, but at least it's not running out of memory now.
The old code had an inverted condition, so
#define a(b)## b
would be accepted while
#define a(b,c) b ## ## c
would be rejected with the confusing error message "'##' invalid at
start of macro".
Quick fix for
http://lists.nongnu.org/archive/html/tinycc-devel/2015-04/msg00160.html.
I don't fully understand the intended semantics of when file->buf_ptr[0]
is valid, but the rest of the code doesn't have any obvious spots with
the same bug.
Feel free to revert this if I'm mistaken or we need to discuss this
change further.
- pop_macro incorrect with initially undefined macro
- horrible implementation (tcc_open_bf)
- crashes eventually (abuse of Sym->prev_tok)
- the (unrelated) asm_label part is the opposite of a fix
(Despite of its name this variable has nothing to do with
the built-in assembler)
This reverts commit 0c8447db79.
* give warning if pragma is unknown for tcc
* don't free asm_label in sym_free(),
it's a job of the asm_free_labels().
The above pragmas are used in the mingw headers.
Thise pragmas are implemented in gcc-4.5+ and current
clang.
library Cello: http://libcello.org/ which uses `$` and several
variations of as macros.
There is also RayLanguage which also uses it as a macro for a kind of
ObjC style message passing: https://github.com/kojiba/RayLanguage
This is a patch from Daniel Holden.
Lets assume that in *.S files a preprocessor directive
follow '#' char w/o spaces between. Otherwise there is
too many problems with the content of the comments.
* tell a right line number in error message
if a #line directive is wrong
* don't print an error message if we preprocess a .S file
and #line directive is wrong. This is the case of
the
# 4026 bytes
comment in *.S file.
* preprocess_skip: skip a line with
if (parse_flags & PARSE_FLAG_ASM_COMMENTS)
p = parse_line_comment(p);
if line starts with # and a preprocessor command not found.
A test program:
#if defined(CONFIG_EDD) || defined(CONFIG_EDD_MODULE)
# This repeats until either a device doesn't exist, or until
#endif
* remove a second definition of the TOK_FLAG_* and PARSE_FLAG_*
from the tccpp.c
in the same pass like
tcc -E one.c two.c three.c -o combined.i
This will allow to speed up a compilation process by using a commamd like
tcc -E *.c | tcc -o program.exe -xc -
It looks that multi-times initialization don't affect anything.
Only call to the free_defines(define_start) in tcc_preprocess()
is removed in assumption that free_defines(NULL) in
tcc_cleanup() will free all defines.
With this option on a defines are included into the output
(inside comments). This will allow to debug a problems like:
In file included from math.c:8:
In file included from /usr/include/math.h:43:
/usr/include/bits/nan.h:52: warning: NAN redefined
tcc -E -P
do not output a #line directive, a gcc compatible option
tcc -E -P1
don't follow a gcc preprocessor style and do output a standard
#line directive. In such case we don't lose a location info when
we going to compile a resulting file wtith a compiler not
understanding a gnu style line info.
A cpp from gcc do this.
A test case:
tcc -E tccasm.c -o tccasm.i
tcc -E tccasm.i -o tccasm.ii
After a patch the line numbers in tccasm.ii are the same
as in tccasm.i
* tccpp.c (parse_number): `shift' should be 1 while parsing binary
floating point number.
* tests/tests2/70_floating_point_literals.c: New test cases for
floating point number parsing.
As per grischka comment, always output a space after macro concatenation
instead of trying to detect if it's necessary as the current approach
has a huge cost.
This was going wrong (case TOK_LAND in unary: computed labels)
- vset(&s->type, VT_CONST | VT_SYM, 0);
- vtop->sym = s;
This does the right thing and is shorter:
+ vpushsym(&s->type, s);
Test case was:
int main(int argc, char **argv)
{
int x;
static void *label_return = &&lbl_return;
printf("label_return = %p\n", label_return);
goto *label_return; //<<<<< here segfault on linux X86_64 without the memset on vset
printf("unreachable\n");
lbl_return:
return 0;
}
Also::
- Rename "void* CValue.ptr" to more usable "addr_t ptr_offset"
and start to use it in obvious cases.
- use __attribute__ ((noreturn)) only with gnu compiler
- Revert CValue memsets ("After several days searching ...")
commit 4bc83ac393
Doesn't mean that the vsetX/vpush thingy isn't brittle and
there still might be bugs as to differences in how the CValue
union was set and is then interpreted later on.
However the big memset hammer was just too slow (-3% overall).
As was pointed out on tinycc-devel, many uses of get_tok_str gives as
second parameter the value NULL. However, that pointer was
unconditionally dereferenced in get_tok_ptr. This commit explicitely add
support for thas case.
I found the problem it was because CValue stack variables have rubish as it inital values
and assigning to a member that is smaller than the big union item and trying to
recover it later as a different member gives bak garbage.
ST_FUNC void vset(TCCState* tcc_state, CType *type, int r, int v)
{
CValue cval;
memset(&cval, 0, sizeof(CValue));
cval.i = v; //,<<<<<<<<<<< here is the main bug that mix with garbage
vsetc(tcc_state, type, r, &cval);
}
/* store a value or an expression directly in global data or in local array */
static void init_putv(TCCState* tcc_state, CType *type, Section *sec, unsigned long c,
int v, int expr_type)
{
...
case VT_PTR:
if (tcc_state->tccgen_vtop->r & VT_SYM) {
greloc(tcc_state, sec, tcc_state->tccgen_vtop->sym, c, R_DATA_PTR);
}
//<<< on the next line is where we try to get the assigned value to cvalue.i as cvalue.ull
*(addr_t *)ptr |= (tcc_state->tccgen_vtop->c.ull & bit_mask) << bit_pos;
break;
Also this patch makes vla tests pass on linux 32 bits
- tccgen: error out for cast to void, as in
void foo(void) { return 1; }
This avoids an assertion failure in x86_64-gen.c, also.
also fix tests2/03_struct.c accordingly
- Error: "memory full" - be more specific
- Makefiles: remove circular dependencies, lookup tcctest.c from VPATH
- tcc.h: cleanup lib, include, crt and libgcc search paths"
avoid duplication or trailing slashes with no CONFIG_MULTIARCHDIR
(as from 9382d6f1a0)
- tcc.h: remove ";{B}" from PE search path
in ce5e12c2f9 James Lyon wrote:
"... I'm not sure this is the right way to fix this problem."
And the answer is: No, please. (copying libtcc1.a for tests instead)
- win32/build_tcc.bat: do not move away a versioned file
- add quotes: eval opt=\"$opt\"
- use $source_path/conftest.c for OOT build
- add fn_makelink() for OOT build
- do not check lddir etc. on Windows/MSYS
- formatting
config-print.c
- rename to conftest.c (for consistency)
- change option e to b
- change output from that from "yes" to "no"
- remove inttypes.h dependency
- simpify version output
Makefile:
- improve GCC warning flag checks
tcc.h:
- add back default CONFIG_LDDIR
- add default CONFIG_TCCDIR also (just for fun)
tccpp.c:
- fix Christian's last warning
tccpp.c: In function ‘macro_subst’:
tccpp.c:2803:12: warning: ‘*((void *)&cval+4)’ is used uninitialized
in this function [-Wuninitialized]
That the change fixes the warning doesn't make sense but anyway.
libtcc.c:
- tcc_error/warning: print correct source filename/line for
token :paste: (also inline :asm:)
lddir and multiarch logic still needs fixing.
This replaces -> use instead:
-----------------------------------
- tcc_set_linker -> tcc_set_options(s, "-Wl,...");
- tcc_set_warning -> tcc_set_options(s, "-W...");
- tcc_enable_debug -> tcc_set_options(s, "-g");
parse_args is moved to libtcc.c (now tcc_parse_args).
Also some cleanups:
- reorder TCCState members
- add some comments here and there
- do not use argv's directly, make string copies
- use const char* in tcc_set_linker
- tccpe: use fd instead of fp
tested with -D MEM_DEBUG: 0 bytes left
This replaces commit 3d409b0889
- revert old fix in libtcc.c
- #include_next: look up the file in the include stack to see
if it is already included.
Also:
- streamline include code
- remove 'type' from struct CachedInclude (obsolete because we check
full filename anyway)
- remove inc_type & inc_filename from struct Bufferedfile (obsolete)
- fix bug with TOK_FLAG_ENDIF not being reset
- unrelated: get rid of an 'variable potentially uninitialized' warning
For vstack Fabrice used the trick to initialize vtop to &vstack[-1], so
that on first push, vtop becomes &vstack[0] and a value is also stored
there - everything works.
Except that when tcc is compiled with bounds-checking enabled, vstack - 1
returns INVALID_POINTER and oops...
Let's workaround it with artificial 1 vstack slot which will not be
used, but only serve as an indicator that pointing to &vstack[-1] is ok.
Now, tcc, after being self-compiled with -b works:
$ ./tcc -B. -o tccb -DONE_SOURCE -DCONFIG_MULTIARCHDIR=\"i386-linux-gnu\" tcc.c -ldl
$ cd tests
$ ../tcc -B.. -run tcctest.c >1
$ ../tccb -B.. -run tcctest.c >2
$ diff -u 1 2
and note, tcc's compilation speed is not affected:
$ ./tcc -B. -bench -DONE_SOURCE -DCONFIG_MULTIARCHDIR=\"i386-linux-gnu\" -c tcc.c
before: 8270 idents, 47221 lines, 1527730 bytes, 0.152 s, 309800 lines/s, 10.0 MB/s
after: 8271 idents, 47221 lines, 1527733 bytes, 0.152 s, 310107 lines/s, 10.0 MB/s
But note, that `tcc -b -run tcc` is still broken - for example it crashes
on
$ cat x.c
double get100 () { return 100.0; }
$ ./tcc -B. -b -DTCC_TARGET_I386 -DCONFIG_MULTIARCHDIR=\"i386-linux-gnu\" -run \
-DONE_SOURCE ./tcc.c -B. -c x.c
Runtime error: dereferencing invalid pointer
./tccpp.c:1953: at 0xa7beebdf parse_number() (included from ./libtcc.c, ./tcc.c)
./tccpp.c:3003: by 0xa7bf0708 next() (included from ./libtcc.c, ./tcc.c)
./tccgen.c:4465: by 0xa7bfe348 block() (included from ./libtcc.c, ./tcc.c)
./tccgen.c:4440: by 0xa7bfe212 block() (included from ./libtcc.c, ./tcc.c)
./tccgen.c:5529: by 0xa7c01929 gen_function() (included from ./libtcc.c, ./tcc.c)
./tccgen.c:5767: by 0xa7c02602 decl0() (included from ./libtcc.c, ./tcc.c)
that's because lib/bcheck.c runtime needs more fixes -- see next
patches.
This needs to be accepted:
typedef int foo;
void f (void) { foo: return; }
namespaces for labels and types are different. The problem is that
the block parser always tries to find a decl first and that routine
doesn't peek enough to detect this case. Needs some adjustments
to unget_tok() so that we can call it even when we already called
it once, but next() didn't come around restoring the buffer yet.
(It lazily does so not when the buffer becomes empty, but rather
when the next call detects that the buffer is empty, i.e. it requires
two next() calls until the unget buffer gets switched back).
- remove redunant else branch
- zero-terminate linker_arg
- declare cstr_xxx as PUB_FUNC
(which are functions used in tcc.c but not in the libtcc API.
Useful for a tcc(.exe) that uses the libtcc.(so/dll))
- while at it, export PUB_FUNCs from dll
- use {B} to substitute tcc_lih_path (instead of \b)
- expand CONFIG_TCC_CRTPREFIX in CONFIG_TCC_LIBPATHS
which fixes duplicate CONFIG_SYSROOT.
- put default CONFIG_SYSROOT ("") into tcc.h
- remove hack from commit db6fcce78f
because $(tccdir)/include is already in sysincludes
- configure: error out for unrecognized options.
- win32/build-tcc.bat: put libtcc into base dir where it will
find lib/include automatically, and build libtcc_test example.
- fix my prev commit:
put declaration above statements to stay c89 compatible
- replace commit by a blank
#define con(a, b) a/**/b
this should yield a b, not ab
We need a ' ' after subst of m in the following case
#define m(name,r) name ## r
#define m0(a,b,c) int m(a,b) c
#define m1(a,b,c) int m(a,b)c
m0(a, b, c);
m1(a, b, c);
* Replace the save/load_buffer_state by a dynarray approach:
- Filename and libname are added to a dynarray when first encountered
- Load repeatedly the files in the dynarray until no new undefined
symbol are encountered
* Replace snprintf by sprintf in libname_to_filename
* Use tcc_fileextension in filename_to_libname
* Introduce a tcc_strcpy_part fonction to copy only a subset of a
string
* Move new_undef_syms declaration from tcc.h to tccelf.c
* search file from INPUT and GROUP commands in the library path in
addition to the current directory
* handle libraries specified by -lfoo options
* Search lib in GROUP command repeatedly
