Arg substitution leaves placeholder marker in the stream for
empty arguments. Those need to be skipped when searching for
a fnlike macro invocation in the replacement list itself. See
testcase.
- define_start: set above preprocess_start because now
preprocess_start is defining macros.
- free "cmd_include_files"
- free defines always (after error-longjmps)
- close all files (after error-longjmps)
- tccpe.c: free imports always
- libtcc.c: call tcc_memstats only after all states have
been deleted.
- tccgen.c/tcc.h: allow function declaration after use:
int f() { return g(); }
int g() { return 1; }
may be a warning but not an error
see also 76cb1144ef
- tccgen.c: redundant code related to inline functions removed
(functions used anywhere have sym->c set automatically)
- tccgen.c: make 32bit llop non-equal test portable
(probably not on C67)
- dynarray_add: change prototype to possibly avoid aliasing
problems or at least warnings
- lib/alloca*.S: ".section .note.GNU-stack,"",%progbits" removed
(has no effect)
- tccpe: set SizeOfCode field (for correct upx decompression)
- libtcc.c: fixed alternative -run invocation
tcc "-run -lxxx ..." file.c
(meant to load the library after file).
Also supported now:
tcc files ... options ... -run @ arguments ...
Once-allocated buffers (here a string) that aren't explicitely freed
at program end but rather freed at _exit about 1 nanosecond later
are regarded a leak with MEM_DEBUG, so explicitely free it. Blaeh :-/
handle mms-bitfields as sub-options of -m. (-mfloat-abi
is still special because it requires arguments)
tcc.c: help():
- list -mms-bitfields under 'Target specific options'
libtcc.c/MEM_DEBUG
- add check for past buffer writes
Also ...
tcctest.c:
- exclude stuff that gcc doesn't compile on windows.
libtcc.c/tccpp.c:
- use unsigned for memory sizes to avoid printf format warnings
- use "file:line: message" to make IDE error parsers happy.
tccgen.c: fix typo
See testcase. We must always paste tokens (at least if not
currently substing a normal argument, which is a speed optimization
only now) but at the same time must not regard a ## token
coming from argument expansion as the token-paste operator, nor
if we constructed a ## token due to pasting itself (that was already
checked by pp/01.c).
Our preprocessor throws away # line-comments in asm mode.
It did so also inside preprocessor directives, thereby
removing stringification. Parse defines in non-asm mode (but
retain '.' as identifier character inside macro definitions).
The include directive needs to be parsed as pp-tokens, not
as token (i.e. no conversion to TOK_STR or TOK_NUM). Also fix
parsing computed includes using quoted strings.
This option includes a file as if '#include "file"' is the first
line of compiled files. It's processed after all -D/-U options
and is processed per input file.
When tokens in macro definitions need cstr_buf inside get_tok_str,
the second might overwrite the first (happens when tokens are
multi-character non-identifiers, see testcase) in macro_is_equal,
failing to diagnose a difference. Use a real local buffer.
Also:
- regenerate all tests/pp/*.expect with gcc
- test "insert one space" feature
- test "0x1E-1" in asm mode case
- PARSE_FLAG_SPACES: ignore \f\v\r better
- tcc.h: move some things
#ifndef guards are cached, however after #elif on the
same level, the file must be re-read.
Also: preprocess asm as such even when there is no
assembler (arm).
- "utf8 in identifiers"
from 936819a1b9
- CValue: remove member str.data_allocated
- make tiny allocator private to tccpp
- allocate macro_stack objects on heap
because otherwise it could crash after error/setjmp
in preprocess_delete():end_macro()
- mov "TinyAlloc" defs to tccpp.c
- define_push: take int* str again
Also:
- allow more than one item per line
- respect "quoted items" and escaped quotes \"
(also for LIBTCCAPI tcc_setoptions)
- cleanup some copy & paste
after several "fixes" and "improvements"
b3782c3cf55fb57bead4
feature did not work at all
- Use 'once' flag, not 'ifndef_macro'
- Ignore filename letter case on _WIN32
- Increment global pp_once for each compilation
fixes 5c35ba66c5
Implementation was consistent within tcc but incompatible
with the ABI (for example library functions vprintf etc)
Also:
- tccpp.c/get_tok_str() : avoid "unknown format "%llu" warning
- x86_64_gen.c/gen_vla_alloc() : fix vstack leak
This may be used to preprocess Fabrice Bellards initial revision
in this repository to demonstrate its capability to compile and
run itself (on i386 32-bit linux or windows).
Initial revision: 27f6e16bae
Also needed:
* an empty stdio.h
* a wrapper named tc.c with
void expr(void);
void decl(int);
void next(void);
#include "tcc.c"
* an hello.c such as
int main()
{
printf("Hello World\n");
return 0;
}
All files with unix LF only lines. Then ...
* preprocess the source
$ tcc -E -P10 -I. tcc.c -o tc1.c
* compile the compiler
$ tcc -w -I. tc.c -o tc -ldl
* run it to compile and
run itself to compile and
run itself to compile and
run itself to compile and
run hello.c
$ ./tc tc1.c tc1.c tc1.c hello.c
--> Hello World!
------------------------------------------------------
* On i386 windows this may be added to the tc.c wrapper
#ifdef _WIN32
#include <windows.h>
void *dlsym(int x, const char *func)
{
if (0 == strcmp(func, "dlsym"))
return &dlsym;
return GetProcAddress(LoadLibrary("msvcrt"), func);
}
#endif
Insert a space when it is required to prevent mistokenisation of
the output, and also in a few cases where it is not strictly
required, imitating GCC's behaviour.
* correct -E output for the case ++ + ++ concatenation
do this only for expanded from macro string
and only when tcc_state->output_type == TCC_OUTPUT_PREPROCESS
- uses new `TinyAlloc`-ators for small `TokenSym`, `CString` and
`TokenString` instances
- conditional `TAL_DEBUG` for mem leaks and double frees detection
- on `TAL_DEBUG` collects allocation origin (file + line)
- conditional `TAL_INFO` for allocators stats (in release mode too)
- chain a new allocator twice current capacity on buffer exhaustion
parse_print_line_comment() and parse_print_comment() are
combined and made more simply:
* don't worry about speed with -E option
* don't handle straya in comments
Do we need to handle strays in regular
parse_line_comment() and
parse_comment() ?
A problem was in TOK_ASMDIR_text:
- sprintf(sname, ".%s", get_tok_str(tok1, NULL));
+ sprintf(sname, "%s", get_tok_str(tok1, NULL));
When tok1 is '.text', then sname is '..text'
From: Vlad Vissoultchev
Date: Mon, 11 Apr 2016 01:26:32 +0300
Subject: Fix pragma once guard when compiling multiple source files
When compiling multiple source files directly to executable cached
include files guard was incorrectly checked for TOK_once in ifndef_macro
member.
If two source files included the same header guarded by pragma once, then
the second one erroneously skipped it as `cached_includes` is not cleared
on second `tcc_compile`
When tccboot kernels compiles with
'Identifiers can start and/or', this kernel don't start.
It is hard to find what is wrong.
PS: there was no test for identifiers in *.S with '.'
There was already support for -dD option but in contrast -dM dumps only `#define` directives w/o actual preprocessor output.
The original -dD output differs from gcc output by additional comment in front of `#define`s so this quirk is left for -dM as well.
made like in pcc
(pcc.ludd.ltu.se/ftp/pub/pcc-docs/pcc-utf8-ver3.pdf)
We treat all chars with high bit set as alphabetic.
This allow code like
#include <stdio.h>
int Lefèvre=2;
int main() {
printf("Lefèvre=%d\n",Lefèvre);
return 0;
}
modified version of the old one which don't allow '.'
in #define Identifiers. This allow correctly preprocess
the following code in *.S
#define SRC(y...) \
9999: y; \
.section __ex_table, "a"; \
.long 9999b, 6001f ; \
// .previous
SRC(1: movw (%esi), %bx)
6001:
A test included.
remove non-existent or duplicate directories from include paths
if -fnormalize-inc-dirs is specified. This will help
to compile current coreutils package
- Identifiers can start and/or contain '.' in PARSE_FLAG_ASM_FILE
- Move all GAS directives under TOK_ASMDIR prefix
This patches breaks compilation of the tccboot (linux 2.4.26
kernel). A test.S which fails with this patches:
#define SRC(y...) \
9999: y; \
.section __ex_table, "a"; \
.long 9999b, 6001f<---->; \
.previous
SRC(1:<>movw (%esi), %bx<------>)
// 029-test.S:7: error: macro 'SRC' used with too many args
A CString used to be copied into a token string, which is an int array.
On a 64-bit architecture the pointers were misaligned, so ASan gave
lots of warnings. On a 64-bit architecture that required memory
accesses to be correctly aligned it would not work at all.
The CString is now included in CValue instead.
tcc_normalize_inc_dirs: normally no problem to be absolutly
gcc compatible as long as it can be done the tiny way.
This reverts to the state before recent related commits and
reimplements a (small) part of it to fix the reported problem.
Also: Revert "parsing "..." sequence"
c3975cf27c
&& p[1] == '.'
is not a reliable way to lookahead
- avoid memory allocation by using its (int) token number
- avoid additional function parameter by using Attribute
Also: fix some strange looking error messages
include dirs are prepared as in gcc
- for each duplicate path keep just the first one
- remove each include_path that exists in sysinclude_paths
include_next streamlined by introducing inc_path_index
in the BufferedFile
This version looks rigth. Comparing to the original
algorithm:
1) Loop breaking. We remember a start point after wich
we can try next path. Do not search include stack after
this.
2) But compare next file patch with the start point.
Skip if it the same. Remove "./" before comparing.
PS: a problems with compaling a coreutils-8.24.51-8802e
remain. There are errors messages like:
src/chgrp
src/chown-core.c:42: multiple definition of `make_timespec'
src/chgrp.c:42: first defined here
A problem is in the lib/config.h
#define _GL_INLINE_ extern inline // gcc
#define _GL_INLINE_ inline // tcc
A long description from the lib/config.h
* suppress extern inline with HP-UX cc, as it appears to be broken
* suppress extern inline with Sun C in standards-conformance mode
* suppress extern inline on configurations that mistakenly use
'static inline' to implement functions or macros in standard
C headers like <ctype.h>.
GCC and Clang are excluded from this list. Why not tcc?
don't give an error and simply ingnore directive
if we detect a loop of the #include_next.
With this aproach coreutils-8.24.51-8802e
compiles, but with errors:
lib/libcoreutils.a: error: 'xnmalloc' defined twice
lib/libcoreutils.a: error: 'xnrealloc' defined twice
don't panic with
error: '.' expected
if there is only two '.' chars. Return tok='.' in such case.
An asm code to test:
jz do_move0 # .. then we have a normal low
# .. or else we have a high
* Documentation is now in "docs".
* Source code is now in "src".
* Misc. fixes here and there so that everything still works.
I think I got everything in this commit, but I only tested this
on Linux (Make) and Windows (CMake), so I might've messed
something up on other platforms...
functionality was broken some time ago and was removed
by the "tccpp: fix issues, add tests"
fix: LINE_MACRO_OUTPUT_FORMAT_NONE in pp_line()
means: output '\n' and not "don't output at all"
In the linux kernel sources:
#ifdef __STDC__
#define SYMBOL_NAME_LABEL(X) X##:
#else
#define SYMBOL_NAME_LABEL(X) X/**/:
#endif
tcc is a STDC compiler and must handle 'X##:' case.
* fix some macro expansion issues
* add some pp tests in tests/pp
* improved tcc -E output for better diff'ability
* remove -dD feature (quirky code, exotic feature,
didn't work well)
Based partially on ideas / researches from PipCet
Some issues remain with VA_ARGS macros (if used in a
rather tricky way).
Also, to keep it simple, the pp doesn't automtically
add any extra spaces to separate tokens which otherwise
would form wrong tokens if re-read from tcc -E output
(such as '+' '=') GCC does that, other compilers don't.
* cleanups
- #line 01 "file" / # 01 "file" processing
- #pragma comment(lib,"foo")
- tcc -E: forward some pragmas to output (pack, comment(lib))
- fix macro parameter list parsing mess from
a3fc543459a715d7143d
(some coffee might help, next time ;)
- introduce TOK_PPSTR - to have character constants as
written in the file (similar to TOK_PPNUM)
- allow '\' appear in macros
- new functions begin/end_macro to:
- fix switching macro levels during expansion
- allow unget_tok to unget more than one tok
- slight speedup by using bitflags in isidnum_table
Also:
- x86_64.c : fix decl after statements
- i386-gen,c : fix a vstack leak with VLA on windows
- configure/Makefile : build on windows (MSYS) was broken
- tcc_warning: fflush stderr to keep output order (win32)
This requires moving TOK_PLCHLDR handling, but the new logic should make
things easier even if (when?) GNU comma handling is removed.
(Somewhat confusingly, GCC no longer supports GNU commas. See
http://gcc.gnu.org/onlinedocs/cpp/Variadic-Macros.html for a description
of past and current GCC behaviour.)
Fix handling of escape characters, spaces, and line feeds in macros or
macro arguments that might yet be subject to # stringification.
Should this be an -f option? I think memory usage increases only very
slightly (in particular, while line feeds, stray \s, and spaces are
preserved, comments are not), so it's probably not worth it to make it
one.
Note that macro_subst now checks for stray \s which are still left in
the input stream after macro substitution, if desired.
This patch depends on the previous patch, so if you revert that, please
revert this patch, too.
See http://lists.nongnu.org/archive/html/tinycc-devel/2015-05/msg00002.html
This adds a PARSE_FLAG_ACCEPT_STRAYS parse flag to accept stray
backslashes in the source code, and uses it for pure preprocessing.
For absolutely correct behaviour of # stringification, we need to use
this flag when parsing macro definitions and in macro arguments, as
well; this patch does not yet do so. The test case for that is something
like
#define STRINGIFY2(x) #x
#define STRINGIFY(x) STRINGIFY2(x)
STRINGIFY(\n)
which should produce "\n", not a parse error or "\\n".
See http://lists.nongnu.org/archive/html/tinycc-devel/2015-05/msg00002.html
Perhaps a better fix would be to ensure tok is set to TOK_EOF rather
than 0 at the end of a macro stream.
This partially fixes test2 of the examples given in:
http://lists.nongnu.org/archive/html/tinycc-devel/2015-05/msg00002.html
It's still failing, but at least it's not running out of memory now.
The old code had an inverted condition, so
#define a(b)## b
would be accepted while
#define a(b,c) b ## ## c
would be rejected with the confusing error message "'##' invalid at
start of macro".
Quick fix for
http://lists.nongnu.org/archive/html/tinycc-devel/2015-04/msg00160.html.
I don't fully understand the intended semantics of when file->buf_ptr[0]
is valid, but the rest of the code doesn't have any obvious spots with
the same bug.
Feel free to revert this if I'm mistaken or we need to discuss this
change further.
- pop_macro incorrect with initially undefined macro
- horrible implementation (tcc_open_bf)
- crashes eventually (abuse of Sym->prev_tok)
- the (unrelated) asm_label part is the opposite of a fix
(Despite of its name this variable has nothing to do with
the built-in assembler)
This reverts commit 0c8447db79.
* give warning if pragma is unknown for tcc
* don't free asm_label in sym_free(),
it's a job of the asm_free_labels().
The above pragmas are used in the mingw headers.
Thise pragmas are implemented in gcc-4.5+ and current
clang.
library Cello: http://libcello.org/ which uses `$` and several
variations of as macros.
There is also RayLanguage which also uses it as a macro for a kind of
ObjC style message passing: https://github.com/kojiba/RayLanguage
This is a patch from Daniel Holden.
Lets assume that in *.S files a preprocessor directive
follow '#' char w/o spaces between. Otherwise there is
too many problems with the content of the comments.
* tell a right line number in error message
if a #line directive is wrong
* don't print an error message if we preprocess a .S file
and #line directive is wrong. This is the case of
the
# 4026 bytes
comment in *.S file.
* preprocess_skip: skip a line with
if (parse_flags & PARSE_FLAG_ASM_COMMENTS)
p = parse_line_comment(p);
if line starts with # and a preprocessor command not found.
A test program:
#if defined(CONFIG_EDD) || defined(CONFIG_EDD_MODULE)
# This repeats until either a device doesn't exist, or until
#endif
* remove a second definition of the TOK_FLAG_* and PARSE_FLAG_*
from the tccpp.c
in the same pass like
tcc -E one.c two.c three.c -o combined.i
This will allow to speed up a compilation process by using a commamd like
tcc -E *.c | tcc -o program.exe -xc -
It looks that multi-times initialization don't affect anything.
Only call to the free_defines(define_start) in tcc_preprocess()
is removed in assumption that free_defines(NULL) in
tcc_cleanup() will free all defines.
With this option on a defines are included into the output
(inside comments). This will allow to debug a problems like:
In file included from math.c:8:
In file included from /usr/include/math.h:43:
/usr/include/bits/nan.h:52: warning: NAN redefined
tcc -E -P
do not output a #line directive, a gcc compatible option
tcc -E -P1
don't follow a gcc preprocessor style and do output a standard
#line directive. In such case we don't lose a location info when
we going to compile a resulting file wtith a compiler not
understanding a gnu style line info.
A cpp from gcc do this.
A test case:
tcc -E tccasm.c -o tccasm.i
tcc -E tccasm.i -o tccasm.ii
After a patch the line numbers in tccasm.ii are the same
as in tccasm.i
* tccpp.c (parse_number): `shift' should be 1 while parsing binary
floating point number.
* tests/tests2/70_floating_point_literals.c: New test cases for
floating point number parsing.
As per grischka comment, always output a space after macro concatenation
instead of trying to detect if it's necessary as the current approach
has a huge cost.
This was going wrong (case TOK_LAND in unary: computed labels)
- vset(&s->type, VT_CONST | VT_SYM, 0);
- vtop->sym = s;
This does the right thing and is shorter:
+ vpushsym(&s->type, s);
Test case was:
int main(int argc, char **argv)
{
int x;
static void *label_return = &&lbl_return;
printf("label_return = %p\n", label_return);
goto *label_return; //<<<<< here segfault on linux X86_64 without the memset on vset
printf("unreachable\n");
lbl_return:
return 0;
}
Also::
- Rename "void* CValue.ptr" to more usable "addr_t ptr_offset"
and start to use it in obvious cases.
- use __attribute__ ((noreturn)) only with gnu compiler
- Revert CValue memsets ("After several days searching ...")
commit 4bc83ac393
Doesn't mean that the vsetX/vpush thingy isn't brittle and
there still might be bugs as to differences in how the CValue
union was set and is then interpreted later on.
However the big memset hammer was just too slow (-3% overall).
As was pointed out on tinycc-devel, many uses of get_tok_str gives as
second parameter the value NULL. However, that pointer was
unconditionally dereferenced in get_tok_ptr. This commit explicitely add
support for thas case.
I found the problem it was because CValue stack variables have rubish as it inital values
and assigning to a member that is smaller than the big union item and trying to
recover it later as a different member gives bak garbage.
ST_FUNC void vset(TCCState* tcc_state, CType *type, int r, int v)
{
CValue cval;
memset(&cval, 0, sizeof(CValue));
cval.i = v; //,<<<<<<<<<<< here is the main bug that mix with garbage
vsetc(tcc_state, type, r, &cval);
}
/* store a value or an expression directly in global data or in local array */
static void init_putv(TCCState* tcc_state, CType *type, Section *sec, unsigned long c,
int v, int expr_type)
{
...
case VT_PTR:
if (tcc_state->tccgen_vtop->r & VT_SYM) {
greloc(tcc_state, sec, tcc_state->tccgen_vtop->sym, c, R_DATA_PTR);
}
//<<< on the next line is where we try to get the assigned value to cvalue.i as cvalue.ull
*(addr_t *)ptr |= (tcc_state->tccgen_vtop->c.ull & bit_mask) << bit_pos;
break;
Also this patch makes vla tests pass on linux 32 bits
- tccgen: error out for cast to void, as in
void foo(void) { return 1; }
This avoids an assertion failure in x86_64-gen.c, also.
also fix tests2/03_struct.c accordingly
- Error: "memory full" - be more specific
- Makefiles: remove circular dependencies, lookup tcctest.c from VPATH
- tcc.h: cleanup lib, include, crt and libgcc search paths"
avoid duplication or trailing slashes with no CONFIG_MULTIARCHDIR
(as from 9382d6f1a0)
- tcc.h: remove ";{B}" from PE search path
in ce5e12c2f9 James Lyon wrote:
"... I'm not sure this is the right way to fix this problem."
And the answer is: No, please. (copying libtcc1.a for tests instead)
- win32/build_tcc.bat: do not move away a versioned file
- add quotes: eval opt=\"$opt\"
- use $source_path/conftest.c for OOT build
- add fn_makelink() for OOT build
- do not check lddir etc. on Windows/MSYS
- formatting
config-print.c
- rename to conftest.c (for consistency)
- change option e to b
- change output from that from "yes" to "no"
- remove inttypes.h dependency
- simpify version output
Makefile:
- improve GCC warning flag checks
tcc.h:
- add back default CONFIG_LDDIR
- add default CONFIG_TCCDIR also (just for fun)
tccpp.c:
- fix Christian's last warning
tccpp.c: In function ‘macro_subst’:
tccpp.c:2803:12: warning: ‘*((void *)&cval+4)’ is used uninitialized
in this function [-Wuninitialized]
That the change fixes the warning doesn't make sense but anyway.
libtcc.c:
- tcc_error/warning: print correct source filename/line for
token :paste: (also inline :asm:)
lddir and multiarch logic still needs fixing.
