1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
|
<html>
<head>
<title>Clang Driver Manual</title>
<link type="text/css" rel="stylesheet" href="../menu.css" />
<link type="text/css" rel="stylesheet" href="../content.css" />
<style type="text/css">
td {
vertical-align: top;
}
</style>
</head>
<body>
<!--#include virtual="../menu.html.incl"-->
<div id="content">
<h1>Driver Design & Internals</h1>
<ul>
<li><a href="#intro">Introduction</a></li>
<li><a href="#features">Features and Goals</a></li>
<ul>
<li><a href="#gcccompat">GCC Compatibility</a></li>
<li><a href="#components">Flexible</a></li>
<li><a href="#performance">Low Overhead</a></li>
<li><a href="#simple">Simple</a></li>
</ul>
<li><a href="#design">Design</a></li>
<ul>
<li><a href="#int_intro">Internals Introduction</a></li>
<li><a href="#int_overview">Design Overview</a></li>
<li><a href="#int_notes">Additional Notes</a></li>
<ul>
<li><a href="#int_compilation">The Compilation Object</a></li>
<li><a href="#int_unified_parsing">Unified Parsing & Pipelining</a></li>
<li><a href="#int_toolchain_translation">ToolChain Argument Translation</a></li>
<li><a href="#int_unused_warnings">Unused Argument Warnings</a></li>
</ul>
<li><a href="#int_gcc_concepts">Relation to GCC Driver Concepts</a></li>
</ul>
</ul>
<!-- ======================================================================= -->
<h2 id="intro">Introduction</h2>
<!-- ======================================================================= -->
<p>This document describes the Clang driver. The purpose of this
document is to describe both the motivation and design goals
for the driver, as well as details of the internal
implementation.</p>
<!-- ======================================================================= -->
<h2 id="features">Features and Goals</h2>
<!-- ======================================================================= -->
<p>The Clang driver is intended to be a production quality
compiler driver providing access to the Clang compiler and
tools, with a command line interface which is compatible with
the gcc driver.</p>
<p>Although the driver is part of and driven by the Clang
project, it is logically a separate tool which shares many of
the same goals as Clang:</p>
<p><b>Features</b>:</p>
<ul>
<li><a href="#gcccompat">GCC Compatibility</a></li>
<li><a href="#components">Flexible</a></li>
<li><a href="#performance">Low Overhead</a></li>
<li><a href="#simple">Simple</a></li>
</ul>
<!--=======================================================================-->
<h3 id="gcccompat">GCC Compatibility</h3>
<!--=======================================================================-->
<p>The number one goal of the driver is to ease the adoption of
Clang by allowing users to drop Clang into a build system
which was designed to call GCC. Although this makes the driver
much more complicated than might otherwise be necessary, we
decided that being very compatible with the gcc command line
interface was worth it in order to allow users to quickly test
clang on their projects.</p>
<!--=======================================================================-->
<h3 id="components">Flexible</h3>
<!--=======================================================================-->
<p>The driver was designed to be flexible and easily accomodate
new uses as we grow the clang and LLVM infrastructure. As one
example, the driver can easily support the introduction of
tools which have an integrated assembler; something we hope to
add to LLVM in the future.</p>
<p>Similarly, most of the driver functionality is kept in a
library which can be used to build other tools which want to
implement or accept a gcc like interface. </p>
<!--=======================================================================-->
<h3 id="performance">Low Overhead</h3>
<!--=======================================================================-->
<p>The driver should have as little overhead as possible. In
practice, we found that the gcc driver by itself incurred a
small but meaningful overhead when compiling many small
files. The driver doesn't do much work compared to a
compilation, but we have tried to keep it as efficient as
possible by following a few simple principles:</p>
<ul>
<li>Avoid memory allocation and string copying when
possible.</li>
<li>Don't parse arguments more than once.</li>
<li>Provide a few simple interfaces for efficiently searching
arguments.</li>
</ul>
<!--=======================================================================-->
<h3 id="simple">Simple</h3>
<!--=======================================================================-->
<p>Finally, the driver was designed to be "as simple as
possible", given the other goals. Notably, trying to be
completely compatible with the gcc driver adds a significant
amount of complexity. However, the design of the driver
attempts to mitigate this complexity by dividing the process
into a number of independent stages instead of a single
monolithic task.</p>
<!-- ======================================================================= -->
<h2 id="design">Internal Design and Implementation</h2>
<!-- ======================================================================= -->
<ul>
<li><a href="#int_intro">Internals Introduction</a></li>
<li><a href="#int_overview">Design Overview</a></li>
<li><a href="#int_notes">Additional Notes</a></li>
<li><a href="#int_gcc_concepts">Relation to GCC Driver Concepts</a></li>
</ul>
<!--=======================================================================-->
<h3><a name="int_intro">Internals Introduction</a></h3>
<!--=======================================================================-->
<p>In order to satisfy the stated goals, the driver was designed
to completely subsume the functionality of the gcc executable;
that is, the driver should not need to delegate to gcc to
perform subtasks. On Darwin, this implies that the Clang
driver also subsumes the gcc driver-driver, which is used to
implement support for building universal images (binaries and
object files). This also implies that the driver should be
able to call the language specific compilers (e.g. cc1)
directly, which means that it must have enough information to
forward command line arguments to child processes
correctly.</p>
<!--=======================================================================-->
<h3><a name="int_overview">Design Overview</a></h3>
<!--=======================================================================-->
<p>The diagram below shows the significant components of the
driver architecture and how they relate to one another. The
orange components represent concrete data structures built by
the driver, the green components indicate conceptually
distinct stages which manipulate these data structures, and
the blue components are important helper classes. </p>
<center>
<a href="DriverArchitecture.png" alt="Driver Architecture Diagram">
<img width=400 src="DriverArchitecture.png">
</a>
</center>
<!--=======================================================================-->
<h3><a name="int_stages">Driver Stages</a></h3>
<!--=======================================================================-->
<p>The driver functionality is conceptually divided into five stages:</p>
<ol>
<li>
<b>Parse: Option Parsing</b>
<p>The command line argument strings are decomposed into
arguments (<tt>Arg</tt> instances). The driver expects to
understand all available options, although there is some
facility for just passing certain classes of options
through (like <tt>-Wl,</tt>).</p>
<p>Each argument corresponds to exactly one
abstract <tt>Option</tt> definition, which describes how
the option is parsed along with some additional
metadata. The Arg instances themselves are lightweight and
merely contain enough information for clients to determine
which option they correspond to and their values (if they
have additional parameters).</p>
<p>For example, a command line like "-Ifoo -I foo" would
parse to two Arg instances (a JoinedArg and a SeparateArg
instance), but each would refer to the same Option.</p>
<p>Options are lazily created in order to avoid populating
all Option classes when the driver is loaded. Most of the
driver code only needs to deal with options by their
unique ID (e.g., <tt>options::OPT_I</tt>),</p>
<p>Arg instances themselves do not generally store the
values of parameters. In many cases, this would
simply result in creating unnecessary string
copies. Instead, Arg instances are always embedded inside
an ArgList structure, which contains the original vector
of argument strings. Each Arg itself only needs to contain
an index into this vector instead of storing its values
directly.</p>
<p>The clang driver can dump the results of this
stage using the <tt>-ccc-print-options</tt> flag (which
must preceed any actual command line arguments). For
example:</p>
<pre>
$ <b>clang -ccc-print-options -Xarch_i386 -fomit-frame-pointer -Wa,-fast -Ifoo -I foo t.c</b>
Option 0 - Name: "-Xarch_", Values: {"i386", "-fomit-frame-pointer"}
Option 1 - Name: "-Wa,", Values: {"-fast"}
Option 2 - Name: "-I", Values: {"foo"}
Option 3 - Name: "-I", Values: {"foo"}
Option 4 - Name: "<input>", Values: {"t.c"}
</pre>
<p>After this stage is complete the command line should be
broken down into well defined option objects with their
appropriate parameters. Subsequent stages should rarely,
if ever, need to do any string processing.</p>
</li>
<li>
<b>Pipeline: Compilation Job Construction</b>
<p>Once the arguments are parsed, the tree of subprocess
jobs needed for the desired compilation sequence are
constructed. This involves determining the input files and
their types, what work is to be done on them (preprocess,
compile, assemble, link, etc.), and constructing a list of
Action instances for each task. The result is a list of
one or more top-level actions, each of which generally
corresponds to a single output (for example, an object or
linked executable).</p>
<p>The majority of Actions correspond to actual tasks,
however there are two special Actions. The first is
InputAction, which simply serves to adapt an input
argument for use as an input to other Actions. The second
is BindArchAction, which conceptually alters the
architecture to be used for all of its input Actions.</p>
<p>The clang driver can dump the results of this
stage using the <tt>-ccc-print-phases</tt> flag. For
example:</p>
<pre>
$ <b>clang -ccc-print-phases -x c t.c -x assembler t.s</b>
0: input, "t.c", c
1: preprocessor, {0}, cpp-output
2: compiler, {1}, assembler
3: assembler, {2}, object
4: input, "t.s", assembler
5: assembler, {4}, object
6: linker, {3, 5}, image
</pre>
<p>Here the driver is constructing seven distinct actions,
four to compile the "t.c" input into an object file, two to
assemble the "t.s" input, and one to link them together.</p>
<p>A rather different compilation pipeline is shown here; in
this example there are two top level actions to compile
the input files into two separate object files, where each
object file is built using <tt>lipo</tt> to merge results
built for two separate architectures.</p>
<pre>
$ <b>clang -ccc-print-phases -c -arch i386 -arch x86_64 t0.c t1.c</b>
0: input, "t0.c", c
1: preprocessor, {0}, cpp-output
2: compiler, {1}, assembler
3: assembler, {2}, object
4: bind-arch, "i386", {3}, object
5: bind-arch, "x86_64", {3}, object
6: lipo, {4, 5}, object
7: input, "t1.c", c
8: preprocessor, {7}, cpp-output
9: compiler, {8}, assembler
10: assembler, {9}, object
11: bind-arch, "i386", {10}, object
12: bind-arch, "x86_64", {10}, object
13: lipo, {11, 12}, object
</pre>
<p>After this stage is complete the compilation process is
divided into a simple set of actions which need to be
performed to produce intermediate or final outputs (in
some cases, like <tt>-fsyntax-only</tt>, there is no
"real" final output). Phases are well known compilation
steps, such as "preprocess", "compile", "assemble",
"link", etc.</p>
</li>
<li>
<b>Bind: Tool & Filename Selection</b>
<p>This stage (in conjunction with the Translate stage)
turns the tree of Actions into a list of actual subprocess
to run. Conceptually, the driver performs a top down
matching to assign Action(s) to Tools. The ToolChain is
responsible for selecting the tool to perform a particular
action; once selected the driver interacts with the tool
to see if it can match additional actions (for example, by
having an integrated preprocessor).
<p>Once Tools have been selected for all actions, the driver
determines how the tools should be connected (for example,
using an inprocess module, pipes, temporary files, or user
provided filenames). If an output file is required, the
driver also computes the appropriate file name (the suffix
and file location depend on the input types and options
such as <tt>-save-temps</tt>).
<p>The driver interacts with a ToolChain to perform the Tool
bindings. Each ToolChain contains information about all
the tools needed for compilation for a particular
architecture, platform, and operating system. A single
driver invocation may query multiple ToolChains during one
compilation in order to interact with tools for separate
architectures.</p>
<p>The results of this stage are not computed directly, but
the driver can print the results via
the <tt>-ccc-print-bindings</tt> option. For example:</p>
<pre>
$ <b>clang -ccc-print-bindings -arch i386 -arch ppc t0.c</b>
# "i386-apple-darwin9" - "clang", inputs: ["
|
