1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<title>How To Release LLVM To The Public</title>
<link rel="stylesheet" href="llvm.css" type="text/css">
</head>
<body>
<div class="doc_title">How To Release LLVM To The Public</div>
<p class="doc_warning">NOTE: THIS DOCUMENT IS A WORK IN PROGRESS!</p>
<ol>
<li><a href="#introduction">Introduction</a></li>
<li><a href="#process">Release Process</a></li>
<li><a href="#dist_targets">Distribution Targets</a></li>
</ol>
<div class="doc_author">
<p>Written by <a href="mailto:rspencer@x10sys.com">Reid Spencer</a>,
<a href="mailto:criswell@cs.uiuc.edu">John Criswell</a></p>
</div>
<!-- *********************************************************************** -->
<div class="doc_section"><a name="introduction">Introduction</a></div>
<!-- *********************************************************************** -->
<div class="doc_text">
<p>This document collects information about successfully releasing LLVM to the
public. It is the release manager's guide to ensuring that a high quality build
of LLVM is released. Mostly, it's just a bunch of reminders of things to do at
release time so we don't inadvertently ship something that is utility
deficient.</p>
<p>
There are three main tasks for building a release of LLVM:
<ol>
<li>Create the LLVM source distribution.</li>
<li>Create the LLVM GCC source distribtuion.</li>
<li>Create a set of LLVM GCC binary distribtuions for each supported
platform. These binary distributions must include compiled versions
of the libraries found in <tt>llvm/runtime</tt> from the LLVM
source distribution created in Step 1.</li>
</ol>
</p>
</div>
<!-- *********************************************************************** -->
<div class="doc_section"><a name="process">Release Process</a></div>
<!-- *********************************************************************** -->
<!-- ======================================================================= -->
<div class="doc_subsection"><a name="overview">Process Overview</a></div>
<div class="doc_text">
<ol>
<li><a href="#updocs">Update Documentation</a></li>
<li><a href="#merge">Merge Branches</a></li>
<li><a href="#deps">Make LibDeps.txt</a></li>
<li><a href="#settle">Settle LLVM HEAD</a></li>
<li><a href="#tag">Tag LLVM and Create the Release Branch</a></li>
<li><a href="#verchanges">Update LLVM Version </a></li>
<li><a href="#build">Build LLVM</a></li>
<li><a href="#check">Run 'make check'</a></li>
<li><a href="#test">Run LLVM Test Suite</a></li>
<li><a href="#dist">Build the LLVM Source Distributions</a></li>
<li><a href="#rpm">Build RPM Packages (optional)</a></li>
<li><a href="#llvmgccbin">Build the LLVM GCC Binary Distribution</a></li>
<li><a href="#webupdates">Update the LLVM Website</a></li>
</ol>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection"><a name="updocs">Update Documentation</a></div>
<div class="doc_text">
<p>
Review the documentation and ensure that it is up to date. The Release Notes
must be updated to reflect bug fixes, new known issues, and changes in the
list of supported platforms. The Getting Started Guide should be updated to
reflect the new release version number tag avaiable from CVS and changes in
basic system requirements.
</p>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection"><a name="merge">Merge Branches</a></div>
<div class="doc_text">
<p>
Merge any work done on branches intended for release into mainline. Finish and
commit all new features or bug fixes that are scheduled to go into the release.
Work that is not to be incorporated into the release should not be merged from
branchs or commited from developer's working directories.
</p>
<p>
From this point until the release branch is created, developers should
<em>not</em>
commit changes to the llvm and llvm-gcc CVS repositories unless it is a bug
fix <em>for the release</em>.
</p>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection"><a name="deps">Make LibDeps.txt</a></div>
<div class="doc_text">
<p>Rebuild the <tt>LibDeps.txt</tt> target in <tt>utils/llvm-config</tt>. This
makes sure that the <tt>llvm-config</tt> utility remains relevant for the
release, reflecting any changes in the library dependencies.</p>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection"><a name="settle">Settle CVS HEAD</a></div>
<div class="doc_text">
<p>
Use the nightly test reports and 'make check' (deja-gnu based tests) to
ensure that recent changes and merged branches have not destabilized LLVM.
Platforms which are used less often should be given special attention as they
are the most likely to break from commits from the previous step.
</p>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection"><a name="tag">CVS Tag And Branch</a></div>
<div class="doc_text">
<p>Tag and branch the CVS HEAD using the following procedure:</p>
<ol>
<li>
Request all developers to refrain from committing. Offenders get commit
rights taken away (temporarily).
</li>
<li>
The Release Manager updates his/her llvm, llvm-test, and llvm-gcc source
trees with the
latest sources from mainline CVS. The Release Manage may want to consider
using a new working directory for this to keep current uncommitted work
separate from release work.
</li>
<li>
The Release Manager tags his/her llvm, llvm-test, and llvm-gcc working
directories with
"ROOT_RELEASE_XX" where XX is the major and minor
release numbers. So, for Release 1.2, XX=12 and for Release 1.10, XX=110.
<p>
<tt>cvs tag ROOT_RELEASE_XX</tt><br>
</p>
</li>
<li>
Immediately create cvs branches based on the ROOT_RELEASE_XX tag. The tag
should be "release_XX" (where XX matches that used for the ROOT_RELEASE_XX
tag). This is where the release distribution will be created.
<p>
cvs tag -b -r ROOT_RELEASE_XX release_XX
</p>
</li>
<li>
Advise developers they can work on CVS HEAD again.
</li>
<li>
The Release Manager and any developers working on the release should switch
to the release branch (as all changes to the release will now be done in
the branch). The easiest way to do this is to grab another working copy
using the following commands:
<p>
<tt>cvs -d <CVS Repository> co -r release_XX llvm</tt><br>
<tt>cvs -d <CVS Repository> co -r release_XX llvm-test</tt><br>
<tt>cvs -d <CVS Repository> co -r release_XX llvm-gcc</tt><br>
</p>
</li>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection"><a name="verchanges">Update LLVM Version</a></div>
<div class="doc_text">
<p>
After creating the llvm release branch, update the release branch's autoconf/configure.ac
version from X.Xsvn to just X.X. Update it on mainline as well to be the next version
(X.X+1svn).
</p>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection"><a name="build">Build LLVM</a></div>
<div class="doc_text">
<p>
Build both debug and release (optimized) versions of LLVM on all
platforms. Ensure the build is warning and error free on each platform.
</p>
<p>
Build a new version of the LLVM GCC front-end after building the LLVM tools.
Once that is complete, go back to the LLVM source tree and build and install
the <tt>llvm/runtime</tt> libraries.
</p>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection"><a name="check">Run 'make check'</a></div>
<div class="doc_text">
<p>Run <tt>make check</tt> and ensure there are no unexpected failures. If
there are, resolve the failures, commit them back into the release branch,
and restart testing by <a href="#build">re-building LLVM</a>.
</p>
<p>
Ensure that 'make check' passes on all platforms for all targets. If certain
failures cannot be resolved before release time, determine if marking them
XFAIL is appropriate. If not, fix the bug and go back. The test suite must
complete with "0 unexpected failures" for release.
</p>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection"><a name="test">LLVM Test Suite</a></div>
<div class="doc_text">
<p>Run the llvm-test suite and ensure there are no unacceptable failures.
If there are, resolve the failures and go back to
<a href="#build">re-building LLVM</a>. The test suite
should be run in Nightly Test mode. All tests must pass.
</div>
<!-- ======================================================================= -->
<div class="doc_subsection"><a name="dist">Build the LLVM Source Distributions</a></div>
<div class="doc_text">
<p>
Create source distributions for LLVM, LLVM GCC, and the LLVM Test Suite by
exporting the source
from CVS and archiving it. This can be done with the following commands:
</p>
<p>
<tt>cvs -d <CVS Repository> export -r release_XX llvm</tt><br>
<tt>cvs -d <CVS Repository> export -r release_XX llvm-test</tt><br>
<tt>cvs -d <CVS Repository> export -r release_XX llvm-gcc</tt><br>
<tt>mkdir cfrontend; mv llvm-gcc cfrontend/src</tt><br>
<tt>tar -cvf - llvm | gzip > llvm-X.X.tar.gz</tt><br>
<tt>tar -cvf - llvm-test | gzip > llvm-test-X.X.tar.gz</tt><br>
<tt>tar -cvf - cfrontend/src | gzip > cfrontend-X.X.source.tar.gz</tt><br>
</p>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection"><a name="rpm">Building RPM packages (optional)</a></div>
<div class="doc_text">
<p>You can, optionally, create source and binary RPM packages for LLVM. These
may make it easier to get LLVM into a distribution. This can be done with
the following commands:</p>
<pre>
make dist # Build the distribution source tarball
make dist-check # Check that the source tarball can build itself.
cp llvm-M.m.tar.gz /usr/src/redhat/SOURCES # Required by rpmbuild
make srpm # for source rpm
make rpm # for binary rpm
</pre>
<p>First, use "make dist" to simply build the distribution. Any
failures need to be corrected (on the branch). Once "make dist" can be
successful, do "make dist-check". This target will do the same thing as the
'dist' target but also test that distribution to make sure it can build itself
and runs "make check" as well. This ensures that needed files are not
missing and that the src tarball can be successfully unpacked, built,
installed, and cleaned. Once you have a reliable tarball, you need to copy
it to the /usr/src/redhat/SOURCES directory which is a requirement of the
rpmbuild tool. The last two "make" invocations just run rpmbuild to build
either a source (<tt>srpm</tt>) or binary (<tt>rpm</tt>) RPM package.</p>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection"><a name="llvmgccbin">Build the LLVM GCC Binary Distribution</a></div>
<div class="doc_text">
<p>
Creating the LLVM GCC binary distribution requires performing the following
steps for each supported platform:
</p>
<ol>
<li>
Build the LLVM GCC front-end. The LLVM GCC front-end must be installed in
a directory named <tt>cfrontend/<platform>/llvm-gcc</tt>. For
example, the Sparc/Solaris directory is named
<tt>cfrontend/sparc/llvm-gcc</tt>.
</li>
<li>
Build the libraries in <tt>llvm/runtime</tt> and install them into the
created LLVM GCC installation directory.
</li>
<li>
For systems with non-distributable header files (e.g. Solaris), manually
remove header files that the GCC build process has "fixed." This process
is admittedly painful, but not as bad as it looks; these header files are
almost always easily identifiable with simple grep expressions and are
installed in only a few directories in the GCC installation directory.
</li>
<li>
Add the copyright files and header file fix script.
</li>
<li>
Archive and compress the installation directory. These can be found in
previous releases of the LLVM-GCC front-end.
</li>
</ol>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection"><a name="webupdates">Update the LLVM Website</a></div>
<div class="doc_text">
<p>
Check out the <tt>website</tt> module from Subversion. Create a new
subdirectory X.X in the releases directory. Place the llvm, llvm-test,
llvm-gcc source, and llvm-gcc
binaries in this new directory. Copy the llvm/docs and LICENSE.txt files
into this new directory. Update the releases/download.html file with the new release.
Update the releases/index.html with the new release. Finally, update the main page (
index.html and sidebar) to point to the new release and release announcement. Make
sure this all gets commited back into cvs.
</p>
</div>
<!--
<div class="doc_subsection"><a name="release">Release</a></div>
<div class="doc_text">
<p>Release the distribution tarball to the public. This consists of generating
several tarballs. The first set, the source distributions, are automatically
generated by the "make dist" and "make dist-check". There are gzip, bzip2, and
zip versions of these bundles.</p>
<p>The second set of tarballs is the binary release. When "make dist-check"
succeeds, it will have created an _install directory into which it installed
the binary release. You need to rename that directory as "llvm" and then
create tarballs from the contents of that "llvm" directory.</p>
<p>Finally, use rpm to make an rpm package based on the llvm.spec file. Don't
forget to update the version number, documentation, etc. in the llvm.spec
file.</p>
</div>
-->
<!-- *********************************************************************** -->
<div class="doc_section"><a name="dist_targets">Distribution Targets</a></div>
<!-- *********************************************************************** -->
<!-- ======================================================================= -->
<div class="doc_subsection">Overview</div>
<div class="doc_text">
<p>The first thing you need to understand is that there are multiple make
targets to support this feature. Here's an overview, we'll delve into the
details later.</p>
<ul>
<li><b>distdir</b> - builds the distribution directory from which the
distribution will be packaged</li>
<li><b>dist</b> - builds each of the distribution tarballs (tar.gz,
tar.bzip2, .zip). These can be built individually as well, with separate
targets.</li>
<li><b>dist-check</b> - this is identical to <tt>dist</tt> but includes a
check on the distribution that ensures the tarball can: unpack successfully,
compile correctly, pass 'make check', and pass 'make clean'.</li>
<li><b>dist-clean</b>- this just does a normal clean but also cleans up the
stuff generated by the other three <tt>dist</tt> targets (above).</li>
</ul>
<p>Okay, that's the basic functionality. When making a release, we want to
ensure that the tree you build the distribution from passes
<tt>dist-check</tt>. Beyond fixing the usual bugs, there is generally one
impediment to making the release in this fashion: missing files. The
<tt>dist-check</tt> process guards against that possibility. It will either
fail and that failure will indicate what's missing, or it will succeed
meaning that it has proved that the tarballs can actually succeed in
building LLVM correctly and that it passes <tt>make check</tt>.</p>
<!-- ======================================================================= -->
<div class="doc_subsection">distdir</div>
<p>This target builds the distribution directory which is the directory from
which the tarballs are generated. The distribution directory has the same
name as the release, e.g. LLVM-1.7). This target goes through the following
process:
<ol>
<li>First, if there was an old distribution directory (for the current
release), it is removed in its entirety and you see <tt>Removing old
LLVM-1.7</tt></li>
<li>Second, it issues a <tt>make all ENABLE_OPTIMIZED=3D1</tt> to ensure
that the everything in your tree can be built in release mode. Often times
there are discrepancies in building between debug and release modes so it
enforces release mode first. If that fails, the <tt>distdir</tt> target
fails too. This is preceded by the message <tt>Making 'all' to verify
build</tt>.</li>
<li>Next, it traverses your source tree and copies it to a new directory
that has the name of the release (<tt>LLVM-M.m</tt> in our current case).
This is the directory that will get tar'd. It contains all the software
that needs to be in the distribution. During the copying process, it omits
generated files, CVS directories, and any other "cruft" that's in your
build tree. This is done to eliminate the possibility of huge distribution
tarballs that include useless or irrelevant stuff in them. This is the
trickiest part of making the distribution. Done manually you will either
include stuff that shouldn't be in the distribution or exclude stuff that
should. This step is preceded by the message <tt>Building Distribution
Directory LLVM-1.7</tt></li>
<li>The distribution directory is then traversed and all <tt>CVS</tt> or
<tt>.svn</tt> directories are removed. You see: <tt>Eliminating CVS/.svn
directories from distribution</tt></li>
<li>The recursive <tt>dist-hook</tt> target is executed. This gives each
directory a chance to modify the distribution in some way (more on this
below).</li>
<li>The distribution directory is traversed and the correct file
permissions and modes are set based on the type of file.</li>
</ol>
<p>To control the process of making the distribution directory correctly,
each Makefile can utilize two features:</p>
<ol>
<li><b><tt>EXTRA_DIST</tt></B> - this make variable specifies which files
it should distribute. By default, all source files are automatically
included for distribution as well as certain <tt>well known</tt> files
(see DistAlways variable in Makefile.rules for details). Each Makefile
specifies, via the <tt>EXTRA_DIST</tt> variable, which additional files
need to be distributed. Only those files that are needed to build LLVM
should be added to <tt>EXTRA_DIST</tt>. <tt>EXTRA_DIST</tt> contains a
list of file or directory names that should be distributed. For example,
the top level Makefile contains
<tt>EXTRA_DIST := test llvm.spec include</tt>.
This means that in addition to regular things that are distributed at the
top level (<tt>CREDITS.txt, LICENSE.txt</tt>, etc.) the distribution should
contain the entire <tt>test</tt> and <tt>include</tt> directories as well
as the <tt>llvm.spec</tt> file.</li>
<li><b><tt>dist-hook</tt></B> - this make target can be used to alter the
content of the distribution directory. For example, in the top level
Makefile there is some logic to eliminate files in the <tt>include</tt>
subtree that are generated by the configure script. These should not be
distributed. Similarly, any <tt>dist-hook</tt> target found in any
directory can add or remove or modify things just before it gets packaged.
Any transformation is permitted. Generally, not much is needed.
</ol>
<p>You will see various messages if things go wrong:</p>
<ol>
<li>During the copying process, any files that are missing will be flagged
with: <tt>===== WARNING: Distribution Source 'dir/file' Not Found!</tt>
These must be corrected by either adding the file or removing it from
<tt>EXTRA_DIST</tt>.
<li>If you build the distribution with <tt>VERBOSE=1</tt>, then you might
also see: <tt>Skipping non-existent 'dir/file'</tt> in certain cases where
its okay to skip the file.</li>
<li>The target can fail if any of the things it does fail. Error messages
should indicate what went wrong.</li>
</ol>
<!-- ======================================================================= -->
<div class="doc_subsection">dist</div>
<p>This target does exactly what <tt>distdir</tt> target does, but also
includes assembling the tarballs. There are actually four related targets
here:<p>
<ul>
<li><b><tt>dist-gzip</tt></b>: package the gzipped distribution tar
file. The distribution directory is packaged into a single file ending in
<tt>.tar.gz</tt> which is gzip compressed.</li>
<li><b><tt>dist-bzip2</tt></b>: package the bzip2 distribution tar file.
The distribution directory is packaged into a single file ending in
<tt>.tar.bzip2</tt> which is bzip2 compressed.</li>
<li><b><tt>dist-zip</tt></b>: package the zip distribution file. The
distribution directory is packaged into a single file ending in
<tt>.zip</tt> which is zip compressed.</li>
<li><b><tt>dist</tt></b>: does all three, dist-gzip, dist-bzip2,
dist-zip</li>
</ul>
<!-- ======================================================================= -->
<div class="doc_subsection">dist-check</div>
<p>This target checks the distribution. The basic idea is that it unpacks the
distribution tarball and ensures that it can build. It takes the following
actions:</p>
<ol>
<li>It depends on the <tt>dist-gzip</tt> target which, if it hasn't already
been built, builds the gzip tar bundle (see dist and distdir above).</li>
<li>removes any pre-existing <tt>_distcheckdir</tt> at the top level.</li>
<li>creates a new <tt>_distcheckdir</tt> directory at the top level.</li>
<li>creates a <tt>build</tt> subdirectory and an <tt>install</tt>
subdirectory under <tt>_distcheckdir</tt>.</li>
<li>unzips and untars the release tarball into <tt>_distcheckdir</tt>,
creating <tt>LLVM-1.7</tt> directory (from the tarball).</li>
<li>in the build subdirectory, it configures with appropriate options to build
from the unpacked source tarball into the <tt>build</tt> directory with
installation in the <tt>install</tt> directory.</li>
<li>runs <tt>make all</tt></li>
<li>runs <tt>make </tt><tt>check</tt></li>
<li>runs <tt>make install</tt></li>
<li>runs <tt>make uninstall</tt></li>
<li>runs <tt>make dist</tt></li>
<li>runs <tt>make clean</tt></li>
<li>runs <tt>make dist-clean</tt></li>
</ol>
<p>If it can pass all that, the distribution will be deemed distribution
worth y and you will see:<p>
<pre>===== LLVM-1.7.tar.gz Ready For Distribution =====</pre>
<p>This means the tarball should then be tested on other platforms and have the
nightly test run against it. If those all pass, THEN it is ready for
distribution.</p>
<p>
A note about disk space: using <tt>dist-check</tt> will easily triple the
amount of disk space your build tree is using. You might want to check
available space before you begin.</p>
<!-- ======================================================================= -->
<div class="doc_subsection">dist-clean</div>
<h2>dist-clean</h2>
<p>In addition to doing a normal <tt>clean</tt>, this target will clean up the
files and directories created by the distribution targets. In particular the
distribution directory <tt>(LLVM-X.X</tt>), check directory
(<tt>_distcheckdir</tt>), and the various tarballs will be removed. You do
this after the release has shipped and you no longer need this stuff in your
build tree.</p>
</div>
<!-- *********************************************************************** -->
<hr>
<address>
<a href="http://jigsaw.w3.org/css-validator/check/referer"><img
src="http://jigsaw.w3.org/css-validator/images/vcss" alt="Valid CSS!"></a>
<a href="http://validator.w3.org/check/referer"><img
src="http://www.w3.org/Icons/valid-html401" alt="Valid HTML 4.01!" /></a>
<a href="mailto:rspencer@x10sys.com">Reid Spencer</a><br>
<a href="http://llvm.cs.uiuc.edu">The LLVM Compiler Infrastructure</a>
<br/>
Last modified: $Date$
</address>
</body>
</html>
|
