aboutsummaryrefslogtreecommitdiff
path: root/lib/Transforms/Vectorize
AgeCommit message (Collapse)Author
2013-07-08Fix for a regression caused by the LoopVectorizer whenTom Stellard
vectorizing loops with memory accesses to non-zero address spaces. It simply dropped the AS info. Fixes PR16306. Merged from r184103 Author: Pekka Jaaskelainen <pekka.jaaskelainen@tut.fi> Date: Mon Jun 17 18:49:06 2013 +0000 git-svn-id: https://llvm.org/svn/llvm-project/llvm/branches/release_33@185869 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-04Merging r183035:Bill Wendling
------------------------------------------------------------------------ r183035 | arnolds | 2013-05-31 12:53:50 -0700 (Fri, 31 May 2013) | 7 lines LoopVectorize: PHIs with only outside users should prevent vectorization We check that instructions in the loop don't have outside users (except if they are reduction values). Unfortunately, we skipped this check for if-convertable PHIs. Fixes PR16184. ------------------------------------------------------------------------ git-svn-id: https://llvm.org/svn/llvm-project/llvm/branches/release_33@183189 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-28Merging r182656:Bill Wendling
------------------------------------------------------------------------ r182656 | d0k | 2013-05-24 11:05:35 -0700 (Fri, 24 May 2013) | 3 lines LoopVectorize: LoopSimplify can't canonicalize loops with an indirectbr in it, don't assert on those cases. Fixes PR16139. ------------------------------------------------------------------------ git-svn-id: https://llvm.org/svn/llvm-project/llvm/branches/release_33@182785 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-22Merging r182485:Bill Wendling
------------------------------------------------------------------------ r182485 | arnolds | 2013-05-22 09:54:56 -0700 (Wed, 22 May 2013) | 7 lines LoopVectorize: Make Value pointers that could be RAUW'ed a VH The Value pointers we store in the induction variable list can be RAUW'ed by a call to SCEVExpander::expandCodeFor, use a TrackingVH instead. Do the same thing in some other places where we store pointers that could potentially be RAUW'ed. Fixes PR16073. ------------------------------------------------------------------------ git-svn-id: https://llvm.org/svn/llvm-project/llvm/branches/release_33@182492 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-08Merging r181286:Bill Wendling
------------------------------------------------------------------------ r181286 | arnolds | 2013-05-06 21:37:05 -0700 (Mon, 06 May 2013) | 7 lines LoopVectorize: getConsecutiveVector must respect signed arithmetic We were passing an i32 to ConstantInt::get where an i64 was needed and we must also pass the sign if we pass negatives numbers. The start index passed to getConsecutiveVector must also be signed. Should fix PR15882. ------------------------------------------------------------------------ git-svn-id: https://llvm.org/svn/llvm-project/llvm/branches/release_33@181455 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-06Update the comment to mention that we use TTI.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181178 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-05LoopVectorize: Print values instead of pointers in debug output.Benjamin Kramer
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181157 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-05LoopVectorize: Add support for floating point min/max reductionsArnold Schwaighofer
Add support for min/max reductions when "no-nans-float-math" is enabled. This allows us to assume we have ordered floating point math and treat ordered and unordered predicates equally. radar://13723044 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181144 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-05LoopVectorizer: Cleanup of miminimum/maximum pattern match codeArnold Schwaighofer
No need for setting the operands. The pointers are going to be bound by the matcher. radar://13723044 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181142 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-05LoopVectorize: We don't need an identity element for min/max reductionsArnold Schwaighofer
We can just use the initial element that feeds the reduction. max(max(x, y), z) == max(max(x,y), max(x,z)) radar://13723044 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181141 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-05Add ArrayRef constructor from None, and do the cleanups that this ↵Dmitri Gribenko
constructor enables Patch by Robert Wilhelm. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181138 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-03LoopVectorizer: Add support for if-conversion of PHINodes with 3+ incoming ↵Nadav Rotem
values. By supporting the vectorization of PHINodes with more than two incoming values we can increase the complexity of nested if statements. We can now vectorize this loop: int foo(int *A, int *B, int n) { for (int i=0; i < n; i++) { int x = 9; if (A[i] > B[i]) { if (A[i] > 19) { x = 3; } else if (B[i] < 4 ) { x = 4; } else { x = 5; } } A[i] = x; } } git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181037 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-01This patch breaks up Wrap.h so that it does not have to include all of Filip Pizlo
the things, and renames it to CBindingWrapping.h. I also moved CBindingWrapping.h into Support/. This new file just contains the macros for defining different wrap/unwrap methods. The calls to those macros, as well as any custom wrap/unwrap definitions (like for array of Values for example), are put into corresponding C++ headers. Doing this required some #include surgery, since some .cpp files relied on the fact that including Wrap.h implicitly caused the inclusion of a bunch of other things. This also now means that the C++ headers will include their corresponding C API headers; for example Value.h must include llvm-c/Core.h. I think this is harmless, since the C API headers contain just external function declarations and some C types, so I don't believe there should be any nasty dependency issues here. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180881 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-30Fix a typoNadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180806 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-26LoopVectorizer: Calculate the number of pointers to disambiguate at runtime ↵Nadav Rotem
based on the numbers of reads and writes. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180593 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-25LoopVectorizer: No need to generate pointer disambiguation checks between ↵Nadav Rotem
readonly pointers. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180570 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-24LoopVectorizer: Change variable name Stride to ConsecutiveStrideArnold Schwaighofer
This makes it easier to read the code. No functionality change. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180197 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-24LoopVectorize: Scalarize padded typesArnold Schwaighofer
This patch disables memory-instruction vectorization for types that need padding bytes, e.g., x86_fp80 has 10 bytes store size with 6 bytes padding in darwin on x86_64. Because the load/store vectorization is performed by the bit casting to a packed vector, which has incompatible memory layout due to the lack of padding bytes, the present vectorizer produces inconsistent result for memory instructions of those types. This patch checks an equality of the AllocSize of a scalar type and allocated size for each vector element, to ensure that there is no padding bytes and the array can be read/written using vector operations. Patch by Daisuke Takahashi! Fixes PR15758. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180196 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-24LoopVectorizer: Bail out if we don't have datalayout we need itArnold Schwaighofer
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180195 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-23LoopVectorizer: Fix 15830. When scalarizing and unrolling stores make sure ↵Nadav Rotem
that the order in which the elements are scalarized is the same as the original order. This fixes a miscompilation in FreeBSD's regex library. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180121 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-23Call the potentially costly isAnnotatedParallel() only once. Pekka Jaaskelainen
Made the uniform write test's checks a bit stricter. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180119 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-23Refuse to (even try to) vectorize loops which have uniform writes,Pekka Jaaskelainen
even if erroneously annotated with the parallel loop metadata. Fixes Bug 15794: "Loop Vectorizer: Crashes with the use of llvm.loop.parallel metadata" git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180081 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-22Move C++ code out of the C headers and into either C++ headersEric Christopher
or the C++ files themselves. This enables people to use just a C compiler to interoperate with LLVM. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180063 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-21SLPVectorize: Add support for vectorization of casts.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179975 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-21SLPVectorizer: Fix a bug in the code that scans the tree in search of nodes ↵Nadav Rotem
with multiple users. We did not terminate the switch case and we executed the search routine twice. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179974 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-20Fix PR15800. Do not try to vectorize vectors and structs.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179960 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-20VecUtils: Clean up uses of dyn_cast.Benjamin Kramer
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179936 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-20SLPVectorizer: Strength reduce SmallVectors to ArrayRefs.Benjamin Kramer
Avoids a couple of copies and allows more flexibility in the clients. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179935 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-20SLPVectorizer: Reduce the compile time by eliminating the search for some of ↵Nadav Rotem
the more expensive patterns. After this change will only check basic arithmetic trees that start at cmpinstr. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179933 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-20refactor tryToVectorizePair to a new method that supports vectorization of ↵Nadav Rotem
lists. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179932 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-20Fix an unused variable warning.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179931 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-20SLPVectorizer: Improve the cost model for loop invariant broadcast values.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179930 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-20Report the number of stores that were found in the debug message.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179929 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-20Fix the header comment.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179928 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-20Use 64bit arithmetic for calculating distance between pointers.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179927 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-19LoopVectorizer: Use matcher from PatternMatch.h for the min/max patternsArnold Schwaighofer
Also make some static function class functions to avoid having to mention the class namespace for enums all the time. No functionality change intended. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179886 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-18Fix a -Wdocumentation warningDmitri Gribenko
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179789 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-18LoopVectorizer: Recognize min/max reductionsArnold Schwaighofer
A min/max operation is represented by a select(cmp(lt/le/gt/ge, X, Y), X, Y) sequence in LLVM. If we see such a sequence we can treat it just as any other commutative binary instruction and reduce it. This appears to help bzip2 by about 1.5% on an imac12,2. radar://12960601 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179773 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-18LoopVectorize: Use a set to avoid longer cycles in the reduction chain too.Benjamin Kramer
Fixes PR15748. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179757 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-15SLPVectorizer: Make it a function pass and add code for hoisting the ↵Nadav Rotem
vector-gather sequence out of loops. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179562 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-15SLPVectorizer: Add support for vectorizing trees that start at compare ↵Nadav Rotem
instructions. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179504 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-14Miscellaneous cleanups for VecUtils.hBenjamin Kramer
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179483 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-14SLP: Document the scalarization cost method.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179479 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-14SLPVectorizer: Add support for trees that don't start at binary operators, ↵Nadav Rotem
and add the cost of extracting values from the roots of the tree. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179475 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-14SLPVectorizer: add initial support for reduction variable vectorization.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179470 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-12SLPVectorizer: add support for vectorization of diamond shaped trees. We now ↵Nadav Rotem
perform a preliminary traversal of the graph to collect values with multiple users and check where the users came from. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179414 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-12Add debug prints.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179412 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-12LoopVectorizer: integer division is not a reduction operationArnold Schwaighofer
Don't classify idiv/udiv as a reduction operation. Integer division is lossy. For example : (1 / 2) * 4 != 4/2. Example: int a[] = { 2, 5, 2, 2} int x = 80; for() x /= a[i]; Scalar: x /= 2 // = 40 x /= 5 // = 8 x /= 2 // = 4 x /= 2 // = 2 Vectorized: <80, 1> / <2,5> //= <40,0> <40, 0> / <2,2> //= <20,0> 20*0 = 0 radar://13640654 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179381 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-11Rename the C function to create a SLPVectorizerPass to something sane and ↵Benjamin Kramer
expose it in the header file. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179272 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-10Make the SLP store-merger less paranoid about function calls. We check for ↵Nadav Rotem
function calls when we check if it is safe to sink instructions. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179207 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.2.3-18-g5258 (git 2.32.0) at 2025-04-29 19:16:46 +0000