aboutsummaryrefslogtreecommitdiff
path: root/test/Transforms/LoopVectorize
AgeCommit message (Collapse)Author
2013-01-11ARM Cost Model: Modify the target independent cost model to askNadav Rotem
the target if it supports the different CAST types. We didn't do this on X86 because of the different register sizes and types, but on ARM this makes sense. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172245 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-11ARM Cost Model: We need to detect the max bitwidth of types in the loop in ↵Nadav Rotem
order to select the max vectorization factor. We don't have a detailed analysis on which values are vectorized and which stay scalars in the vectorized loop so we use another method. We look at reduction variables, loads and stores, which are the only ways to get information in and out of loop iterations. If the data types are extended and truncated then the cost model will catch the cost of the vector zext/sext/trunc operations. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172178 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-10LoopVectorizer: Fix a bug in the vectorization of BinaryOperators. The ↵Nadav Rotem
BinaryOperator can be folded to an Undef, and we don't want to set NSW flags to undef vals. PR14878 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172079 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-09ARM Cost model: Use the size of vector registers and widest vectorizable ↵Nadav Rotem
instruction to determine the max vectorization factor. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172010 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-09ARM Cost Model: Add a basic vectorization unrolling test.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171931 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-09Remove the -licm pass from the loop vectorizer test because the loop ↵Nadav Rotem
vectorizer does it now. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171930 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-09Cost Model: Move the 'max unroll factor' variable to the TTI and add initial ↵Nadav Rotem
Cost Model support on ARM. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171928 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-07LoopVectorizer: Add support for floating point reductionsNadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171812 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-07LoopVectorizer: When we vectorizer and widen loops we process many elements ↵Nadav Rotem
at once. This is a good thing, except for small loops. On small loops post-loop that handles scalars (and runs slower) can take more time to execute than the rest of the loop. This patch disables widening of loops with a small static trip count. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171798 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-05Fix a typo. Remove the duplicated test.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171584 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-05iLoopVectorize: Non commutative operators can be used as reduction variables ↵Nadav Rotem
as long as the reduction chain is used in the LHS. PR14803. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171583 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-05Force a fixed unroll count on the target independent tests.Nadav Rotem
This should fix clang-native-arm-cortex-a9. Thanks Renato. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171582 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-04Do not vectorize loops with subtraction reductionsPaul Redmond
Since subtraction does not commute the loop vectorizer incorrectly vectorizes reductions such as x = A[i] - x. Disabling for now. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171537 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-04LoopVectorizer:Nadav Rotem
1. Add code to estimate register pressure. 2. Add code to select the unroll factor based on register pressure. 3. Add bits to TargetTransformInfo to provide the number of registers. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171469 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-03LoopVectorizer: Test the unrolling flag.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171446 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-02Avoid vectorization when the function has the "noimplicitflot" attribute.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171429 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-30LoopVectorizer: Fix a bug in the code that updates the loop exiting block.Nadav Rotem
LCSSA PHIs may have undef values. The vectorizer updates values that are used by outside users such as PHIs. The bug happened because undefs are not loop values. This patch handles these PHIs. PR14725 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171251 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-26If all of the write objects are identified then we can vectorize the loop ↵Nadav Rotem
even if the read objects are unidentified. PR14719. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171124 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-26LoopVectorizer: Optimize the vectorization of consecutive memory access when ↵Nadav Rotem
the iteration step is -1 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171114 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-25LoopVectorize: Enable vectorization of the fmuladd intrinsicHal Finkel
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171076 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-24Fix typo "Makre" -> "Make".Nick Lewycky
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171043 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-24LoopVectorizer: When checking for vectorizable types, also checkNadav Rotem
the StoreInst operands. PR14705. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171023 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-24LoopVectorizer: Fix an endless loop in the code that looks for reductions.Nadav Rotem
The bug was in the code that detects PHIs in if-then-else block sequence. PR14701. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171008 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-23CostModel: Change the default target-independent implementation for findingNadav Rotem
the cost of arithmetic functions. We now assume that the cost of arithmetic operations that are marked as Legal or Promote is low, but ops that are marked as custom are higher. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171002 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-23Loop Vectorizer: Update the cost model of scatter/gather operations and makeNadav Rotem
them more expensive. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170995 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-21Fix a bug in the code that checks if we can vectorize loops while using dynamicNadav Rotem
memory bound checks. Before the fix we were able to vectorize this loop from the Livermore Loops benchmark: for ( k=1 ; k<n ; k++ ) x[k] = x[k-1] + y[k]; git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170811 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-20LoopVectorize: Fix a bug in the scalarization of instructions.Nadav Rotem
Before if-conversion we could check if a value is loop invariant if it was declared inside the basic block. Now that loops have multiple blocks this check is incorrect. This fixes External/SPEC/CINT95/099_go/099_go git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170756 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-19Make TargetLowering::getTypeConversion more resilient against odd illegal MVTs.Benjamin Kramer
- An MVT can become an EVT when being split (e.g. v2i8 -> v1i8, the latter doesn't exist) - Return the scalar value when an MVT is scalarized (v1i64 -> i64) Fixes PR14639ff. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170546 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-18LoopVectorize: Emit reductions as log2(vectorsize) shuffles + vector ops ↵Benjamin Kramer
instead of scalar operations. For example on x86 with SSE4.2 a <8 x i8> add reduction becomes movdqa %xmm0, %xmm1 movhlps %xmm1, %xmm1 ## xmm1 = xmm1[1,1] paddw %xmm0, %xmm1 pshufd $1, %xmm1, %xmm0 ## xmm0 = xmm1[1,0,0,0] paddw %xmm1, %xmm0 phaddw %xmm0, %xmm0 pextrb $0, %xmm0, %edx instead of pextrb $2, %xmm0, %esi pextrb $0, %xmm0, %edx addb %sil, %dl pextrb $4, %xmm0, %esi addb %dl, %sil pextrb $6, %xmm0, %edx addb %sil, %dl pextrb $8, %xmm0, %esi addb %dl, %sil pextrb $10, %xmm0, %edi pextrb $14, %xmm0, %edx addb %sil, %dil pextrb $12, %xmm0, %esi addb %dil, %sil addb %sil, %dl git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170439 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-13Teach the cost model about the optimization in r169904: Truncation of ↵Nadav Rotem
induction variables costs the same as scalar trunc. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170051 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-12LoopVectorizer: Use the "optsize" attribute to decide if we are allowed to ↵Nadav Rotem
increase the function size. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170004 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-11PR14574. Fix a bug in the code that calculates the mask the converted PHIs ↵Nadav Rotem
in if-conversion. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169916 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-11Loop Vectorize: optimize the vectorization of trunc(induction_var). The ↵Nadav Rotem
truncation is now done on scalars. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169904 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-11Fix PR14565. Don't if-convert loops that have switch statements in them.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169813 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-10Add support for reverse induction variables. For example:Nadav Rotem
while (i--) sum+=A[i]; git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169752 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-09LoopVectorize: support vectorizing intrinsic callsPaul Redmond
- added function to VectorTargetTransformInfo to query cost of intrinsics - vectorize trivially vectorizable intrinsic calls such as sin, cos, log, etc. Reviewed by: Nadav git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169711 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04Fix a bug in vectorization of if-converted reduction variables. If theNadav Rotem
reduction variable is not used outside the loop then we ran into an endless loop. This change checks if we found the original PHI. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169324 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04Add support for reduction variables when IF-conversion is enabled. Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169288 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04Add the last part that is needed for vectorization of if-converted code.Nadav Rotem
Added the code that actually performs the if-conversion during vectorization. We can now vectorize this code: for (int i=0; i<n; ++i) { unsigned k = 0; if (a[i] > b[i]) <------ IF inside the loop. k = k * 5 + 3; a[i] = k; <---- K is a phi node that becomes vector-select. } git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169217 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-25Add support for pointer induction variables even when there is no integer ↵Nadav Rotem
induction variable. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168558 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-17LoopVectorizer: Add initial support for pointer induction variables (for ↵Nadav Rotem
example: *dst++ = *src++). At the moment we still require to have an integer induction variable (for example: i++). git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168231 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-13Relax the restrictions on vector of pointer types, and vector getelementptr.Duncan Sands
Previously in a vector of pointers, the pointer couldn't be any pointer type, it had to be a pointer to an integer or floating point type. This is a hassle for dragonegg because the GCC vectorizer happily produces vectors of pointers where the pointer is a pointer to a struct or whatever. Vector getelementptr was restricted to just one index, but now that vectors of pointers can have any pointer type it is more natural to allow arbitrary vector getelementptrs. There is however the issue of struct GEPs, where if each lane chose different struct fields then from that point on each lane will be working down into unrelated types. This seems like too much pain for too little gain, so when you have a vector struct index all the elements are required to be the same. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167828 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-09Add support for memory runtime check. When we can, we calculate array bounds.Nadav Rotem
If the arrays are found to be disjoint then we run the vectorized version of the loop. If they are not, we run the scalar code. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167608 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-06Cost Model: add tables for some avx type-conversion hacks.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167480 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-05Code Model: Improve the accuracy of the zext/sext/trunc vector cost estimation.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167412 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-05Implement the cost of abnormal x86 instruction lowering as a table.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167395 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31LoopVectorize: Preserve NSW, NUW and IsExact flags.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167174 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31Fix a bug in the cost calculation of vector casts. Detect situations where ↵Nadav Rotem
bitcasts cost zero. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167170 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31LoopVectorize: Do not vectorize loops with tiny constant trip counts.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167101 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31Add support for loops that don't start with Zero.Nadav Rotem
This is important for loops in the LAPACK test-suite. These loops start at 1 because they are auto-converted from fortran. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167084 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.2.3-18-g5258 (git 2.32.0) at 2025-05-01 22:13:52 +0000