aboutsummaryrefslogtreecommitdiff
path: root/test/Transforms/LoopVectorize
AgeCommit message (Collapse)Author
2012-12-23CostModel: Change the default target-independent implementation for findingNadav Rotem
the cost of arithmetic functions. We now assume that the cost of arithmetic operations that are marked as Legal or Promote is low, but ops that are marked as custom are higher. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171002 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-23Loop Vectorizer: Update the cost model of scatter/gather operations and makeNadav Rotem
them more expensive. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170995 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-21Fix a bug in the code that checks if we can vectorize loops while using dynamicNadav Rotem
memory bound checks. Before the fix we were able to vectorize this loop from the Livermore Loops benchmark: for ( k=1 ; k<n ; k++ ) x[k] = x[k-1] + y[k]; git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170811 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-20LoopVectorize: Fix a bug in the scalarization of instructions.Nadav Rotem
Before if-conversion we could check if a value is loop invariant if it was declared inside the basic block. Now that loops have multiple blocks this check is incorrect. This fixes External/SPEC/CINT95/099_go/099_go git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170756 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-19Make TargetLowering::getTypeConversion more resilient against odd illegal MVTs.Benjamin Kramer
- An MVT can become an EVT when being split (e.g. v2i8 -> v1i8, the latter doesn't exist) - Return the scalar value when an MVT is scalarized (v1i64 -> i64) Fixes PR14639ff. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170546 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-18LoopVectorize: Emit reductions as log2(vectorsize) shuffles + vector ops ↵Benjamin Kramer
instead of scalar operations. For example on x86 with SSE4.2 a <8 x i8> add reduction becomes movdqa %xmm0, %xmm1 movhlps %xmm1, %xmm1 ## xmm1 = xmm1[1,1] paddw %xmm0, %xmm1 pshufd $1, %xmm1, %xmm0 ## xmm0 = xmm1[1,0,0,0] paddw %xmm1, %xmm0 phaddw %xmm0, %xmm0 pextrb $0, %xmm0, %edx instead of pextrb $2, %xmm0, %esi pextrb $0, %xmm0, %edx addb %sil, %dl pextrb $4, %xmm0, %esi addb %dl, %sil pextrb $6, %xmm0, %edx addb %sil, %dl pextrb $8, %xmm0, %esi addb %dl, %sil pextrb $10, %xmm0, %edi pextrb $14, %xmm0, %edx addb %sil, %dil pextrb $12, %xmm0, %esi addb %dil, %sil addb %sil, %dl git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170439 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-13Teach the cost model about the optimization in r169904: Truncation of ↵Nadav Rotem
induction variables costs the same as scalar trunc. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170051 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-12LoopVectorizer: Use the "optsize" attribute to decide if we are allowed to ↵Nadav Rotem
increase the function size. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170004 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-11PR14574. Fix a bug in the code that calculates the mask the converted PHIs ↵Nadav Rotem
in if-conversion. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169916 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-11Loop Vectorize: optimize the vectorization of trunc(induction_var). The ↵Nadav Rotem
truncation is now done on scalars. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169904 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-11Fix PR14565. Don't if-convert loops that have switch statements in them.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169813 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-10Add support for reverse induction variables. For example:Nadav Rotem
while (i--) sum+=A[i]; git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169752 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-09LoopVectorize: support vectorizing intrinsic callsPaul Redmond
- added function to VectorTargetTransformInfo to query cost of intrinsics - vectorize trivially vectorizable intrinsic calls such as sin, cos, log, etc. Reviewed by: Nadav git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169711 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04Fix a bug in vectorization of if-converted reduction variables. If theNadav Rotem
reduction variable is not used outside the loop then we ran into an endless loop. This change checks if we found the original PHI. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169324 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04Add support for reduction variables when IF-conversion is enabled. Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169288 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04Add the last part that is needed for vectorization of if-converted code.Nadav Rotem
Added the code that actually performs the if-conversion during vectorization. We can now vectorize this code: for (int i=0; i<n; ++i) { unsigned k = 0; if (a[i] > b[i]) <------ IF inside the loop. k = k * 5 + 3; a[i] = k; <---- K is a phi node that becomes vector-select. } git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169217 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-25Add support for pointer induction variables even when there is no integer ↵Nadav Rotem
induction variable. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168558 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-17LoopVectorizer: Add initial support for pointer induction variables (for ↵Nadav Rotem
example: *dst++ = *src++). At the moment we still require to have an integer induction variable (for example: i++). git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168231 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-13Relax the restrictions on vector of pointer types, and vector getelementptr.Duncan Sands
Previously in a vector of pointers, the pointer couldn't be any pointer type, it had to be a pointer to an integer or floating point type. This is a hassle for dragonegg because the GCC vectorizer happily produces vectors of pointers where the pointer is a pointer to a struct or whatever. Vector getelementptr was restricted to just one index, but now that vectors of pointers can have any pointer type it is more natural to allow arbitrary vector getelementptrs. There is however the issue of struct GEPs, where if each lane chose different struct fields then from that point on each lane will be working down into unrelated types. This seems like too much pain for too little gain, so when you have a vector struct index all the elements are required to be the same. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167828 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-09Add support for memory runtime check. When we can, we calculate array bounds.Nadav Rotem
If the arrays are found to be disjoint then we run the vectorized version of the loop. If they are not, we run the scalar code. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167608 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-06Cost Model: add tables for some avx type-conversion hacks.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167480 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-05Code Model: Improve the accuracy of the zext/sext/trunc vector cost estimation.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167412 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-05Implement the cost of abnormal x86 instruction lowering as a table.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167395 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31LoopVectorize: Preserve NSW, NUW and IsExact flags.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167174 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31Fix a bug in the cost calculation of vector casts. Detect situations where ↵Nadav Rotem
bitcasts cost zero. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167170 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31LoopVectorize: Do not vectorize loops with tiny constant trip counts.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167101 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31Add support for loops that don't start with Zero.Nadav Rotem
This is important for loops in the LAPACK test-suite. These loops start at 1 because they are auto-converted from fortran. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167084 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30LoopVectorize: Add support for write-only loops when the write destination ↵Nadav Rotem
is a single pointer. Speedup SciMark by 1% git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167035 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30LoopVectorize: Fix a bug in the initialization of reduction variables. AND ↵Nadav Rotem
needs to start at all-one while XOR, and OR need to start at zero. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167032 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-271. Fix a bug in getTypeConversion. When a *simple* type is split, we need to ↵Nadav Rotem
return the type of the split result. 2. Change the maximum vectorization width from 4 to 8. 3. A test for both. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166864 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-26Refactor the VectorTargetTransformInfo interface.Nadav Rotem
Add getCostXXX calls for different families of opcodes, such as casts, arithmetic, cmp, etc. Port the LoopVectorizer to the new API. The LoopVectorizer now finds instructions which will remain uniform after vectorization. It uses this information when calculating the cost of these instructions. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166836 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-26Move the target-specific tests, which require specific backends, to dirs ↵Nadav Rotem
that only run if the target is present. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166796 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25Fix the cost-model test.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166722 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25Add the cpu model to the test.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166718 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25Add support for additional reduction variables: AND, OR, XOR.Nadav Rotem
Patch by Paul Redmond <paul.redmond@intel.com>. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166649 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24Implement a basic cost model for vector and scalar instructions. Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166642 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24LoopVectorizer: Add a basic cost model which uses the VTTI interface.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166620 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-23Use the AliasAnalysis isIdentifiedObj because it also understands mallocs ↵Nadav Rotem
and c++ news. PR14158. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166491 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22Don't crash if the load/store pointer is not a GEP.Nadav Rotem
Fix by Shivarama Rao <Shivarama.Rao@amd.com> git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166427 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22Vectorizer: optimize the generation of selects. If the condition is uniform, ↵Nadav Rotem
generate a scalar-cond select (i1 as selector). git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166409 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21Fix a bug in the vectorization of wide load/store operations.Nadav Rotem
We used a SCEV to detect that A[X] is consecutive. We assumed that X was the induction variable. But X can be any expression that uses the induction for example: X = i + 2; git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166388 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21Add support for reduction variables that do not start at zero.Nadav Rotem
This is important for nested-loop reductions such as : In the innermost loop, the induction variable does not start with zero: for (i = 0 .. n) for (j = 0 .. m) sum += ... git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166387 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21Vectorizer: fix a bug in the classification of induction/reduction phis.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166384 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-20Fix an infinite loop in the loop-vectorizer.Nadav Rotem
PR14134. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166379 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-20Vectorize: teach cavVectorizeMemory to distinguish between A[i]+=x and ↵Nadav Rotem
A[B[i]]+=x. If the pointer is consecutive then it is safe to read and write. If the pointer is non-loop-consecutive then it is unsafe to vectorize it because we may hit an ordering issue. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166371 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-19Vectorizer: Add support for loop reductions.Nadav Rotem
For example: for (i=0; i<n; i++) sum += A[i] + B[i] + i; git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166351 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-19vectorizer: Add support for reading and writing from the same memory location.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166255 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-18Vectorizer: Add support for loops with an unknown count. For example:Nadav Rotem
for (i=0; i<n; i++){ a[i] = b[i+1] + c[i+3]; } git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166165 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-17Add a loop vectorizer.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166112 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.2.3-18-g5258 (git 2.32.0) at 2025-06-22 01:40:40 +0000