aboutsummaryrefslogtreecommitdiff
path: root/lib/Transforms/Vectorize
AgeCommit message (Collapse)Author
2012-11-14Fix the largest offender of determinism in BBVectorizeHal Finkel
Iterating over the children of each node in the potential vectorization plan must happen in a deterministic order (because it affects which children are erased when two children conflict). There was no need for this data structure to be a map in the first place, so replacing it with a vector is a small change. I believe that this was the last remaining instance if iterating over the elements of a Dense* container where the iteration order could matter. There are some remaining iterations over std::*map containers where the order might matter, but so long as the Value* for instructions in a block increase with the order of the instructions in the block (or decrease) monotonically, then this will appear to be deterministic. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167942 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-14use the getSplat API. Patch by Paul Redmond.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167892 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-13BBVectorize: Remove temporary assert used for debuggingHal Finkel
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167817 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-13BBVectorize: Don't vectorize vector-manipulation chainsHal Finkel
Don't choose a vectorization plan containing only shuffles and vector inserts/extracts. Due to inperfections in the cost model, these can lead to infinite recusion. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167811 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-12BBVectorize: Only some insert element operand pairs are free.Hal Finkel
This fixes another infinite recursion case when using target costs. We can only replace insert element input chains that are pure (end with inserting into an undef). git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167784 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-12BBVectorize: Use a more sophisticated check for input costHal Finkel
The old checking code, which assumed that input shuffles and insert-elements could always be folded (and thus were free) is too simple. This can only happen in special circumstances. Using the simple check caused infinite recursion. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167750 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-12BBVectorize: Check the types of compare instructionsHal Finkel
The pass would previously assert when trying to compute the cost of compare instructions with illegal vector types (like struct pointers). git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167743 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-12BBVectorize: Check the input types of shuffles for legalityHal Finkel
This fixes a bug where shuffles were being fused such that the resulting input types were not legal on the target. This would occur only when both inputs and dependencies were also foldable operations (such as other shuffles) and there were other connected pairs in the same block. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167731 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-11Fix a comment typo and add comments.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167684 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-09Add support for memory runtime check. When we can, we calculate array bounds.Nadav Rotem
If the arrays are found to be disjoint then we run the vectorized version of the loop. If they are not, we run the scalar code. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167608 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-02Fix sign compare warning. Patch by Mahesha HS.Chandler Carruth
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167282 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-01BBVectorize: Use target costs for incoming and outgoing values instead of ↵Hal Finkel
the depth heuristic. When target cost information is available, compute explicit costs of inserting and extracting values from vectors. At this point, all costs are estimated using the target information, and the chain-depth heuristic is not needed. As a result, it is now, by default, disabled when using target costs. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167256 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-01BBVectorize: Account for internal shuffle costsHal Finkel
When target costs are available, use them to account for the costs of shuffles on internal edges of the DAG of candidate pairs. Because the shuffle costs here are currently for only the internal edges, the current target cost model is trivial, and the chain depth requirement is still in place, I don't yet have an easy test case. Nevertheless, by looking at the debug output, it does seem to do the right think to the effective "size" of each DAG of candidate pairs. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167217 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31LoopVectorize: Preserve NSW, NUW and IsExact flags.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167174 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31Put the threshold magic number in a variable.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167134 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31Remove enum values since they are not used anymore.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167131 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31BBVectorize: Choose pair ordering to minimize shufflesHal Finkel
BBVectorize would, except for loads and stores, always fuse instructions so that the first instruction (in the current source order) would always represent the low part of the input vectors and the second instruction would always represent the high part. This lead to too many shuffles being produced because sometimes the opposite order produces fewer of them. With this change, BBVectorize tracks the kind of pair connections that form the DAG of candidate pairs, and uses that information to reorder the pairs to avoid excess shuffles. Using this information, a future commit will be able to add VTTI-based shuffle costs to the pair selection procedure. Importantly, the number of remaining shuffles can now be estimated during pair selection. There are some trivial instruction reorderings in the test cases, and one simple additional test where we certainly want to do a reordering to avoid an unnecessary shuffle. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167122 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31LoopVectorize: Do not vectorize loops with tiny constant trip counts.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167101 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31Add support for loops that don't start with Zero.Nadav Rotem
This is important for loops in the LAPACK test-suite. These loops start at 1 because they are auto-converted from fortran. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167084 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30Add documentation.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167055 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30BBVectorize: Cache fixed-order pairs instead of recomputing pointer info.Hal Finkel
Instead of recomputing relative pointer information just prior to fusing, cache this information (which also needs to be computed during the candidate-pair selection process). This cuts down on the total number of SE queries made, and also is a necessary intermediate step on the road toward including shuffle costs in the pair selection procedure. No functionality change is intended. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167049 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30BBVectorize: Fix a small bug introduced in r167042.Hal Finkel
We need to make sure that we take the correct load/store alignment when the inputs are flipped. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167044 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30BBVectorize: Simplify how input swapping is handled.Hal Finkel
Stop propagating the FlipMemInputs variable into the routines that create the replacement instructions. Instead, just flip the arguments of those routines. This allows for some associated cleanup (not all of which is done here). No functionality change is intended. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167042 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30BBVectorize: Don't make calls to SE when the result is unused.Hal Finkel
SE was being called during the instruction-fusion process (when the result is unreliable, and thus ignored). No functionality change is intended. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167037 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30LoopVectorize: Add support for write-only loops when the write destination ↵Nadav Rotem
is a single pointer. Speedup SciMark by 1% git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167035 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30LoopVectorize: Fix a bug in the initialization of reduction variables. AND ↵Nadav Rotem
needs to start at all-one while XOR, and OR need to start at zero. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167032 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30LoopVectorizer: change debug prints: Print the module identifier when ↵Nadav Rotem
deciding to vectorize. When deciding not to vectorize do not print the called function name because it can be null. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166989 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-29LoopVectorize: Update and preserve the dominator tree info.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166970 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-27Update BBVectorize to use the new VTTI instr. cost interfaces.Hal Finkel
The monolithic interface for instruction costs has been split into several functions. This is the corresponding change. No functionality change is intended. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166865 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-271. Fix a bug in getTypeConversion. When a *simple* type is split, we need to ↵Nadav Rotem
return the type of the split result. 2. Change the maximum vectorization width from 4 to 8. 3. A test for both. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166864 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-26Refactor the VectorTargetTransformInfo interface.Nadav Rotem
Add getCostXXX calls for different families of opcodes, such as casts, arithmetic, cmp, etc. Port the LoopVectorizer to the new API. The LoopVectorizer now finds instructions which will remain uniform after vectorization. It uses this information when calculating the cost of these instructions. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166836 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-26Use VTTI->getNumberOfParts in BBVectorize.Hal Finkel
This change reflects VTTI refactoring; no functionality change intended. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166752 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-26Disable generation of pointer vectors by BBVectorize.Hal Finkel
Once vector-of-pointer support works, then this can be reverted. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166741 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25BBVectorize, when using VTTI, should not form types that will be split.Hal Finkel
This is needed so that perl's SHA can be compiled (otherwise BBVectorize takes far too long to find its fixed point). I'll try to come up with a reduced test case. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166738 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25Begin incorporating target information into BBVectorize.Hal Finkel
This is the first of several steps to incorporate information from the new TargetTransformInfo infrastructure into BBVectorize. Two things are done here: 1. Target information is used to determine if it is profitable to fuse two instructions. This means that the cost of the vector operation must not be more expensive than the cost of the two original operations. Pairs that are not profitable are no longer considered (because current cost information is incomplete, for intrinsics for example, equal-cost pairs are still considered). 2. The 'cost savings' computed for the profitability check are also used to rank the DAGs that represent the potential vectorization plans. Specifically, for nodes of non-trivial depth, the cost savings is used as the node weight. The next step will be to incorporate the shuffle costs into the DAG weighting; this will give the edges of the DAG weights as well. Once that is done, when target information is available, we should be able to dispense with the depth heuristic. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166716 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25LoopVectorize: Teach the cost model to query scalar costs as scalar types ↵Nadav Rotem
and not vectors of 1. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166715 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25Add support for additional reduction variables: AND, OR, XOR.Nadav Rotem
Patch by Paul Redmond <paul.redmond@intel.com>. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166649 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24Implement a basic cost model for vector and scalar instructions. Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166642 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24whitespaceNadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166622 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24LoopVectorizer: Add a basic cost model which uses the VTTI interface.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166620 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24Back out r166591, not sure why this made it through since I cancelled the ↵Micah Villmow
command. Bleh, sorry about this! git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166596 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24Delete a directory that wasn't supposed to be checked in yet.Micah Villmow
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166591 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-23Use the AliasAnalysis isIdentifiedObj because it also understands mallocs ↵Nadav Rotem
and c++ news. PR14158. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166491 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22Don't crash if the load/store pointer is not a GEP.Nadav Rotem
Fix by Shivarama Rao <Shivarama.Rao@amd.com> git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166427 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22BBVectorize should ignore unreachable blocks.Hal Finkel
Unreachable blocks can have invalid instructions. For example, jump threading can produce self-referential instructions in unreachable blocks. Also, we should not be spending time optimizing unreachable code. Fixes PR14133. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166423 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22Rename a variable.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166410 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22Vectorizer: optimize the generation of selects. If the condition is uniform, ↵Nadav Rotem
generate a scalar-cond select (i1 as selector). git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166409 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22Update the loop vectorizer docs.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166408 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21Avoid an extra hash lookup when inserting a value into the widen map.Anders Carlsson
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166395 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21Simplify code. No functionality change.Jakub Staszak
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166393 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.2.3-18-g5258 (git 2.32.0) at 2025-04-30 11:55:44 +0000