aboutsummaryrefslogtreecommitdiff
path: root/lib/Transforms/Vectorize/LoopVectorize.cpp
AgeCommit message (Collapse)Author
2013-02-21Allow GlobalValues to vectorize with AliasAnalysisRenato Golin
Storing the load/store instructions with the values and inspect them using Alias Analysis to make sure they don't alias, since the GEP pointer operand doesn't take the offset into account. Trying hard to not add any extra cost to loads and stores that don't overlap on global values, AA is *only* calculated if all of the previous attempts failed. Using biggest vector register size as the stride for the vectorization access, as we're being conservative and the cost model (which calculates the real vectorization factor) is only run after the legalization phase. We might re-think this relationship in the future, but for now, I'd rather be safe than sorry. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175818 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-13LoopVectorize: Simplify code for clarity.Benjamin Kramer
No functionality change. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175076 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-13Metadata for annotating loops as parallel. The first consumer for this Pekka Jaaskelainen
metadata is the loop vectorizer. See the documentation update for more info. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175060 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-08Typos.Jakob Stoklund Olesen
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174723 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-08ARM cost model: Address computation in vector mem ops not freeArnold Schwaighofer
Adds a function to target transform info to query for the cost of address computation. The cost model analysis pass now also queries this interface. The code in LoopVectorize adds the cost of address computation as part of the memory instruction cost calculation. Only there, we know whether the instruction will be scalarized or not. Increase the penality for inserting in to D registers on swift. This becomes necessary because we now always assume that address computation has a cost and three is a closer value to the architecture. radar://13097204 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174713 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-08Test CommitMichael Kuperstein
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174709 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-07fix 80-col violation and fix the docs.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174671 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-07Loop Vectorizer: Refactor Memory Cost ComputationArnold Schwaighofer
We don't want too many classes in a pass and the classes obscure the details. I was going a little overboard with object modeling here. Replace classes by generic code that handles both loads and stores. No functionality change intended. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174646 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-05Loop Vectorizer: Refactor code to compute vectorized memory instruction costArnold Schwaighofer
Introduce a helper class that computes the cost of memory access instructions. No functionality change intended. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174422 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-05Loop Vectorizer: Handle pointer stores/loads in getWidestType()Arnold Schwaighofer
In the loop vectorizer cost model, we used to ignore stores/loads of a pointer type when computing the widest type within a loop. This meant that if we had only stores/loads of pointers in a loop we would return a widest type of 8bits (instead of 32 or 64 bit) and therefore a vector factor that was too big. Now, if we see a consecutive store/load of pointers we use the size of a pointer (from data layout). This problem occured in SingleSource/Benchmarks/Shootout-C++/hash.cpp (reduced test case is the first test in vector_ptr_load_store.ll). radar://13139343 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174377 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-29LoopVectorize: convert TinyTripCountVectorThreshold constantPekka Jaaskelainen
to a command line switch. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173837 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-29LoopVectorize: Clean up ValueMap a bit and avoid double lookups.Benjamin Kramer
No intended functionality change. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173809 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-28Vectorization Factor clarificationRenato Golin
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173691 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-25LoopVectorize: Refactor the code that vectorizes loads/stores to remove ↵Nadav Rotem
duplication. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173500 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-25LoopVectorize: Simplify code. No functionality change.Benjamin Kramer
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173475 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-25LoopVectorizer: Refactor more code to use the IRBuilder. Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173471 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-25Refactor some code to use the IRBuilder.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173467 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-23Add support for reverse pointer induction variables. These are loops that ↵Nadav Rotem
contain pointers that count backwards. For example, this is the hot loop in BZIP: do { m = *--p; *p = ( ... ); } while (--n); git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173219 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-21Fix a comment. Induction vars dont need to start at zero.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173061 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-20LoopVectorize: Fix a C++11 incompatibility.Benjamin Kramer
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172990 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-20Fix a build error.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172971 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-20LoopVectorizer: Implement a new heuristics for selecting the unroll factor.Nadav Rotem
We ignore the cpu frontend and focus on pipeline utilization. We do this because we don't have a good way to estimate the loop body size at the IR level. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172964 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-19LoopVectorizer: Emit memory checks into their own basic block.Benjamin Kramer
This separates the check for "too few elements to run the vector loop" from the "memory overlap" check, giving a lot nicer code and allowing to skip the memory checks when we're not going to execute the vector code anyways. We still leave the decision of whether to emit the memory checks as branches or setccs, but it seems to be doing a good job. If ugly code pops up we may want to emit them as separate blocks too. Small speedup on MultiSource/Benchmarks/MallocBench/espresso. Most of this is legwork to allow multiple bypass blocks while updating PHIs, dominators and loop info. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172902 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-15LoopVectorizer cost model. Honor the user command line flag that selects the ↵Nadav Rotem
vectorization factor even if the target machine does not have any vector registers. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172544 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-13Fix PR14547. Handle induction variables of small sizes smaller than i32 (i8 ↵Nadav Rotem
and i16). git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172348 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-11ARM Cost Model: We need to detect the max bitwidth of types in the loop in ↵Nadav Rotem
order to select the max vectorization factor. We don't have a detailed analysis on which values are vectorized and which stay scalars in the vectorized loop so we use another method. We look at reduction variables, loads and stores, which are the only ways to get information in and out of loop iterations. If the data types are extended and truncated then the cost model will catch the cost of the vector zext/sext/trunc operations. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172178 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-10LoopVectorizer: Fix a bug in the vectorization of BinaryOperators. The ↵Nadav Rotem
BinaryOperator can be folded to an Undef, and we don't want to set NSW flags to undef vals. PR14878 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172079 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-09ARM Cost model: Use the size of vector registers and widest vectorizable ↵Nadav Rotem
instruction to determine the max vectorization factor. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172010 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-09Cost Model: Move the 'max unroll factor' variable to the TTI and add initial ↵Nadav Rotem
Cost Model support on ARM. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171928 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-08Code cleanup: refactor the switch statements in the generation of reduction ↵Nadav Rotem
variables into an IR builder call. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171871 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-08Rename the enum members to match the LLVM coding style.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171868 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-07LoopVectorizer: Add support for floating point reductionsNadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171812 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-07LoopVectorizer: When we vectorizer and widen loops we process many elements ↵Nadav Rotem
at once. This is a good thing, except for small loops. On small loops post-loop that handles scalars (and runs slower) can take more time to execute than the rest of the loop. This patch disables widening of loops with a small static trip count. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171798 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-07Simplify LoopVectorize to require target transform info and rely on itChandler Carruth
being present. Make a member of one of the helper classes a reference as part of this. Reformatting goodness brought to you by clang-format. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171726 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-07Merge the unused header file for LoopVectorizer into the source file.Chandler Carruth
This makes the loop vectorizer match the pattern followed by roughly all other passses. =] Notably, this header file was braken in several regards: it contained a using namespace directive, global #define's that aren't globaly appropriate, and global constants defined directly in the header file. As a side benefit, lots of the types in this file become internal, which will cause the optimizer to chew on this pass more effectively. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171723 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-07Fix the enumerator names for ShuffleKind to match tho coding standards,Chandler Carruth
and make its comments doxygen comments. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171688 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-07Move TargetTransformInfo to live under the Analysis library. This noChandler Carruth
longer would violate any dependency layering and it is in fact an analysis. =] git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171686 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-05Switch the loop vectorizer from VTTI to just use TTI directly.Chandler Carruth
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171620 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-05iLoopVectorize: Non commutative operators can be used as reduction variables ↵Nadav Rotem
as long as the reduction chain is used in the LHS. PR14803. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171583 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-04Do not vectorize loops with subtraction reductionsPaul Redmond
Since subtraction does not commute the loop vectorizer incorrectly vectorizes reductions such as x = A[i] - x. Disabling for now. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171537 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-04Fix a warningNadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171525 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-04LoopVectorizer:Nadav Rotem
1. Add code to estimate register pressure. 2. Add code to select the unroll factor based on register pressure. 3. Add bits to TargetTransformInfo to provide the number of registers. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171469 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-03LoopVectorizer: Add support for loop-unrolling during vectorization for ↵Nadav Rotem
increasing the ILP. At the moment this feature is disabled by default and this commit should not cause any functional changes. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171436 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-02Avoid vectorization when the function has the "noimplicitflot" attribute.Nadav Rotem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171429 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-02Move all of the header files which are involved in modelling the LLVM IRChandler Carruth
into their new header subdirectory: include/llvm/IR. This matches the directory structure of lib, and begins to correct a long standing point of file layout clutter in LLVM. There are still more header files to move here, but I wanted to handle them in separate commits to make tracking what files make sense at each layer easier. The only really questionable files here are the target intrinsic tablegen files. But that's a battle I'd rather not fight today. I've updated both CMake and Makefile build systems (I think, and my tests think, but I may have missed something). I've also re-sorted the includes throughout the project. I'll be committing updates to Clang, DragonEgg, and Polly momentarily. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171366 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-01Add IRBuilder::CreateVectorSplat and use it to simplify code.Benjamin Kramer
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171349 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-30Remove the Function::getFnAttributes method in favor of using the AttributeSetBill Wendling
directly. This is in preparation for removing the use of the 'Attribute' class as a collection of attributes. That will shift to the AttributeSet class instead. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171253 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-30LoopVectorizer: Fix a bug in the code that updates the loop exiting block.Nadav Rotem
LCSSA PHIs may have undef values. The vectorizer updates values that are used by outside users such as PHIs. The bug happened because undefs are not loop values. This patch handles these PHIs. PR14725 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171251 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-26If all of the write objects are identified then we can vectorize the loop ↵Nadav Rotem
even if the read objects are unidentified. PR14719. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171124 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-26LoopVectorizer: Optimize the vectorization of consecutive memory access when ↵Nadav Rotem
the iteration step is -1 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171114 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.2.3-18-g5258 (git 2.32.0) at 2025-04-27 10:25:50 +0000