I'm working on a new language using the LLVM C++ API and would like to take advantage of optimization passes. (Note: I'm currently using the latest from source LLVM which I believe equates to 3.8)
I have yet to find any examples that use the new PassManager and even Clang is still utilizing the LegacyPassManager.
I have come across posts such as this that are several years old now that mention the new PassManager, but they all still use the legacy system.
Is there any examples/tutorials on how to use this new(ish) PassManager? Should new LLVM projects prefer PassManager to LegacyPassManager? Does Clang plan on migrating or is this why the Legacy system has stuck around?
From what I've gathered with help from the #llvm IRC:
FunctionPassManager FPM;
//Use the PassInfoMixin types
FPM.addPass(InstCombinePass());
//Register any analysis passes that the transform passes might need
FunctionAnalysisManager FAM;
//Use the AnalysisInfoMixin types
FAM.registerPass([&] { return AssumptionAnalysis(); });
FAM.registerPass([&] { return DominatorTreeAnalysis(); });
FAM.registerPass([&] { return BasicAA(); });
FAM.registerPass([&] { return TargetLibraryAnalysis(); });
FPM.run(*myFunction, FAM);
But to avoid the hassle of manually registering each pass you can use PassBuilder to register the analysis passes
FunctionPassManager FPM;
FPM.addPass(InstCombinePass());
FunctionAnalysisManager FAM;
PassBuilder PB;
PB.registerFunctionAnalyses(FAM);
FPM.run(*myFunction, FAM);
Extending Lukes answer, with PassBuilder you can build predefined "out of box" simplification pipelines with different optimization levels:
llvm::FunctionAnalysisManager FAManager;
llvm::PassBuilder passBuilder;
passBuilder.registerFunctionAnalyses(FAManager);
passBuilder.buildFunctionSimplificationPipeline(
llvm::PassBuilder::OptimizationLevel::O2,
llvm::PassBuilder::ThinLTOPhase::None);
which will add a bunch of passes to FunctionAnalysisManager. This may simplify your life. The best place to see the full set of passes added for each OptimizationLevel is the original sources.
Related
I'm trying to upgrade my AutoFFI project by making it more elegant and use Clang's ASTMatchers more extensively. I'd like to create a matcher that filters on the file path that was specified. Is it possible to do such a thing, or do I need to add custom logic outside of the matcher for this to work? As far as I can see, there's no way to get the SourceManager and use it to create a FullSourceLoc, but maybe I'm missing something.
Some relevant links:
https://clang.llvm.org/doxygen/classclang_1_1FullSourceLoc.html
https://github.com/llvm-mirror/clang/blob/f3b7928366f63b51ffc97e74f8afcff497c57e8d/include/clang/ASTMatchers/ASTMatchersMacros.h#L28
If someone could tell me whether this is a limitation to Clang's ASTMatcher API or not I'd be very grateful!
Never mind, I've found the answer by looking at the source of isExpansionInMainFile:
AST_POLYMORPHIC_MATCHER(isExpansionInMainFile,
AST_POLYMORPHIC_SUPPORTED_TYPES(Decl, Stmt, TypeLoc)) {
auto &SourceManager = Finder->getASTContext().getSourceManager();
return SourceManager.isInMainFile(
SourceManager.getExpansionLoc(Node.getBeginLoc()));
}
Turns out I missed the getASTContext in MatchFinder, which holds on to the source manager.
Currently I'm using FastTree for binary classification, but I would like to give SVM a try and compare metrics.
All the docs mention LinearSvm, but I can't find code example anywhere.
mlContext.BinaryClassification.Trainers does not have public SVM trainers. There is LinearSvm class and LinearSvm.TrainLinearSvm static method, but they seem to be intended for different things.
What am I missing?
Version: 0.7
For some reason there is no trainer in the runtime API but there is a linear SVM trainer in the Legacy API (for v0.7) found here. They might be generating a new one for the upcoming API, so my advice is to either use the legacy one, or wait for a newer API.
At this stage, ML.Net is very much in development.
Copy pasting the response I got on Github:
I have two answers for you: What the status of the API is, and how to use the LinearSVM in the meantime.
First, we have LinearSVM in the ML.NET codebase, but we do not yet have samples or the API extensions to place it in mlContext.BinaryClassification.Trainers. This is being worked through in issue #1318. I'll link this to that issue, and mark it as a bug.
In the meantime, you can use direct instantiation to get access to LinearSVM:
var arguments = new LinearSvm.Arguments()
{
NumIterations = 20
};
var linearSvm = new LinearSvm(mlContext, arguments);
var svmTransformer = linearSvm.Fit(trainSet);
var scoredTest = svmTransformer.Transform(testSet);
This will give you an ITransformer, here called svmTransformer that you can use to operate on IDataView objects.
We are currently writing a module for Sitecore and have ran into a problem.
We have a pipeline in which we do the following check:
if (Sitecore.Context.PageMode.IsExperienceEditor)
{
return;
}
The problem is that one of our clients are running and older version of Sitecore (8.0 update 5) where the property IsExperienceEditor does not exist yet. See Sitecore release notes for next update where it is introduced.
To quickly fix the error we used the older deprecated property which is this:
if (Sitecore.Context.PageMode.IsPageEditor)
{
return;
}
Now the question is, is there any way in which we can test for the Sitecore version so we can have backwards compatibility in the module?
You can use the code which is executed in Sitecore in background of both properties mentioned by you:
if (Sitecore.Context.Site.DisplayMode == Sitecore.Sites.DisplayMode.Edit)
{
return;
}
I know that using Sitecore.Context.PageMode.IsExperienceEditor (or Sitecore.Context.PageMode.IsPageEditor) is more elegant, but in a situation when you need to support both old and new Sitecore versions, that's sounds like a good option.
The deprecated property of IsPageEditor is still present specifically for the purpose of backward compatibility. IsExperienceEditor is just a renamed property that does the same thing that IsPageEditor does.
However you can check for the existence of a property like this:
public static bool HasProperty(this object obj, string propertyName)
{
return obj.GetType().GetProperty(propertyName) != null;
}
Another option is to make two different versions of the module, if the implementation becomes significantly different for the different versions of Sitecore.
I'm new to LLVM. I am using the clang c++ API to compile multiple stub files (in c) to IR, and then stick them together using IR builder (after linking them) to eventually run via JIT.
All this works great, unless I add a functionInlining pass to my optimizations, at which point one of these function calls made in IR builder will trigger the following exception when the pass manager is run:
Assertion failed: (New->getType() == getType() && "replaceAllUses of value with new value of different type!"), function replaceAllUsesWith, file /Users/mike/Development/llvm/llvm/lib/IR/Value.cpp, line 356.
This is how I make the call instruction (pretty straight forward):
Function *kernelFunc = mModule->getFunction( (kernel->Name() + StringRef("_") + StringRef(funcName)).str());
if (kernelFunc){
CallInst* newInst = builder.CreateCall(kernelFunc, args);
}
Later the module is optimized:
legacy::PassManager passMan;
PassManagerBuilder Builder;
Builder.OptLevel = 3;
//Builder.Inliner = llvm::createFunctionInliningPass(); //commenting this back in trigger the exception
Builder.populateModulePassManager(passMan);
passMan.run( *mModule ); //exception occurs before this call returns
Any ideas what to look for?
Try running llvm::verifyModule on your module to see if it's correct. You might have an error and have been getting lucky beforehand but it tripped something up in the inliner.
In general assertions check a subset of things that can be wrong with your module but verify checks a lot more.
It could be a bug in LLVM but more than likely it's a bad module, it's easy to happen.
So I finally setup my dev environment so I could inspect the assertion call in the debugger. I turns out the basic block being replaced had a different context set than the one it was being replaced with. going back and making sure IRBuilder was using the same context as the IR parsers solved the problem.
I'd like to get list of all Upgrade codes of all installed products on Windows box. The question is: is there a dedicated MSI function to address this request?
There is MsiEnumProducts() that enumerates all installed products and MsiEnumRelatedProducts() to enumerate all products for the given Upgrade code. But I can't find a function to get all Upgrade codes in the system.
The workaround I can imagine is use MsiEnumProducts() to get list of all installed products, open each with MsiOpenProduct() function and read "UpgradeCode" property with MsiGetProductProperty(). But this should be very slow due to multiple MsiOpenProduct() calls.
I believe MsiEnumProducts loop with MsiOpenProduct and then MsiGetProductProperty is the correct official sequence. If you really need faster and are willing to bypass the API's you could read the registry directly at HKCR\Installer\UpgradeCodes. You'll have to reverse the Darwin Descriptors though. This isn't technically supported but the reality is these keys have been there for 16 years and MSFT has been doing ZERO development on The Windows Installer. Ok, maybe they updated the version number and removed ARM support in Windows 10 LOL.
FWIW, I like to use C# not C++ but the concept is the same. The following snippet ran on my developer machine in about 2 seconds.
using System;
using Microsoft.Deployment.WindowsInstaller;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
foreach (var productInstallation in ProductInstallation.AllProducts)
{
using(var database = new Database(productInstallation.LocalPackage, DatabaseOpenMode.ReadOnly))
{
Console.WriteLine(database.ExecutePropertyQuery("UpgradeCode"));
}
}
}
}
}
According to the DTF documentation, ProductInstallation.AllProducts uses MsiEnumProducts. The Database class constructor is using MsiOpenDatabase and ExecutePropertyQuery is a higher level call that basically abstracts doing a SELECT Value from Property WHERE Property = '%s'. So it'll be calling APIs to create, execute and fetch results from views. All these classes implement IDisposable to call the correct APIs to free resources also.
Ya... that's why I love managed code. :)