Does anybody know how to compile a single file containing various shaders (PixelShader, VertexShader, GeometryShader) using the fxc.exe tool provided by the DirectX 11 SDK?
The shader is used to create a tesselation-effect in a C++ programmed Enviroment:
The result should be a .fxo shader File.
Thx in advance :)
You run the compiler separately for each effect source file (one source file per effect, including the various shaders and helper routines). There will be a separate shader object file for each effect, similarly. The command line depends on what you actually want to compile but something like this:
fxc.exe /T ps_2_0 /nologo /E main /Fo"Effect.fxo" "Effect.fx"
or
fxc.exe /T fx_4_0 /nologo /Fo"Effect.fxo" "Effect.fx"
Related
I'm struggling to figure out how to set up my hlsl shaders to work with Premake5 and Visual Studio 2017.
I have no idea how to tell Premake5 to compile my hlsl shaders, as a pre-build step.
Here are my goals for this pre-build step:
Specify shader model
Specify debug/release compilation
Only compile files that have changed
Produce dxasm files
Place resulting *.asms and *.cso in appropriate release/debug folders
Update 1:
Investigating a little further I found that Premake5 has buildaction which makes direct reference to FxCompile.
Update 2:
Thanks to mukunda. I was able to configure my project perfectly!
I had to build premake5 from source in order to get this premake script to run.
Because as of March, 2019 the binary distributed doesn't support shader model greater than 5.
hf.deactivate_filter()
files(src_dir.."shaders/*.hlsl")
shadermodel("6.3")
shaderassembler("AssemblyCode")
local shader_dir = _WORKING_DIR.."/"..src_dir.."shaders/%{cfg.buildcfg}/"
-- HLSL files that don't end with 'Extensions' will be ignored as they will be
-- used as includes
filter("files:**.hlsl")
flags("ExcludeFromBuild")
shaderobjectfileoutput(shader_dir.."%{file.basename}"..".cso")
shaderassembleroutput(shader_dir.."%{file.basename}"..".asm")
filter("files:**_ps.hlsl")
removeflags("ExcludeFromBuild")
shadertype("Pixel")
filter("files:**_vs.hlsl")
removeflags("ExcludeFromBuild")
shadertype("Vertex")
hf.deactivate_filter()
-- Warnings as errors
shaderoptions({"/WX"})
Update 3:
I figured out how to handle command line parameters.
I was just messing around with this today. For my HLSL files I have a simple naming scheme:
*-p.hlsl for pixel shader files.
*-v.hlsl for vertex shader files.
*.hlsl for generic files meant to be included by the shader programs. I just use the hlsl extension so that it shows up with proper HLSL syntax highlighting in the editor.
You don't need custom build rules to compile them. Premake seems to be able to output a proper block in the Visual Studio project for using the shader compiler, and then things like only recompiling files that have changed (with #include dependencies) are handled just fine. Here's what my configuration block looks like:
filter { "files:**.hlsl" }
flags "ExcludeFromBuild"
shadermodel "5.0"
filter { "files:**-p.hlsl" }
removeflags "ExcludeFromBuild"
shadertype "Pixel"
shaderentry "ForPixel"
filter { "files:**-v.hlsl" }
removeflags "ExcludeFromBuild"
shadertype "Vertex"
shaderentry "ForVertex"
filter {}
Basically sets up the shadermodel and entry points for my files (For DirectX 12 you probably want shadermodel "6.0"). Then I can add my shaders to the project through files:
files { "shaders/**.hlsl"; }
This is all very new stuff in premake, so you won't find much documentation on it. I saw these options here: https://github.com/premake/premake-core/blob/master/modules/vstudio/_preload.lua.
As for your question about exporting the dxasm files, there are some other options such as shaderassembleroutput and shaderassembler, the latter I believe to be the switch that exports the assembly code to the location specified, but I have not done any testing with these to give an elaborate answer.
Unfortunately I'm not really sure how to enable debugging information in the output files, as there doesn't seem to be a premake option for that yet. I was digging through the documentation to look for any way to add the /Zi (enable debug symbols) flag for shader compilation, but I'm not entirely sure if it's possible.
I have HLSL shaders added to my Visual Studio project and they're getting compiled automatically to cso files as part of the build process. The problem which I'm facing is, that I need to compile them multiple times using different Shader models because i.e. "/4_0_level_9_3" works on DirectX 11, but not on DirectX 9, while "/3_0" only works on DirectX 9 but not on DirectX 11.
Adding the same files more than once to the Visual Studio project is not working and I would like to avoid copying the HLSL source files as it increases maintenance efforts and potential error causes. I can't create separate targets/configuration in Visual Studio to achieve this either, because I need to support both DirectX versions in the executable (switchable at runtime).
Is there any way to specify multiple compilations (different shader models and different output file) for a single HLSL file in Visual Studio?
Put your shader code into an .hlsli file, add that to your project. Create for each shader and model combination an .hlsl file which does a #include of your .hlsli file. Add each .hlsl to your project and set the file settings appropriately.
File 1 (set to exclude from build)
// MyShader.hlsli
PS_INPUT VertexShader( VS_INPUT input )
{
...
}
float4 PixelShader( PS_INPUT input)
{
...
}
File 2 (set to build as a Vertex Shader, Shader Model 9_3, Entry-point VertexShader)
// MyShader_VS.hlsl
#include "MyShader.hlsl"
File 3 (set to build as a Pixel Shader, Shader Model 9_3, Entry-point PixelShader)
// MyShader_PS.hlsl
#include "MyShader.hlsl"
File 4 (set to build as a Vertex Shader, Shader Model 4.0, Entry-point VertexShader)
// MyShader_VS4.hlsl
#include "MyShader.hlsl"
File 5 (set to build as a Pixel Shader, Shader Model 4.0, Entry-point PixelShader)
// MyShader_PS4.hlsl
#include "MyShader.hlsl"
Note that you can make life a little easier by manually editing your vcxproj file with a text editor. Look for the Label="UserMacros" and then in the ItemDefinitionGroup for each configuration right after that add a section for <FXCompile> to set better defaults (rather than 4.0_level_9_1):
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<Link>
...
</Link>
<ClCompile>
...
</ClCompile>
<FXCompile>
<ShaderModel>4.0_level_9_3</ShaderModel>
</FXCompile>
Unless you are specifically targeting Windows XP systems, there's not much value in using the legacy Direc3D 9 API at all. Just use DirectX 11 Feature Level 9.3+ to target Shader Model 2.0/3.0 era video cards. The main value of Shader Model 3.0 was vertex texture fetch which was never implemented by one of the major vendors anyhow. The 4_0_level_9_3 profile already builds the shader as both 2.0 and 4.0 in the same blob.
I was wondering if anyone can help me. I'm currently working on a game engine project which involves its own c++ compiler. For this I'm "borrowing" the visual studio 2013 c++ compiler. everything works fine. The problem I am having is a cant figure out how I would pass commands to the elevated program in a batch file.
Let me Explain, right now I am using a program which calls the "vcvarsall.bat" file and passes in "x86" as a parameter. This is great for manual entry as it then allows me to input the commands to compile files. E.G "cl /EHsc <cpp files>"
As of now, when I add commands after I call "vcvarsall.bat", they just give me a command reference error saying the command is not recognized.
What I want to achieve is being able to call one bat file which executes and compiles all of my code for me. instead of having to manually type in the commands every time. This way the entire process is easier for the user.
Any help would be appreciated,
Thank you in advance!
when I add commands after I call "vcvarsall.bat"
Maybe it has been too long since I last did a batch file .. hope the following gets you started:
I think any .bat file will accept parameters, and internally, the .bat writer (i.e. you) uses special identifiers. Often they are named something like %1 and %2, etc. (some scripting languages use $1, and probably a few other approaches)
Without consuming these parameters in your .bat file, the command line interpreter tries to use the parameter as another command (so you get 'command not recognized')
Search all .bat files on your system for %1 (or $1 or whatever) ... maybe you'll find enough hints.
Thank you all for the help, the way I solved the problem was by finding the last batch file which was called and making the end of the file call another batch file in the main compile directory, this means I can programatically generate this batch file making it incredibly easy to generate custom compilations. thank you all,
Alister
I have a project that I converted from a makefile that has a source file that expects the command line options from the compiler. For example for when the project was built with gcc if you did program --help it would spit out the gcc command line used to compile the program.
How can I do the same thing in Visual Studio, so that it spits out the cl command line used to compile the program? Basically I want to hit F7 (to build solution) and have the whole thing automated. I can't find a macro for it. Thanks
edit; I mean programatically, so for example I want when I run the program for its output to contain the cl.exe command string that is used. You can see the command line at Configuration Properties > C/C++ > Command Line > All Options but I can't find a macro for it or some way to encapsulate it in a file.
Since VS switched the underlying build system to MsBuild the command line as shown in that dialog is created programatically within VS only. It might not even be the exact command line passed to cl: MsBuild itself invokes CL via a task and as such there is no direct link with what is shown in VS nor is there a way to get the command line out of it.
Anyway, there is no such thing as the command line since each source file might have different options. Furthermore I doubt you want the full commandline including the absolute include paths etc. Nobody is interested in that. Now if you make clever use of the macros from http://msdn.microsoft.com/en-us/library/b0084kay.aspx you can sort of recreate the command line yourself since most options are there:
std::string CompilerCommandLineOptions()
{
std::string cmd;
#ifdef _CHAR_UNSIGNED
cmd += " /J";
#endif
#ifdef __cplusplus_cli
cmd += " /clr";
#endif
#ifdef _CPPRTTI
cmd += " /GR"
#endif
//etc
return cmd;
}
Note: are you sure it's worth the hassle? Is there really somebody interested in the command line? It's not even sufficient to build the project. Why not the linker options as well then?
A .vcxproj is a Visual Studio project in the MSBuild project format. You can build it by running msbuild.exe, devenv.exe, devenv.com, using the Visual Studio GUI or the MSBuild API.
Visual Studio GUI uses the MSBuild API. In doing so, it limits the MSBuild output.
If you want more details, change your user settings in Visual Studio:
Tools > Options > Project and Solutions > Build and Run > two verbosity settings
Detailed will show the cl.exe command lines.
The closest thing which I came across cl command line which msuild executes is "hacking" the rsp file used while calling cl.exe.
Using Override compiler solution, I changed ClCompile ToolExe to custom mycl.bat script and this script received an argument which was #tmp-1234xxx.rsp file. This rsp file contained whole command line except cl.exe path, something like -
rsp file
/P /DDEBUG Source.cpp
Then after making desired changes in the rsp file by calling a separate bash script which were very minor for me, I called cl.exe with contents of my rsp file. So, whenever user hits the build button, this script executes.
mycl.bat script
#echo off
SET PATH=%PATH%;/usr/bin //to call cygwin bash
set parameter=%1
set parameter=%parameter:~1% //to remove # in the beginning
c:/cygwin/bin/bash process.sh %parameter%
process.sh
iconv -f UCS-2 -t UTF-8 <$1 >$1.conv //file converted to UTF-8, else bash wasn't handling it well
contents=`cat $1.conv`
#Processing on file contents here
path/to/cl.exe $(contents)
Very nasty solution, but it worked for my use case. I wanted to change the names of the file on the go based on some logic.
The problem I faced is Visual Studio uses tlogs written by CL Task to check while file needs to to be rebuilt on incremental build and my target's tlogs files were not enough. There were tlogs of every command in batch and bash scripts but not for whole target. So, it was building whole thing on incremental builds also.
I'm hoping someone points out something wrong or a workaround for what we are experiencing.
When compiling a project with /MP it appears that only files in the same folder are compiled concurrently. I used process explorer to swipe the command line and confirm the behavior.
Project filters seem to have all no impact on the what gets compiled concurrently.
Project structure on disk:
Folder\
project.vcxproj
source\
foo.cpp
foo1.cpp
other_folder\
bar.cpp
bar1.cpp
bar3.cpp
Initial process tree:
MSBuild.exe
cl.exe ( passed: source\foo.cpp source\foo1.cpp )
cl.exe ( passed: source\foo.cpp )
cl.exe ( passed: source\foo1.cpp )
After the 2 child instances of cl.exe complete the parent closes and the following process tree appears:
MSBuild.exe
cl.exe ( passed: other_folder\bar.cpp other_folder\bar1.cpp other_folder\bar2.cpp )
cl.exe ( passed: other_folder\bar.cpp )
cl.exe ( passed: other_folder\bar1.cpp )
cl.exe ( passed: other_folder\bar2.cpp )
Our source is nicely organized in many levels of nested folders that match the layout of the headers on disk - I would hate to have to give that up to take advantage of /MP.
The use of %(RelativeDir) in "Object File Name" ( in the vcxproj XML, /Fo on the CL.exe command line ) project causes msbuild to batch the cpp files to cl.exe on a per directory basis. This can have a significant impact on the benefits gained from using /MP.
Note that if your project uses %(RelativeDir) for object files its likely that the configuration is trying to avoid colliding .obj files that from cpp files with the same name in different folders.
The /Fo command line parameter is typically a folder that the compiler dumps the obj files into - there is only ONE passed, so all of the cpp files for a given directory can only be passed to CL.exe at a time.
That was a pain - but I'm glad there is a reason and a solution. Hope it helps.
Update
A team mate found that anytime an MSBuild parameter is sent to CL.exe it seems to break or severely limit /MP. This is most likely because for /MP to work well the top level CL.exe needs to have a bundle of cpp files.
Our solution was to not use any msbuild params ( I think its the %params% ) for 'Object File Name'. This required that we rename some cpp files so they did not collide.
Hope this has changed in VS2012 or VS2013.
According it MSDN, the files should be compiled when ever there is a thread to handle them, it at the same time makes no guarantee's about the order in which files are to be compiled:
The source files might not be compiled in the same order in which they
appear on the command line. Although the compiler creates a set of
processes that contain copies of the compiler, the operating system
schedules when each process executes. Consequently, you cannot
guarantee that the source files will be compiled in a particular
order.
A source file is compiled when a process is available to compile it.
If there are more files than processes, the first set of files is
compiled by the available processes. The remaining files are processed
when a process finishes handling a previous file and is available to
work on one of the remaining files.
It also states that the number of created process will be bound by the number of threads and files in the command line:
That value is the lesser of the number of source files that you specify on the command line
Combining these to things, we can see that the compiler handles the compilation incrementally(file wise) so that it can dispatch work to children correctly, it dose this folder by folder.
You might be able to get around this if you generated a custom make file, where you should be able to get more than one folder being processed at the same time (or try using the MSBUILD.exe tool).
I've addressed this issue here with a new approach for avoiding .obj clobbering conflicts
https://stackoverflow.com/a/26935613/4253427
With the above in mind, I can elaborate on "anytime an MSBuild parameter is sent to CL.exe it seems to break or severely limit /MP". /MP works one CL.exe invocation at a time. When you "send an msbuild parameter" what you actually do is create multiple CL.exe invocations with commandlines tailored to each source file. These can't be batched. The above linked solution attempts to work around that by tailoring a commandline to a minimal set of output directories.
This should solve your problem by preventing clobbers while maintaining a high degree of batching, so long as your project doesn't contain 100 files all named x.cpp in different directories... usually in my experience there are only a few colliding .obj in a much larger project.
Can confirm the issue, and the cause wasn't easy to find, so let's get some more keywords in for other users stumbling on it.
I've been noticing that rebuilds on my latest MSVC project take way too long. In particular, the CPU cores are barely utilized, the number on cl.exe processes in Task Manager varies wildly and the Output window shows the source files being compiled in what seem like batches. Anywhere from one to 16 are being compiled at a time, a small pause, then another set of files, and so on. In comparison, on my older projects the CPU is pretty much fully used and the Output window shows a continuous stream of source files being compiled.
Now, the big difference in my new project was making a better use of namespaces with a matching directory structure, which meant ending up with some classes having identical names and causing conflicts due to the different .obj files heading for the same directory, which lead to changing the Object File Name in C/C++ -> Output Files.
Updates to the Output window also match the directory structure. If there's a namespace/directory with one source file inside it, then VS shows just that one file being compiled at a time. The next directory has 10 source files and VS shows all 10 being compiled simultaneously.
There aren't many solutions. Either avoid classes with the same name and don't change the Object File Name, or use the the workaround posted by zeromus, it functions great. My rebuild time went from 03:15 to 01:20, which is quite a difference and matches the CPU utilization going from ~35% to a flatline 100% during most the compilation.
VS 2015, 2017 and 2019 all behave in this way, so not much hope for a change.