How to get Rcpp to include the right header file path - c++

Question

I was trying to run a simple testing C++ file in my R package development directory with Rcpp located at say the directory "my_R_package_dir/src/my_file_location.cpp":
#include <Rcpp.h>
// [[Rcpp::export]]
int plusone() {
return 1 + 1;
}
But when running Rcpp::sourceCpp("src/my_file_location.cpp") the R console gives the following warning:
Warning message:
In normalizePath(path.expand(path), winslash, mustWork) :
path[1]="C:/Users/myself/my_R_package_dir/src/../inst/include": file not found
The function exported plusone() did work, but I can see clearly that its for some reason trying to access a non-existent file named "include" in the "inst" directory which I do have. I'm guessing it wants to access the Rcpp.h file but I probably need to somehow indicate it the path to point at? How do I do that?
P.S. This also causes the package build to fail and part of the message also indicates that theres no such file or directory Rcpp.h.
I thought I needed a Makevars.win and Makevars file, and I put them both into src and they look like this:
PKG_CPPFLAGS=`Rscript -e 'Rcpp:::CxxFlags()'`
PKG_LIBS=`Rscript -e 'Rcpp:::LdFlags()'`
CC=C:\msys64\mingw64\bin\g++.exe
CXX=C:\msys64\mingw64\bin\gcc.exe
But with no avail. I am on a windows device with Rtools 4.3, gcc, and g++.