I have a strategy that calculates entry/exit criteria based on Daily timeframe, but I use 15min timeframe to actually trade against those daily values. It works fine, EXCEPT when I try to go farther back in time to backtest than the current 15min chart will show.
As a simple example, is there a way in Pinescript to:
Calculate EMA 20 cross above EMA 200 on a DAILY timeframe
Backtest trades that on a 15min timeframe go long when this occurs or short during the opposite.
I can make that work, but I cannot for example go back to Jan 1, 2014 to June 1, 2014 15-min historical data and backtest the same criteria as Tradingview just resets the timeframe to Weekly to be able to show that far back.
Any ideas are greatly appreciated as I currently do things in excel and it takes a while per symbol.
Thanks
Related
Please, I created my table using hour time partition. Please, I would like would like to fetch data that was stored in my table in the last X minutes, eg last 5 minutes.
I tried using this command
SELECT *
FROM mydataset.mytable
FOR SYSTEM_TIME AS OF TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 5 MINUTE);
But, it returns a lot more rows than what is expected. I typically store 500 rows every 2 minutes, but this query is returning more than 30000 rows
As #Samuel mentioned in the comment, below example query can be considered to fetch data for a specific time period in BigQuery.
Select * from `dataset.table`
WHERE col_timestamp < TIMESTAMP_SUB(CURRENT_TIMESTAMP(),
INTERVAL 5 MINUTE)
Posting the answer as community wiki for the benefit of the community that might encounter this use case in the future.
Feel free to edit this answer for additional information.
I am working on a forecasting graph that will be pulling in data from SQL through power query. The graph will have slicers and filters available to the user to analyze different populations. I am currently being asked to provide a rolling average for each day of the next month but will update daily as more data becomes available. I need 30 days of rolling average that occur after the current date. Using the prior days rolling average calculation. This has to be completed in dax to allow the filters and slicers to work appropriately. The rolling average looks 14 days back.
Any ideas on how to best proceed would be greatly appreciated.
Below is a table of data to show how this looks in excel:
I have issues with Data sorting
I have spend data by period which is a combination of year and month. For example if the period is March 2019 then period would show 201903. Data type of this period field is "Number". I have spend data starting from period 201501 to 201903.
Now I am creating a line chart to show spend trend with period in X axis and spend in Y axis. I have sorted (ascending) the chart by period, however sorting works ok partially, means period that I see in the chart are mentioned below:-
201708,201801,201806,201807,201808,201809,201810,201811,201812,201901,201902,201903
Not sure why sorting option is not working
Also in the Visual Level filter I want to show trailing 12 months spend, details of Visual Level filters that i selected are as follows:-
Visual Level Filter - Period
Filter Type - Top N
Show item - Bottom 12
By Value - Spend Amount
The expected result that I want in the line chart is :-
201804,201805,201806,201807,201808,201809,201810,201811,201812,201901,201902,201903
I checked the data and confirm it has all periods starting from 201501 to 201903
Any help would be highly appreciated
Can we use query while retrieving the data from the dataset in AWS IoT Analytics, I want data between 2 timestamps. Im using boto3 to fetch the data. I didn't see any option to use query in get dataset content Below is the boto3 code:
response = client.get_dataset_content(
datasetName='string',
versionId='string'
)
Does anyone have suggestions how to use query or how rerieve the data between 2 timestamp in AWS IoT Analytics?
Thanks,
Pankaj
There could be a few ways to do this depending on what your workflow is, if you have a few more details, that would be helpful.
Possible approaches are;
1) Create a scheduled query to run every hour (for example) where the query looks something like this;
SELECT * FROM my_datastore WHERE __dt >= current_date - interval '1' day
AND my_timestamp >= now() - interval '1' hour
You may need to adjust the format of the timestamp to suit depending on how you are storing it (epoch seconds, epoch milliseconds, ISO8601 etc. If you set this to run every hour, each time it executes, you will get the last one hour of data. Note that the __dt constraint just helps your query run faster (and cheaper) by limiting the scan to the most recent day only.
2) You can improve on the above by using the delta window function of the dataset which lets you get the data that has arrived since the query last ran more easily. You could then simplify your query to look like;
select * from my_datastore where __dt >= current_date - interval '1' day
And configure the delta time window to look at your timestamp field. You then control how much data is retrieved by the frequency at which you execute the query (every 15 mins, every hour etc).
3) If you have a more general purpose requirement to fetch the data between 2 timestamps that you are calculating programatically, and may not be of the form now() - some interval, the way you could do this is to create a dataset and then update the dataset with the revised SQL expression before running it with create-dataset-content. That way the dataset content is updated with just the results you need with each execution. If this is of interest, I can expand upon the actual python required.
4) As Thomas suggested, it can often be just as easy to pull out a larger chunk of data with the dataset (for example the last day) and then filter down to the timestamp you want in code. This is particularly easy if you are using panda dataframes for example and there are plenty of related questions such as this one that have good answers.
Frankly, the easiest thing would be to do your own time filtering (the result of get_dataset_content is a csv file).
That's what QuickSight does to allow you to navigate the dataset in time.
If this isn't feasible the alternative is to reprocess the datastore with an updated pipeline that filters out everything except the time range you're interested in (more information here). You should note that while it's tempting to use the startTime and endTime parameters for StartPipelineReprocessing, these are only approximate to the nearest hour.
I am currently developing a sentiment index using Google search frequencies taken from Google Trends.
I am using Stata 12 on Windows.
My approach is as following:
I downloaded approx ~150 business-related search queries from Googletrends from Jan 2004 to Dec 2013
I now want to construct an index using the 30 at that point in time most relevant queries related to the market I observe
To achieve that I want to use monthly expanding backward rolling regressions of each query on the market
Thus I need to regress 150 items one-by-one on the market 120 times (12 months x 10 years), using different time windows and then extract the 30 queries with the most negative t-test.
To exemplify the procedure, if I would want to construct the sentiment for January 2010 I would regress the query terms on the market during the period from Jan 2004 to December 2009 and then extract the 30 queries with the most negative t-statistic.
Now I am looking for a way to make this as automatized as possible. I guess should be able to run the 150 items at once, and I can specify the time window using the time stamps. Using Excel commands and creating a do-file with all the regression commands in it (which would be quite large) I could probably create the regressions relatively efficiently (although it depends on how much Stata can handle - any experience on that?).
What I would need to make the data extraction much easier is a command which I can use to rank the results of the regression according to their t-statistics. Does someone have an efficient approach to this? Or has general advice?
If you are using Stata, once you run a ttest, you can type return list and you will get scalars that stata stores. Once you run a loop you can store these values in a number of different ways. check out the post command.