I am wondering if it is possible to specify the time range for past data in GFS, NAM, RAP, HRRR, and the NDFD forecasting models! the current option is recalling data for the present daya as stated here
`start = pd.Timestamp(datetime.date.today(), tz=tz)`
end = start + pd.Timedelta(days=7)
The source code for the model.get_data() function indicates yes. The only stipulation is that the start and end arguments must be datetime or timestamp datatypes to be compatible with the netcdf2pandas library
Related
I would like to store a time range (without dates) like "HH:MM-HH:MM".
Can you give me a hint, how can I implement it most easily? Maybe there is a way to use DateRangeField for this aim or something else.
Thanks for your spend time!
Time without date doesn't make much since if you ever need a range that span mid-night (days) You could always convert to text using to_char(<yourtimestamp>,'hh24.mi:ss') or extract the individual parts. Unfortunately Postgres does not provide an extract(time from <yourtimestamp>) function. The following function provides essentially that.
create or replace
function extract_tod(date_time_in timestamp without time zone)
returns time
language sql
as $$
select to_char(date_time_in, 'hh24:mi:ss')::time;
$$;
See here for timestatamp ranges and here for their associated functions. As for how to store then just store with the date
as a standard TIMESTAMP (date + time).
Wrote something in .NET; it works well. Now I am trying to rewrite it as a shell extension with the Win32 API. Ultimately I want to convert FILETIMEs to and from ISO-8601 strings. This is doable without fuss, using GetTimeZoneInformation and FileTimeToSystemTime and SystemTimeToTzSpecificLocalTime and StringCchPrintf to assemble the members of the SYSTEMTIME and TIME_ZONE_INFORMATION structs into a string.
The problem, as usual when working with date/times, is Daylight Saving Time. Using GetTimeZoneInformation tells me the UTC offset that's in effect now. Using .NET's DateTime.ToString("o") takes into account the daylight saving time at the time represented in the DateTime.
Example for the same FILETIME:
Output of ToString("o"): 2017-06-21T12:00:00.0000000-05:00
Output of chained APIs: 2017-06-21T12:00:00-06:00
The UTC offset is wrong coming from the chained API calls. How does .NET's DateTime do it? How do we replicate that in Win32? Is there something like GetTimeZoneInformationForYear, but instead of for a year, for a moment in local time?
First, I use DYNAMIC_TIME_ZONE_INFORMATION structure and GetDynamicTimeZoneInformation
DYNAMIC_TIME_ZONE_INFORMATION and TIME_ZONE_INFORMATION also has a DaylightBias member:
The bias value to be used during local time translations that occur
during daylight saving time. This member is ignored if a value for the
DaylightDate member is not supplied.
This value is added to the value of the Bias member to form the bias
used during daylight saving time. In most time zones, the value of
this member is –60.
So, if the date is in daylight saving time, you need to add this DaylightBias to Bias.
In addition, you can determine whether the current date is daylight saving time according to the description in DaylightDate:
To select the correct day in the month, set the wYear member to zero,
the wHour and wMinute members to the transition time, the wDayOfWeek
member to the appropriate weekday, and the wDay member to indicate the
occurrence of the day of the week within the month (1 to 5, where 5
indicates the final occurrence during the month if that day of the
week does not occur 5 times).
If the wYear member is not zero, the transition date is absolute; it
will only occur one time. Otherwise, it is a relative date that occurs
yearly.
I have a dataset that I am trying to manipulate in GraphLab. I want to convert a UNIX Epoch timestamp from the input file (converted to an SFrame) into a human readable format so I can do analysis based on hour of day and day of week.
time_array is the column/feature of the SFrame sf representing the timestamp, I have broken out just the EPOCH time to simplify things. I know how to convert the time of one row, but I want a vector operation. Here is what I have for one row.
time_array = sf['timestamp']
datetime.datetime.fromtimestamp(time_array[0]).strftime('%Y-%m-%d %H')
You can also get parts of the time from the timestamp to create another column, by which to filter (e.g., get the hour):
sf['hour'] = [x.strftime('%H')for x in sf['timestamp']]
So after staring at this for awhile and then posting the question it came to me, hopefully someone else can benefit as well. Use the .apply method with the datetime.datetime() function
sf['date_string'] = sf['timestamp'].apply(lambda x: datetime.datetime.fromtimestamp(x).strftime('%Y-%m-%d %H'))
you can also use the split_datetime API to split the timestamp to multiple columns:
split_datetime('timestamp',limit=['hour','minute'])
I'm reading timestamp fields from a PostgreSQL database. The timestamp column is defined as:
my_timestamp TIMESTAMP WITH TIME ZONE DEFAULT NOW()
When reading from the database, I convert it to a boost timestamp like this:
boost::posix_time::ptime pt( boost::posix_time::time_from_string( str ) );
The problem seems to be that boost::posix_time::time_from_string() ignores the timezone.
For example:
database text string == "2013-05-30 00:27:04.8299-07" // note -07 timezone
boost::posix_time::to_iso_extended_string(pt) == "2013-05-30T00:27:04.829900"
When I do arithmetic with the resulting ptime object, the time is off by exactly 7 hours. Is there something better I should be doing to not lose the timezone information?
I think you should be using boost::local_date_time, which handles time zones. There is an example in the documentation that is very similar to what you're trying to do: http://www.boost.org/doc/libs/1_41_0/doc/html/date_time/examples.html#date_time.examples.seconds_since_epoch
EDIT: Boost supports date parsing with specific formats. http://www.boost.org/doc/libs/1_40_0/doc/html/date_time/date_time_io.html#date_time.format_flags
string inp("2013-05-30 00:27:04.8299-07");
string format("%Y-%m-%d %H:%M:%S%F%Q");
date d;
d = parser.parse_date(inp,
format,
svp);
// d == 2013-05-30 00:27:04.8299-07
I originally asked this question so many years ago, I don't even remember doing it. But since then, all my database date/time code on the client side has been greatly simplified. The trick is to tell PostgreSQL the local time zone when the DB connection is first established, and let the server automatically add or remove the necessary hours/minutes when it sends back timestamps. This way, timestamps are always in local time.
You do that with a 1-time call similar to this one:
SET SESSION TIME ZONE 'Europe/Berlin';
You can also use one of the many timezone abbreviations. For example, these two lines are equivalent:
SET SESSION TIME ZONE 'Asia/Hong_Kong';
SET SESSION TIME ZONE 'HKT';
The full list of timezones can be obtained with this:
SELECT * FROM pg_timezone_names ORDER BY name;
Note: there are over 1000 timezone names to pick from!
I have more details on PostgreSQL and timezones available on this post: https://www.ccoderun.ca/programming/2017-09-14_PostgreSQL_timestamps/index.html
Similar to:
Convert historical time to GMT
This time my scenario is:
I need to convert some string times in the format "2011061411322100" into GMT. The problem is that the times are coming from another PC and is a historical time. So I am not getting the times in real time so I cannot simply get the GMT from the local time on the box that my code is running.
The times represent Start and End times. If my code is running during a time change, the time change will bre applied on the remote box where I am getting the times. So if the change happens betwwen the start and end time, then the end time will be offset.
I assume I must first convert to tm:
// add to tm struct
tm tmTime;
tmTime.tm_hour = hour;
tmTime.tm_min = min;
tmTime.tm_sec = sec;
tmTime.tm_mday = day;
tmTime.tm_mon = (month-1);
tmTime.tm_year = (year - 1900);
Then convert to time_t
time_t tmInTime_t = mktime( &tmTime );
Then use gmtime:
struct tm *gmt = gmtime( &tmInTime_t );
This will still cause large delta if time change happens between start and end. How do I fix?
Do I need to set .tm_isdst? How do I know what to set to?
You ABSOLUTELY must know what the reference timezone was on the source data. Simply saying it was local time on the source machine doesn't provide enough info.
Consider (for one example out of many): The DST rules changed in the US in 2007, and any date calculations spanning across that range must take that into account. When you start dealing with timezones across political borders things get vastly more complicated.
http://www.boost.org/doc/libs/1_47_0/doc/html/date_time.html
is an example of a robust time/date library written in C++ and works on most platforms. It will allow you to perform various 'arithmetic' operations on dates and times. It also includes support for the TZ database, which will allow you to perform operations on datetimes spanning timezone rule changes.