I have an application in which allow users to query some data based on date. The application supports different westerns languages and the user is allowed to input the query in their format.
the various formats includes
15.10.2012
15/10/2012
10/15/2012
15-Oct-2012
15-Okt-2012
The last one is a different language and the look up table doesn't fit well for this because I need to define the lookup tables to the N number of languages I support.
I believe there could be some solution MFC/C++ Library/Windows platform itself to extract the date information in various valid format. Please help if there are some libraries/APIs sort of.
In most of the modern GUIs the user are not required to enter the date/time by keying-in. Usually there are some sort of calender controls which helps the user to enter the date-time. Can't you use some of these controls (for e.g : http://msdn.microsoft.com/en-us/library/hhdec0z1(v=vs.80).aspx) to restrict the user input? And you can always display the date/time back to the user in the current locale using the microsoft function strftime().
Related
I am trying to add data to a Map with Danish postcodes, e.g. 1000-9000 (we have four digits in Denmark).
When I add that into a Map, it scatters all over the world, as Power BI do not recognize it as Danish locations, even my Power BI is setup in Danish and the Map has Danish spelled city names.
I tried to add the regions Jylland, Fyn, Sjælland as a country hierarchy, but doing that moved Jylland (Jutland) as a place in Norway...
I also tried to use city names instead of post codes, but then a city shows up in Sweden...
It does not change whether the post code format is Text or Number format, and I have no option to use a Location format in the query.
Can anyone help me use Danish post codes for Map visualization? : )
Thanks
Ok, I solved it myself!
I found the place in the modelling part where I could force PowerBI to accept my city names, region names etc. and it now works.
More detailed: go into the middle of the three left side windows called Data (not Report, not Model), and click on the column you want to change format of. Then find the Tools section and change the Data Category to for example Address, or Country etc. Hope that helps
I need your help.
I work for a survey company and I am responsible for creating its architecture and modeling a data warehouse that analyzes the results of an international survey (50 countries).
For the architecture, we decided to create a tabular model in PowerBI to analyze our data and to create our reports.
Here below is the model as I thought:
However, I have a design problem.
Since the survey is international, the wording of my dimensions differs from country to country.
My 1st question:
-Would it make more sense to create only one PowerBI embedded model for all countries or 50 PowerBI reports?
My 2nd question:
My model must be multilingual
With my 50 countries, I have several languages (5 languages) and for the same language, I have several variants.
The British English labels differ from the US English labels.
For example, for the Response dimension for France the IdReponse = 1 has the wording 'Vrai' while for the USA the wording is 'True' and for the Britain is 'OK'.
Do you know how to model multi language in a data warehouse?
About question #1 - It's always better, if there is only one model. It will be much easier to maintain. It isn't clear from your question will these 50 reports show the same data (excluding the internationalization of texts like Vrai/True/OK), or each report/country should show it's own subset of the data. In case all reports will show the same data, then definitely it will be better to make one common model and all report use it. You can do this with Power BI by making one "master" report and publishing it, and then the rest of your "per country" reports use it as a data source. And you will need separate reports per country, because you will need to translate the texts (column names, static texts, etc.).
About question #2 - You can create lookup tables in your model (maybe even in the database, it's up to you). The key value (1) will be linked to the key of the table, and there will be columns per language. Depending on the language of the current report, you will select the appropriate column (e.g. French, British, etc.) and even you can fallback to let's say US English, in case there is no translation entered for the current language (e.g. by making a computed column). It is also an option to make separate lookup table per language, but I think it will be more cumbersome to maintain this way.
About question #1: Yes you need only one data model.
About question #2: You Load a question in the language it is asked and the response you get as is in the response DIM. You should create a new column in your response DIM such as Clean_response where you transformed original response to a uniformed value. for example "Vrai", "OK", "True" has same meaning so you may chose to put "Yes" in the Clean_response column. You can also convert different variation of "No", "Nada", "noops", "nah" to a clean value of "No", but keep the original value too.
Labeling a column in the report should be handle in the report code. For example writing a report in French should use your dim column name "Question" and show it as "interroger" as a heading on the report.
I have faced this problem in other projects, but right now we're working in Django / Python with Vue.js and PostgreSQL.
Nearly every 'date' field in our model is truly a date, not a datetime. The business users don't care about a time-part, in fact storing any such value is misleading. A good example is the effective date on a tax rate. From the user point of view, it takes effect at midnight on the specified date.
If we store this as a Django DateTimeField or DateField, it requires a time-part. It could be 0000h, but the value has no meaning to the business, because they will never need to see or compare anything with the time-part. The Django functionality is great if we want to store a real datetime, put it into the database as UTC, and then display it to users in their local time through the automatic time zone conversion. But we don't. If the effective date is March 1, 2019, it should display as such regardless of the user's timezone. If the date is stored as a datetime (2019, 3, 1, 0, 0, 0) and entered by someone in Vancouver, it will appear as the next calendar day for another user in Toronto. Definitely not what we want. We could kludge it by setting the time-part to 1200h, but really?
We also have potential problems, depending on the internal representation in the database, when using SQL or tools that access the schema directly (e.g. BI tools). How do we know what time zone applies to the datetime value?
So, we're thinking of using Django CharField with ISO 8601 format (YYYY-MM-DD) instead. It will sort properly, it can be compared easily (or directly in some tools like SQL), and can be displayed without reformatting if the client is willing to use the standard. If we need to do date arithmetic, we can use the Python Standard Libraries datetime and calendar to convert from/to string. We'll need to use those to catch SQL injection attacks anyway.
We will also need to deal with date entry through a Datepicker, converting to the ISO 8601 string before storing and back again when displaying for edit.
It appears to be a better way to represent what the business needs, and it gets rid of any timezone conversion issues.
There is certainly a lot of comment on datetime and time zone handling, but I haven't found anyone taking this approach to storing true dates. Am I missing an important 'gotcha'? We're early enough in the project that we can go either way, so I'm hoping to confirm that this will work before refactoring becomes a big job.
Have you considered using DateField?
This will only store the date part and not the time.
I am wondering how I can efficiently manage formats in SAS for a reporting office that takes in data from various sources, some with proper lookup tables / metadata, and some without.
For data sources that have proper metadata, joining tables for value descriptions works fine, but when metadata doesn't exist and needs to be maintained separately, how should that be done? Some straightforward examples/ideas:
Plain .sas files with a native PROC FORMAT step that is maintained separately.
External files (e.g., Excel, CSV) that are maintained separately and imported into SAS to create a format library.
Database tables maintained separately that can be read from to create a format library.
In addition to just the formatted values, managing value changes (i.e., effective dates for certian values) is also a concern.
Any help in conventions or standards that work well for this type of task is greatly appreciated.
I'm not sure there's a single best solution here - it depends largely on your environment, your users, etc.
If you have fairly naive users, then I'd definitely recommend a single complete repository if possible; whether that is a .sas7bcat file if you are using a single SAS version/OS/bitness, or a ready-made table/dataset to input into PROC FORMAT (and a .sas file included in their autoexec to do the importing). The biggest drawbacks to this are that you have to manage it actively (you cannot allow users to write their own formats to the master format dataset, for example, as they may overwrite other ones), and that there will be additional work to ensure format names do not conflict - YNF. might be 1=YES 2=NO or 1=YES 0=NO or something else. This also doesn't allow you to very easily handle effective dates; but it's possible this is better for your users (and then just handle the documentation separately).
If you have more advanced users, then you might consider a table/dataset that is more relational in nature. A hybrid approach might include a dataset with columns:
Dataset Name (qualified as needed to ensure uniqueness)
Format Name
Start
Label
Other elements (Type, HLO, etc.)
Effective date
That would allow users to make their own modifications (assuming you trust them enough to add dataset name properly, anyway - or set up a stored proc to do the adding from a temp table that checked for conflicts) and allow you to handle format names that conflicted. You'd still have to have a way for the user to handle using multiple datasets, if that's necessary (such as by adding some unique element to the format name by default, like 'dataset ID').
In my mind, however, the best option is using a data dictionary to handle the metadata, which combines self-documentation with metadata management. Similar to above, you have a table with dataset and format elements, but add columns for descriptive text (question description, for example) and other useful information, depending on your use cases. This can be maintained in a database table or dataset, or perhaps more usefully in an excel or similar document that can be shared with non-programmers and easily edited. I use this method for several projects, and it has paid off by allowing my users to help write the documentation for my code, keeping my programs accurate and up to date, while minimizing back-and-forth discussions of updates. I just import the spreadsheet and run a proc format each time I run my data.
You can then have one spreadsheet per dataset, one tab, or one full spreadsheet with all datasets in them - whichever is easiest to use. This easily handles 'effective date' type issues as well - or even versioning, as that can be handled in the spreadsheet.
I need to match on a date using the LIKE operator. The date should be in current user's local format, not in the general format.
So I use the strftime function like this:
WHERE strftime('%d.%m.%Y %H:%M', createdDate, 'localtime') LIKE '%{searchTerm}%'
Unfortunately this works only for fixed format '%d.%m.%Y %H:%M'. But I want to use the user's current locale format.
So I need to either:
1) Get the strftime format string from a C++ locale object
2) Make SQLite format the date using current locale itself
Spent already several hours on this to no avail.
Would be grateful if anyone would point me in the right direction on how to tackle this problem.
Example:
Given current locale is German I want to get something like "%d.%m.%Y %H:%m".
For an US locale I want to get "%m/%d/%Y %H:%m"
Normally the local date format can be obtained with the Windows GetDateFormat API (or GetDateFormatEx API for Vista). Your program could interrogate the API then transform the date accordingly. Following that, the date can be recorded in SQLite.
However, once can question the validity of storing timestamps in a specific format. That basically means a lot of code to manipulate each date, or no date manipulation at all. May I suggest, if it is possible, storing in a plain format (say ISO or UNIX timestamp) and working from that, outputing with whichever flavour of GetDateFormat is required?
OK, different answer.
Suppose you have your MyTable:
CREATE TABLE MyTable (
MyPrimaryKeyHnd INTEGER PRIMARY KEY,
...
CreatedDate TEXT);
(Where CreatedDate is in ISO format. I suppose you could also use a Unix timestamp. Your choice.)
Then a list of possible formats
CREATE TABLE TimeFormat (
TimeFormatHnd INTEGER PRIMARY KEY,
TimeFormatString TEXT NOT NULL,
TimeFormatDescriptor TEXT);
You allow your user to chose a format of their choice and keep that in a seperate table or INI file. TimeformatString would be your strftime() compatible format string (such as '%d.%m.%Y %H:%M'). You just need to build your query with whatever the user's choice is.