I would like to convert a column called lauder from int to float in Big Query. My table is called historical. I have been able to use this SQL query
SELECT *, CAST(lauder as float64) as temp
FROM sandbox.dailydev.historical
The query works but the changes are not saved into the table. What should I do?
If you use SELECT * you will scan the whole table and thus will be the cost. If table is small this shouldn't be a problem, but if it is big enough to be concern about cost - below is another approach:
apply ALTER TABLE ADD COLUMN to add new column of needed data type
apply UPDATE for new column
UPDATE table
SET new_column = CAST(old_column as float64)
WHERE true
Do you want to save them in a temporary table to use it later?
You can save it to a temporary table like below and then refer to "temp"
with temp as
( SELECT *, CAST(lauder as float64)
FROM sandbox.dailydev.historical)
You can not change a columns data type in a table
https://cloud.google.com/bigquery/docs/manually-changing-schemas#changing_a_columns_data_type
What you can do is either:
Create a view to sit on top and handle the data type conversion
Create a new column and set the data type to float64 and insert values into it
Overwrite the table
Options 2 and 3 are outlined well including pros and cons in the link I shared above.
Your statement is correct. But tables columns in Big Query are immutable. You need to run your query and save results to a new table with the modified column.
Click "More" > "Query settings", and in "Destination" select "Set a destination table for query results" and fill the table name. You can even select if you want to overwrite the existing table with generated one.
After these settings are set, just "Run" your query as usual.
You can use CREATE or REPLACE TABLE to write the structural changes along with data into the same table:
CREATE OR REPLACE TABLE sandbox.dailydev.historical
AS SELECT *, CAST(lauder as float64) as temp FROM sandbox.dailydev.historical;
In this example, historical table will be restructured with an additional column temp.
In some cases you can change column types:
CREATE TABLE mydataset.mytable(c1 INT64);
ALTER TABLE mydataset.mytable ALTER COLUMN c1 SET DATA TYPE NUMERIC;
Check conversion rules.
And google docs.
Related
My dimension tables contains more rows than my fact table, I would like my dim table fields to show only the values in the fact table when used as a filter in the filter panel.What cleaning/modeling steps are the best to achieve this.
I also know how to write sql if that is an option for an answer.
Rather than use a table for your dimension, use a view that has an inner join to the fact table
I have created BQ table with two columns:
col1 string nullable
col2 string required
And next populated table with some dummy data:
insert into `test.test` values ('val1', 'val2')
insert into `test.test` values (null, 'val2')
insert into `test.test` values ('val1', 'val2')
After that I dropped single column:
Alter table `test.test` drop column col2
And after that I would like to add new column:
alter table `test.test` add column col3 string
HINT: New column that I am trying to add is named differently than the one I deleted.
BQ raises an error to me:
Column `col2` was recently deleted in the table `test`. Deleted column name is reserved for up to the time travel duration, use a different column name instead.
It seems to be not right. I know that deleted columns is still kept somewhere in BQ world, but I am trying to add column with different name than the deleted one.
Any idea?
This is a known issue and is being worked upon by Bigquery Engineering team. You may click on +1 to bring more attention on the issue and STAR the issue so that you can be notified for updates.
Meanwhile, as a workaround you try adding new field via
BigQuery UI
bq command line
API or Client Library
Is there any way to store the output of BigQuery merge in a new data table? If not, is it possible to view the results somewhere in GCP? I don't want to modify the target table.
This is the query I'm using-
MERGE dataset.target T
USING dataset.source S
ON T.c1 = S.c1
WHEN NOT MATCHED BY SOURCE THEN
DELETE
If you want to see the rows deleted, you can copy dataset.target into dataset.target2, then execute your MERGE on dataset.target, then see the difference between dataset.target2 and dataset.target:
SELECT * FROM dataset.target2
EXCEPT
SELECT * FROM dataset.target
I want to create a second table from the first table using filters with dates and other variables as follows. How can I create this?
Following is the expected table and original table,
Go to Edit Queries. Lets say our base table is named RawData. Add a blank query and use this expression to copy your RawData table:
=RawData
The new table will be RawDataGrouped. Now select the new table and go to Home > Group By and use the following settings:
The result will be the following table. Note that I didnt use the exactly values you used to keep this sample at a miminum effort:
You also can now create a relationship between this two tables (by the Index column) to use cross filtering between them.
You could show the grouped data and use the relationship to display the RawDate in a subreport (or custom tooltip) for example.
I assume you are looking for a calculated table. Below is the workaround for the same,
In Query Editor you can create a duplicate table of the existing (Original) table and select the Date Filter -> Is Earliest option by clicking right corner of the Date column in new duplicate table. Now your table should contain only the rows which are having minimum date for the column.
Note: This table is dynamic and will give subsequent results based on data changes in the original table, but you to have refresh both the table.
Original Table:
Desired Table:
When I have added new column into it, post to refreshing dataset I have got below result (This implies, it is doing recalculation based on each data change in the original source)
New data entry:
Output:
Today is my first day to use PowerBI 2.0 Desktop.
Is there any way to add new columns from external data into the existing table in my PowerBI?
Or is there anyway to add new columns from another table in PowerBI?
It seems that, in PowerQuery, all the tabs Add Custom Column, Add Index Column and Duplicate Column are all using the existing columns in the same table.....
You can use Merge Queries to join together two queries, which will let you bring in the other table's columns.
Also, Add Custom Column accepts an arbitrary expression, so you can reference other tables in that expression. For example, if Table1 and Table2 had the same number of rows, I could copy over Table2's column by doing the following:
Add an Index Column. Let's call it Index.
Add a Custom Column with the following expression: Table2[ColumnName]{[Index]}