In my current project I'm trying to check a MySQL database: a MySQL database is updated by another program, so my C++ program needs to select only the new rows. It is not going to be a small table (>10000 rows), so I do not want to search each row. i.e. checking a column like isNew=0 or =1. I already found:
Query to find tables modified in the last hour
http://www.codediesel.com/mysql/how-to-check-when-a-mysql-table-was-last-updated/
However, in this example you can only get the table which is updated. How can I only select the new rows from a table?
How can I only select the new rows from a table?
Assuming new rows means newly inserted, and if you can change the database schema, you could use an auto increment column. By remembering the largest value each time your program selects a result set, it could save that value for the next query:
select * from table where id > 123
I would recommend adding an isNew column to the table with default value 1 and add an index on it. The index will prevent your query from checking all rows. After you have processed the row, set isNew to 0.
Related
I have created BQ table with two columns:
col1 string nullable
col2 string required
And next populated table with some dummy data:
insert into `test.test` values ('val1', 'val2')
insert into `test.test` values (null, 'val2')
insert into `test.test` values ('val1', 'val2')
After that I dropped single column:
Alter table `test.test` drop column col2
And after that I would like to add new column:
alter table `test.test` add column col3 string
HINT: New column that I am trying to add is named differently than the one I deleted.
BQ raises an error to me:
Column `col2` was recently deleted in the table `test`. Deleted column name is reserved for up to the time travel duration, use a different column name instead.
It seems to be not right. I know that deleted columns is still kept somewhere in BQ world, but I am trying to add column with different name than the deleted one.
Any idea?
This is a known issue and is being worked upon by Bigquery Engineering team. You may click on +1 to bring more attention on the issue and STAR the issue so that you can be notified for updates.
Meanwhile, as a workaround you try adding new field via
BigQuery UI
bq command line
API or Client Library
I have created a query on three tables. Here there is a column GUID (which is actually unique), then there is a column date/time and a column with a phone number.
Now if the same phone number calls more than once, get duplicates with always the same GUID. Can I filter this in Power BI so that the ID only appears once?
If I understand your requirement correct, There is an option in Power Query Editor to remove all duplicate rows. You can first select your all 3 columns- guid, date_time and phone_number. Now right click on any of the Column's header and select Remove Duplicate from the list as shown in the below sample image-
This should keep only 1 rows per distinct combination considering 3 columns.
I want to create a second table from the first table using filters with dates and other variables as follows. How can I create this?
Following is the expected table and original table,
Go to Edit Queries. Lets say our base table is named RawData. Add a blank query and use this expression to copy your RawData table:
=RawData
The new table will be RawDataGrouped. Now select the new table and go to Home > Group By and use the following settings:
The result will be the following table. Note that I didnt use the exactly values you used to keep this sample at a miminum effort:
You also can now create a relationship between this two tables (by the Index column) to use cross filtering between them.
You could show the grouped data and use the relationship to display the RawDate in a subreport (or custom tooltip) for example.
I assume you are looking for a calculated table. Below is the workaround for the same,
In Query Editor you can create a duplicate table of the existing (Original) table and select the Date Filter -> Is Earliest option by clicking right corner of the Date column in new duplicate table. Now your table should contain only the rows which are having minimum date for the column.
Note: This table is dynamic and will give subsequent results based on data changes in the original table, but you to have refresh both the table.
Original Table:
Desired Table:
When I have added new column into it, post to refreshing dataset I have got below result (This implies, it is doing recalculation based on each data change in the original source)
New data entry:
Output:
Today is my first day to use PowerBI 2.0 Desktop.
Is there any way to add new columns from external data into the existing table in my PowerBI?
Or is there anyway to add new columns from another table in PowerBI?
It seems that, in PowerQuery, all the tabs Add Custom Column, Add Index Column and Duplicate Column are all using the existing columns in the same table.....
You can use Merge Queries to join together two queries, which will let you bring in the other table's columns.
Also, Add Custom Column accepts an arbitrary expression, so you can reference other tables in that expression. For example, if Table1 and Table2 had the same number of rows, I could copy over Table2's column by doing the following:
Add an Index Column. Let's call it Index.
Add a Custom Column with the following expression: Table2[ColumnName]{[Index]}
I have a table in prod which is having huge data, recently I have added new column for that table and loaded surrogate values for new rows. Existing rows are null's.
Now I want to update this new columns with max value+1 incremental values where new columns are null. Can any one please provide solution for this scenario.
Thanks in advance...