Redshift derive column value based on other column - amazon-web-services

Question

Let's say, I have 3 date columns (d1,d2,d3) in a redshift table.
d1 = max(d2,d3)
Instead of my application computing the value and setting it, during insert, if I specify only d2 and d3, can redshift auto-populate d1 = max(d2,d3)?

Answer 1

There are two ways to load data into Amazon Redshift.
The first is via the COPY command, when data is loaded from files stored in Amazon S3. Each column in a file will be mapped to one column in a table, so you cannot 'compute' a column during this process.
The second is via an INSERT command. This is not very efficient when used with Amazon Redshift and preferably is used to insert bulk rows rather than one row at a time.
A common practice is to load the data into a staging table, manipulate it as desired, then re-insert it into the target table.
You might even be able to do some fancy stuff with Redshift Spectrum where you can SELECT directly from files in S3 and insert into a table. This would also allow you to include terms, eg:
INSERT INTO normal-table
SELECT max(d2,d3), d2, d3 FROM spectrum-table
An alternative is to load the data, then use an UPDATE command to set the value of the extra column based upon existing columns.
Update:
It appears that using an UPDATE statement in Amazon Redshift (and, in fact, in any columnar database) is not a good idea. This is because each column is stored separately but in the same order. Updating one value requires the whole row to be re-written at the end of the storage space, rather than updated in-place. Thus, you'd need to VACUUM the database after such updates.

Answer 2

In postgresql (on which redshift is based), you can do what you want like this:
create table test (a int, b int, c int);
insert into test (a, b, c)
values (1, 2, greatest(1, 2))
(4, 1, greatest(4, 1));
It should also work in redshift, although I can't verify that at the moment. But this won't work for bulk loading data via the copy command.
If the above doesn't work, the other option would be to insert data, and then set column c using an update query.
insert into test (a, b) values (1, 2);
update test set c = greatest(a, b) where c is null;
for bulk loading, it is necessary to load data into columns a & b first using the copy command, then use the update query to set the value of column c