I am currently using puckel airflow but now the apache airflow image is also available.
Which one is better and reliable. And given I need to start from scratch, which option would be better?
Official. Puckel is no longer updated
I started with puckel airlfow but as Javier mentioned its no longer updated. Last updated version was 1.10.9. Its easier to start with this image and following updates and mimicking required behaviours from official docker image you can build on it.
Related
Up until today, my deploy process has worked fine. Today when I go to deploy a new revision, I get stuck at the Deploying... text with a spinning indicator, and it says One or more of the referenced revisions does not yet exist or is deleted. I've tried a number of different images and flags -- all the same.
See Viewing the list of revisions for a service, in order to undo whatever you may have done.
Probably you have the wrong project selected, if it does not know any of the revisions.
I know I provided scant information, but just to follow up with an answer: it looks like the issue was that I was deploying a revision, and then immediately trying to tag it using gcloud alpha run services update-traffic <service_name> --set-tags which looks to have caused some sort of race, where it complained that the revision was not yet deployed, and would hang indefinitely. Moving the set-tag into the gcloud alpha run deploy seemed to fix it.
I recently started a Spark cluster on Google Cloud Dataproc using the 'preview' image. According to the documentation, the preview image's Spark version is '2.1.0', however running spark-shell --version reveals that the cluster is in fact running Spark 2.2.0. This is a problem for us, because our version of spark-avro is not compatible with Spark 2.2.0. Is anyone else experiencing this issue? I haven't been able to find any trace of an official announcement from Google regarding the version bump.
Sorry about that, it appears the minor release notes for the recent preview image update got lost in the ether; the documentation should hopefully be updated by tomorrow. Indeed you're right that the current Dataproc preview version is now Spark 2.2.0. If you need to pin to a known working older preview image, you can try:
gcloud dataproc clusters create --image https://www.googleapis.com/compute/v1/projects/cloud-dataproc/global/images/dataproc-1-2-20170227-145329
That should contain Spark 2.1.0. That said, keep in mind that in general it's always possible that incompatible changes may be made in new preview images, and pinning to that older preview image isn't guaranteed to continue working long term.
In your case, do you happen to know whether you're hitting this issue filed on spark-avro or is it something specific to your version? Ideally we should get you updated to Spark 2.2, since an official (non-preview) image version is going to be imminent with Spark 2.2.
I have a tool which requires boto3's direct connect functionality. However I cannot tell from which version onwards boto3 includes this functionality.
Is there any way to obtain the older version of the docs to see at which point direct connects were added?
At this point in time the latest is 1.4.4.
boto3 development is an open source initiative. So you can always checkout botocre development repository for the release notes, or ask question there.
changelog.rst will show a brief list of changes.
Then use following command to switch to specify tag version and look for the date of commit.
git checkout tags/<version_name>
I've just setup wercker for python3.4, it seems good - but I am not sure how I should get it to install/configure postgis and django.
There is some information related: http://blog.wercker.com/2013/11/18/django-16.html and https://github.com/wercker/wercker-django-example/blob/master/wercker.yml - but it seems rather outdated.
The docs you refer to are of the "old" wercker. They've transitioned to a Docker based build system now, which is the recommended way to go.
The new (docker based) documentation can be found here.
So basically all you need to do is build using a docker image that already contains Django and Postgis. This one might work.
The Sitecore documentation provides some pretty clear instructions on how to configure a Sitecore instance as a processing server:
https://doc.sitecore.net/sitecore_experience_platform/xdb_configuration/configure_a_processing_server
However, many of those steps require enabling/disabling of files manually on the installed server. Has anybody seen or built a patch file (similar to SwitchMasterToWeb) that can disable/enable the appropriate functionality as a patch? I would rather not touch the default Sitecore install and instead rely on automated deployment of configuration patches.
I haven't seen this as a patch and not sure if its possible to do this with just one patch (would love to be proved wrong), but for something like this I've used a Powershell script.
I set up Octopus Deploy to run a Powershell script step after deploy to disable files and change settings if patch files can't do the job.
I can highly recommend the Powercore tools for this kind of thing.
https://github.com/adoprog/Sitecore-PowerCore/tree/master/Framework/ConfigUtils
If anybody else winds up looking for this, I've posted some work up on GitHub for patch files for a variety of versions for 8.0:
https://github.com/jst-cyr/Sitecore-Role-Configs
The patches there will do the 'disable/enable/change' for authoring, delivery, or processing. I don't have one for the reporting server.
Sitecore has evaluated POC for same. At this point of time applciable for Sitecore CMS 8.1 rev. 160302 (Update-2). See here-
https://github.com/Sitecore/Sitecore-Configuration-Roles