How to use packer export_opts? - virtualbox

I build a VirtualBox VM using Packer and I would like to set some VM meta data (e.g. description, version) using the export_opts parameter. The docs say
export_opts (array of strings) - Additional options to pass to the VBoxManage export. This can be useful for passing product information to include in the resulting appliance file.
I am trying to do this in a bash script calling packer:
desc=' ... some ...'
desc+=' ... multiline ...'
desc+=' ... description ...'
# this is actually done using printf, shortened for clarity
export_opts='[ "version", "0.2.0", "description", "${desc}" ]'
# the assembled string looks OK
echo "export_opts: ${export_opts}"
packer build \
... (more options) ...
-var "export_opts=${export_opts}" \
... (more options) ...
<packer configuration file>
I also tried --version instead of version and putting version and the value into the same string, but none of this works; once exported and re-imported, the VM description is empty.
Does anyone have some working sample code or can help me out with what I'm doing wrong ?
Thank you very much.
Update:
Following Anthony Staunton's approach, I figured out that adding
"export_opts": [ "--vsys", "0", "--version", "0.2.0", "--description", "some test description" ],
to the Packer JSON file does work; passing the same string as --var to Packer does not work.

Fixed the problem at long last, updated the packer documentation with the example below, pull requests pending:
Packer JSON configuration file example:
{
"type": "virtualbox-ovf",
"export_opts":
[
"--manifest",
"--vsys", "0",
"--description", "{{user `vm_description`}}",
"--version", "{{user `vm_version`}}"
],
"format": "ova",
}
A VirtualBox VM description may contain arbitrary strings; the GUI interprets HTML formatting. However, the JSON format does not allow arbitrary newlines within a value. Add a multi-line description by preparing the string in the shell before the packer call like this (shell > continuation character snipped for easier copy & paste):
vm_description='some
multiline
description'
vm_version='0.2.0'
packer build \
-var "vm_description=${vm_description}" \
-var "vm_version=${vm_version}" \
"packer_conf.json"

You may have to specify the data as
in your packer json file
"export_opts": [ "--vsys 0 --version \"0.2.0\"", "{{.Name}} --description \"${desc}\" " ],

Related

Handling dynamic parameter value while creating a stack in AWS

Can someone help me out in handling a dynamic "ParameterValue" in parameter.json file.
I'm running "cloudformation create-stack" and passing in --parameters a parameter.json file, there are few "ParameterValue" in the file that needs to be dynamic for example, timestamp and appending index values from loop etc... so, how can i modify the parameters.json file to handle dynamic values.
Alternate way i could go with is to just not use the parameters.json file and pass in the key, value like below to the create-stack command inside the loop in the script,
--parameters ParameterKey="XYZ",ParameterValue="${someval}${index}"
I would create parameters.json.template file to hold the values in their parameterized form like you show:
[
{
"ParameterKey": "XYZ",
"ParameterValue": "{someval}{index}"
},
{
"ParameterKey": "ABC",
"ParameterValue": "staticval-{suffix}"
}
]
I am assuming you are doing this on the cli, based on the use of the --parameters flag. In that case, I would create a script to merge the template file with the values (into a generated file) and call the create-stack cli command after that.
Something like this on linux:
#! /bin/bash
# create output file from template
cp templates/parameters.json.template generated/parameters.json
# merge dynamic values into templated file
sed -i "s/{someval}/$SOME_VAL/g" generated/parameters.json
sed -i "s/{index}/$INDEX/g" generated/parameters.json
sed -i "s/{suffix}/$SUFFIX/g" generated/parameters.json
aws cloudformation create-stack ... --parameters generated/parameters.json ...
This of course assumes your script has access to your dynamic values.

Is there a way to get the parameters that were passed to a GCP Dataflow job from the CLI/API

I have tried the describe command listed here and I don't see the parameters. Is there another command that I should use to get this information, or some other API that would provide it?
TL;DR - You're missing the --full argument to the gcloud dataflow jobs describe command.
FLAGS
--full
Retrieve the full Job rather than the summary view
View full job info
If you're using gcloud to view the information about the GCP Dataflow job, this command will show the full info (which is actually quite a lot of info) about the job including any parameters that were passed to the job:
gcloud dataflow jobs describe JOB_ID --full
All the options are under the hierarchy environment.sdkPipelineOptions.options
View all options as JSON
To view all the options passed to the job (which prints actually more than just the command line arguments BTW) as a JSON, you can do:
$ gcloud dataflow jobs describe JOB_ID --full --format='json(environment.sdkPipelineOptions.options)'
{
"environment": {
"sdkPipelineOptions": {
"options": {
"apiRootUrl": "https://dataflow.googleapis.com/",
"appName": "WordCount",
"credentialFactoryClass": "com.google.cloud.dataflow.sdk.util.GcpCredentialFactory",
"dataflowEndpoint": "",
"enableCloudDebugger": false,
"enableProfilingAgent": false,
"firstArg": "foo",
"inputFile": "gs://dataflow-samples/shakespeare/kinglear.txt",
"jobName": "wordcount-tuxdude-12345678",
"numberOfWorkerHarnessThreads": 0,
"output": "gs://BUCKET_NAME/dataflow/output",
"pathValidatorClass": "com.google.cloud.dataflow.sdk.util.DataflowPathValidator",
"project": "PROJECT_NAME",
"runner": "com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner",
"secondArg": "bar",
"stableUniqueNames": "WARNING",
"stagerClass": "com.google.cloud.dataflow.sdk.util.GcsStager",
"stagingLocation": "gs://BUCKET_NAME/dataflow/staging/",
"streaming": false,
"tempLocation": "gs://BUCKET_NAME/dataflow/staging/"
}
}
}
}
View all options as a table
$ gcloud dataflow jobs describe JOB_ID --full --format='flattened(environment.sdkPipelineOptions.options)'
environment.sdkPipelineOptions.options.apiRootUrl: https://dataflow.googleapis.com/
environment.sdkPipelineOptions.options.appName: WordCount
environment.sdkPipelineOptions.options.credentialFactoryClass: com.google.cloud.dataflow.sdk.util.GcpCredentialFactory
environment.sdkPipelineOptions.options.dataflowEndpoint:
environment.sdkPipelineOptions.options.enableCloudDebugger: False
environment.sdkPipelineOptions.options.enableProfilingAgent: False
environment.sdkPipelineOptions.options.firstArg: foo
environment.sdkPipelineOptions.options.inputFile: gs://dataflow-samples/shakespeare/kinglear.txt
environment.sdkPipelineOptions.options.jobName: wordcount-tuxdude-12345678
environment.sdkPipelineOptions.options.numberOfWorkerHarnessThreads: 0
environment.sdkPipelineOptions.options.output: gs://BUCKET_NAME/dataflow/output
environment.sdkPipelineOptions.options.pathValidatorClass: com.google.cloud.dataflow.sdk.util.DataflowPathValidator
environment.sdkPipelineOptions.options.project: PROJECT_NAME
environment.sdkPipelineOptions.options.runner: com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
environment.sdkPipelineOptions.options.secondArg: bar
environment.sdkPipelineOptions.options.stableUniqueNames: WARNING
environment.sdkPipelineOptions.options.stagerClass: com.google.cloud.dataflow.sdk.util.GcsStager
environment.sdkPipelineOptions.options.stagingLocation: gs://BUCKET_NAME/dataflow/staging/
environment.sdkPipelineOptions.options.streaming: False
environment.sdkPipelineOptions.options.tempLocation: gs://BUCKET_NAME/dataflow/staging/
Get the value of just a single option
To get the value of just a single option named --argName (whose value BTW is MY_ARG_VALUE), you can do:
$ gcloud dataflow jobs describe JOB_ID --full --format='value(environment.sdkPipelineOptions.options.argName)'
MY_ARG_VALUE
gcloud formatting
gcloud in general supports a wide range of formatting options in the output which is applicable to most gcloud commands which pull info from the server. You can read about them here.

Is there any easy way / API to find out the number of pipelines on a gocd server?

Sorry for the brief question, but just wondering if there's an API to find out the number of pipelines on a GoCD server.
The Pipeline Groups API will give you what you need after some JSON parsing.
$ curl 'https://ci.example.com/go/api/config/pipeline_groups' \
-u 'username:password'
Returns:
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
[
{
"pipelines": [
{
"stages": [
{
"name": "up42_stage"
}
],
"name": "up42",
"materials": [
{
"description": "URL: https://github.com/gocd/gocd, Branch: master",
"fingerprint": "2d05446cd52a998fe3afd840fc2c46b7c7e421051f0209c7f619c95bedc28b88",
"type": "Git"
}
],
"label": "${COUNT}"
}
],
"name": "first"
}
]
You can grab the config.xml file and parse it. from the config repo or via http.
As an alternative, you can just get the cctray file from your server at http://yourgoserver/go/cctray.xml and parse it.
It contains information about all the pipelines (including its stages)
I would recommend using yagocd:
from yagocd import Yagocd
go = Yagocd(server='https://build.gocd.io')
# login as guest
go._session.get('https://build.gocd.io/go/plugin/interact/gocd.guest.user.auth.plugin/index')
print(len(list(go.pipelines)))
Yes, of course. You can get the desired output in different ways. The first easy way to get the number of pipelines and other statistical information from the GoCD support URL (https://example.com/go/api/support) which requires admin privilege.
If the user does not have the admin privilege, we need to go with the GoCD pipeline_groups API. The below command should give you the exact result with jq(JSON processor)
$ curl 'https://example.com/go/api/config/pipeline_groups' -u 'username:password' | jq -r '.[] | .pipelines[].name' | wc -l
NOTE: Still Go Administrator users can get the actual number of pipelines.

how to locate 'imqi.hpp' from node-gyp

I am trying to use "nan" module to call MQ_CONNECT() from node.js
See
Node.js and C/C++ integration: how to properly implement callbacks?
and
https://github.com/nodejs/nan
When I use "node-gyp" it says it can not find "imqi.hpp", the MQ header
As far as I can see, the path to MQ includes has to be provided in "binding.gyp", and I have tried this without success:
{
"targets": [
{
"target_name": "mqconn",
"sources": [
"initall.cc",
"mqconn.cc"
],
"include_dirs": [
"<!(node -e \"require('nan')\")",
"c:\MQ\tools\cplus\include"
]
}
]
}
Does anyboby have a clue on how to fix this ?
Sebastian.
PD.- of course, the file is where the path indicates:
c:\>dir c:\MQ\tools\cplus\include\imqi.hpp
Volume in drive C is OS
Volume Serial Number is 12AA-0601
Directory of c:\MQ\tools\cplus\include
27/06/2013 02:00 1.538 imqi.hpp
Because binding.gyp is in JSON, the String "c:\MQ\tools\cplus\include"is a standard JavaScript String, and therefore the \ needs to be escaped to \\.
So you should replace "c:\MQ\tools\cplus\include" into "c:\\MQ\\tools\\cplus\\include".
I hope that fixes the problem...

How to use Sencha SDK for ExtJS?

I am using ExtJS 4.1 and I am deploying my simple HelloExt program on GlassFish V3.1.
I am trying to create a build from Sencha SDK.
I have used the following two commands...
C:\>sencha create jsb -a http://localhost:8080/HelloExt/index.jsp -p appname.jsb
3 -v
C:\>sencha build -p appname.jsb3 -v -d .
As per the documentation, it will create app-all.js file. But where does it create the file?
How can I know IF build are created successfully or not?
Where are the generated JS files?
I made a search but I can not found anything like app-all.js.
For more information:
I am using JDK 1.6.0_12 and GlassFish V3.1 application server.
Here are the edited content of the question ....
And when I am trying to use the sencha SDK, It generates a .dpf file into the class path.
The contents of the .dpf file as as below ...
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE glassfish-web-app PUBLIC "-//GlassFish.org//DTD GlassFish Application Server 3.1 Servlet 3.0//EN" "http://glassfish.org/dtds/glassfish-web-app_3_0-1.dtd">
<glassfish-web-app error-url="">
<context-root>/HelloExt</context-root>
<class-loader delegate="true"/>
<jsp-config>
<property name="keepgenerated" value="true">
<description>Keep a copy of the generated servlet class' java code.</description>
</property>
</jsp-config>
</glassfish-web-app>
Can anyone tell me Why here it generated .DPF file ? Why its not generating the app-all.js file ?
Try running the command from inside the app root directory and then using a relative path:
0) open cmd window
1) run in cmd window: "cd C:\[webserver_webapp_root]\[app_name]"
In other words change the cmd directory to the app root. Fill in the bracketed text above with the correct paths.
2) run in cmd window: "sencha create jsb -a index.html -p app.jsb3 -v"
The app.jsb3 should be created in your app's root directory (C:\[webserver_webapp_root]\[app_name]). Open it up and make sure it contains all of your app classes, it should look something like this:
{
"projectName": "Project Name",
"licenseText": "Copyright(c) 2012 Company Name",
"builds": [
{
"name": "All Classes",
"target": "all-classes.js",
"options": {
"debug": true
},
"files": [
{
"clsName": "YourApp.view.Viewport",
"name": "Viewport.js",
"path": "app/view/"
},
// plus ALOT more classes...
]
},
{
"name": "Application - Production",
"target": "app-all.js",
"compress": true,
"files": [
{
"path": "",
"name": "all-classes.js"
},
{
"path": "",
"name": "app.js"
}
]
}
],
"resources": []
}
If everything looks fine then you can go onto the next step, if not then there is something wrong with your app directory structure and you need to fix it per Sencha recommended ExtJS application architecture.
You can also use any error messages to help identify the problem.
3) update placeholders ("Project Name", etc) at the top of app.jsb3
4) run in cmd window: "sencha build -p app.jsb3 -d . -v"
The app-all.js file should also be created in the app's root directory. If the cmd window doesn't give any errors before it says "Done Building!" then you are all done. You can now change your index.html script link to point to app-all.js instead of app.js.
If there are errors then you have to fix those and run this again.
Other things you can try:
In response to your last comment, your -p switch parameter should be a jsb3 file not jsb.
Make sure that the web server is running and that your app runs without any errors before you try to use the SDK Tools.
Then try these:
C:\Projects\HelloExt\build\web>sencha create jsb -a index.jsp -p HelloExt.jsb3 -v
C:\Projects\HelloExt>sencha create jsb -a index.jsp -p HelloExt.jsb3 -v
C:\>sencha create jsb -a [actual IP address]:8080/HelloExt/index.jsp -p HelloExt.jsb3 -v
Fill in your actual IP address where the brackets are (not localhost).
This should produce the jsb3 file shown in #2 above then you can move on to step #3 above.