I want to run my bat file on windows with the help of GCP Composer, but i am not sure if we can communicate with windows machine as composer is fully based on linux environment. Please help me if you have any solution.
There are a couple possible solutions described in this thread, basically:
Installing ssh into your Windows machine and then connecting to run commands remotely using the Airflow’s ssh operator.
Install a package like pywinrm, which allows you to run Windows commands on a target machine from Python code. Then, use the Python operator, within your DAG, to make the call. You may refer to the GCP documentation for steps on installing additional Python packages in Composer.
Related
I want to deploy a python project to ubuntu instance on aws from a windows operating system, but all tutorials I have encountered either use ubuntu or mac as their development/local machine.
Is the deployment from windows the same i.e. after createing the instance all I would then be doing from the local windows system would be running inside the ubuntu instance?
is there any tutorial which can help me achieve my objective?
Note i am deploying directly without git.
ANy help would be appreciated
To transfer files to an ubuntu instance you could use SSH, from windows you could download an SSH client such as Mobaxterm (https://mobaxterm.mobatek.net/)! or download Windows Subsystem for Linux (https://learn.microsoft.com/en-us/windows/wsl/install-win10)! then use SCP to copy files (https://www.computerhope.com/unix/scp.htm)!. Both options require that you have the .pem security file for your instance
Your best bet is going to be installing WSL(Windows Subsystem for Linux) and using that to run bash commands. This will make your life a whole lot easier as you won't have to look for Windows specific tutorials and you can now follow Ubuntu/Linux tutorials.
What is WSL? It is essentially a Linux VM built into Windows. It will provide you with a terminal running Ubuntu or pretty much any other Linux distro you could want.
How to install WSL
I am looking for a help on GCP where I want to create a Windows VM and which will have Java and some browsers like say Chrome. Once this is done I wanted to integrate this VM to Jenkins such that whenever a automated build runs in Jenkins it will run those automated tests say Selenium on VM machine and creates the reports and so on. Is it possible via GCP. Please let me know and guide me on this and please share any tutorial for sample.
Thanks a lot.
I don't think any of the images provided by GCP have that software installed, I mean you need to install manually or you can use startup-script to automate some of this task,
this is a quick information to get you started:
Create Windows Instance
Install java or JDK
Install chrome
Install jenkins
Automate the task with jenkins and windows
As alternative you can deploy from Marketplace, find the Jenkis which is installed on a Windows VM and then install the other components(chrome & java)
considers that some marketplace solutions has an additional cost
I'm using AWS Lambda, which involves creating an archive of my node.js script, including the node_modules folder and uploading that to their infrastructure to run.
This works fine, except when it comes to node modules with native bindings (using node-gyp). Because the binding was complied and project archived on my local computer (OS X), it is not compatible with AWS's (Amazon Linux) servers.
How can I cross-compile/install a node module (specifically, node-sqlite3) so when I upload it to another server arch it runs?
While not really a solution to your problem, a very easy workaround could be to simply compile the native addons on a Linux machine.
For your particular situation, I would use Vagrant. Vagrant can create virtual machines and configure them within seconds.
Find an OS image that resembles Amazon's Linux distro (Fedora, CentOS, others that use yum as package manager - see Wiki)
Use a simple configuration script that, when run by Vagrant on machine startup, will run npm install (optionally it might also remove the node_modules folder before to ensure a clean installation)
For extra comfort, the script can also create the zip file for deployment
Once the installation finishes, the script will shutdown the VM to avoid unnecessary consumption of system resources
Deploy!
It might require some tuning if the linked libraries are not at the same place on the target machine but generally this seems to me like the best and quickest solution.
While installing the app using Vagrant might be sufficient in some cases, I have found it necessary to build the app on Linux which is as close to Lambda's Amazon Linux AMI as possible.
You can read the original answer here: https://stackoverflow.com/a/34019739/303184
Steps to make it work:
Spawn new EC2 instance. Make sure it is based on exactly the same image as your AWS Lambda runtime. You can review Lambda env details here: http://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html. In our case, it was Amazon Linux AMI called amzn-ami-hvm-2015.03.0.x86_64-gp2.
Install nvm and use it to install the same version of Node.js as on the AWS Lambda. At the time of writing this, it was v0.10.36. You can refer to http://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html again to find out.
You will probably need to install git & g++ compiler on the EC2. You can do this running
sudo yum install git gcc-c++
Finally, clone your app to your new EC2 and install your app's dependecies:
nvm use 0.10.36
npm install --production
You can then easily download the node_modules using scp or such.
Same lines as Robert's answer, when I had to work on my MAC in a different OS I use vm ware like Oracle's free virtualizer VirtualBox to get a linux on my mac, no cost to me. Or sign up for a new AWS account, you get a micro for a year free. Use that to get your linux box, do whatever you need there.
AWS has a page describing how to deal with native NPM modules: https://aws.amazon.com/blogs/compute/nodejs-packages-in-lambda/
I'm trying to automate deploying code to my 3 GCE Linux VM's. I read this article Scripting with gcloud: a beginner’s guide to automating GCP tasks, it shows how to make a script. Now I assume that means saving the code as a .sh file (it even has a shebang on top), now how do I run that. Do I type the script file name in the Google Cloud SDK Shell? I tried it, it does not seem to work. can someone help me? I will really appreciate.
Here is an image of my google cloud shell where I am trying to use the script files.
You're able to install Google Cloud SDK on variety of operation systems such as Linux, macOS and Windows. After that, you'll be able to use same commands like gcloud, gsutil and bq. Meanwhile, scripting relies on the command-line interpreters: you can use bash with Linux and macOS, but for Windows you should use cmd and PowerShell. You can run examples provided at the article, you've mentioned, and at the documentation Scripting gcloud CLI commands with bash on Linux and macOS, so the error messages you've got were expected. You can't run .sh scripts on windows naively, as it was mentioned by #Pievis at the comment section.
As a possible workaround you can install Windows Subsystem for Linux (WSL) for Windows 10 (usually you can choose between WSL2 and WSL1, but it depends on build version of your Windows 10) to get some interoperability between Windows and Linux.
If you need to transfer files to you VM instances please follow the documentation Transferring files to VMs.
If you are interested in automation with GCP, please have a look on the documentation Infrastructure as code to "automate repeatable tasks like provisioning, configuration, and deployments".
Usually the developer can use Softbanks own software Choreography to give programs to Pepper robot.
Isn't there a way to setup a different development environment? e.g. Access via SSH and creating Python scripts with a simple text editor and starting the script manually? It means writing and starting Python scripts for Pepper without using Choreography.
You can also use qibuild (pip install qibuild) : https://github.com/aldebaran/qibuild
It contains a qipkg command, just run
qipkg deploy-package path/to/your/file.pml --url USER#IP:/home/nao
A pml file is a project, it is created by Choregraph, or you can use this tool :
https://github.com/pepperhacking/robot-jumpstarter
in order to get a sample app.
Of course, using Choregraphe is not an obligation, you can use the different SDKs directly.
You can for instance create a python script on your computer, copy it on the robot
scp path/to/script/myscript.py nao#robotIp
And then ssh onto the robot and launch the script
ssh nao#robotIp
python myscript.py
You can also ssh onto the robot, create a script (using nano for instance) and launch it from there.
I've been using Pycharm Pro for 6 months and I am happy with it. You get automatic deployment and remote debugging. The most basic setup must still be done with Choregraphe, but it takes less than one minut.