I have a scheduling program running on Server A running Windows 2008 RS. Server B is my SAS server under Windows 2008 R2. How do I kick-off a job on SAS server from my scheduling server? I can either use the sas.exe or a batch file to start my job. Owners of the SAS server tell me that I cannot add an application or Windows service to the SAS server. Is this even possible?
Below is a copy of my answer to a slightly different question (source: http://www.runsubmit.com/questions/260/hide-sas-batch-jobs-winxp). I'm copy/pasting it here for perpetuity and also because it's more likely to help people searching:
You can use PsExec which is part of Microsoft/Sysinternals list of utility programs. This file will go on the scheduling server. Grab it from here:
http://technet.microsoft.com/en-us/sysinternals/bb897553.aspx
The tool is designed to allow you to execute jobs on remote machines. For example, if you want to launch a SAS program from the command line you could run:
psexec \\machinename sas.exe -sysin remotedrivename:\remotefolder\myprogram.sas
This would launch SAS.EXE on the remote machine and run the supplied program that exists on the remote machine. When it launches SAS it appears to launch it within a PsServ service. Because it's running within a service no interface will be displayed. I'm not even sure if you would see it appear as it's own process or application in windows task manager. If you use SysInternals other program, ProcessExplorer, instead of Task Manager you can see this happening.
Note that the REMOTE MACHINE and the LOCAL machine can be the same machine.
PROS: Many other uses for this technique. It's free. PsExec is only required on the machine that is making the call, not both machines.
CONS: Its a bit of a roundabout way to do things. Need to install a third party program (although it is now a MS tool). Some antivirus programs/network admins may not allow it.
Note that if your SAS jobs access network resources then you will probably need to make the network resource available first using the net map command. I suggest running your sas job in a batch file like so (or use the 'x' command from within your SAS file to call the 'net use' commands):
Command executed from local machine:
psexec \\machinename -sysin remotedrivename:\remotefolder\myprogram.BAT
Contents of batch file on remote machine:
net use m: \\fileserver\sharedfolder /USER:mynetworkdomainname\myusername mypassword
sas.exe -sysin remotedrivename:\remotefolder\myprogram.sas
net use m: /delete
Related
I am trying to train a deep learning model on GCP VM which is supposed to run 10-12 hours. But every time it runs for few epoch (few hours), it gives ssh timeout error.
I have set the following parameters on the config file but still the same error.
Can this be a code issue or something to do with VM config parameter.
The first step is to understand the problem, then select the solution.
When you create an SSH connection to a host (your VM), this opens a TCP connection on port 22. The SSH server then runs a shell program in this SSH session. When the SSH session closes or is interrupted, the shell program is killed, which kills most processes that the shell started.
Home Internet providers often reset long-running connections. This is to prevent you from running servers or large downloads over a home Internet connection. There are other reasons that the Internet provider might reset connections that are normal. You need to understand that this will happen and select options to manage this.
There are several programs that are available. Look into tmux and screen.
I use both programs and prefer tmux. Both programs are free and very popular. There are additional methods and programs from running your program in the background & to using nohup.
From Wikipedia:
tmux is a terminal multiplexer for Unix-like operating systems. It allows multiple terminal sessions to be accessed simultaneously in a single window. It is useful for running more than one command-line program at the same time. It can also be used to detach processes from their controlling terminals, allowing SSH sessions to remain active without being visible.
GNU Screen is a terminal multiplexer, a software application that can be used to multiplex several virtual consoles, allowing a user to access multiple separate login sessions inside a single terminal window, or detach and reattach sessions from a terminal. It is useful for dealing with multiple programs from a command line interface, and for separating programs from the session of the Unix shell that started the program, particularly so a remote process continues running even when the user is disconnected.
I'm new at GCP and I'm trying to keep my process running on Jupyter Notebook after shutting down my local PC. Does anyone know how can I do it? Nowaday I open a terminal on my VM run jupter notebook and then after start the process on jupyter I'd like to turn my machine off.
I keep following the process on my cellphone and shutdown on there. Does anyone know how to turn this off automatically when it stops?
Sorry to make two questions at once, but I think that one is related with another. If it does not I can edit and make another one.
This is a technical limitation of Jupyter Notebooks unfortunately. The browser window contains the code which updates the notebook itself, so if you close the browser window then there is not process running to update the notebook.
However, there is one workaround which you may find useful.
There is a library called Fairing that you can use with GCP's new AI Platform Notebooks which allows you to pack up your notebook and run it remotely, and that library will save the results of that execution in a GCP Storage bucket. No active internet connection required (once you kick of the notebook run).
You can learn how to use it by creating a new GCP AI Platform Notebook and looking at the tutorials folder inside it. You can also find additional tutorials for Fairing here
Typically to keep your remote sessions up in the event of network connectivity loss (which also covers shutting down the local computer) you'd use a terminal multiplexer application. From Known issues:
Intermittent disconnects: At this time, we do not offer a specific SLA for connection lifetimes. Use terminal multiplexers like tmux
or screen if you plan to keep the terminal window open for an
extended period of time.
But these multiplexers are terminal/text-mode apps, so you'd have to launch the notebook with the --no-browser and then connect your local browser to its port.
You can find a recipe based on tmux and a local browser connection to the notebook using an SSH tunnel at Using Jupyter notebooks securely on remote linux machines.
As for shutting down the session - you'd just have to instruct the multiplexer application to end the session (or terminate the multiplexer app itself) - which you could do automatically via a wrapper script first invoking your process and immediately after the process ends invoking the commands to shutdown the session.
Normally I've developed locally (on my own machine) and pushed to wherever things needed to go via mapped drives, ftp, github, etc. I have done a bit of work with vagrant/virtualbox (but again, locally) with a shared/mirrored folder.
I am now in a situation where everyone here has access to their own dev box (a vm on the network). I see some working in Vim directly via SSH, I believe, but I'm not there yet. So I'm left with the question: What's the best way for (more of a front end guy) to approach this?
I have heard of doing an SSH-mount from my workstation... if that's a viable thing. I'm curious what everyone's take on this kind of environment is and (perhaps) any best practices. Tips, links, and reading is highly welcome and appreciated, too... any pointing in a good direction would be wonderful.
Thank you.
The best answer will come from what virtual resources do you want to capitalize on for the virtual networked VMs. If you just want the storage space, then share the VM's drives, and mount them locally, treat them as local, end of story. If you want to run all the processing on the remote machine, and connect from a thin client, you have a couple of options, but they all take the same form. Connect to the machine, edit the files on the remote machine. Depending on your OS, you will have different options available.
If the remote machine doesn't have an graphical client installed you are stuck with either, mounting the remote share locally (you can use whatever editor you want) or ssh to the remote machine and using a commandline editor (vim, nano, emacs).
If there is a graphical client installed you have more options:
Remote in the server using any visual viewer (mstsc for windows, vnc is an option), and then use any remotely installed editor of your choice.
Remote in using ssh -X, and then run the remotely installed editor. Assuming you have an X-Server locally (if you are running linux you already do), the GUI part of the application will be run on the client side of the ssh tunnel, and the process will be run on the server. This is probably the best option.
So:
Make sure the remote server has a desktop client software (gtk, kde, gnome, almost any windows os, etc...)
install GUI editor of your choice on that server
ssh -X to that server
install sublime text, geany, or your choice of editor
run subl, geany, or other to start the application.
SSH mounting would indeed allow you to use all of the files on the VM as if they were stored in your local machine, letting you edit and update files without having to manually copy them every time you perform changes. You will run into a speed bump though, since files changed will have to be synchronized/copied to your remote machine every time and that takes a couple of seconds. Check this post by DigitalOcean, they explain how to get the SSH mount working.
A better option you have (IMHO) is to use an IDE in your local machine that allows you to push changes to a server after saving or by manually doing so. This would allow you to develop faster by using your local resources (local web server) since no files would have to be copied over the network to the remote VM; and would also allow you to test on that remote VM when needed by uploading the files when you are ready to test on that environment.
PS: Exporting visual apps or environments form the remote machine to your local one can be slow (depending on your network and the VM host load running your machine). If you still like that approach, you could also install something to access that VM over something more standard and lightweight like RDP for GNU/Linux (xrdp).
I'm using an Amazon compute instance with Windows Server 2012 R2 to run some executable I own for data processing.
Right now, what I do it to send my data via FTP (I set up an FTP server on the remote Windows machine), and manually start the data processing. When the processing is completed, I download the outputs back from FTP and manually stop the remote Amazon computing instance.
I want to automate this process. Namely, I want to find a way to automatically start the remote machine when I start sending my data, then automatically trigger the processing (this I can handle via scripting), and then send back the data and shut down the machine automatically (this I think I also can handle).
So, to sum up, I need to know how can I automatically start the machine when I send my data to it.
I am using an FTP server on that machine and an EBS drive, but there may be a better way. Also, does anyone have any more suggestions on this setup?
Thank you
There are many ways to automate this. Is your control machine (from where you will be controlling the EC2 instance) a linux or windows machine?
Ansible: It is the easiest and the most straightforward if you are familiar with ansible. Barely 20 lines of code to achieve what you want. And it is free. You will be using EC2 module to start/stop your instances and one of many modules to transfer files. However, there is a bit of learning.
AWS CLI: A one line command to start (or stop) your instance. Once the instance is up and running, you can automate the file transfer part
My problem is simple, I have 1 computer conected to many powerfull servers. I want to execute the app locally but run the process (heavy load) in the remote servers.
The app+settings vary a lot, and I want that this exactly version of the app+settings folder to be used by the remote instances.
My approach so far:
Launch the app locally
Use PSEXEC to remote launch the same executable as it is running in local -> in the servers (with a random port number passed by argument)
Contect to them via sockets
Send commands to execute remotely and get the results
My problem relies in the config files, wich are many(50+) and some of them +4MB. This config files are TXT files in a config folder.
What is the proper way to do it? Is it possible to use PSEXEC to copy remotely a whole folder? Can I do any good trick on the sockets to directly pass a copy of the local files to remote?
I would like all the process to be semi-transparent. Since many people will use it with different versions and settings at the same time. So manually copying the files to 20+servers is NOT an option.
Thank you!
Put the program/script that you want to execute by all machines on one common location on local network (put your configs there too). On all servers create a batch file say 'runme.bat' that will execute your program directly from network location.
This way you can use psexec to run runme.bat essentially executing your program/script on any server you want.
Since often - there are issues using psexec - you may invoke your scripts from Task Scheduler etc.
I do that for 500+ servers and it works. If working for me it will work for you.
You might want to look at HTCondor (http://research.cs.wisc.edu/htcondor/) which could perhaps manage all of this for you.